Publications

The vehicle control systems domain encounters an increasing number of parameters and calibration targets considering the emerging technologies such as connected vehicles and automated driving. Accordingly, the calibration processes for such systems have become more complex and thus error prone and tedious. Moreover, the derived control policies are not easily transferable between different vehicle configurations, hence, the calibration effort is increasing dramatically with each configuration change. Therefore, the reduction of such efforts needed to setup the control policy is inevitable to further reduce cost of drivetrain development. The proposed methodology therefore is an important means to make BEVs (Battery Electric Vehicles) more attractive for car buyers.

The fast development of the Artificial Intelligence (AI) domain is opening the door to numerous opportunities and applications in the automotive industry. We utilize Reinforcement Learning (RL) techniques to design appropriate control strategies for different vehicle systems, thus improving the conventional approaches and reducing the development effort. Combining the expertise in simulation and big data, we propose a cloud-based solution that runs a high-fidelity simulation to train, test and deploy the thermal management control strategy for a fleet of BEVs. The benefits, among others, are that the RL-based control policies can be designed more rapidly and run more efficient than the traditional rule-based approaches. After deploying the initial model trained against the simulation, we have the capability of collecting data from a fleet of vehicles operating with the latest control strategy. Using collected data, we iteratively train and customize the strategy throughout the operation time.

We have tested the above-mentioned framework on the use case of cabin heating mode selection for BEVs. Our RL agents are trained and evaluated in a model-in-the-loop simulation environment. The policy evaluation is based on the agents’ performance on representative vehicle test measurements (drive cycles). The metrics are selected to quantify the energy efficiency and comfort individually, as well as aggregated to enable a fair comparison. Notably the trained agents achieved better results than the original control policy on most of the individual metrics and significantly better results on the aggregated metric.

At this moment, our framework is tested against simulated vehicle fleet. The first reasonable research question is if the trained control system can be directly transferred to the real vehicle, or whether additional adjustments must be performed to achieve the needed flexibility. Another open question refers to the adequate combination of RL algorithms to achieve even better performance on telemetry data from a connected fleet. Time will tell if the idealized case with continuing on-policy training, or the more complex case using the policy-agnostic offline algorithms, will provide the stronger solution. The main technical contribution of our work is the use case agnostic framework which iteratively improves a conventional rule-based control strategy. Following the automotive V-model, the design-, implementation-, and testing -phase is strictly separated from the in-use phase of a vehicle function. To leverage historic data from the in-use phase, our framework disrupts this classical model and embeds the DevOps and ML-Ops practices into the automotive engineering process.

In this article we suggest a framework which automates the complex and time-consuming creation process of control strategies. The trained control policies provide better results and allow for a continuous improvement after they are finally deployed on a fleet of vehicles.

As the adoption and development of battery electric vehicles (BEV) increases globally with the aim to eliminate tail-pipe emissions of the incoming generation of vehicles, the automotive industry faces new challenges to maintain cost and performance parity of the new battery vehicles with their internal combustion engine (ICE) counterparts. These challenges include the temperature sensitivity of powertrain components, user range anxiety, bottlenecks in charging infrastructure and crucially, the additional burden of auxiliary consumers such as climate control.

Yet, these challenges coincide with a new era of high-speed communication and connectivity which pose many opportunities for connected vehicles to solve and optimize complex control problems. Intelligent control solutions based on predicted knowledge of future driving conditions can be used to reduce the overall energy consumption of a vehicle journey, increasing potential range, and allowing the use of smaller, cheaper powertrain components. Furthermore, such intelligent control solutions when applied to the thermal management system could help reduce stress on thermally sensitive components, increasing their efficiency and lifespan.

We capitalized on a proposed holistic BEV thermal management system model developed at AVL to study the potential gains in energy consumption if using connected control optimization. The system model utilizes the idea that three key domains can be interchangeably defined as ‘heat sinks’ or ‘heat rejectors’, thus allowing for the transfer of excess heat energy from where it is not required to where it is. The three key domains include the battery, electric machine, and cabin air temperature. Depending on the state of each domain and the ambient conditions, heat energy can be transferred via heat pumps, allowing for holistic control of these three thermal states. This thermal management system operates categorically and is made of up of eight pre-engineering control modes which determine how heat energy is transferred around the vehicle. For heating purposes, if sufficient heat energy is not present in the system or atmospheric conditions, the electric heater is used, drawing energy from the battery.

Selecting which mode to use and when poses a complex control problem, but if implemented well, can reduce battery energy consumption while ensuring passenger comfort. Future knowledge of the journey makes it possible to model the predicted state conditions and apply optimization techniques to select the most suitable sequence of control modes for a given journey. This paper evaluates the application of Dynamic Programming (DP) for use with this categorical type of control system. The DP algorithm views the problem as a whole, therefore providing a global optimum solution based on a full journey, as opposed to a rule-based system which makes decisions based only on current and past state conditions. DP is computationally intensive and is therefore typically applied as an “offline” optimizer from which to benchmark, thus, this research acts as a precursor to the development of a real-time optimized controller. As the DP results identify the best possible system gains, these will be used as a benchmark when implementing an online optimized controller, and to assess the level of investment appropriate considering the available gains.

This study has used a thermal management model and integrated this with the highly optimized Dynamic Programming MATLAB toolbox, “DPM”, developed at ETH Zurich by Sundstrom, O et al. Initial simulations over the Worldwide Harmonized Light Vehicles Test Procedure (WLTP) resulted in a maximum improvement in cold ambient conditions where the powertrain components are at their least efficient. The potential gains diminish at higher ambient temperatures as the powertrain components are closer to their operating windows. The results of this study provide a valuable insight into the optimization of the thermal management of a BEV which is potentially critical for their success as a long-term viable alternative to ICE powered vehicles.