Improvement of Vehicle Stability Using Reinforcement Learning

Janaína R. Amaral; Harald Göllinger; Thiago A. Fiorentin

doi:10.5753/eniac.2018.4420

Janaína R. Amaral UFSC
Harald Göllinger Technische Hochschule Ingolstadt
Thiago A. Fiorentin UFSC

DOI: https://doi.org/10.5753/eniac.2018.4420

Resumo

This paper presents a preliminary study on the use of reinforcement learning to control the torque vectoring of a small rear wheel driven electric race car in order to improve vehicle handling and vehicle stability. The reinforcement learning algorithm used is Neural Fitted Q Iteration and the sampling of experiences is based on simulations of the vehicle behavior using the software CarMaker. The cost function is based on the position of the states on the phase-plane of sideslip angle and sideslip angular velocity. The resulting controller is able to improve the vehicle handling and stability with a significant reduction in vehicle sideslip angle.

Referências

Akbari, A. A. and Goharimanesh, M. (2014). Yaw Moment Control Using Fuzzy Reinforcement Learning. In International Symposium on Advanced Vehicle Control.

Benbrahim, H. and Franklin, J. A. (1997). Biped dynamic walking using reinforcement learning. In Robotics and Autonomous Systems 22, pages 283-302. Elsevier.

Desjardins, C. and Chaib-Draa, B. (2011). Cooperative adaptive cruise control: A reinforcement learning approach. In IEEE Transactions on intelligent transportation systems, pages 1248-1260. IEEE.

Duan Y., Cui B. and Yang H. (2008). Robot Navigation Based on Fuzzy RL Algorithm. In: Sun F., Zhang J., Tan Y., Cao J. and Yu W. (eds) Advances in Neural Networks. Lecture Notes in Computer Science, vol 5263. Springer, Berlin, Heidelberg.

Gillespie, T. D. (1992), Fundamentals of Vehicle Dynamics, Society of Automotive Engineers.

Goharimanesh, M. and Akbari, A. A. (2017). Improving lateral dynamic of vehicle using direct yaw moment controller by differential brake torques based on quantitative feedback theory. In Scientia Iranica. Transaction B, Mechanical Engineering, pages 662-672.

Guo, J., Chu, L., Liu, H., Shang, M. and Fang, Y. (2010). Integrated control of active front steering and electronic stability program. In International Conference on Advanced Computer Control (ICACC), pages 449-453. IEEE.

He, J. (2005). Integrated vehicle dynamics control using active steering, driveline and braking (Doctoral dissertation, University of Leeds).

Howell, M. N., Frost, G. P., Gordon, T. J. and Wu, Q. H. (1997). Continuous action reinforcement learning applied to vehicle suspension control. In Mechatronics, pages 263–276. Elsevier.

Inagaki, S., Kshiro, I. and Yamamoto, M. (1994). Analysis on vehicle stability in critical cornering using phase-plane method. In International Symposium on Advanced Vehicle Control, pages 287-292.

Janusz, B. and Riedmiller, M. (1995). Self-learning neural control of a mobile robot. In IEEE International Conference on Neural Networks, Vol. 5, pages 2358-2363. IEEE.

Lange, S., Gabel, T. and Riedmiller, M. (2012) “Batch Reinforcement Learning”, In: M. Reinforcement Learning. Adaptation, Learning, and Optimization, Edited by Marco Wiering and Matijn van Otterlo, vol 12, Springer, Berlin, Heidelberg.

LeCun, Y., Bottou, L., Orr, G. B. and Muller, K.R. (1998). Efficient BackProp. In Neural Networks: Tricks of the trade. Springer.

Lee, M., Hwang, K. and Suh, I. S. (2015). Independent and Integrated Torque Control of 4-Wheel Drive Electric Vehicle for Automated Driving. In International Electric Vehicle Symposium and Exhibition.

Lu, Q., Gentile, P., Tota, A., Sorniotti, A., Gruber, P., Costamagna, F. and De Smet, J. (2016). Enhancing vehicle cornering limit through sideslip and yaw rate control. In Mechanical Systems and Signal Processing, pages 455-472. Elsevier.

Oh, S. Y., Lee, J. H. and Choi, D. H. (2000). A new reinforcement learning vehicle control architecture for vision-based road following. In IEEE Transactions on Vehicular Technology, pages 997-1005. IEEE.

Pietquin, O., Tango, F. and Aras, R. (2011). Batch reinforcement learning for optimizing longitudinal driving assistance strategies. In IEEE Symposium on Computational intelligence in vehicles and transportation systems, pages 73-79. IEEE.

Riedmiller, M. (2005). Neural fitted Q iteration–first experiences with a data efficient neural reinforcement learning method. In: European Conference on Machine Learning, pages 317-328. Springer, Berlin, Heidelberg.

Riedmiller, M., Montemerlo, M. and Dahlkamp, H. (2007). Learning to drive a real car in 20 minutes. In Frontiers in the Convergence of Bioscience and Information Technologies, pages 645-650. IEEE.

Riedmiller, M., Gabel, T., Hafner, R., and Lange, S. (2009). Reinforcement learning for robot soccer. In Autonomous Robots, pages 55–73. Springer.

Sutton, R. S. and Barto, A. G. (1998), Reinforcement Learning: an Introduction, MIT Press.

UN ECE (2014). Addendum 12-H: Regulation No. 13-H, third edition.

Vincent, I. and Sun, Q. (2012). A combined reactive and reinforcement learning controller for an autonomous tracked vehicle. In Robotics and Autonomous Systems, pages 599-608. Elsevier.

Wang, P., Chan, C. Y. and de La Fortelle, A. (2018). A Reinforcement Learning Based Approach for Automated Lane Change Maneuvers. In 2018 IEEE Intelligent Vehicles Symposium. IEEE.

Wiering, M. and Van Otterlo, M. (2012), Reinforcement learning. Adaptation, learning, and optimization, Springer, v. 12.

Yu, G. and Sethi, I. K. (1995). Road-following with continuous learning. In Intelligent Vehicles' 95 Symposium, pages 412-417. IEEE.