Reinforcement Learning for Semi-Active Vertical Dynamics Control with Real-World Tests
<p>The DLR’s test vehicle AFM on a four-post test rig (adopted from [<a href="#B20-applsci-14-07066" class="html-bibr">20</a>]).</p> "> Figure 2
<p>Overview of the whole reinforcement learning toolchain utilized in this work (adapted from [<a href="#B20-applsci-14-07066" class="html-bibr">20</a>]).</p> "> Figure 3
<p>Damper force–velocity characteristics for different damper currents for AFM’s (<b>a</b>) front axle and (<b>b</b>) rear axle (compare [<a href="#B20-applsci-14-07066" class="html-bibr">20</a>]).</p> "> Figure 4
<p>Input-to-force dynamics for different current steps and different damper velocities with (<b>a</b>) a rising current step and (<b>b</b>) a falling current step. The variables are depicted as fraction of their start or end value over time. In addition to the measurement data, the input signal and the result of the fit are plotted.</p> "> Figure 5
<p>Comparison of AFM’s frequency response from (<b>a</b>) road displacement <math display="inline"><semantics> <mrow> <msub> <mrow> <mi>z</mi> </mrow> <mrow> <mi>r</mi> <mi>o</mi> <mi>a</mi> <mi>d</mi> </mrow> </msub> </mrow> </semantics></math> to the acceleration of the vehicle body <math display="inline"><semantics> <mrow> <msub> <mrow> <mi>a</mi> </mrow> <mrow> <mi>b</mi> <mi>o</mi> <mi>d</mi> <mi>y</mi> </mrow> </msub> </mrow> </semantics></math> for different constant damper currents and (<b>b</b>) from road displacement <math display="inline"><semantics> <mrow> <msub> <mrow> <mi>z</mi> </mrow> <mrow> <mi>r</mi> <mi>o</mi> <mi>a</mi> <mi>d</mi> </mrow> </msub> </mrow> </semantics></math> to the dynamic wheel load <math display="inline"><semantics> <mrow> <msub> <mrow> <mi>F</mi> </mrow> <mrow> <mi>z</mi> </mrow> </msub> </mrow> </semantics></math> for different constant damper currents for the front left side of the vehicle. Each subplot visualizes the measurement data, the data obtained from an optimized simple QVM model, as well as the resulting data obtained from the optimized best QVM model structure.</p> "> Figure 6
<p>Comparison of AFM’s frequency response from (<b>a</b>) road displacement <math display="inline"><semantics> <mrow> <msub> <mrow> <mi>z</mi> </mrow> <mrow> <mi>r</mi> <mi>o</mi> <mi>a</mi> <mi>d</mi> </mrow> </msub> </mrow> </semantics></math> to the acceleration of the vehicle body <math display="inline"><semantics> <mrow> <msub> <mrow> <mi>a</mi> </mrow> <mrow> <mi>b</mi> <mi>o</mi> <mi>d</mi> <mi>y</mi> </mrow> </msub> </mrow> </semantics></math> for different constant damper currents and (<b>b</b>) from road displacement <math display="inline"><semantics> <mrow> <msub> <mrow> <mi>z</mi> </mrow> <mrow> <mi>r</mi> <mi>o</mi> <mi>a</mi> <mi>d</mi> </mrow> </msub> </mrow> </semantics></math> to the dynamic wheel load <math display="inline"><semantics> <mrow> <msub> <mrow> <mi>F</mi> </mrow> <mrow> <mi>z</mi> </mrow> </msub> </mrow> </semantics></math> for different constant damper currents for the rear left side of the vehicle. Each subplot visualizes the measurement data, the data obtained from an optimized simple QVM model as well as the resulting data obtained from the optimized best QVM model structure.</p> "> Figure 7
<p>Basic RL agent environment setting (adapted from [<a href="#B27-applsci-14-07066" class="html-bibr">27</a>]).</p> "> Figure 8
<p>Illustration of the force jump reward term <math display="inline"><semantics> <mrow> <msub> <mrow> <mi>r</mi> </mrow> <mrow> <mi mathvariant="normal">f</mi> <mi mathvariant="normal">j</mi> </mrow> </msub> <mo>(</mo> <msub> <mrow> <mi>v</mi> </mrow> <mrow> <mi mathvariant="normal">d</mi> </mrow> </msub> <mo>,</mo> <mi mathvariant="sans-serif">Δ</mi> <mi>u</mi> <mo>)</mo> </mrow> </semantics></math> for parameters <math display="inline"><semantics> <mrow> <msub> <mrow> <mi>k</mi> </mrow> <mrow> <mi mathvariant="normal">f</mi> <mi mathvariant="normal">j</mi> </mrow> </msub> <mo>=</mo> <mn>20</mn> </mrow> </semantics></math>, <math display="inline"><semantics> <mrow> <msub> <mrow> <mi>θ</mi> </mrow> <mrow> <msub> <mrow> <mi>v</mi> </mrow> <mrow> <mi mathvariant="normal">d</mi> </mrow> </msub> </mrow> </msub> <mo>=</mo> <mn>0.01</mn> </mrow> </semantics></math>, and <math display="inline"><semantics> <mrow> <msub> <mrow> <mi>θ</mi> </mrow> <mrow> <mi mathvariant="sans-serif">Δ</mi> <mi>u</mi> </mrow> </msub> <mo>=</mo> <mn>0.01</mn> </mrow> </semantics></math>. The colors emphasize the values of <math display="inline"><semantics> <mrow> <msub> <mrow> <mi>r</mi> </mrow> <mrow> <mi mathvariant="normal">f</mi> <mi mathvariant="normal">j</mi> </mrow> </msub> </mrow> </semantics></math> beginning from 0.0 in dark blue to 1.0 in yellow.</p> "> Figure 9
<p>Structure of the benchmark controller.</p> "> Figure 10
<p>Verification and application toolchain for the trained RL agents.</p> "> Figure 11
<p>(<b>a</b>) Rendering of the AFM’s full-vehicle model simulation setup (adapted from [<a href="#B20-applsci-14-07066" class="html-bibr">20</a>]) and (<b>b</b>) part of the ISO 8608 type B road height profile used as excitation for verification.</p> "> Figure 12
<p>Time domain plots of the FVM simulation setup subject to excitation with ISO 8608 road type B with a velocity of <math display="inline"><semantics> <mrow> <mn>95</mn> <mo> </mo> <mrow> <mrow> <mi mathvariant="normal">k</mi> <mi mathvariant="normal">m</mi> </mrow> <mo>/</mo> <mrow> <mi mathvariant="normal">h</mi> </mrow> </mrow> </mrow> </semantics></math>. All signals are exemplary, shown for the front left side of the vehicle.</p> "> Figure 13
<p>Real-world test drive on a bumpy road.</p> "> Figure 14
<p>Normalized performance metrics of the trained controller as pareto plots on (<b>a</b>) sine sweep, (<b>b</b>) synchronous synthetic road excitations, (<b>c</b>) real-road replays, and (<b>d</b>) asynchronous synthetic road excitations. Metrics smaller than <math display="inline"><semantics> <mrow> <mn>1</mn> </mrow> </semantics></math> represent a superior performance of the RL agent, and metrics greater than <math display="inline"><semantics> <mrow> <mn>1</mn> </mrow> </semantics></math> correspond to a superior performance of the benchmark controller. (Remark: Due to a corrupted measurement, road type B is missing in subplot (<b>b</b>)).</p> ">
Abstract
:1. Introduction
1.1. Related Work
1.2. Contribution and Overview of This Work
- We address the complete process of applying DRL to the semi-active suspension control problem in great detail. This process includes taking measurements, deriving a training model, training the controller, verifying the controller in simulation, and conducting real-world tests.
- In our approach, we propose to optimize the QVM model structure as well as the QVM parameters in order to obtain an accurate training model. Additionally, we show that the optimized model structure is able to approximate the real measurement data better than a standard two-mass QVM.
- We propose to train the controller on different QVMs, which represent the different corners of the vehicle to avoid overfitting. Additionally, we train on different excitation types to make the resulting controllers more robust. The whole training process, including the design of the reward function and the selection of the trained agent, is presented in great detail.
- We evaluate the resulting controller in real-world tests on a four-post test rig. The selected RL agent was able to outperform an offline-optimized benchmark controller on road-like excitations, improving the comfort criterion by about 2.5% and the road-holding criterion by about 2.0% on average.
2. The Vertical Dynamics RL Controller Design Process
3. Modeling and Parameter Optimization of the Training Model
3.1. Selection of the Training Model Structure
3.2. Damper Identification and Modeling
3.3. Quarter-Vehicle Modeling
4. Training the Controller
4.1. The Reinforcement Learning Setting
4.2. Application to the Vertical Dynamics Problem
4.2.1. The Training Setup
4.2.2. Environment Interface
- acceleration sensors at all four-wheel carriers;
- acceleration sensors at the front left, front right, and rear left chassis;
- displacement sensors between the chassis and wheel carrier at all four wheels;
- current sensor for each damper.
4.2.3. Reward Function Design
4.2.4. Agent Performance Assessment
5. Control System Verification
5.1. Full-Vehicle Model
5.2. Vertical Dynamics Control System
- Prediction module for vehicle body accelerations:
- Feed-forward control:
- Safety module:
5.3. Design and Optimization of the Benchmark Controller
5.4. Verification Toolchain
5.5. Verification Simulation
6. Real-World Test
6.1. Results from the Test Drives
6.2. Results from the Four-Post Test Rig
7. Summary and Outlook
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
Appendix A
Appendix A.1. Results of the QVM Structure and Parameter Optimization
Front Left (FL) | Front Right (FR) | Rear Left (RL) | |
---|---|---|---|
QVM Structure | Engine QVM | Engine QVM | Topmount QVM |
Body mass | |||
Wheel mass | |||
Suspension spring stiffness | |||
Tire spring stiffness | |||
Tire damping | |||
Spring ratio | |||
Spring ratio | |||
Damper ratio | |||
Damper ratio | |||
Engine mass | |||
Engine bearing stiffness | |||
Engine bearing damping | |||
Topmount bearing stiffness [ ] | |||
Topmount bearing damping | |||
Damper friction force |
Appendix A.2. List of Measured Signals during the Four-Post Test Rig Experiments
Measurand | No. of Signals | Sensor Type | Comments | Availability | |
---|---|---|---|---|---|
Test Rig | Vehicle | ||||
Post position | 4 | N.A. | Integrated test rig sensors | ✓ | ✗ |
Post velocity | 4 | N.A. | ✓ | ✗ | |
Post acceleration | 4 | Accelerometer | ✓ | ✗ | |
Wheel load | 4 | Load cell | ✓ | ✗ | |
Wheel acceleration | 4 | Accelerometer | - | ✓ | ✓ |
Body acceleration | 3 | Accelerometer | At rear axle only one sensor on the left-hand side | ✓ | ✓ |
Deflection wheel–chassis | 4 | Linear potentiometer | Only available on test rig | ✓ | ✗ |
Engine deflection | 3 | Linear potentiometer | - | ✓ | ✓ |
Engine acceleration | 1 | Accelerometer | - | ✓ | ✓ |
Damper deflection | 4 | Rotary potentiometers | - | ✓ | ✓ |
Tire deflection | 2 | Laser sensor | Single-sided, front and rear. Only available on test rig | ✓ | ✗ |
Damper current | 4 | Hall effect sensor | - | ✓ | ✓ |
Appendix A.3. Hyperparameters Used for Training the RL Agent
Hyperparameter | Value |
---|---|
n-timesteps | 106 |
policy | “MlpPolicy” |
policy_kwargs | “dict(net_arch = [64, 64])” |
learning_rate | 10−5 |
buffer_size | 106 |
learning_starts | 100 |
batch_size | 256 |
tau | 0.005 |
gamma | 0.99 |
train_freq | 1 |
gradient_steps | 1 |
ent_coef | “auto” |
use_sde | false |
Appendix A.4. Parametrization of the Reward Function
Parameter | Value |
---|---|
0.5 | |
2 | |
References
- Karnopp, D.; Crosby, M.; Harwood, R. Vibration Control Using Semi-Active Force Generators. J. Eng. Ind. 1974, 96, 619–626. [Google Scholar] [CrossRef]
- Valášek, M.; Novák, M.; Šika, Z.; Vaculín, O. Extended Ground-Hook—New Concept of Semi-Active Control of Truck’s Suspension. Veh. Syst. Dyn. 1997, 27, 289–303. [Google Scholar] [CrossRef]
- Poussot-Vassal, C.; Spelta, C.; Sename, O.; Savaresi, S.; Dugard, L. Survey and performance evaluation on some automotive semi-active suspension control methods: A comparative study on a single-corner model. Annu. Rev. Control 2012, 36, 148–160. [Google Scholar] [CrossRef]
- Savaresi, S.; Spelta, C. Mixed Sky-Hook and ADD: Approaching the Filtering Limits of a Semi-Active Suspension. J. Dyn. Syst. Meas. Control. 2007, 129, 382–392. [Google Scholar] [CrossRef]
- Jenelten, F.; He, J.; Farshidian, F.; Hutter, M. DTC: Deep Tracking Control. Sci. Robot. 2024, 9, eadh5401. [Google Scholar] [CrossRef] [PubMed]
- François-Lavet, V.; Henderson, P.; Islam, R.; Bellemare, M.; Pineau, J. An Introduction to Deep Reinforcement Learning. Found. Trends® Mach. Learn. 2018, 11, 219–354. [Google Scholar] [CrossRef]
- Ruggaber, J.; Ahmic, K.; Brembeck, J.; Baumgartner, D.; Tobolář, J. AI-For-Mobility—A New Research Platform for AI-Based Control Methods. Appl. Sci. 2023, 13, 2879. [Google Scholar] [CrossRef]
- Howell, M.; Frost, G.; Gordon, T.; Wu, Q. Continuous action reinforcement learning applied to vehicle suspension control. Mechatronics 1997, 7, 263–276. [Google Scholar] [CrossRef]
- Tognetti, S.; Savaresi, S.; Spelta, C.; Restelli, M. Batch Reinforcement Learning for semi-active suspension control. In Proceedings of the 2009 IEEE International Conference on Control Applications (CCA), St. Petersburg, Russia, 8–10 July 2009; pp. 582–587. [Google Scholar] [CrossRef]
- Dessort, R.; Chucholowski, C. Explicit model predictive control of semi-active suspension systems using Artificial Neural Networks (ANN). In 8th International Munich Chassis Symposium; Pfeffer, P., Ed.; Springer Fachmedien Wiesbaden: Munich, Germany, 2017; pp. 207–228. [Google Scholar] [CrossRef]
- Savaia, G.; Formentin, S.; Panzani, G.; Corno, M.; Savaresi, S. Enhancing skyhook for semi-active suspension control via machine learning. IFAC J. Syst. Control 2021, 17, 100161. [Google Scholar] [CrossRef]
- Ming, L.; Yibin, L.; Xuewen, R.; Shuaishuai, Z.; Yanfang, Y. Semi-Active Suspension Control Based on Deep Reinforcement Learning. IEEE Access 2020, 8, 9978–9986. [Google Scholar] [CrossRef]
- Liang, G.; Zhao, T.; Wei, Y. DDPG based self-learning active and model-constrained semi-active suspension control. In Proceedings of the IEEE 5th CAA International Conference on Vehicular Control and Intelligence (CVCI), Tianjin, China, 29–31 October 2021; pp. 1–6. [Google Scholar] [CrossRef]
- Han, S.-Y.; Liang, T. Reinforcement-Learning-Based Vibration Control for a Vehicle Semi-Active Suspension System via the PPO Approach. Appl. Sci. 2022, 12, 3078. [Google Scholar] [CrossRef]
- Kim, S.; Kim, C.; Shin, S.; Kim, S.-W. Deep Reinforcement Learning for Semi-Active Suspension: A Feasibility Study. In Proceedings of the IEEE 2023 International Conference on Electronics, Information, and Communication (ICEIC), Singapore, 5–8 February 2023; pp. 1–5. [Google Scholar] [CrossRef]
- Wang, Y.; Wang, C.; Zhao, S.; Guo, K. Research on Deep Reinforcement Learning Control Algorithm for Active Suspension Considering Uncertain Time Delay. Sensors 2023, 23, 7827. [Google Scholar] [CrossRef] [PubMed]
- Lee, D.; Jin, S.; Lee, C. Deep Reinforcement Learning of Semi-Active Suspension Controller for Vehicle Ride Comfort. IEEE Trans. Veh. Technol. 2023, 72, 327–339. [Google Scholar] [CrossRef]
- Yong, H.; Seo, J.; Kim, J.; Kim, M.; Choi, J. Suspension Control Strategies Using Switched Soft Actor-Critic Models for Real Roads. IEEE Trans. Ind. Electron. 2023, 70, 824–832. [Google Scholar] [CrossRef]
- ISO 8608:2016; Mechanical Vibration—Road Surface Profiles—Reporting of Measured Data. International Organization for Standardization: Geneva, Switzerland, 2016.
- Ultsch, J.; Tobolar, J.; Ruggaber, J.; Pfeiffer, A.; Kamp, T.; Brembeck, J.; Baumgartner, D.; Ziegler, M.; Wahl, U.; Rohn, M.; et al. Sachbericht zum Projekt “KI-basierte Fahrwerksregelung KIFAHR”; DLR Institut für Systemdynamik und Regelungstechnik: Oberpfaffenhofen, Germany, 2023. [Google Scholar] [CrossRef]
- Peng, X.; Andrychowicz, M.; Zaremba, W.; Abbeel, P. Sim-to-Real Transfer of Robotic Control with Dynamics Randomization. In Proceedings of the 2018 IEEE International Conference on Robotics and Automation, ICRA, Brisbane, Australia, 21–25 May 2018; pp. 3803–3810. [Google Scholar] [CrossRef]
- Modelica Association. Modelica—A Unified Object-Oriented Language for Systems Modeling—Language Specification. Available online: https://specification.modelica.org/maint/3.6/MLS.html (accessed on 5 June 2024).
- Modelica Association. Functional Mock-Up Interface Specification. Available online: https://fmi-standard.org/ (accessed on 12 June 2024).
- Ultsch, J.; Ruggaber, J.; Pfeiffer, A.; Schreppel, C.; Tobolář, J.; Brembeck, J.; Baumgartner, D. Advanced Controller Development Based on eFMI with Applications to Automotive Vertical Dynamics Control. Actuators 2021, 10, 301. [Google Scholar] [CrossRef]
- Fleps-Dezasse, M.; Tobolar, J.; Pitzer, J. Modelling and parameter identification of a semi-active vehicle damper. In Proceedings of the 10th International Modelica Conference, Lund, Sweden, 10–12 March 2014; Linköping University Electronic Press: Linköping, Sweden, 2014; pp. 283–292. [Google Scholar] [CrossRef]
- Pfeiffer, A. Optimization Library for Interactive Multi-Criteria Optimization Tasks. In Proceedings of the 9th International Modelica Conference, Munich, Germany, 3–5 September 2012; Linköping University Electronic Press: Linköping, Sweden, 2012; pp. 669–680. [Google Scholar] [CrossRef]
- Sutton, R.S.; Barto, A.G. Reinforcement Learning: An Introduction, 2nd ed.; The MIT Press: Cambridge, MA, USA; London, UK, 2018; ISBN 9780262039246. [Google Scholar]
- Mnih, V.; Kavukcuoglu, K.; Silver, D.; Graves, A.; Antonoglou, I.; Wierstra, D.; Riedmiller, M. Playing Atari with Deep Reinforcement Learning. arXiv 2013, arXiv:1312.5602v1. [Google Scholar] [CrossRef]
- Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; Klimov, O. Proximal Policy Optimization Algorithms. arXiv 2017, arXiv:1707.06347v2. [Google Scholar] [CrossRef]
- Lillicrap, T.; Hunt, J.; Pritzel, A.; Heess, N.; Erez, T.; Tassa, Y.; Silver, D.; Wierstra, D. Continuous control with deep reinforcement learning. In Proceedings of the 4th International Conference on Learning Representations (ICLR), San Juan, PR, USA, 2–4 May 2016. [Google Scholar] [CrossRef]
- Haarnoja, T.; Zhou, A.; Abbeel, P.; Levine, S. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. In Proceedings of the 35th International Conference on Machine Learning, ICML, Stockholm, Sweden, 10–15 July 2018; Dy, J., Krause, A., Eds.; PMLR: New York, NY, USA, 2018; pp. 1861–1870. [Google Scholar]
- Liang, E.; Liaw, R.; Nishihara, R.; Moritz, P.; Fox, R.; Goldberg, K.; Gonzalez, J.; Jordan, M.; Stoica, I. RLlib: Abstractions for Distributed Reinforcement Learning. In Proceedings of the 35th International Conference on Machine Learning, ICML, Stockholm, Sweden, 10–15 July 2018; Dy, J., Krause, A., Eds.; PMLR: New York, NY, USA, 2018; pp. 3053–3062. [Google Scholar]
- Raffin, A.; Hill, A.; Gleave, A.; Kanervisto, A.; Ernestus, M.; Dormann, N. Stable-Baselines3: Reliable Reinforcement Learning Implementations. J. Mach. Learn. Res. 2021, 22, 1–8. [Google Scholar]
- Brockman, G.; Cheung, V.; Pettersson, L.; Schneider, J.; Schulman, J.; Tang, J.; Zaremba, W. OpenAI Gym. arXiv 2016, arXiv:1606.01540v1. [Google Scholar] [CrossRef]
- Towers, M.; Terry, J.; Kwiatkowski, A.; Balis, J.; Cola, G.; Deleu, T.; Goulão, M.; Kallinteris, A.; Arjun, K.; Krimmel, M.; et al. Gymnasium. 2023. Available online: https://zenodo.org/records/8127026 (accessed on 5 June 2024).
- Ultsch, J.; Brembeck, J.; de Castro, R. Learning-Based Path Following Control for an Over-Actuated Robotic Vehicle. In VDI-AUTOREG; VDI Verlag: Düsseldorf, Germany, 2019; pp. 25–46. [Google Scholar] [CrossRef]
- Ultsch, J.; Mirwald, J.; Brembeck, J.; de Castro, R. Reinforcement Learning-based Path Following Control for a Vehicle with Variable Delay in the Drivetrain. In Proceedings of the 2020 IEEE Intelligent Vehicles Symposium, IV, Las Vegas, NV, USA, 19 October–13 November 2020; pp. 532–539. [Google Scholar] [CrossRef]
- Raffin, A. RL Baselines3 Zoo. Available online: https://github.com/DLR-RM/rl-baselines3-zoo (accessed on 12 June 2024).
- Haarnoja, T.; Zhou, A.; Hartikainen, K.; Tucker, G.; Ha, S.; Tan, J.; Kumar, V.; Zhu, H.; Gupta, A.; Abbeel, P.; et al. Soft Actor-Critic Algorithms and Applications. arXiv 2019, arXiv:1812.05905v2. [Google Scholar] [CrossRef]
- Kirkpatrick, J.; Pascanu, R.; Rabinowitz, N.; Veness, J.; Desjardins, G.; Rusu, A.; Milan, K.; Quan, J.; Ramalho, T.; Grabska-Barwinska, A.; et al. Overcoming catastrophic forgetting in neural networks. Proc. Natl. Acad. Sci. USA 2017, 114, 3521–3526. [Google Scholar] [CrossRef] [PubMed]
- Savaresi, S.; Poussot-Vassal, C.; Spelta, C.; Sename, O.; Dugard, L. Semi-Active Suspension Control Design for Vehicles; Butterworth-Heinemann: Oxford, UK, 2010; ISBN 978-0-08-096678-6. [Google Scholar]
- Enders, E.; Burkhard, G.; Fent, F.; Lienkamp, M.; Schramm, D. Objectification methods for ride comfort. Forsch. Ingenieurwesen 2019, 83, 885–898. [Google Scholar] [CrossRef]
- ISO 2631-1:1997; Mechanical Vibration and Shock—Evaluation of Human Exposure to Whole-Body Vibration. Part 1: General Requirements. International Organization for Standardization: Geneva, Switzerland, 1997.
- Bünte, T.; Rill, G.; Ruggaber, J.; Tobolář, J. Modelling and Validation of the TMeasy Tyre Model for Extreme Parking Manoeuvres. In Advances in Dynamics of Vehicles on Roads and Tracks II; Orlova, A., Cole, D., Eds.; Springer International Publishing: Cham, Switzerland, 2022; pp. 1015–1025. [Google Scholar] [CrossRef]
- Múčka, P. Simulated Road Profiles According to ISO 8608 in Vibration Analysis. J. Test. Eval. 2017, 46, 405–418. [Google Scholar] [CrossRef]
Front Axle | Rear Axle | ||
---|---|---|---|
Tf,falling * | |||
df,falling * |
QVM Structure Name | Nonlinear Spring/Damper Transmission | Topmount Bushing as Linear Spring/Damper Element | Engine Mass with Linear Spring/Damper Bearing |
---|---|---|---|
Simple QVM | ✗ | ✗ | ✗ |
Transmission QVM | ✓ | ✗ | ✗ |
Topmount QVM | ✓ | ✓ | ✗ |
Engine QVM | ✓ | ✗ | ✓ |
Topmount engine QVM | ✓ | ✓ | ✓ |
Reward Weights | Reward Terms | ||
---|---|---|---|
- | - | Force jump reward term | |
Chassis motion weight | Chassis motion reward term | ||
Control signal jump weight | Control signal jump reward term | ||
Control signal weight | Control signal reward term |
Excitation | Type | Adapted ISO 2631 Comfort Criterion | Road-Holding Criterion |
---|---|---|---|
Exponential Sine-Sweep (1–30 Hz|50 mm/s) | Sweep | 0.971 | 1.017 |
Exponential Sine-Sweep (1–30 Hz|100 mm/s) | Sweep | 1.019 | 1.028 |
Exponential Sine-Sweep (1–30 Hz|150 mm/s) | Sweep | 1.083 | 0.921 |
Exponential Sine-Sweep (1–30 Hz|200 mm/s) | Sweep | 1.108 | 0.871 |
Exponential Sine-Sweep (1–30 Hz|250 mm/s) | Sweep | 1.092 | 0.842 |
ISO 8608 Type A Road (asynchronous) | Road-like | 0.931 | 0.954 |
ISO 8608 Type B Road (asynchronous) | Road-like | 0.967 | 0.980 |
ISO 8608 Type C Road (asynchronous) | Road-like | 0.998 | 0.993 |
ISO 8608 Type D Road (asynchronous) | Road-like | 1.024 | 0.991 |
ISO 8608 Type A Road (synchronous) | Road-like | 0.947 | 0.943 |
ISO 8608 Type C Road (synchronous) | Road-like | 0.993 | 0.995 |
ISO 8608 Type D Road (synchronous) | Road-like | 1.032 | 0.991 |
Real Road Replay: Rough Bump | Road-like | 0.924 | 0.982 |
Real Road Replay: Rough Road | Road-like | 0.962 | 0.992 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ultsch, J.; Pfeiffer, A.; Ruggaber, J.; Kamp, T.; Brembeck, J.; Tobolář, J. Reinforcement Learning for Semi-Active Vertical Dynamics Control with Real-World Tests. Appl. Sci. 2024, 14, 7066. https://doi.org/10.3390/app14167066
Ultsch J, Pfeiffer A, Ruggaber J, Kamp T, Brembeck J, Tobolář J. Reinforcement Learning for Semi-Active Vertical Dynamics Control with Real-World Tests. Applied Sciences. 2024; 14(16):7066. https://doi.org/10.3390/app14167066
Chicago/Turabian StyleUltsch, Johannes, Andreas Pfeiffer, Julian Ruggaber, Tobias Kamp, Jonathan Brembeck, and Jakub Tobolář. 2024. "Reinforcement Learning for Semi-Active Vertical Dynamics Control with Real-World Tests" Applied Sciences 14, no. 16: 7066. https://doi.org/10.3390/app14167066