Vehicle Platooning using Multi-Agent Reinforcement Learning: A Study on Autonomous Driving in the CARLA Simulator

We successfully trained a model using reinforcement learning that enables a car to identify and follow a leader vehicle, therefore creating a vehicle platoon in a simulation.
For this thesis, we are using CARLA, an open-source 3D environment simulator for autonomous driving research based on the Unreal Engine.

Specifically, we implemented the Double Deep Q-Network algorithm to train the policy.
The state is solely represented as the sensor input taken from the follower car's front camera.
From that, an Atari-style Convolutional Neural Network predicts the actions' Q-values .
A custom reward function, that combines the relative longitudinal and lateral positions of the two vehicles in the platoon, provides the necessary feedback of each action's performance.
Actions, discrete by definition of a DQN, comprise left and right steering, acceleration and slowing down, as well as braking.

The resulting model is capable of forming and maintaining a platoon.
While navigating in a multi-lane scenario, it even manages to cross intersections successfully.
It enables the follower car to identify the leader vehicle and imitate its actions in a previously unseen environment.