Robust Autonomous Vehicle Pursuit without Expert Steering Labels

Jiaxin Pan*1, Changyao Zhou*1, Mariia Gladkova1, Qadeer Khan1,2, Daniel Cremers1,2,3
Technical University of Munich1, Munich Center for Machine Learning2, University of Oxford3
IEEE Robotics and Automation Letters
*Indicates Equal Contribution
Paper Code arXiv

Abstract

In this work, we present a learning method for both lateral and longitudinal motion control of an ego-vehicle for the task of vehicle pursuit. The car being controlled does not have a pre-defined route, rather it reactively adapts to follow a target vehicle while maintaining a safety distance. To train our model, we do not rely on steering labels recorded from an expert driver, but effectively leverage a classical controller as an offline label generation tool. In addition, we account for the errors in the predicted control values, which can lead to a loss of tracking and catastrophic crashes of the controlled vehicle. To this end, we propose an effective data augmentation approach, which allows to train a network that is capable of handling different views of the target vehicle. During the pursuit, the target vehicle is firstly localized using a Convolutional Neural Network. The network takes a single RGB image along with cars' velocities and estimates target vehicle's pose with respect to the ego-vehicle. This information is then fed to a Multi-Layer Perceptron, which regresses the control commands for the ego-vehicle, namely throttle and steering angle. We extensively validate our approach using the CARLA simulator on a wide range of terrains. Our method demonstrates real-time performance, robustness to different scenarios including unseen trajectories and high route completion.

Overview

MY ALT TEXT

Our proposed framework consists of preparatory offline steps (left) and online pipeline (right), which is conducted respectively during training and testing of our system.

Left: Firstly, we perform label generation using Model Predictive Controller. MPC takes location and orientation of the target with respect to the ego-vehicle estimated by a LiDAR-based 3D detector. Secondly, data augmentation is conducted by utilizing estimated dense depth maps for novel view synthesis.

Right: Our learning method is trained to predict accurate lateral and longitudinal control values by leveraging obtained labels, augmented image dataset and velocities of both ego and target vehicles. During test time our approach requires only RGB image sequence and velocities as input to control the ego-vehicle.

Results

Here we show videos to show the qualitative results of our approach.

We tested our model in different maps, in a city (left) and in the countryside (right). To summarize, the first vehicle (red) in the video is controlled by autopilot. The ego-vehicle (black) is controlled autonomously with our method to follow the red target vehicle.

MY ALT TEXT

We further tested our model with ego vehicles. As can be seen in the ego vehicle, the first vehicle (red) is controlled by autopilot, the first ego-vehicle (gray) is controlled by our model to follow the red target vehicle. And the second ego-vehicle (black) is controlled by the same model to follow the gray ego-vehicle.

MY ALT TEXT

Though the model is trained only with samples in sunny weather, it fits different weather conditions quite well. Here we show two examples, in dark night (left) and rainy weather (right).

MY ALT TEXT

Data Augmentation

Example of synthesized views with longitudinal, lateral and rotational offset

MY ALT TEXT

BibTeX


      @article{vehiclepursuit2023,
        author = {J Pan and C Zhou and M Gladkova and Q Khan and D Cremers},
        title = {Robust Autonomous Vehicle Pursuit without Expert Steering Labels},
        journal = {{IEEE} Robotics and Automation Letters ({RA-L})},
        year = {2023},
        keywords = {vehicle pursuit, deep learning},
       }