Session: 12-11: Software-Defined Vehicles, Digital Twins, and Connected AI Platforms

Paper Number: 197134

197134 - Trainable Digital Twin for Guided Teleoperated Driving 

Abstract: 

Teleoperated driving is a critical fail-safe for autonomous vehicles, enabling remote human intervention in complex scenarios. However, transmitting sensor data and control commands over wireless networks introduces significant latencies. Combined with processing and actuation delays, these latencies can severely degrade the stability and performance of teleoperated maneuvers.

To address this challenge, we develop and evaluate a digital twin based on a neural dynamic model designed to assist remote operators. Our system determines if current network conditions can safely support specific maneuvers via a two-step process. First, we calibrate the neural dynamic model using a baseline maneuver, such as a lane change. This enables the model to learn the system's overall latency and the remote operator's unique response characteristics. Second, the trained model continuously evaluates the stability of ongoing teleoperated maneuvers in real time. If it detects impending instability, the system proactively issues an alert or intervenes to stabilize the vehicle.

The core of our approach is the neural dynamic model: a specialized neural network that explicitly incorporates both network delay and vehicle dynamics. This architecture ensures rapid training and facilitates real-time stability analysis. In this talk, we demonstrate how the model is trained using human-in-the-loop teleoperated driving data and illustrate its effectiveness in assisting operators during teleoperated lane-tracking tasks.

Presenting Author: Sergei Avedisov Toyota North America R&D, InfoTech Labs

Presenting Author Biography: Sergei S. Avedisov is a Principal Researcher a Toyota InfoTech Labs and an integral member of the Communication Systems Research (CSR) team. Sergei specializes in cooperative automated driving including cooperative localization, cooperative perception, and cooperative maneuvering; and examines the effects of these technologies on automated vehicle performance. At the conference he will share how a new type of wireless vehicle-to-vehicle messages, intent-sharing, may benefit platooning through a reinforcement learning framework.

Authors:

Sergei Avedisov Toyota North America R&D, InfoTech Labs
Mariam Nour Toyota North America R&D, InfoTech Labs
Mohammad Irfan Khan Toyota North America R&D, InfoTech Labs
Takayuki Shimizu Toyota North America R&D, InfoTech Labs
Onur Altintas Toyota North America - InfoTech Labs