Efficient Driving Behavior Narration and Reasoning on Edge Device Using Large Language Models

Abstract

Large language models (LLMs) with robust reasoning capabilities have significantly advanced the development of autonomous driving technologies, particularly in the narration and reasoning of driving behaviors, which hold substantial importance for accident analysis and traffic management. However, traditional deployment of these models relies on cloud servers, resulting in high latency and training costs, making it challenging to meet the stringent real-time requirements of autonomous driving scenarios. Recent studies suggest that edge computing, by deploying models closer to the data source, offers a promising solution to these issues. While existing general-purpose LLMs excel in video understanding and task reasoning, their generalization capabilities in rapidly changing traffic scenarios remain questionable. This paper provides a valuable reference for deploying LLMs at the edge in autonomous driving contexts. By leveraging real-world 5G networks for rapid deployment, we validate the performance and response speeds of various models in autonomous driving scenarios. Furthermore, we introduce an innovative prompt engineering strategy that enhances model performance by 25% without changing model parameters through minimal prompt tuning. Experimental results demonstrate that LLMs deployed on edge devices achieve satisfactory response times. Tests on the OpenDV-YouTube dataset further confirm that our prompt strategy significantly improves the performance of driving behavior narration and reasoning.