YOLOv8-LiDAR Fusion: Increasing Range Resolution Based on Image Guided-Sparse Depth Fusion in Self-Driving Vehicles

Abstract

Self-driving vehicles are significant in industrial and commercial applications, primarily driven by the development of environmental awareness systems. The need for real-time object recognition, segmentation, perception, projection, and position has significantly increased in object and line tracking, obstacle avoidance, and route planning. The primary sensors used are high-resolution cameras, Light Detection and Ranging (LiDAR), and high-precision GPS/IMU inertial navigation systems. However, out of all these sensors, LiDARs and cameras have a vital function in perception and comprehensive situations. Although LiDAR is capable of providing precise depth information, its resolution is constrained. On the other hand, cameras provide abundant semantic information but do not offer precise assessments of the distance to objects. This work presents the incorporation of YOLOv8, an advanced object identification method, into the fusion process. We specifically investigate the notion of Camera-LiDAR Projection and provide a thorough explanation of the process of projecting LiDAR point clouds onto an image coordinate frame. This is achieved by utilizing transformation matrices that establish the relationship between the LiDAR and the camera. This project aims to improve the range resolution and perception capabilities of autonomous driving systems by combining YOLOv8-based object recognition with LiDAR point cloud data by using the KITTI object detection benchmark.