Real-Time Object Detection and Distance Measurement Enhanced with Semantic 3D Depth Sensing Using Camera–LiDAR Fusion

Abstract

Camera and LiDAR data fusion has been a popular research area, especially in the field of autonomous vehicles. This study evaluates the efficiency and accuracy of different depth point extraction methods, including Point-by-Point (PbyP), Complete Region Depth Extraction (CoRDE), Central Region Depth Extraction (CeRDE), and Grid Central Region Depth Extraction (GCRDE), across object categories such as person, bicycle, car, bus, and truck, and occlusion levels ranging from 0 to 3. The approaches are assessed based on extraction time, accuracy, and root mean squared error (RMSE). Bounding box-based methods, such as PbyP and CoRDE, consistently show slower extraction times compared to segmentation mask methods, with CeRDE being the most efficient in terms of computational speed. However, segmentation mask methods, particularly CeRDE and GCRDE, offer superior accuracy, especially for complex objects like trucks and cars, where bounding box methods struggle, particularly at higher occlusion levels. In terms of RMSE, segmentation mask methods consistently outperform bounding box methods, providing more precise depth estimations, particularly for larger and more occluded objects. Overall, segmentation mask methods are preferred for applications where accuracy is critical, despite their slower processing speed, while bounding box methods are suitable for real-time applications requiring faster depth extraction. GeRDE offers a balance between speed and accuracy, making it ideal for tasks needing both efficiency and precision.