A Pipeline for Evaluation of Keypoint-Based Bounding Boxes for Multi-Scale Pedestrians

Abstract

Pedestrians account for approximately 25% of traffic accidents, many of which can be prevented using autonomous driving systems (ADS). Although pedestrian detection has advanced significantly, intent prediction still lags behind human perception. A key challenge is predicting the intent of smaller pedestrians, who are harder to detect and analyse using 2D pose estimation techniques because of their tendency to blend into the background. However, human joint keypoints (e.g., head, shoulders, elbows, and knees) can reliably predict pedestrian movement, such as gait, limb motion, and head orientation. This paper introduces the Keypoint Evaluation (KeyEval) pipeline, a new technique for generating high-quality pedestrian keypoints using a 2D pose estimator. Leveraging ground-truth bounding box annotations from the JAAD and PIE datasets, we assess keypoint accuracy and apply state-of-the-art fine-tuning, achieving a 19% improvement in average precision (76%) over the baseline. This suggests KeyEval can enhance predictions of pedestrian intent—crossing, waiting, or changing direction—particularly for smaller pedestrians. The KeyEval pipeline can be seamlessly integrated into ADS to proactively reduce vehicle-pedestrian accidents, as demonstrated in our previous work.