Deep Learning based Pothole Monocular Depth Estimation and Segmentation Using 3D Scanner-Derived Depth Maps

Abstract

Potholes pose significant safety risks to drivers and cause damage to vehicles. This paper modified a novel approach called the monocular depth estimation and segmentation (modified 3DPredicNet) network [1] to accurately estimate depth maps and segment potholes. To facilitate model training and evaluation, a comprehensive dataset consists of RGB images captured using a DSLR camera and corresponding 3D scan data for generating depth maps. The depth maps derived from the 3D scans are utilized for pothole depth estimation, while masks are used for pothole segmentation. The evaluation results reveal the model's ability to accurately predict and segment potholes in RGB images, achieving a minimum absolute relative error (ARel) of 0.062, square relative error (SRel) of 0.011, and root mean square error (RMSE) of 0.118 when tested on the newly developed dataset. Moreover, when tested on the newly developed dataset, the model demonstrates good pothole segmentation performance, attaining a high mean intersection over union (mIoU) of 81.05. Furthermore, when utilizing the publicly available dataset, the modified 3DPredicNet achieved accurate depth estimation with ARel of 0.093 and SRel of 0.033.