Self- and Domain-Awareness for Machine Learning in NDE and SHM

Abstract

Given any input, a machine learning (ML) algorithm will produce an output, but the confidence and accuracy of that output are currently not well understood or quantified. This poses serious risks in the case of automated safety-critical nondestructive evaluation (NDE) or structural health monitoring (SHM) applications. This work aims to provide a comprehensive approach for quantifying the uncertainty of ML models when used for inspection purposes (self-awareness), in addition to equipping these models with domain-awareness features (i.e., the ability to detect out-of-distribution or “bad” input data). An ensemble of models is trained as part of the procedure, and the reliability of the predictions is measured using the statistics of the models. This information is also used to assess the quality of the input data and evaluate whether it comes from an out-of-distribution source or falls within the domain that the models were trained on. However, this process encompasses making a good choice of the ML model’s type and complexity to ensure model effectiveness and stability. It also involves understanding the trade-off between the ensemble size (number of models within the ensemble) and the prediction’s uncertainty and statistical significance. A robust framework for the whole process is provided and discussed in this paper. The methodology is demonstrated using simulated ultrasound data for corrosion-profile assessment. The presented findings are a step forward towards having more confidence in using ML for NDE and SHM applications. Some current shortcomings are discussed and suggested for future investigation.