ROC-based Performance Analysis and Interpretation of Image-based Damage Diagnostic Tools for Underwater Inspections

It is of practical importance for inspectors to have knowledge of the efficiency of Non-Destructive Testing (NDT) tools when applied commercially. It has become common practice to model the performance of NDT tools in a probabilistic manner in terms of Probability of Detection (PoD), Probability of False Alarm (PFA) and eventually by Receiver Operating Characteristic (ROC) Curves. Traditionally, these quantities are estimated from training data, however, there are often doubts about the validity of these estimates when the sample size is small. In the case of underwater inspections, the scarcity of good quality training data means that this scenario arises more often than not. Comprehensive studies around the on-site performance of image-based damage diagnostic tools have only recently been made possible through the availability of online resources such as the Underwater Lighting and Turbidity Image Repository (ULTIR), which contains photographs of various damages forms captured under controlled visibility conditions. This paper shows how meaningful information can be extracted from this repository and used to construct ROC curves that can be related to the on-site performance of image-based NDT methods for detecting various damage forms and under a range of environmental conditions. The ability to draw connections between image-based techniques applied in real underwater inspections with ROC curves that can be constructed on-demand provide the engineer/ inspector with a clear and systematic route for assessing the reliability of data obtained from image-based methods. As a case study, the general approach has been applied to characterise the performance of image-based techniques for identifying instances of corrosion and cracks on marine structures. A discussion around how the results can be used for further analysis is provided. This includes looking at how the results can be fed into in the decision chain and can be used for risk analysis, intervention and work scheduling, and eventually understanding the value of information.