An Effective Evaluation Tool for Hyperspectral Target Detection: 3D Receiver Operating Characteristic Curve Analysis

Abstract

Receiver operating characteristic (ROC) analysis is performed by a curve, called ROC curve, plotted based on detection probability, PD, versus false alarm probability, PF, and has been widely used as an evaluation tool for signal detection. Specifically, the area under an ROC curve (AUC) is calculated and used as a detection measure. Unfortunately, finding distributions of PDand PFto generate a continuous ROC curve is practically infeasible. This article investigates approaches to generating a discrete 2D ROC curve of ( PD, PF) without appealing for probability distributions. Since PDand PFare determined by the same threshold τ to specify a detector, an ROC curve of ( PD, PF) can only be used to evaluate the effectiveness of a detector but not target detectability (TD) and also not background suppressibility (BS). To address this issue, a 3D ROC curve is generated as a function of ( PD, PF, τ) by introducing a specific threshold parameter τ as a third independent variable. As a result, a 3D ROC curve along with its derived three 2D ROC curves of ( PD, PF), ( PD, τ), and ( PF, τ) can further be used to design new quantitative measures to evaluate the effectiveness of a detector and its TD and BS. To demonstrate the full utility of 3D ROC analysis in target detection, extensive experiments are performed on two types of targets, prior target detection and anomaly detection, to conduct a comprehensive analysis on 3D ROC curves using new designed detection measures to evaluate target/anomaly detection performance.