Theoretical performance analysis of the W-ABORT detector

In a recent paper we introduced a modification of the adaptive beamformer orthogonal rejection test (ABORT) for adaptive detection of signals in unknown noise, by supposing under the null hypothesis the presence of signals orthogonal to the nominal steering vector in the whitened observation space. We will refer to this new receiver as the whitened adaptive beamformer orthogonal rejection test (W-ABORT). Through Monte Carlo simulations this new detector was shown to provide better rejection capabilities of mismatched (e.g., sidelobe) signals than existing ones, like ABORT or the adaptive coherence estimator (ACE), but at the price of a certain loss in terms of detection of matched (i.e., mainlobe) signals. The aim of this paper is to provide a theoretical validation of this fact. We consider both the case of distributed targets and point-like targets. We provide a statistical characterization of the W-ABORT test statistic, under the null hypothesis, and for matched and mismatched signals under the alternative hypothesis. For distributed targets, the probability of false alarm and the probability of detection can only be expressed in terms of multi-dimensional integrals, and are thus very complicated to obtain; in contrast, for point-like targets, such probabilities can be easily calculated by numerical integration techniques. The theoretical expressions derived herein corroborate the simulation results obtained previously