The correct estimate of the probability of false detection of the matched filter in weak-signal detection problems

• Published in: Astronomy and astrophysics (Berlin) Vol. 627
• Main Authors: Vio, R., Andreani, P., Biggs, A., Hayatsu, N.
• Format: Journal Article
• Language: English
• Published: EDP Sciences 2019
• Subjects: methods: data analysis; methods: statistical
• ISSN: 0004-6361; 1432-0746; 1432-0746
• DOI: 10.1051/0004-6361/201834854

The matched filter (MF) represents one of the main tools to detect signals from known sources embedded in the noise. In the Gaussian isotropic case, the noise can be assumed to be the realization of a Gaussian random field (GRF). The most important property of the MF, the maximization of the probability of detection subject to a constant probability of false detection or false alarm (PFA), makes it one of the most popular techniques. However, the MF technique relies upon the a priori knowledge of the number and the position of the searched signals in the GRF (e.g. an emission line in a spectrum or a point-source on a map), which usually are not available. A typical way out is to assume that, if present, the position of a signal coincides with one of the peaks in the matched filtered data. A detection is claimed when the probability that a given peak is due only to the noise (i.e. the PFA) is smaller than a prefixed threshold. This last step represents a critical point in the detection procedure. Since a signal is searched for amongst the peaks, the probability density function (PDF) of the amplitudes of the latter has to be used for the computation of the PFA. Such a PDF, however, is different from the Gaussian. Moreover, the probability that a detection is false depends on the number of peaks present in the filtered GRF. This is because the greater the number of peaks in a GRF, the higher the probability of peaks due to the noise that exceed the detection threshold. If this fact is not taken into account, the PFA can be severely underestimated. In statistics this is a well-known problem named the multiple comparisons, multiple testing, or multiple hypotheses problem, whereas in other fields it is known as the look-elsewhere effect. Many solutions have been proposed to this problem. However, most of them are of a non-parametric type hence not able to exploit all the available information. Recently, this limitation has been overcome by means of two efficient parametric approaches. One is explicitly based on the PDF of the peak amplitudes of a smooth and isotropic GRF whereas the other makes use of the Gumbel distribution, which represents the asymptotic PDF of the corresponding extreme. On the basis of numerical experiments as well of an application to an interferometric map obtained with the Atacama Large Millimeter/submillimeter Array (ALMA), we show that, although the two methods produce almost identical results, the first is more flexible and at the same time allows us to check the reliability of the detection procedure.