Abstract
Loud transient signals in underwater acoustic data increase the bias and variance of background noise power spectral density (PSD) estimates based on the sample mean, such as Welch’s Overlapped Segment Averaging. Order statistics filters (OSFs) mitigate the effect of loud transients by removing data samples above a chosen threshold rank order statistics (OS) from the PSD estimate computation. Notably, Schwock and Abadi’s Welch Percentile (SAWP) estimator replaces WOSA’s averaging step with a normalized single order statistic of consecutive periodograms. The SAWP estimator avoids the increased bias and variance caused by outliers, but raises two new challenges. Firstly, the SAWP discards the information in the samples below the threshold rank, samples that are free of outlier bias and could help further reduce the variance of the estimator. Secondly, the unpredictability of the loud transients in the underwater environment makes the selection of a single threshold rank OS a challenging problem in practical real-time applications. To address the first challenge, this dissertation proposes the Truncated Linear Order Statistics Filter (TLOSF), an OSF that estimates the background PSD with a weighted sum of all the OS up to the threshold rank. The TLOSF weights result from a least-squares optimization that minimizes variance while maintaining the unbiased property of the resulting estimator. To address the second challenge, this dissertation proposes universal versions of the OSFs that blend the fixed-rank OSFs instead of choosing a single threshold rank. The blending weights are sequentially adjusted over time to favor the OSFs performing best over a recent time window, thus circumventing the challenges of threshold rank selection. In theoretical analyses and simulations, the TLOSF further reduces its variance compared to SAWP, while the performance of universal OSFs provably approaches the performance of the best fixed-rank OSF one would choose in hindsight. In a test with spectrograms of real underwater acoustic data, both the TLOSF and the universal OSFs effectively filter loud clicks in the recording while preserving the background noise structure.