Comparison between wavelet transforms and short-time Fourier transform in estimating a nonparametric regression function using simulation

Abstract

Nonparametric regression is considered a suitable alternative to parametric regression, as it does not require assuming a specific functional form of the relationship between variables. However, traditional nonparametric methods suffer from high sensitivity to fluctuations and noise, as well as weak performance when dealing with non-stationary data. To overcome these challenges, signal processing techniques such as wavelet transforms and the Short-Time Fourier Transform (STFT) can be employed, as they provide a more accurate representation of the temporal and frequency characteristics of the data. This integration enhances estimation accuracy and reduces both bias and variance, granting this advanced methodology a clear advantage over classical nonparametric regression approaches, particularly in the analysis of complex economic, financial, and natural data.
In this study, the Discrete Wavelet Transform (DWT) is employed, along with flexible thresholding values that handle coefficients at each level separately, unlike universal thresholds that process all levels simultaneously. Furthermore, the Short-Time Fourier Transform (STFT) and the Adaptive Short-Time Fourier Transform (ASTFT) are utilized with newly proposed thresholding rules. Through simulation experiments using test functions, sample sizes (128, 256, 512), and noise ratios (5%, 10%), the findings reveal that the most efficient estimation method is the DWT combined with the Fast Fourier Transform (FFT-DWT), followed by the DWT combined with the Adaptive Fast Fourier Transform (AFFT-DWT), as evaluated by the Average Mean Squared Error (AMSE) criterion.