Wiener Filter

The Wiener filter is an optimization-based filtering technique used in signal processing and image processing. As illustrated in Figure 1, it was developed to restore a noisy signal or image to its closest approximation of the original. This technique is widely applied in fields such as radar, medical imaging, and telecommunications. Its fundamental principle is to estimate the optimal filter coefficients that minimize the mean squared error (MSE) by leveraging prior statistical knowledge about the signals in the environment. In a sense, this filter predicts future behavior based on how the signal has behaved in the past. Two mathematical expressions are essential to understanding the Wiener Filter.

1. Autocorrelation: This measures how similar a signal is to its own past. For example, does a previous sound wave in an audio recording provide clues about the next one?
2. Cross-correlation: This examines the relationship between a clean signal and a corrupted (noisy) signal, attempting to find the similarity between the original signal and the noise-contaminated version.

For the filtering process to be effective, certain statistical properties of the signal and noise must be known in advance.

^{Figure 1. Denoising of a one-dimensional signal using the Wiener Filter}

The Wiener filter is a linear and time-invariant filter used to estimate the original form of a signal that has been corrupted by noise or shifted in time. Therefore, although it is referred to as a filter, it can also be used for prediction. In the design of the Wiener filter, the input signal $x[n]$

1. $x[n]$ is "wide-sense stationary," meaning that its variance and expected value are constant and finite, and its autocorrelation function depends only on the time lag fixed and is given by only time.
2. The expected value of all signals (input, output, and noise) is zero.

The Wiener filter can be designed in different categories: continuous-time versus discrete-time, and causal versus non-causal.

The Wiener Filter was developed by Norbert Wiener in 1942. Wiener designed this method to reduce noise in radar signals. Subsequently, the technique was adopted in electronics and other fields, and over time it was adapted for digital signal and image processing applications.

Applications

1. Signal Processing: Used to enhance radar and sonar signals acquired in noisy environments. For example, it is applied in submarine sonar systems to detect targets.
2. Image Processing: Commonly preferred for noise reduction and detail enhancement in digital images (Figure 2). It is successfully used in medical imaging devices (MRI and CT).
3. Telecommunications: Preferred to reduce error rates in data signals degraded by noise.
4. Audio Processing: Applied to improve noisy audio recordings and increase their intelligibility.

^{Figure 2. Result of image processing using the Wiener Filter}

The designed system is represented schematically in Figure 3. Here, $y[n]$ denotes the input signal, $\hat{y}[n]$ denotes the estimated signal, and $x[n]$ represents the input signal. The MSE is defined as:

$MSE=E[e^2[n]]=E[(\hat{y}[n]-y[n]{)}^2]=E[{\hat{y}}^2[n]]-2E[y[n]\hat{y}[n]]+E[y^2[n]]={\Phi }_{\hat{y}\hat{y}}[0]-2{\Phi }_{y\hat{y}}[0]+{\Phi }_{yy}[0]$

$E[\cdot ]$ is the expectation value operator, where ${\Phi }_{\hat{y}\hat{y}}$

$\hat{y}[n]=\sum _{m=-\infty }^{\infty }h[m]x[n-m]$

^{Figure 3:  Linear Time-Invariant (LTI), discrete-time, non-causal Wiener filter, with impulse response} $h[n]$

Based on this information, the autocorrelation and cross-correlation values forming the MSE are computed as follows:

${\Phi }_{y\hat{y}}[0]=E[y[n]\hat{y}[n]]=E[\sum _{m=-\infty }^{\infty }h[m]x[n-m]y[n]]=\sum _{m=-\infty }^{\infty }h[m]{\Phi }_{xy}[m]$

${\Phi }_{\hat{y}\hat{y}}[0]=[\hat{y}[n]\hat{y}[n]]=E[\sum _{m=-\infty }^{\infty }h[m]x[n-m]\sum _{l=-\infty }^{\infty }h[l]x[n-l]]=\sum _{m=-\infty }^{\infty }\sum _{l=-\infty }^{\infty }h[m]h[l]{\Phi }_{xx}[l-m]$

Using the computed autocorrelation and cross-correlation values, the MSE is reformulated as:

$MSE={\Phi }_{yy}[0]-2\sum _{m=-\infty }^{\infty }h[m]{\Phi }_{xy}[m]+\sum _{m=-\infty }^{\infty }\sum _{l=-\infty }^{\infty }h[m]h[l]{\Phi }_{xx}[l-m]$

Let us compute the first and second derivatives of the MSE with respect to the system h:

$\frac{dMSE}{dh[n_i]}=-2{\Phi }_{xy}\left[n_i\right]+2\sum _{m=-\infty }^{\infty }h[m]{\Phi }_{xx}[n_i-m]$

$\frac{d^2\mathrm{MSE}}{dh[n_i{]}^2}=2{\Phi }_{xx}[0]$

${\Phi }_{xx}[0]=E[x^2[n]]$

As observed, the autocorrelation function is quadratic in form and therefore always non-negative. Consequently, the MSE is a convex function. Since the MSE is convex, the point where its derivative equals zero corresponds to the minimum MSE. Using this property, the minimum MSE condition is derived as:

$-2{\Phi }_{xy}[n_i]+2{\sum }_{m=-\infty }^{\infty }h[m]{\Phi }_{xx}[n_i-m]=0,\forall n_i$

$\sum _{m=-\infty }^{\infty }h[m]{\Phi }_{xx}[n_i-m]={\Phi }_{xy}[n]$

The above equation can be expressed in closed form using the convolution operator:

$h[n]\ast {\Phi }_{xx}[n]={\Phi }_{xy}[n]$

For convenience, a transition to the z-domain is performed. The Z-transforms of the correlation functions in the time domain yield spectral power functions, denoted as $S(\cdot )$.

$H(z)S_{xx}(z)=S_{xy}(z)$

$H(z)=S_{xy}(z)S_{xx}^{-1}(z)$

Thus, the Wiener filter is generally designed. The Wiener filter can also be expressed in the Laplace domain as:

$H(s)=S_{xy}(s)S_{xx}^{-1}(s)$

Thus, filters designed in the Laplace and z domains can be converted respectively into continuous-time and discrete-time domains.

Example

_{Figure 4: Example of a non-causal, linear time-invariant (LTI) discrete-time Wiener filter. All signals are generally wide-sense stationary and there is no correlation between the signal} $y[n]$ _{and the noise} $v[n]$_.

In the example shown in Figure 4, there is no correlation between $y[n]$ and $v[n]$, and all signals are generally wide-sense stationary (WSS). The filter $h[n]$ is the Wiener filter. Previously, we found that the Wiener filter in the $z$ domain is given by:

$H(z)=S_{xy}(z)S_{xx}^{-1}(z)$. The functions $S_{xy}(z)$ and $S_{xx}^{-1}(z)$ are calculated as follows:

${\Phi }_{xx}[k]=E[x[n]x[n+k]]$

$=E\left[\right(y[n]+v[n]\left)\right(y[n+k]+v[n+k]\left)\right]$

$=E[y[n]y[n+k]]+E[y[n]v[n+k]]+E[v[n]y[n+k]]+E[v[n]v[n+k]]$

$={\Phi }_{yy}[k]+0+0+{\Phi }_{vv}[k]$

$S_{xx}(z)=S_{yy}(z)+S_{vv}(z)$

${\Phi }_{xy}[k]=E[x[n]y[n+k]]$

$=E\left[\right(y[n]+v[n])y[n+k]]=E[y[n]y[n+k]]+E[v[n]y[n+k]]$

$={\Phi }_{yy}[k]+0$

$S_{xy}(z)=S_{yy}(z)$

Thus, the Wiener filter is obtained as follows:

$H(z)=S_{xy}(z)S_{xx}^{-1}(z)=\frac{S_{yy}(z)}{S_{yy}(z)+S_{vv}(z)}$

The Wiener filter uses the power spectral densities of both the input signal and the noise to optimize a noisy signal or image. It preserves the signal at frequencies with a high signal-to-noise ratio (SNR) and suppresses noise at frequencies with low SNR.

The approach developed by Shannon and Bode can be used for causal Wiener filter design:

1. The autocorrelation of the input signal is computed, and its stable poles and zeros are separated from the unstable poles and zeros into ${\Phi }_{xx}^{++}[k]$ and ${\Phi }_{xx}^{--}[k]$ respectively.
2. ${\Phi }_{xx}^{++}[k]$ and ${\Phi }_{xx}^{--}[k]$ are transformed respectively into the Z or Laplace domain as $Z$ and $S_{xx}^{++}$ and $S_{xx}^{--}$.
3. The inverse of the stable system is taken to obtain the whitening filter: $\frac{1}{S_{xx}^{++}(z)}$ and whitening filter is obtained.
4. A new filter is constructed using the unstable system: $\hat{h}=\frac{S_{xy}(z)}{S_{xx}^{--}(z)}$
5. The filter $\hat{h}$ is transformed into the time domain, and its stable components are separated. The stable portion is then transformed back into the Z or Laplace domain: $\hat{h}=\frac{S_{xy}(z)}{S_{xx}^{--}(z)}$ .
6. The causal Wiener filter is obtained as: ${\hat{h}}^{+}=\frac{1}{S_{xx}^{++}(z)}{\left[\frac{S_{xy}(z)}{S_{xx}^{--}(z)}\right]}^{+}$ .

Advantages:

- Provides the optimal result in terms of minimum mean square error (MSE).

- Can be applied in both analog and digital implementations.

- Achieves high success rates when the noise level and the power spectral densities of the signal are known in advance.

Limitations:

- The statistical properties of the signal and noise must be known a priori as complete.

- If the power spectral densities are estimated inaccurately, filtering may be erroneous.

- Due to complex computational requirements, processing load may increase in real-time applications as true.

The Wiener filter can be applied in both the time and frequency domains. Frequency domain implementation is performed using the fast Fourier transform (FFT). The mathematical steps of the filter are as follows:

1. Take the Fourier transform of the noisy signal.
2. Apply weighting using the Wiener filter.
3. Perform the inverse Fourier transform to return to the time domain.

As an example, the removal of salt-and-pepper noise from an image follows these steps:

1. Compute the power spectrum of the noisy image.
2. Estimate the power spectral density of the noise.
3. Apply the Wiener filter to optimize the image.

The Wiener filter is becoming more flexible and dynamic through integration with emerging artificial intelligence and machine learning techniques. In particular, adaptive filtering methods based on the Wiener filter learn signal and noise characteristics in real time to optimize performance. These advancements offer broader application potential in fields such as medicine, autonomous vehicles, and space technology.