Question

In the context of kernel density estimation, if the chosen bandwidth is significantly too small, what is the inevitable trade-off regarding the bias and variance of the resulting density estimate?

Accepted Answer

In kernel density estimation, the bandwidth is the parameter that controls the width of the kernel function, which determines how much weight is assigned to data points surrounding any given location. When the bandwidth is significantly too small, the estimate suffers from low bias but high variance, a state often described as overfitting. Low bias means the model is highly flexible and closely follows the individual data points in the sample, accurately capturing the local peaks and valleys of the specific dataset at hand. However, the resulting estimate is highly sensitive to the random noise inherent in that specific sample. This high variance means that if you were to collect a new set of data from the same source, the resulting density plot would look significantly different, showing jagged, spurious spikes where the original sample happened to be dense. Because the estimate tracks the noise rather than the underlying smooth probability distribution, it fails to generalize well, resulting in a model that is technically precise for the current sample but unreliable for predicting the broader population structure.

Home → All Courses → Science and Mathematics Courses → Computational Statistics → Flashcard

In the context of kernel density estimation, if the chosen bandwidth is significantly too small, what is the inevitable trade-off regarding the bias and variance of the resulting density estimate?