AI Engineering Degree Practice Exam 2025 - Free AI Engineering Practice Questions and Study Guide

When the value of K in KNN (K-Nearest Neighbors) is too small, the model becomes highly sensitive to small fluctuations in the data, which results in increased complexity. A very small K, such as 1, means that the model makes decisions based solely on the nearest neighbor. This can lead to overfitting, as the model captures noise and anomalies in the training data rather than generalizing from the underlying pattern.

This complexity arises because the model contours itself tightly around the training samples, reacting to every data point, including outliers. As a result, while it may fit the training data perfectly, its ability to generalize to unseen data is compromised, making the model perform poorly in practical applications, especially when there is noise in the dataset.

In contrast, as K increases, the model becomes more general and less sensitive to noise, thus reducing its complexity and overfitting potential. This illustrates why a small K leads to a highly complex model.