Question

In the context of model performance metrics, why is a high F1 score often preferred over basic accuracy when dealing with imbalanced datasets?

Accepted Answer

Accuracy measures the ratio of correct predictions to the total number of cases, which is misleading when a dataset is imbalanced. In an imbalanced dataset, one class significantly outnumbers the other. For instance, if 99 percent of transactions are legitimate and 1 percent are fraudulent, a model can achieve 99 percent accuracy simply by labeling every transaction as legitimate, even though it failed to identify a single fraudulent case. The F1 score is preferred because it calculates the harmonic mean of precision and recall, providing a more balanced assessment. Precision measures how many of the positive predictions were actually correct, while recall measures how many of the actual positive cases were correctly identified by the model. By combining these two metrics, the F1 score penalizes models that ignore the minority class. Because the harmonic mean is highly sensitive to low values, a model must achieve both high precision and high recall to obtain a high F1 score, ensuring that the model is performing well on the minority class rather than just relying on the abundance of the majority class.

Home → All Courses → Engineering and Technology Courses → AI Product Management → Flashcard

In the context of model performance metrics, why is a high F1 score often preferred over basic accuracy when dealing with imbalanced datasets?