How to evaluate clustering with ground truth?

This paper provides a review of external validity indexes used to evaluate clustering results when ground truth data is available. The author examines various set-matching-based measures to help researchers choose the most appropriate evaluation method depending on whether they need to assess the quality of clusters as a whole or the accuracy of individual data points.

Evaluating cluster quality

When ground truth is available, external indexes allow researchers to measure how well a clustering algorithm has performed. The paper focuses on set-matching-based measures, which compare the clusters produced by an algorithm against the known, correct groupings. The author highlights that the choice of index depends on the specific goals of the evaluation, such as whether the focus is on the overall structure of the clusters or the specific placement of individual points.

Recommended approaches

The author recommends the Centroid Index (CI) as a primary tool for evaluation. This measure is highlighted for being intuitive and providing results that are easy to explain at the cluster level. For researchers requiring a more granular, point-level assessment, the paper suggests the Pair-set Index (PSI). The PSI is noted for providing a normalized score that remains reliable even when cluster sizes vary, avoiding the bias that can occur with other metrics.

Selecting the right metric

The paper clarifies that the "best" metric depends on the researcher's priorities. If the objective is to ensure that every point is treated with equal importance, the author suggests using clustering accuracy (ACC) or other similar set-matching measures. By categorizing these indexes based on their strengths—whether they are better suited for cluster-level analysis or point-level precision—the paper serves as a guide for selecting the most effective evaluation strategy for center-based clustering tasks.