Model evaluation is difficult and generally relies on analysis that can mask compensating errors. This paper defines new metrics, using clusters generated from a machine learning algorithm, to estimate mean and compensating errors in different model runs. As a test case, we investigate the Southern Ocean shortwave radiative bias using clusters derived by applying self-organizing maps to satellite data. In particular, the effects of changing cloud phase parameterizations in the MetOffice Unified Model are examined. Differences in cluster properties show that the regional radiative biases are substantially different than the global bias, with two distinct regions identified within the Southern Ocean, each with a different signed bias. Changing cloud phase parameterizations can reduce errors at higher latitudes but increase errors at lower latitudes of the Southern Ocean. Ranking the parameterizations often shows a contrast in mean and compensating errors, notably in all cases large compensating errors remain.