At the Pandora plenary meeting in Caen, I gave a short talk on how to characterize difficult examples in order to better understand datasets and improve machine learning models. The talk highlighted why prediction errors should be analyzed beyond a simple binary view, since they can arise from different causes such as ambiguity, atypicality, mislabeling, or out-of-distribution samples.
I also discussed how training dynamics can help identify and study these difficult cases, and why combining data-centric and model-centric perspectives is useful for understanding persistent model failures.