Interpretability
Understanding why a model predicted what it did, and being honest about how far that understanding goes.
3 articles, newest first.
- Partial dependence plots and what they quietly assume PDPs average over the rest of the data, which means they can describe combinations that never occur. ICE curves, correlated features, and how to read both without overclaiming.
- Feature importance is not explanation Impurity importance, permutation importance, and SHAP each answer a narrower question than people think — and none of them answers "what causes what".
- Reading SHAP values correctly SHAP explains one prediction against a baseline, not the world. What the numbers mean, what the plots hide, and the readings that get a model into trouble.