Chapter 9: Metrics & Model Interpretation
Metrics & Model Interpretation in ML Software Engineering: Interview Concept Review.
Learning Objectives
By the end of this chapter, you will be able to:
- Relate Metrics & Model Interpretation to common ML software engineering interview questions and trade-offs.
- Explain when this topic deserves a deeper pass through another tutorial on this site versus staying at recap depth.
- Surface assumptions, pitfalls, and follow-up probes an interviewer is likely to use.
Offline metrics are policy choices
Accuracy misleads skewed labels. Pair metrics with slice analysis (region, device, user tenure) to mirror business risk.
Calibration matters when downstream systems threshold predicted probabilities (ads bidding, risk scoring).
ROC versus precision–recall trade space
ROC AUC tolerates massive negatives; PR focuses on rare positives—default to PR narrative in heavy imbalance unless interviewer clarifies uniform prior.
SHAP / feature importances serve communication, not causation. Say that clearly to avoid “explainable AI” mythology traps.
Go deeper on this site
Machine Learning Fundamentals (evaluation language) · Model Relationships (selection narratives)
1. Massive negative class, small positive—first plot family?