Chapter 8: Ensembles & Boosting Mental Models - ML Software Engineering: Interview Concept Review

Learning Objectives

By the end of this chapter, you will be able to:

Relate Ensembles & Boosting Mental Models to common ML software engineering interview questions and trade-offs.
Explain when this topic deserves a deeper pass through another tutorial on this site versus staying at recap depth.
Surface assumptions, pitfalls, and follow-up probes an interviewer is likely to use.

Bootstrap aggregating decorrelates learner errors via data subsampling + averaging predictions—classic random forests average tree votes.

Boosting sequentially reweights hardest examples; reduces bias aggressively but stresses calibration and tuning if outliers dominate.

Stacking trains a meta learner on cross-validated base predictions—higher variance in small data; mention leakage guard via out-of-fold inputs.

1. Gradient boosting ensembles primarily reduce ______ relative to shallow trees?

Bias-dominated approximation error when tuned thoughtfully.
Inference RAM usage always to zero.