Choosing ML Algorithms — Problem Shape to Model Family
Senior interviewers do not want an algorithm encyclopedia. They want you to map problem shape to model family under constraints: tabular vs sequence, interpretability vs accuracy, latency, data size, and drift. Start with a baseline; ship the simplest model that meets the metric and the ops budget.
- 1Gist
- 2Maps
- 3Q&A
- 4Sandbox
Voice readout needs Web Speech Synthesis in this browser.
Interviewer: which model do you use?
Prefer
Shape, then constraints, then family, then a baseline bake-off
Name the data geometry and the ops budget before XGBoost. Beat mean / majority / linear on a leakage-safe split, then escalate.
- Tabular, sequence, and pixels want different inductive biases.
- Explainability, p99 latency, and drift beat leaderboard folklore.
- Libraries are a last-mile choice after the family is right.
Alternative
Jump to a deep net or a default GBM
Skipping baselines hides data bugs and overfits the interview. Deep nets rarely win first on flat tabular under 50k rows.
- Train-set accuracy is not a decision.
- Random splits on time-ordered data lie.
- Feature importance is not causal.
Interview order of operations
Family first. Library last. A linear baseline is not optional.
- 1
Name the labels
Supervised, unsupervised, ranking, classification, regression, multilabel. - 2
Name the modality
Tabular, sequence, text, image. Inductive bias must match structure. - 3
Check time
Does shuffling destroy skill? If yes, walk-forward — never random K-fold. - 4
List constraints
Explainability, p99, CPU vs GPU, n vs p, missingness, drift. - 5
Baseline, then escalate
Mean / linear / seasonal naive, then tree / RF / GBM / classical TS / deep sequences.
Overview
Senior SWE / ML interviewers do not ask you to recite every algorithm. They ask you to defend a choice under constraints: tabular vs sequence, interpretability vs accuracy, latency budget, data size, and concept drift.
This hub stays under AI / ML. Classical ML belongs here — not under system design. It does not duplicate Structured Outputs, and it does not re-teach causal identification.
You should be able to:
- Walk a problem-shape checklist in interview order.
- Point at trees, forests, boosting, forecasting, and the playbook as separate full lessons.
- Beat a dumb baseline before claiming SOTA.
Problem-shape checklist
- Supervised vs unsupervised vs ranking — labels, or pairwise preferences?
- Classification vs regression vs multilabel
- Tabular vs sequence vs text/image — inductive bias must match structure
- Data size and feature types — n much smaller than p, categoricals, missingness, high cardinality
- Interpretability and compliance — can you ship a black box?
- Latency and footprint — batch vs online p99; CPU vs GPU
- Stationarity and drift — i.i.d. tabular vs temporal / non-stationary
When each family wins
- Linear / GLM — strong baselines; sparse or linear signal; coefficient audits
- Single decision tree — rules, debug, tiny data, explainability; high variance alone
- Random forest / bagging — robust medium-tabular default; parallel; often loses to GBM on SOTA tabular
- Gradient boosting — tabular workhorse; sequential residuals; watch calibration and latency
- Classical TS (ARIMA / ETS / Prophet) — clear seasonality or trend, short univariate series
- Trees on lagged features — many exogenous covariates plus mild temporal structure
- Deep sequences / foundation TS — long horizons, multimodal; heavier ops
- Deep nets (vision / NLP) — pixels and tokens; rarely first for flat tabular
Rule of thumb: beat mean / majority / linear first; ship the simplest model that meets the metric and the ops budget.
Architecture (problem shape to family)
Single-column chooser. Always keep a linear baseline on the board.
Decisions
- ?
1 Labels?
- supervised2 Modality?
- unsupervisedLinear / GLM baseline
- ?
2 Modality?
- tabular3 Explain / latency / drift?
- text / imageVision / NLP nets
- sequence3 Truly temporal?
- 3
Linear / GLM baseline
- ?
3 Explain / latency / drift?
- rules / debugSingle tree
- robust defaultRandom forest
- tabular accuracyGradient boosting
- always startLinear / GLM baseline
- 5
Vision / NLP nets
- ?
3 Truly temporal?
- autocorrelationARIMA / ETS / Prophet
- covariates winGradient boosting
- long multimodalDeep sequence / foundation
- 7
ARIMA / ETS / Prophet
- 8
Gradient boosting
- 9
Deep sequence / foundation
- 10
Single tree
- 11
Random forest
Lesson map
Choosing ML Algorithms — Problem Shape to Model Family
Senior interviewers do not want an algorithm encyclopedia. They want you to map problem shape to model family under constraints: tabular vs sequence, interpretability vs accuracy, latency, data size, and drift. Start with a baseline; ship the simplest model that meets the metric and the ops budget.
Architecture. Architecture
Select a node to see why it exists, or an edge to see the protocol, direction, effect, and consequence.
Mermaid export
flowchart TB q1["1 Labels?"] q2["2 Modality?"] c1["3 Explain / latency / drift?"] q3["3 Truly temporal?"] q1 -->|supervised| q2 q2 -->|tabular| c1 q2 -->|sequence| q3
- Rules and debug: decision trees
- Robust default: random forests
- Tabular accuracy: gradient boosting
- Seasonality and lags: forecasting
- Metrics, leakage, production: playbook
Sandbox: family recommender (Python)
Conceptual router. Always keep a linear baseline in the experiment tracker.
Press Run. Snippets must be self-contained — no network, files, or native modules.
Same gate (TypeScript)
Press Run. Snippets must be self-contained — no network, files, or native modules.
This cluster (five siblings)
- Decision trees — splits, interpretability, when a single tree fails.
- Random forests — bagging, OOB, importance pitfalls.
- Gradient boosted trees — XGBoost / LightGBM / CatBoost tradeoffs.
- Time series forecasting — classical vs ML features vs deep sequences.
- Algorithm selection playbook — metrics, baselines, leakage, production.
LLM numeric forecasting is a niche. Structured Outputs is an interface concern, not a calibrated forecast — do not fold that cluster in here.
Pitfalls
Fraud on 2M tabular rows, 40 mixed features, p99 under 8 ms on CPU, auditors want some explanation. Sketch the checklist, name the baseline, then the family you would try second. What changes if the label is a 7-day-ahead count with weekly seasonality?
Interview Q&A
RF vs GBM on tabular?
When is deep learning the wrong first choice?
Answer
Flat tabular under about 50k rows, auditability needs, or tight CPU latency — trees and linear models usually dominate. Deep nets earn cost on images, text, audio, or long multimodal sequences.
Time series vs tabular with a date column?
Answer
Check autocorrelation and whether shuffling destroys performance. If order matters, use walk-forward validation — never random splits. Depth: forecasting.
LLM forecasting in interviews?
Answer
Niche for narratives or constrained API outputs. Numeric horizons still belong to classical / ML / deep TS. Structured JSON is an interface concern, not a calibrated forecast — see Structured Outputs only as that interface.
What is the first sentence of a senior answer?
Answer
“Here is the problem shape, here is the constraint, here is the baseline I must beat, here is the family I would try next.” A library name without that sentence is junior.
n much smaller than p — what family?
Answer
Regularized linear / GLM, or a heavily pruned tree if you need rules. Deep nets and unregularized GBMs memorize noise. Watch leakage in feature selection.
Need human-readable rules for ops or legal?
Answer
A pruned CART tree or a tiny rule list. Forests and GBMs can yield surrogate trees, but the deployable explanation is the shallow path. Depth: trees.
Unsupervised ask — clustering or a supervised reframe?
Answer
Ask whether a downstream decision has a label you could collect. Clustering and dim-red are diagnostics; many “unsupervised” product asks are ranking or classification in disguise. Keep a linear baseline if labels exist.
Prediction vs ‘did the launch work?’
Answer
Forecasting predicts under status quo. A launch effect is a counterfactual. Cross-link causal time series lightly from the forecasting lesson — do not bake off identification estimators here.
What belongs in the playbook after you pick a family?
Answer
Metric matched to cost, leakage-safe split, calibration if you threshold, drift monitoring, retrain cadence. Depth: playbook.