Data and ML engineering has its own version of the guessed-fix problem, and arguably a worse one: when a pipeline produces bad output or a model's behavior shifts unexpectedly, it's tempting to treat the whole system as an inscrutable black box and reach for a plausible-sounding adjustment rather than an actual diagnosis.
Why "investigate before you fix" matters even more here
A traditional software bug usually has a discoverable, deterministic cause. A data or ML issue might stem from a schema change three pipeline stages upstream, a subtly shifted input distribution, a silently failing upstream job, or an actual model regression — and these look similar from the outside while having completely different fixes. The Investigate skill's insistence on following evidence through the actual pipeline, verifying real state rather than reasoning about it abstractly, and citing a specific cause is precisely the discipline that keeps "the model's being weird" from becoming an acceptable diagnosis.
Full tier for pipeline architecture, light tier for feature iteration
A new feature-engineering step within an existing pipeline, a model hyperparameter change, a new evaluation metric — usually light tier. A new data source integration, a change to the pipeline's core architecture, or a change with real data-migration or backfill risk — full tier, for the same reasons any other system with real architectural risk gets full tier. The tier decision doesn't need special ML-specific rules; the existing criteria already generalize.
Where analysis-first applies to data work
Rebuilding or significantly changing an existing pipeline benefits from the same Design-First discipline as any other redesign: extracting what the current pipeline actually does — its real behavior, not the outdated architecture diagram nobody's updated in a year — before specifying what the new version should do. app-analysis's emphasis on live interaction and actual observed behavior over assumed behavior applies directly here.
Root cause documentation as a defense against recurring incidents
ML and data issues have an especially high recurrence rate when the actual cause is never nailed down — the same upstream schema fragility, the same silent failure mode, shows up again in three months because the first fix addressed a symptom instead of a cause. A findings document that cites the real cause, not just "adjusted the threshold and it seems better now," is what breaks that cycle.
Data and ML systems don't get a pass on root-cause discipline just because they're probabilistic. If anything, the discipline matters more, precisely because "it's probably just noise" is such an easy, and easily wrong, place to stop looking.