The
Law That Explains Everything Wrong With Your Quality Dashboard
In 1975, the British economist Charles Goodhart wrote a sentence that
would become one of the most quoted observations in social science:
“Any observed statistical regularity will tend to collapse once
pressure is placed upon it for control purposes.” Marilyn Strathern
later simplified it into the form most people know: When a
measure becomes a target, it ceases to be a good measure.
If you’ve spent any time in manufacturing quality, you’ve lived this
law. You just didn’t have a name for it.
You set a defect rate target, and your inspectors start reclassifying
defects. You tie bonuses to OEE numbers, and suddenly every minute of
unplanned downtime becomes “planned maintenance.” You track first-pass
yield, and your team discovers that rework doesn’t count as a failure if
they never formally document the defect in the first place.
The measure didn’t break. Your organization broke the measure. And it
did so with the best of intentions, using the systems you designed,
following the incentives you created.
This is Goodhart’s Law in action on the manufacturing floor, and
understanding it might be the single most important thing you can do to
improve how your organization thinks about quality.
Why
Goodhart’s Law Hits Manufacturing Especially Hard
Manufacturing organizations are measurement-obsessed by nature. You
have to be. You’re running processes with tight tolerances, regulatory
requirements, customer specifications, and audit standards that demand
numerical evidence of compliance. The entire quality management system —
from incoming inspection to final audit — is built on the assumption
that if you measure the right things and hit the right numbers, good
quality follows.
This assumption is not wrong. Measurement is essential. Targets are
essential. Without them, you have no way to know whether your processes
are stable, improving, or quietly falling apart.
But Goodhart’s Law describes the gap between “measuring to
understand” and “measuring to hit a number.” And that gap is where most
of your quality problems live.
The reason manufacturing is so vulnerable to Goodhart’s Law is
threefold:
First, the stakes are real. When metrics are tied to
performance reviews, bonuses, shift rankings, or customer scorecards,
people optimize for the metric — not for the underlying quality the
metric was supposed to represent. This isn’t dishonesty. It’s rational
behavior in a system that rewards the wrong thing.
Second, the measurements are complex and
interpretable. Unlike a simple count of “how many units came
off the line,” most quality metrics involve judgment calls. Is that
scratch a cosmetic defect or within specification? Does that dimension
fall inside the tolerance band if you measure it differently? Is that
stoppage really “unplanned”? Where there’s interpretation, there’s
optimization — and where there’s optimization under pressure, the
measurement degrades.
Third, the feedback loops are slow. By the time a
customer complains about a quality problem that your metrics said was
“fine,” months may have passed. The metric looked good the entire time.
The underlying quality was eroding. But nobody noticed because the
number said everything was on target.
The Anatomy of Metric
Collapse
Goodhart’s Law doesn’t happen overnight. It follows a predictable
pattern, and if you know what to look for, you can catch it before it
does real damage.
Phase 1: The Honest
Measurement
A quality metric is introduced with genuine intent. Maybe it’s a
first-pass yield target of 98%. At first, the measurement is honest —
defects are logged accurately, the number reflects reality, and the
organization uses it to make real improvements. This phase can last
months or even years.
Phase 2: The Target Arrives
Then someone decides this metric should be a target. It goes on the
weekly dashboard. It’s discussed in the production meeting. It gets tied
to shift performance, or department rankings, or individual evaluations.
The message is clear: this number matters, and it needs to go up (or
down, or stay steady).
At this point, the metric is still mostly honest. But the pressure
has begun.
Phase 3: The Optimization
Begins
People start making small adjustments. Not fraud, not deception —
just reasonable interpretations that happen to make the number look
better. An inspector classifies a borderline defect as “within spec.” A
supervisor codes a downtime event as “planned” to protect the OEE
number. An engineer adjusts a sampling plan so that it’s less likely to
catch defects during periods when the numbers are being watched
closely.
Each individual adjustment is defensible. Each one makes sense in
context. But collectively, the metric is slowly detaching from
reality.
Phase 4: The Uncoupling
Eventually, the metric has become something entirely different from
what it was designed to measure. The first-pass yield number says 98.5%.
The customer return rate says something different entirely. The OEE
dashboard shows 85%. The maintenance logs tell a story of chronic
breakdowns that somehow never made it into the calculation.
The measure has collapsed. It’s still a number. It’s still on the
dashboard. But it no longer measures quality. It measures the
organization’s ability to produce the number.
Phase 5: The Crisis
And then something breaks. A customer audit finds problems your
metrics said didn’t exist. A warranty claim spike reveals defects that
your dashboards never showed. A regulatory inspection uncovers
systematic issues that your numbers, your reports, and your
certifications all said were under control.
The investigation always goes the same way: “How did we miss this?
The numbers looked fine.” And the answer is always the same: the numbers
were fine. The quality wasn’t.
The Classic Examples
You’ve seen all of these. Maybe you’ve lived them.
The Scrap Rate Target. You set a scrap rate of less
than 2%. Your teams discover that if they rework a part before it’s
formally scrapped, it doesn’t count. Scrap rate drops to 1.3%. Everyone
celebrates. But rework costs have tripled, cycle times have increased,
and the reworked parts have higher failure rates in the field. You hit
the target. You missed the point.
The Customer Complaint Metric. You track customer
complaints per thousand units shipped. The number looks great — under
0.5, best in class. But your customers have stopped complaining because
they’ve learned your response process is so bureaucratic and slow that
it’s not worth the effort. Instead, they’ve quietly started sourcing
from your competitor and will fully switch at the end of the contract.
Your metric didn’t measure satisfaction. It measured tolerance for
paperwork.
The Audit Finding Count. You track the number of
audit findings per audit as a measure of quality system health. The
number trends downward. Your management team is pleased. But the reality
is that your internal auditors have learned that findings create
corrective action paperwork for their own departments. So they’ve gotten
more selective about what they write up. The audit findings decreased.
The actual nonconformances did not.
The Training Completion Rate. You track training
completion as a percentage. It sits at 99.7%. Excellent. But the
training consists of clicking through a slide deck in three minutes and
checking a box. The employees who “completed” the training couldn’t pass
a basic test on the material if their jobs depended on it — which,
ironically, they might. You measured completion. You didn’t measure
competence.
Why This Keeps Happening
The uncomfortable truth is that Goodhart’s Law isn’t a failure of the
people being measured. It’s a failure of the people doing the
measuring.
When you set a target without understanding the behaviors it will
drive, you’re not managing quality. You’re managing a number. And the
people who work for you — smart, resourceful, motivated people who want
to succeed within the system you’ve built — will figure out how to
succeed within that system. If the system rewards hitting the number,
they’ll hit the number. If the number and quality have diverged, that’s
on the system, not on them.
This is particularly dangerous in organizations with strong
performance cultures. The more seriously people take their metrics, the
more pressure they feel to optimize those metrics, and the faster
Goodhart’s Law takes effect. Apathetic organizations are actually
somewhat protected — nobody cares enough about the metrics to optimize
them. It’s the high-performance organizations that are most at risk.
What
Goodhart-Aware Quality Management Looks Like
You can’t eliminate Goodhart’s Law. It’s not a bug you can fix. It’s
a structural feature of any measurement system. But you can design your
quality metrics to be more resistant to it.
Use Multiple Overlapping
Metrics
Never rely on a single metric to capture something as complex as
quality. If you’re tracking first-pass yield, also track rework hours,
customer returns, warranty costs, and inline defect rates. When one
metric starts to diverge from the others, that’s your early warning sign
that Goodhart’s Law is at work.
The key insight: if five independent metrics all point in the same
direction, you can be reasonably confident you’re measuring reality. If
one metric shows dramatic improvement while the others are flat or
declining, you’re almost certainly measuring optimization, not
improvement.
Rotate and Randomize
Measurement
If people know exactly when and how they’ll be measured, they can
optimize for the measurement. Some of the most effective quality systems
build in unpredictability — random sampling times, rotating audit
schedules, unannounced Gemba walks, and measurement points that shift
periodically.
This isn’t about catching people doing things wrong. It’s about
making it impossible to game a single, predictable measurement
window.
Separate Measurement From
Targets
This is the hardest one, but also the most important. The act of
measuring something should be separated, as much as possible, from the
consequences of that measurement.
In practice, this means that the people collecting quality data
should not be the same people whose performance is evaluated based on
that data. The inspector who finds defects should not report to the
supervisor whose metrics depend on low defect rates. The auditor who
writes findings should not be in the same management chain as the
process owner being audited.
Independence of measurement is not just an auditing principle. It’s a
Goodhart defense.
Track the Behavior, Not
Just the Outcome
Instead of measuring “defect rate,” measure whether people are
following the process correctly. Instead of measuring “training
completion,” measure whether operators can demonstrate competency.
Instead of measuring “number of corrective actions closed,” measure
whether the problems actually recurred.
Outcome metrics are easier to game because outcomes can be
reclassified, reinterpreted, or redefined. Behavioral metrics — did you
follow the procedure, did you stop the line when you saw a problem, did
you escalate when the data looked wrong — are harder to fake and more
directly connected to the quality you’re actually trying to build.
Make It Safe to Report Bad
Numbers
The single most powerful accelerator of Goodhart’s Law is fear. When
people are afraid that a bad number means a bad performance review, a
lost bonus, or a public shaming in the production meeting, they will
find ways to make the number look better. This is not a character flaw.
It’s a survival instinct.
The organizations most resistant to Goodhart’s Law are the ones where
bad numbers are treated as information, not as failure. Where a rise in
defect rate triggers curiosity (“What’s happening?”) rather than
punishment (“Who’s responsible?”). Where the first response to a missed
target is to ask whether the process needs help, not whether someone
needs to be blamed.
This is easy to say and extraordinarily hard to do. But it’s the
foundation of everything else.
The Dashboard Test
Here’s a simple exercise. Look at your quality dashboard — the one
you review every week, the one that drives decisions and discussions and
performance evaluations. For each metric on that dashboard, ask yourself
two questions:
- If someone wanted to make this number look better without
actually improving quality, could they do it? - Would anyone notice?
If the answer to the first question is “yes” and the answer to the
second is “probably not,” you have a Goodhart problem. That metric is no
longer measuring quality. It’s measuring your organization’s ability to
produce that metric.
And the really insidious part? The more important that metric is —
the more visibility it has, the more it’s tied to rewards and
consequences — the faster it’s decaying from a measurement into a
performance.
The Deeper Lesson
Goodhart’s Law is ultimately about the relationship between maps and
territories. Your quality metrics are a map. The actual quality of your
products, processes, and systems is the territory. The map is useful —
essential, even. But when you start managing the map instead of the
territory, you get lost while convinced you know exactly where you
are.
The best quality professionals understand this intuitively. They
don’t ignore metrics. They use them constantly. But they never forget
that the metric is a proxy, not the thing itself. They cross-reference.
They verify. They go to the Gemba and look with their own eyes. They
maintain a healthy skepticism about numbers that look too good,
improvements that come too fast, and dashboards that always seem to
confirm that everything is under control.
Because the most dangerous moment in quality management isn’t when
your metrics show a problem. It’s when your metrics show everything is
fine — and you believe them without checking.
Charles Goodhart gave us the law. The manufacturing world keeps
proving it. The only question is whether your organization will learn
from it before the next metric collapse teaches the lesson the hard
way.
Peter Stasko is a Quality Architect with over 25
years of hands-on experience in manufacturing quality systems, process
optimization, and organizational transformation. He has helped companies
across automotive, aerospace, electronics, and medical device industries
build quality systems that actually work — not just systems that look
good on paper. His approach combines deep technical knowledge with a
pragmatic understanding of how real people in real organizations
actually behave.
