Quality
and Goodhart’s Law: When Your Organization’s KPIs Become the Reason Its
Quality Collapses — and the Numbers Everyone Chased Replaced the
Excellence Everyone Wanted
The Day the Dashboard Ate
the Factory
Martin was proud of his dashboards. As Quality Director of a
mid-sized automotive components supplier, he had spent two years
building what he considered the most sophisticated quality metrics
system in the region. Every station, every shift, every operator had a
number. Scrap rate. First-pass yield. Customer complaints per million.
On-time delivery. Audit findings closed. Corrective actions overdue. The
data flowed in real time to a wall-mounted screen in the cafeteria,
updated hourly, color-coded red-yellow-green like a traffic light for
the entire organization.
And for eighteen months, it worked. Every metric trended in the right
direction. The green bars grew. The red bars shrank. Martin presented
the charts at every management review. The CEO quoted the numbers at
town halls. Customers saw the data during audits and nodded approvingly.
The factory won a regional quality award, and Martin was promoted.
Then, on a Tuesday in November, a customer’s engineering team arrived
for an unannounced visit. Not an audit — a visit. Their end-of-line
failure rate had been climbing for six months. They had traced every
failure back to a single tolerance parameter on a single component — the
one component Martin’s factory supplied.
The customer’s lead engineer sat in Martin’s conference room, slid a
stack of nonconformance reports across the table, and asked a single
question: “How is your first-pass yield above 99% when your parts are
failing at our line at four times the rate they were two years ago?”
Martin opened his mouth. Closed it. Opened his browser to the
dashboard. Every number was green. He looked at the screen, then at the
stack of failures, then back at the screen. The numbers said everything
was fine. The parts said everything was not.
What Martin discovered over the next three months of investigation
would reshape his entire understanding of quality management. He hadn’t
built a quality system. He had built a system for producing numbers that
looked like quality. And the distinction — the gap between the metric
and the reality — had cost his organization its most important customer
relationship.
The British economist Charles Goodhart once observed a principle so
simple it sounds almost trivial: When a measure becomes a
target, it ceases to be a good measure. In the world of
finance, where Goodhart articulated his law, this meant that once a
central bank targeted a specific monetary aggregate, the relationship
between that aggregate and the real economy would break down.
But in quality management — where organizations live and die by their
KPIs, where every process is measured, tracked, and incentivized —
Goodhart’s Law isn’t an economic curiosity. It’s an existential
threat.
Why Your Best Metrics Betray
You
Goodhart’s Law operates through a mechanism that is both predictable
and invisible. The cycle works like this:
Phase One: The Measurement Is Honest. An
organization identifies a metric that genuinely reflects quality
performance. First-pass yield really does correlate with process health.
Scrap rate really does indicate waste. Customer complaints really do
signal dissatisfaction. The data is collected, reported, and used to
make better decisions. Improvement follows.
Phase Two: The Measurement Becomes Important.
Leadership notices the metric. It appears on scorecards. It gets
discussed in meetings. It becomes part of performance reviews, bonus
calculations, and supplier scorecards. The metric is no longer just
information — it has consequences.
Phase Three: The Measurement Becomes a Target. Goals
are set. “First-pass yield must exceed 99%.” “Scrap rate must stay below
0.5%.” “Customer complaints must decrease by 20% year over year.” The
metric transitions from a rearview mirror into a steering wheel.
Phase Four: The System Optimizes for the Metric, Not for
Quality. And this is where everything breaks. Because human
beings are optimization engines. When you attach consequences to a
number, intelligent, well-meaning people will find ways to make the
number look good — even if the underlying reality it was supposed to
measure deteriorates.
The inspector who is measured on defects found will find defects —
including ones that aren’t there. The operator whose bonus depends on
first-pass yield will find ways to rework parts offline so they never
appear as failures in the system. The quality engineer whose performance
review includes “corrective actions closed on time” will close
corrective actions on time — by implementing shallow fixes that address
symptoms while the root cause festers. The purchasing manager measured
on cost reduction will find cheaper suppliers — and the incoming quality
problems will show up as someone else’s metric.
None of this is fraud. None of this is malicious. It is rational
behavior in a system that has accidentally incentivized the wrong thing.
And it happens in every organization that has ever set a quality target
without understanding Goodhart’s Law.
The Anatomy of Metric
Corruption
Let me walk you through the specific ways Goodhart’s Law corrupts
quality systems, drawn from twenty-five years of watching it happen
across automotive, aerospace, and pharmaceutical manufacturing.
1. Reclassification Games
When scrap rate becomes a KPI with consequences, organizations
discover a remarkable taxonomy of non-scrap outcomes. Material that once
would have been scrapped is now “reworked to specification.” Or
“downgraded to a lower-grade application.” Or “held for engineering
review” — indefinitely. The scrap rate drops. The physical waste hasn’t
changed. Only the label has.
I worked with a precision machining operation that reduced its scrap
rate from 3.2% to 0.8% in a single quarter. Their CEO celebrated. When I
visited the plant, I found a warehouse area the floor supervisor called
“the waiting room” — pallets of nonconforming material awaiting
disposition decisions that somehow never came. The parts weren’t
scrapped. They weren’t reworked. They weren’t used. They simply existed
in a statistical limbo where they didn’t count against any metric at
all.
2. Boundary Manipulation
When your customer complaint metric is measured per units shipped,
the most effective way to reduce it is to ship more units — even
marginal ones. When your audit finding closure rate is measured as
“percentage closed within 30 days,” the most effective strategy is to
redefine what counts as an audit finding. When your on-time delivery is
measured against the date you promised rather than the date the customer
needed, you can achieve world-class delivery performance by promising
dates you know you can meet — and letting the customer’s production line
wait.
I recall a pharmaceutical contract manufacturer that achieved a
remarkable 98% “right first time” metric by simply expanding the
specification limits on in-process testing. The products still passed
final release testing — barely. But the process capability index, which
nobody was targeting, had quietly dropped from 1.67 to 0.95. They were
operating at the ragged edge of compliance and calling it excellence
because the metric they had chosen to elevate told them they were
excellent.
3. Effort Displacement
Perhaps the most insidious effect of Goodhart’s Law is effort
displacement — the redirection of attention and resources from
unmeasured quality activities toward measured ones.
When organizations measure the number of corrective actions
initiated, they get corrective actions — for problems that might have
been better addressed through process redesign, operator training, or
supplier development. When they measure audit finding counts, they get
audits that find findings — rather than audits that drive systemic
improvement. When they measure training hours completed, they get
training hours — often in the most convenient format, regardless of
whether anyone learned anything.
The unmeasured activities — the quiet conversations with operators
about what’s really going wrong, the Gemba walks that build
understanding rather than data, the cross-functional collaboration that
prevents problems before they start — these get squeezed out. Not
because they’re unimportant. Because they’re invisible to the
measurement system that now drives organizational behavior.
4. Threshold Effects
When a target is binary — pass/fail, above/below, green/red — the
system optimizes around the threshold rather than around genuine
excellence.
If your target is zero customer complaints, you will invest enormous
energy in resolving the last few complaints — energy that might have
been better spent on prevention activities that don’t move any visible
metric. If your target is less than five major audit findings, you will
pour resources into ensuring you get four findings instead of five —
while ignoring a hundred minor findings that collectively represent far
more risk.
The threshold becomes a magnet. Everything above it is “good enough.”
Everything below it triggers panic. And the actual distribution of
quality performance — the shape of the curve, the trends within the
acceptable range, the near-misses that signal emerging problems — gets
ignored because the only question anyone asks is “Are we above the
line?”
The Deeper
Problem: What Metrics Can’t Measure
Goodhart’s Law reveals something uncomfortable about quality
management: the most important aspects of quality are the hardest to
measure.
Can you measure the trust between a quality engineer and a
production supervisor? That trust determines whether problems get
reported early or hidden until they’re catastrophic.
Can you measure the quality culture’s actual strength?
Survey scores measure what people say. Culture reveals itself in what
people do when nobody is watching — and there is no metric for that.
Can you measure the prevention activities that prevented problems
from ever occurring? By definition, you cannot count the defects
that never happened. The most effective quality work is invisible in any
measurement system.
Can you measure the depth of root cause analysis in a corrective
action? You can measure whether the form is filled out correctly.
You can measure whether it was closed on time. But the quality of
thinking that went into it — the rigor of the investigation, the honesty
of the analysis, the completeness of the solution — these resist
quantification.
When organizations build quality systems around what can be measured,
they inadvertently build systems that optimize what can be measured. The
measurable drives out the meaningful. The countable replaces the
important. And the dashboard glows green while the foundation
cracks.
Living With
Goodhart’s Law: A Practical Framework
You cannot eliminate Goodhart’s Law. It is a structural feature of
any measurement system. But you can design around it. Here is a
framework I have used across dozens of organizations to mitigate its
effects.
1. Measure in Triplicate
Never rely on a single metric for any important quality dimension.
For every aspect of performance you care about, maintain at least three
independent measures — and watch for divergence between them.
If first-pass yield is improving but customer returns are flat,
something is wrong with your yield measurement. If audit findings are
decreasing but warranty claims are increasing, your audits have lost
touch with reality. If training hours are up but operator error rates
haven’t changed, your training isn’t working.
The power isn’t in any single metric. It’s in the triangulation. When
metrics that should move together start to diverge, that divergence is
more valuable information than any individual number.
2. Rotate Your Metrics
Organizations develop metric immunity over time. The same KPIs,
measured quarter after quarter, become familiar — and therefore
gameable. One effective countermeasure is to rotate your key metrics
periodically, not randomly, but strategically.
Measure scrap rate for six months. Then shift to measuring rework
hours. Then measure total cost of nonconformance. Each metric
illuminates a different facet of the same underlying reality. The
rotation prevents any single measure from becoming a permanent target,
and the shifts force the organization to engage with the underlying
quality problem rather than optimizing for a specific number.
3. Separate Measurement
From Consequence
This is the hardest recommendation and the most important one.
Goodhart’s Law activates when metrics have consequences — bonuses,
promotions, penalties, public recognition. The more consequential the
metric, the stronger the distortion.
Wherever possible, use metrics for learning rather than judgment.
Share data openly. Discuss trends honestly. Use numbers to ask better
questions, not to deliver verdicts. When people are not afraid of what
the numbers will do to them, they are less likely to manipulate what the
numbers say.
This doesn’t mean abandoning accountability. It means being
thoughtful about where accountability attaches. Hold people accountable
for the quality of their thinking, the rigor of their analysis, and the
honesty of their reporting — not for hitting a number that they can
influence through methods you don’t want.
4. Audit the Measurement
System Itself
Just as you calibrate your measurement instruments, you must
periodically calibrate your organizational measurement system. Ask
yourself:
Has the relationship between this metric and the quality outcome
it represents changed since we started targeting it?
Are there behaviors emerging that make the metric look good
without improving actual quality?
What important quality dimensions does this metric miss?
If we removed this metric tomorrow, would quality improve or
deteriorate? (If quality would improve when the metric is removed,
the metric has become counterproductive.)
5. Invest in Qualitative
Intelligence
Numbers are necessary but insufficient. Every quality system needs
parallel channels of qualitative intelligence — the kind of
understanding that comes from walking the floor, talking to operators,
observing processes directly, and listening to the stories behind the
data.
The most effective quality leaders I have worked with use metrics as
a starting point for investigation, not as a conclusion. When a number
changes, they don’t celebrate or panic. They ask: “What does this number
mean? What is actually happening on the floor? What is this number not
telling me?”
These leaders maintain what I call a “dual operating system” — the
formal measurement system that produces reports for management reviews,
and the informal intelligence network built through relationships,
presence, and curiosity. When the two systems agree, confidence is high.
When they diverge, the qualitative system is usually closer to the truth
— because it is harder to game than a number.
Martin’s Reckoning
Let me return to Martin, the Quality Director with the beautiful
dashboards.
The investigation into his customer’s failures revealed a cascade of
Goodhart-driven distortions. First-pass yield was indeed above 99% —
because operators had learned to rework parts at their stations before
entering them into the system, so failed first attempts never appeared
in the data. Scrap rate was low because nonconforming material was being
reclassified as “rework in progress” and then quietly dispositioned as
“acceptable with deviation” by an engineering team whose own performance
metric included “minimizing formal scrap events.”
Customer complaints were low because the customer service team —
measured on complaint resolution time — had begun categorizing incoming
quality concerns as “information requests” rather than formal
complaints. The customer’s engineering team had been sending emails
about tolerance drift for eight months. Not a single one had been
classified as a complaint.
The most damning finding was the tolerance parameter itself. Martin’s
process engineering team had noticed it drifting six months earlier. But
the parameter wasn’t on the dashboard — it wasn’t one of the tracked
KPIs. So the drift appeared in engineering reports that nobody read, in
SPC charts that nobody reviewed, and in operator logs that nobody
analyzed. It was, in every meaningful sense, invisible — because the
measurement system had been designed to track the metrics that mattered
to management, not the parameters that mattered to the process.
Martin didn’t lose his job. But he dismantled his dashboard. Not
entirely — he kept the metrics, but he changed how they were used. No
more public displays. No more color-coded traffic lights. No more
bonuses tied to specific numbers. Instead, he instituted monthly “metric
health checks” — structured conversations where cross-functional teams
examined whether the numbers still meant what they were supposed to
mean. He created an anonymous channel for operators to report when they
felt pressure to make numbers look good. He began spending three hours a
day on the production floor, watching processes directly, building the
qualitative intelligence network that his dashboard had replaced.
Six months later, the customer’s failure rate had dropped below its
historical baseline. Not because any metric had improved. Because the
actual quality had.
The Paradox at
the Heart of Quality Management
Here is the uncomfortable truth that Goodhart’s Law forces us to
confront: the moment you measure something in order to control it, you
begin to lose control of what you were trying to measure.
This doesn’t mean measurement is futile. It means measurement is
powerful — and power without wisdom is dangerous. The best quality
systems I have seen in twenty-five years share a common trait: they
treat metrics with respect but not reverence. They use numbers as tools
for understanding, not as substitutes for judgment. They maintain the
humility to recognize that every measurement is a simplification, and
that the most important aspects of quality will always resist being
reduced to a number on a dashboard.
The organization that understands Goodhart’s Law doesn’t abandon
metrics. It holds them lightly. It watches for the moment when the
metric starts driving behavior instead of reflecting it. It invests in
the unmeasurable — in trust, culture, honest communication, and the kind
of deep process understanding that no KPI can capture.
Because in the end, quality is not a number. Quality is what happens
when nobody is measuring.
Peter Stasko is a Quality Architect with 25+ years
of experience transforming organizations across automotive, aerospace,
and pharmaceutical industries. He has spent his career helping companies
build quality systems that actually work — not just systems that look
good on dashboards. His approach combines rigorous statistical
methodology with deep understanding of human behavior, organizational
psychology, and the uncomfortable truth that the most important quality
problems are usually the ones your metrics aren’t showing you.
