The Comfort of Being Right
Every quality engineer has been there. You run the numbers, build the
report, present the findings, and everyone nods. The data confirms what
you already suspected. The process is running well. The defect rate is
under control. The supplier is performing within specification. The
corrective action worked.
It feels good. It feels like competence.
And then, three months later, a customer returns an entire shipment.
The defect you said was under control has been quietly growing. The
supplier you certified has been shipping nonconforming material
disguised in conforming packaging. The corrective action you celebrated
was never actually implemented on the shop floor — it existed only in
the PowerPoint presentation.
What happened? You weren’t incompetent. You weren’t lazy. You weren’t
even wrong about the data.
You were just seeing what you expected to see.
Confirmation bias is the tendency to search for, interpret, favor,
and recall information in a way that confirms or supports one’s prior
beliefs or values. In manufacturing quality, it is not merely a
psychological curiosity — it is an operational hazard. It is the reason
organizations collect mountains of data and still miss the signals that
matter. It is the reason experienced engineers make catastrophic
misjudgments. And it is the reason your quality system, no matter how
sophisticated, can be blindsided by problems that were visible all
along.
How Confirmation
Bias Infects Quality Systems
Confirmation bias does not announce itself. It does not show up in
your audit findings or your management review. It operates silently,
shaping the questions you ask, the data you collect, the way you
interpret that data, and the actions you take — or don’t take.
In Inspection and Testing
When an inspector expects a part to be good, they inspect it
differently than when they expect it to be bad. This is not a character
flaw. It is a documented, measurable phenomenon. Studies in visual
inspection have shown that inspectors who are told a batch is “probably
fine” miss significantly more defects than inspectors who are told to
“watch carefully — we’ve had issues with this supplier.”
The confirmation bias works through attention. When you expect
something to be fine, your visual system allocates less scrutiny. Your
brain takes shortcuts. It fills in gaps with assumptions. The part looks
right, so it must be right — until it isn’t.
This is why blind inspections, where the inspector does not know the
source or expected result, consistently outperform informed inspections.
It is also why your best inspectors are often the ones who have been
burned before — they’ve learned, through painful experience, not to
trust their expectations.
In Root Cause Analysis
Confirmation bias is the mortal enemy of effective root cause
analysis. When a team begins an investigation with a strong hypothesis —
“it’s probably the operator,” “it’s always been the material,” “we
changed the machine settings last week” — the investigation subtly but
powerfully bends toward confirming that hypothesis.
The team asks questions that support their theory. They collect data
that supports their theory. They interpret ambiguous evidence in ways
that support their theory. And they discount, dismiss, or simply never
gather the data that would challenge their theory.
This is why the Five Whys so often produces superficial answers. It
is why 8D investigations so often end with “operator error — retrained”
as the root cause. It is why the same problems recur after corrective
actions that everyone was confident would work.
The corrective action confirmed the hypothesis. It did not solve the
problem.
In SPC and Data Analysis
Statistical process control was designed to be objective. Control
charts don’t have opinions. Capability indices don’t play favorites. And
yet, confirmation bias finds its way into SPC through the humans who
create the charts, select the data, and interpret the results.
Consider: which data points get excluded as “special causes” and
which get included as “common cause variation”? Which characteristics
get charted and which get ignored? Which control limits get recalculated
and which stay frozen? Every one of these decisions is a potential entry
point for confirmation bias.
A process engineer who believes a process is capable will
unconsciously set wider control limits, exclude more out-of-control
points, and choose charting methods that smooth variation. The same
engineer, confronted with a process they distrust, will set tighter
limits, investigate every anomaly, and choose methods that highlight
variation.
Same process. Same data. Different conclusions. The difference is the
expectation.
In Supplier Quality
Supplier quality management is fertile ground for confirmation bias.
Once a supplier has been approved and certified, the organization
develops a belief that the supplier is good. This belief then shapes how
incoming inspection is conducted, how audits are performed, and how
deviations are handled.
A supplier with a strong quality history gets the benefit of the
doubt. A minor dimensional deviation is “within measurement error.” A
missed delivery is “just a scheduling fluke.” A packaging defect is
“cosmetic, not functional.”
Meanwhile, a supplier with a poor quality history faces the opposite
bias. Every deviation is evidence of systemic failure. Every late
delivery proves they can’t be trusted. Every minor finding in an audit
becomes a major nonconformance.
Sometimes the poor-quality supplier genuinely has systemic problems.
But sometimes they’ve improved, and your organization can’t see it
because confirmation bias has already convicted them. And sometimes the
“good” supplier has been cutting corners for years, and your
organization can’t see it because confirmation bias has already
acquitted them.
In Management Reviews
and Decision-Making
Management reviews are supposed to be objective assessments of
quality system performance. In practice, they often become exercises in
confirming that the quality system is working as expected. Managers come
to the review with beliefs about what is going well and what is not.
They look at dashboards that have been designed to show the metrics that
matter to them. They ask questions that elicit answers consistent with
their expectations.
When a metric trends downward, the first question is often “is the
data right?” rather than “what is happening?” When a metric trends
upward, the first response is often “great work” rather than “is this
real?”
This is not cynicism. This is how human cognition works under
pressure. Managers have limited time, limited attention, and a strong
need to believe that the systems they oversee are functioning.
Confirmation bias provides that comfort. It also provides the blind
spots that lead to crisis.
The Cost of Seeing What You
Expect
The costs of confirmation bias in quality are enormous, though they
are rarely attributed to the bias itself. They show up instead as:
- Recurring defects that were “solved” by corrective
actions that addressed the wrong root cause. - Customer complaints about issues that internal data
said were under control. - Supplier failures from sources that had been
certified and trusted. - Audit nonconformances that surprise everyone
because “we’ve always done it this way and it’s always been fine.” - Wasted investigation time chasing hypotheses that
confirmed expectations while the real cause went unexamined. - False confidence in process capability, leading to
commitments to customers that cannot be met.
The most insidious cost is the erosion of learning. When confirmation
bias prevents an organization from seeing reality, it prevents the
organization from learning. The same mistakes get made. The same blind
spots persist. The quality system becomes increasingly elaborate and
increasingly disconnected from actual performance.
Structural
Defenses Against Confirmation Bias
You cannot eliminate confirmation bias through willpower. Telling
people to “be objective” does not work. Telling engineers to “consider
all the data” does not work. The bias operates below the level of
conscious awareness. The defenses must be structural — baked into the
systems, processes, and culture.
Blind Analysis
Whenever possible, separate the analyst from the context that
triggers expectations. In inspection, this means blind sampling. In root
cause analysis, it means presenting the team with data before revealing
the hypothesis. In SPC, it means having the control limits set by
someone other than the process owner.
Pre-Registration of
Hypotheses
Before beginning an investigation, require the team to write down
their hypothesis and their planned analysis. This creates a record that
makes it harder to unconsciously revise the hypothesis after seeing the
data. It also makes it possible to distinguish between “the data
supports our hypothesis” and “we found a way to interpret the data as
supporting our hypothesis.”
This is standard practice in clinical trials. It should be standard
practice in root cause analysis.
Red Teams and Devil’s
Advocates
Assign someone — or a small team — the explicit role of challenging
the dominant interpretation. Not as a contrarian exercise, but as a
structured function. Their job is to find the evidence that contradicts
the prevailing theory. They ask: “What would have to be true for us to
be wrong? What data would we expect to see if our hypothesis is
incorrect? Where is that data?”
This role must be valued, not punished. If the devil’s advocate is
treated as an obstacle, the organization will learn to suppress dissent,
and confirmation bias will strengthen.
Diverse Teams
Confirmation bias thrives in homogeneous groups. People with similar
backgrounds, similar training, and similar experiences tend to have
similar expectations. Diverse teams — diverse in discipline, in
experience, in perspective — are more likely to generate competing
interpretations of the same data.
This does not mean putting a random collection of people in a room.
It means deliberately including perspectives that are likely to
challenge the dominant view. If the quality team thinks the problem is
the material, bring in the process engineer. If the process team thinks
the problem is the machine, bring in the operator.
Data Before Narrative
Confirmation bias is most powerful when it operates through narrative
— the stories we tell ourselves about what is happening. The defense is
to require data before narrative. Before any hypothesis is discussed,
the team reviews the raw data — defect rates, control charts,
measurement results, process parameters — without labels, without
context, without interpretation.
Only after the data has been reviewed does the team begin
constructing explanations. This sequence — data first, explanation
second — reduces the likelihood that the explanation will distort the
data.
Systematic Search
for Disconfirming Evidence
Build into every investigation an explicit step: “What evidence would
prove us wrong, and have we looked for it?” This is the single most
powerful antidote to confirmation bias. It forces the team to actively
seek the data that would challenge their conclusion, rather than
passively accepting the data that supports it.
If the team concludes that the root cause is operator error, they
must explicitly look for evidence that it was NOT operator error — and
document what they found. If they conclude the process is capable, they
must explicitly look for evidence that it is NOT capable.
This step is uncomfortable. It feels like undermining the team’s own
work. But that discomfort is precisely the signal that the defense is
working.
The Leadership Challenge
Ultimately, confirmation bias in quality is a leadership challenge.
Leaders set the tone for how data is received, how investigations are
conducted, and how dissenting views are treated.
If leaders reward teams for confirming expectations, confirmation
bias will flourish. If leaders punish teams for bringing bad news,
confirmation bias will flourish. If leaders demand quick answers and
simple narratives, confirmation bias will flourish.
The alternative is a culture of disciplined inquiry — one that values
accuracy over comfort, evidence over narrative, and truth over
consensus. This culture is built one decision at a time: how you respond
to a control chart that contradicts your belief, how you treat the
engineer who challenges the team’s conclusion, how you handle the audit
finding that exposes a blind spot.
Confirmation bias will always be present. It is a feature of human
cognition, not a bug. The question is whether your quality system is
designed to counteract it — or to amplify it.
A Final Observation
The most dangerous form of confirmation bias in manufacturing quality
is the belief that your organization does not have it. Every
organization has it. The ones that manage it well are the ones that
acknowledge it, build defenses against it, and remain perpetually
suspicious of conclusions that feel too comfortable.
If your data always tells you what you expect to hear, your data is
not telling you the truth. The truth is almost always more complicated,
more uncomfortable, and more useful than what confirmation bias allows
you to see.
The defects you miss will not be the ones you looked for and failed
to find. They will be the ones you never thought to look for — because
you already knew they couldn’t be there.
Peter Stasko is a Quality Architect with over 25
years of experience in manufacturing quality management, process
improvement, and quality system design. He specializes in bridging the
gap between theory and practice — making quality tools work in the real
world, not just in the documentation.
