Quality
and Confirmation Bias: When Your Organization Only Sees What It Already
Believes — and the Evidence That Should Have Changed Your Mind Gets
Filtered Out Before It Reaches the Decision That Matters
The
Investigation That Was Over Before It Started
In 2019, a medical device manufacturer in Bavaria received a
complaint that would eventually cost them €14 million in recalls,
regulatory penalties, and lost contracts. A batch of insulin delivery
pumps had been delivering inconsistent dosages — sometimes 15% above the
prescribed amount, sometimes 10% below. For diabetic patients relying on
precise insulin delivery, the consequences ranged from uncomfortable to
life-threatening.
The quality team launched an investigation with commendable speed.
They pulled the batch records, reviewed the process parameters, and
conducted a thorough analysis of the manufacturing data. Within two
weeks, they had their answer: a malfunctioning check valve in the
filling station had caused pressure fluctuations during the dosing
cycle. The valve was replaced, the process was revalidated, and the case
was closed.
Except the case should not have been closed. Because six months
later, the same defect pattern appeared in a different product line —
one that used a completely different filling station with a brand-new
check valve. And then it appeared again in a third line. By the time the
real root cause was identified, the company had spent millions chasing a
ghost.
The actual cause? A software algorithm in the central dosing
controller that had been incorrectly calibrated during a routine
firmware update. The algorithm was applying temperature compensation
using the wrong coefficient, causing systematic dosage drift whenever
ambient temperature fluctuated.
The quality team had found the check valve explanation first. It fit
their existing mental model — mechanical components fail, valves wear
out, pressure fluctuations cause dosing errors. It was a story they
understood, a failure mode they had seen before, and a fix they knew how
to implement. And because it felt right, they stopped looking.
They didn’t ignore contradictory evidence. They simply never noticed
it. The data points that would have pointed to a software issue — the
correlation between defect severity and shift timing (which coincided
with temperature cycles), the identical error pattern across different
mechanical configurations, the absence of physical wear on the “failed”
valves — were all present in the investigation file. But the team’s
brains had already categorized them as irrelevant noise the moment the
check valve narrative took hold.
This is confirmation bias. And it is not a character flaw. It is a
feature of human cognition that your quality system must be specifically
designed to counteract.
What Confirmation Bias
Actually Is
Confirmation bias is the tendency to search for, interpret, favor,
and recall information in a way that confirms or supports one’s prior
beliefs or values. It is not simply “believing what you want to
believe.” It is a systematic distortion of the entire
information-processing pipeline — from what you choose to look at, to
how you interpret what you see, to what you remember afterward.
Psychologist Peter Wason first demonstrated this effect in 1960
through a deceptively simple experiment. Participants were given a
number sequence (2, 4, 6) and asked to determine the rule that generated
it. They could test their guesses by proposing additional sequences and
receiving feedback on whether they matched the rule. Most participants
formed a hypothesis quickly — “increasing even numbers” — and then only
tested sequences that confirmed their guess. They rarely tried sequences
that would disprove their hypothesis, such as odd numbers or
decreasing sequences. The actual rule was far simpler: any three
ascending numbers.
Wason showed that people don’t test ideas by trying to prove
themselves wrong. They test ideas by trying to prove themselves right.
And in a quality investigation, where the stakes are measured in
customer safety and organizational survival, this cognitive default is
catastrophically dangerous.
Confirmation bias operates through four distinct mechanisms:
1. Biased Search: You selectively gather information
that supports your hypothesis while ignoring or underweighting
information that contradicts it. In the insulin pump case, the team
extensively documented the valve’s wear pattern but didn’t run the
simple statistical test that would have shown zero correlation between
valve condition and defect severity.
2. Biased Interpretation: Ambiguous evidence is
interpreted as supportive of your existing belief. A slight pressure
variation that was within normal tolerances became “proof” of valve
failure because the team was already looking for a mechanical cause.
3. Biased Memory: You more easily remember
information that confirms your beliefs and forget or distort information
that contradicts them. After the investigation, team members genuinely
recalled the pressure data as being more abnormal than the records
showed.
4. Biased Questioning: The questions you ask are
structured to produce confirming answers. “Did the valve show signs of
wear?” produces a different answer than “What evidence would prove the
valve was NOT the cause?”
Why Quality
Organizations Are Uniquely Vulnerable
Every human brain is susceptible to confirmation bias. But quality
organizations have structural characteristics that amplify the effect to
dangerous levels.
Expertise as a Trap
The more experienced your quality engineers, the more vulnerable they
are to confirmation bias. This sounds counterintuitive — isn’t expertise
supposed to improve judgment? In many domains, it does. But in root
cause analysis, expertise creates a vast library of past failures that
the brain uses as pattern-matching templates.
When an experienced engineer encounters a new defect, their brain
automatically suggests the most similar past case. This initial
hypothesis then becomes the lens through which all new evidence is
filtered. The expert doesn’t realize this is happening because the
pattern match feels like intuition — and in a culture that venerates
experience, nobody questions the senior engineer’s gut feeling.
A study published in the Journal of Applied Research in Memory and
Cognition found that forensic analysts with more experience were
actually more likely to exhibit confirmation bias than novices,
because their larger repertoire of past cases gave them more ready-made
hypotheses to latch onto. The same dynamic plays out in quality
investigations every day.
The Pressure to Close
Quickly
Quality investigations operate under intense time pressure. Customers
are waiting. Production lines are stopped. Regulatory deadlines are
ticking. Managers want answers by Friday. In this environment, the first
hypothesis that fits the available evidence feels like a gift — and the
instinct to keep investigating feels like a luxury you can’t afford.
Confirmation bias thrives under time pressure. When you need an
answer quickly, your brain doesn’t conduct a thorough, dispassionate
analysis. It grabs the first plausible explanation and starts building a
case for it. The faster you need to decide, the more likely you are to
settle on the first story that makes sense.
Organizational Silence
Structures
Many organizations have an unspoken rule: once the quality team has
identified a root cause and implemented a corrective action, questioning
that conclusion is seen as undermining the team’s competence. This
creates an environment where the only socially acceptable response to a
completed investigation is agreement.
I’ve seen this dynamic play out in organizations where engineers
privately doubted an investigation’s conclusions but said nothing
because “the team had already made their decision.” In one automotive
supplier, three different engineers independently suspected that a
reported root cause was wrong but none of them spoke up because the
quality manager had already presented the findings to the customer. By
the time the real cause emerged eight months later, the customer had
already found an alternative supplier.
The Dashboard Illusion
Modern quality systems generate enormous volumes of data, and the
assumption is that more data leads to better decisions. But confirmation
bias doesn’t care how much data you have — it cares about what you do
with it. A dashboard showing 200 process parameters can be just as
misleading as a single data point if the person reading it is already
convinced they know the answer.
The dashboard becomes a Rorschach test. Two engineers looking at the
same screen will see different things depending on their hypotheses. The
one who believes it’s a material issue will notice the batch-to-batch
variation in raw material specifications. The one who believes it’s an
operator issue will notice the correlation with shift changes. Both
patterns are real. Neither is the root cause. But each engineer will
feel that the data “clearly supports” their interpretation.
The Architecture of
a Biased Investigation
To understand how confirmation bias infiltrates a quality
investigation, let’s walk through a typical 8D process and examine where
the distortion occurs.
D1 — Team Formation: You assemble a team based on
who you think will be relevant. If you suspect a mechanical failure, you
bring in mechanical engineers. If you suspect a software issue, you
bring in developers. The team composition itself reflects the initial
hypothesis, which means you’ve already constrained the investigation
before it begins.
D2 — Problem Description: The way you describe the
problem shapes every subsequent step. “Dosing inconsistency caused by
pressure fluctuations” is a very different starting point than “Dosing
inconsistency of unknown origin.” The first formulation embeds a
hypothesis in the problem statement. The second leaves the investigation
open. Most organizations use the first formulation without even
realizing it.
D3 — Interim Containment: You implement containment
actions based on your initial understanding. These actions are often
expensive and disruptive, which creates a psychological investment in
the hypothesis that justified them. Once you’ve shut down a production
line and quarantined $500,000 worth of inventory based on a theory,
you’re going to be reluctant to admit that theory might be wrong.
D4 — Root Cause Analysis: This is where confirmation
bias does its most systematic damage. The team uses tools like fishbone
diagrams and 5-Why analysis — tools that are supposed to be objective
and structured. But the fishbone categories (Man, Machine, Material,
Method, Measurement, Environment) are populated based on what the team
thinks is relevant. And the 5-Why chain follows the path that seems most
logical to the investigator, which means it follows the path that
confirms their hypothesis.
I once watched a team conduct a 5-Why analysis that beautifully
demonstrated this principle. Their chain went: Why did the part fail?
Because the hardness was out of specification. Why was the hardness out
of specification? Because the heat treatment temperature was too low.
Why was the temperature too low? Because the thermocouple was drifting.
Why was it drifting? Because it hadn’t been calibrated on schedule. Root
cause: missed calibration.
It was elegant. It was logical. It was completely wrong. The real
cause was a material substitution by the supplier that changed the alloy
composition, making the standard heat treatment parameters inadequate.
The thermocouple was actually fine — it was reading the correct
temperature. But the team’s 5-Why chain had led them inexorably to a
calibration failure because that’s where their hypothesis was
pointing.
D5–D8 — Corrective Actions and Verification: Once a
root cause has been identified, the corrective actions are designed to
address it. Verification then tests whether those actions were
effective. But the verification is often structured to confirm success
rather than to challenge the root cause conclusion. If the defect rate
drops after implementing the corrective action, that’s taken as proof
that the root cause was correct — even if the defect rate dropped for
unrelated reasons (seasonal variation, production volume changes, or
simple regression to the mean).
How to
Build an Anti-Confirmation-Bias Quality System
You cannot eliminate confirmation bias through willpower, training,
or exhortation. It is a structural feature of human cognition. The only
way to manage it is through structural countermeasures built into your
processes.
1. Mandate Disconfirming
Evidence
Every root cause investigation should include a formal requirement to
actively seek evidence that would disprove the leading hypothesis. This
means creating a specific section in the investigation template — not as
an afterthought, but as a required deliverable.
The template should ask: “What evidence would prove this root cause
is WRONG?” and “What specific tests or analyses have been conducted to
check for this disconfirming evidence?” If this section is blank, the
investigation is incomplete.
In the insulin pump case, this simple discipline would have forced
the team to ask: “If the check valve were NOT the cause, what would we
expect to see?” The answer — “We would see the same defect pattern
across different filling stations using different valves” — was exactly
what the data showed. But nobody asked the question.
2. Implement Red Team /
Blue Team Reviews
Borrowing from military and cybersecurity practices, formalize a
process where a separate team (the Red Team) is specifically tasked with
challenging the investigation’s conclusions. The Red Team doesn’t
conduct its own investigation — it systematically attacks the logic,
evidence, and reasoning of the original team.
This is not adversarial for the sake of being adversarial. The Red
Team’s job is to ask the questions that the original team didn’t think
to ask, precisely because confirmation bias made those questions
invisible to them.
Effective Red Team reviews include: – “What alternative explanations
could account for the same evidence?” – “What evidence would you need to
see to rule out each alternative?” – “If you had to argue that this root
cause is wrong, what’s your strongest argument?” – “What data points did
you collect but not include in the report, and why?”
3. Hypothesis Blind Analysis
Before sharing findings with the team, have an independent analyst
examine the raw data without being told the working hypothesis. This
analyst should be asked: “What patterns do you see in this data, and
what conclusions would you draw?”
If the independent analyst reaches a different conclusion than the
investigation team, that discrepancy deserves serious attention. It
doesn’t mean the team is wrong — but it means there’s an alternative
interpretation that the team’s hypothesis may have blinded them to.
4. Formalize the Pre-Mortem
Before finalizing a root cause conclusion, conduct a formal
pre-mortem. Gather the team and ask: “Imagine it’s one year from now,
and we’ve just discovered that our root cause conclusion was completely
wrong. What was the actual cause?”
This thought experiment forces the brain out of confirmation mode and
into imagination mode. It liberates team members to consider
alternatives that felt like doubt or disloyalty when the investigation
was “live.” The pre-mortem often surfaces hypotheses that were present
in the data all along but were suppressed by the confirmation bias
surrounding the leading theory.
5. Separate Investigation
from Solution
Many organizations combine root cause analysis and corrective action
design into a single process. This is efficient but dangerous. When the
same team that identified the root cause also designs the corrective
action, they have a psychological investment in both being correct. The
corrective action becomes evidence for the root cause, and the root
cause justifies the corrective action, creating a circular logic that
feels airtight but may be entirely wrong.
Separate these phases. Have the root cause analysis reviewed and
approved by an independent party before the corrective action team
begins its work. This creates a natural checkpoint where confirmation
bias can be interrupted.
6. Track Prediction Accuracy
Most organizations track whether corrective actions were effective.
Almost none track whether their root cause conclusions were accurate
over time. This is a critical omission.
Implement a system where, six months after an investigation is
closed, you revisit the original defect pattern and assess whether the
root cause conclusion has held up. Did the defect truly not recur? Did
it recur in a way that suggests the original root cause was wrong? Did
the corrective action address the symptom rather than the cause?
This longitudinal tracking creates organizational learning about
where confirmation bias most frequently distorts your investigations,
which allows you to target your countermeasures more effectively.
The Leadership Role
None of these structural countermeasures will work in a culture where
changing your mind is seen as weakness. Leaders must model intellectual
humility and make it explicit that revising a conclusion in the face of
new evidence is not a sign of failure — it is the definition of good
quality practice.
This is harder than it sounds. In a manufacturing environment,
decisiveness is rewarded and vacillation is penalized. A quality manager
who says “I was wrong about the root cause” is perceived differently
than one who says “I’ve identified the root cause,” even though the
first statement may represent better engineering judgment.
Leaders must deliberately create space for revision. When presenting
investigation findings to customers or regulators, include the
alternative hypotheses that were considered and the evidence that was
used to rule them out. This demonstrates thoroughness rather than doubt,
and it creates a record that protects the organization if the original
conclusion turns out to be wrong.
The Cost of Not Acting
The Bavarian medical device manufacturer eventually identified the
real root cause of their dosing inconsistency — but not before the
recall expanded to three additional product lines, the regulatory
authority issued a formal warning, and two major hospital networks
switched to a competitor. The total cost exceeded €14 million. The
original investigation had cost €85,000. A thorough investigation that
included disconfirming evidence analysis and an independent review would
have cost perhaps €120,000 — less than 1% of the eventual loss.
But the financial cost, while staggering, was not the worst
consequence. Three patients experienced hypoglycemic episodes severe
enough to require emergency medical attention. All three recovered. But
the margin between the dosing error they experienced and a fatal
overdose was uncomfortably thin.
Confirmation bias in quality is not an academic curiosity. It is a
patient safety issue. It is an aircraft safety issue. It is an
automotive safety issue. Every time a quality investigation settles on
the first plausible explanation without rigorously testing alternatives,
the organization is gambling that its initial guess was correct — and
the stakes of that gamble are measured in human lives.
A Practical
Framework: The Disconfirmation Protocol
For organizations ready to address confirmation bias systematically,
here is a practical protocol that can be integrated into existing
investigation processes:
Step 1 — Before the Investigation Begins: Document
the initial hypothesis and rate your confidence in it on a scale of
1-10. Then explicitly ask: “What would make me reduce this confidence to
below 5?” Write down the specific evidence that would disconfirm your
hypothesis.
Step 2 — During Evidence Collection: For every piece
of evidence you collect, classify it as supporting, contradicting, or
neutral relative to your hypothesis. Track the ratio. If 90% of your
evidence is supporting, you’re probably not looking hard enough for
contradictions.
Step 3 — After Initial Analysis: Generate at least
three alternative hypotheses that could explain the same evidence. For
each alternative, identify what additional evidence would distinguish it
from your leading hypothesis. Then go collect that evidence.
Step 4 — Before Finalizing: Conduct a formal
pre-mortem. Have someone who wasn’t involved in the investigation review
your findings and challenge your reasoning. Specifically ask: “If you
had to convince me that I’m wrong, what argument would you use?”
Step 5 — After Closure: Schedule a six-month review
to assess whether the root cause conclusion has held up. Document
lessons learned about the investigation process itself, not just the
defect.
This protocol adds approximately 20% time to a typical investigation.
It saves approximately 80% of the cost of investigations that reach the
wrong conclusion.
The Uncomfortable Truth
The uncomfortable truth about confirmation bias is that knowing about
it doesn’t make you immune to it. The quality engineers at the Bavarian
device company had all received training on cognitive biases. They could
define confirmation bias, identify it in case studies, and pass a
written test on it with flying colors. And then they went ahead and
exhibited it in their own investigation, because that’s how cognitive
biases work — they operate below the level of conscious awareness, and
they are especially powerful when you’re confident that you’re thinking
clearly.
The only defense is structural. Build the countermeasures into your
processes, enforce them rigorously, and accept that the extra time they
require is not overhead — it is the price of actually being right
instead of merely feeling right.
Your customers — and in some industries, your patients — are
depending on the difference.
Peter Stasko is a Quality Architect with 25+ years
of experience transforming organizations across automotive, aerospace,
and pharmaceutical industries. He has led quality system implementations
on three continents and specializes in bridging the gap between
behavioral science and operational excellence. His approach combines
deep technical expertise with an understanding of the human factors that
determine whether quality systems succeed or become expensive
decorations.
