Quality
MSA: When Your Organization Discovers That the Numbers It Trusted Were
Never Real — and the Measurement System Everyone Assumed Was Accurate
Became the Hidden Source of Every Bad Decision You Ever Made
The Day the Data Died
It was a Tuesday morning when the quality director at a Tier 1
automotive supplier received the phone call that unraveled three years
of process improvement work. The customer’s incoming inspection had
rejected an entire shipment — 12,000 fuel injector housings — for
dimensional nonconformance. The supplier’s own final inspection had
passed every single one of them.
The quality director pulled the records. Their CMM had reported every
part within specification. Their SPC charts showed a beautifully
centered, capable process. Their operators had followed every procedure.
Their quality system had functioned exactly as designed.
And yet, 12,000 parts were wrong.
The investigation took six weeks. It revealed that the CMM’s probe
had been damaged during a routine maintenance event fourteen months
earlier. The damaged probe introduced a systematic bias of 0.012mm —
small enough to be invisible on a control chart that was already
monitoring a process with natural variation of 0.045mm, but large enough
to push parts past the customer’s tolerance when measured with their
properly calibrated equipment.
Three years of SPC data. Hundreds of capability studies. Dozens of
process improvement decisions. All of them based on measurements that
were quietly, consistently wrong.
This is the nightmare that Measurement Systems Analysis exists to
prevent. And the terrifying truth is that most organizations have no
idea whether their measurement systems are telling them the truth.
What
MSA Actually Is — and Why Most People Misunderstand It
Measurement Systems Analysis is the discipline of evaluating whether
your measurement process is capable of producing data that is accurate
enough, precise enough, and stable enough to support the decisions
you’re making with it.
Most quality professionals think they understand MSA. They’ve done a
Gage R&R study. They’ve filled out the spreadsheets. They’ve
reported the percentages to their auditor. They filed the results and
moved on.
But MSA is not a checkbox. It is not a form you fill out to satisfy
IATF 16949 clause 7.1.5.1.1. It is the foundation upon which every
single quality decision rests. And when that foundation is cracked,
everything built on top of it — your SPC, your capability indices, your
process improvements, your customer approvals — is built on sand.
Think of it this way: every number on every control chart, every Cpk
value in every capability study, every pass/fail decision at every
inspection station — all of it is filtered through a measurement system.
If that measurement system is producing noise, bias, or inconsistency,
then every decision derived from that data is contaminated.
MSA asks the question that most organizations never think to ask:
Is our measurement system good enough to tell us what’s actually
happening?
The
Five Characteristics Your Measurement System Must Have
MSA evaluates a measurement system across five critical dimensions.
Understanding these is not academic — it is the difference between
trusting your data and being betrayed by it.
1. Bias
Bias is the systematic difference between your measurement and the
true value. It’s like a scale that always reads two pounds heavy. The
measurements are consistent, but they’re consistently wrong.
Bias creeps into measurement systems through improper calibration,
worn fixtures, environmental changes, and operator technique
differences. The dangerous thing about bias is that it’s invisible
unless you specifically test for it. Your control charts will still look
normal. Your process will appear capable. Your operators will feel
confident. And every number will be shifted from reality by an amount
you never measured.
2. Linearity
Linearity tells you whether your measurement system’s accuracy
changes across its range. A micrometer might be perfectly accurate at
25mm but increasingly biased at 50mm. A pressure gauge might read
correctly at 100 PSI but drift at 300 PSI.
This matters enormously in practice. If you’re using the same
instrument to measure parts across a range of dimensions, and you’ve
only validated it at one point in that range, you may be making
decisions with data that degrades the further you get from your
calibration point.
3. Stability
Stability asks whether your measurement system produces consistent
results over time. A CMM that was accurate last year but has drifted due
to environmental changes, wear, or aging components will give you
measurements that are slowly, silently diverging from reality.
Most calibration systems catch gross stability failures. But gradual
drift — the kind that introduces 0.012mm of bias over fourteen months —
often slips through. Your calibration sticker says you’re fine. Your
data says you’re fine. And your customer’s incoming inspection says
you’re not.
4. Repeatability
Repeatability is the variation you get when the same operator
measures the same part multiple times with the same instrument. It’s the
measurement system’s inherent noise — the random scatter that exists
even under ideal conditions.
Low repeatability means your measurement system cannot reliably
distinguish between parts that are close in value. If your repeatability
variation is larger than the tolerance you’re trying to control, your
inspection is essentially a coin flip wearing a lab coat.
5. Reproducibility
Reproducibility is the variation introduced when different operators
measure the same parts with the same instrument. It captures differences
in technique, interpretation, training, and even motivation.
This is where organizational psychology meets metrology. Two
operators can follow the same written procedure, use the same gage,
measure the same parts, and produce systematically different results —
because one of them applies slightly more force when seating the part,
or reads the dial from a slightly different angle, or interprets the
boundary condition differently.
The Gage R&R
Study: What It Actually Tells You
The most common MSA tool is the Gage Repeatability and
Reproducibility study. But most organizations execute it so poorly that
the results are meaningless.
A proper Gage R&R study requires careful design:
Sample Selection: The parts you use must represent
the actual production variation. Using only good parts, or only parts
from a single batch, will understate the true measurement variation. You
need at least 10 parts that span the expected process range.
Operator Selection: Use the actual operators who
perform the measurement in production — not engineers, not supervisors,
not the quality manager who happens to be available on a Tuesday. The
study must capture real-world operator variation, not idealized
variation.
Randomization: The order in which operators measure
parts must be randomized. If Operator A always measures the parts in the
same sequence, fatigue effects and learning effects will contaminate the
results.
Blinding: Operators should not know what results
others have obtained. If Operator B sees that Operator A measured a part
at 25.03mm, Operator B’s measurement will unconsciously anchor toward
that value.
Replication: Each operator must measure each part at
least twice (typically three times) to separate repeatability from
reproducibility.
The results of a Gage R&R study are typically expressed as
percentages of the tolerance or the process variation:
- Under 10%: The measurement system is
acceptable. - 10-30%: The measurement system is marginal — it may
be acceptable depending on the application, but it deserves
investigation. - Over 30%: The measurement system is unacceptable.
It is producing more noise than signal, and any decisions based on its
data are suspect.
Here’s the uncomfortable reality: in my experience auditing and
consulting across dozens of organizations, a significant percentage of
measurement systems in routine use fall into the 20-40% range when
properly studied. Organizations are making pass/fail decisions,
calculating capability indices, and adjusting processes based on
measurement systems that are generating more variation than the
processes they’re monitoring.
The Attribute Measurement
System Trap
Most MSA attention focuses on variable measurements — dimensions,
weights, pressures. But attribute measurement systems — go/no-go gages,
visual inspections, subjective assessments — are where the really
expensive failures hide.
Attribute MSA studies (often called Attribute Agreement Analysis)
evaluate whether inspectors consistently make the same pass/fail
decisions on the same parts. The methodology is straightforward: present
a set of known reference parts (some good, some borderline, some clearly
defective) to multiple inspectors multiple times, and measure
agreement.
The results are routinely shocking. I’ve seen visual inspection
systems where the agreement rate between inspectors was below 50%. I’ve
seen organizations where the same inspector made different decisions on
the same part 30% of the time. I’ve seen incoming inspection departments
that were rejecting good parts and accepting bad ones at rates that made
their existence a net negative for quality.
Attribute measurement failures are particularly dangerous because
they’re invisible in the data. Your defect tracking system records what
the inspector decided, not what was actually there. If your inspectors
are inconsistent, your defect data is contaminated, your Pareto charts
are wrong, and your improvement priorities are based on inspection noise
rather than process reality.
The Hidden Costs of Poor
Measurement
When your measurement system is inadequate, the costs cascade through
your organization in ways that rarely show up on any cost report:
False Rejects: You’re scrapping or reworking parts
that were actually within specification. This is the most visible cost —
and often the one that triggers the investigation. One automotive
supplier I worked with discovered that their final inspection was
rejecting 8% of production due to measurement system noise. The actual
defect rate was 1.2%. They were throwing away 6.8% of their output —
millions of dollars per year — because their measurement system couldn’t
reliably distinguish good parts from bad ones.
False Accepts: You’re shipping parts that are
outside specification. This is the cost that usually doesn’t appear
until the customer finds it. And when they do, the cost isn’t just the
returned parts — it’s the containment, the sort, the line shutdown, the
warranty claims, the lost trust, and the competitive disadvantage that
follows.
Process Tampering: When your measurement system
produces noisy data, your SPC charts generate false signals. Operators
chase assignable causes that don’t exist. Engineers adjust processes
that were running fine. The adjustments introduce real variation that
wasn’t there before, and the process gets worse — not better — as a
direct result of trying to improve it based on bad data. This is
Deming’s funnel experiment playing out in real factories, every single
day.
Misallocated Resources: Your continuous improvement
team spends months optimizing a process that wasn’t the problem. Your
capital budget buys new equipment to fix a capability gap that was
actually a measurement gap. Your training program re-educates operators
on a process that was performing exactly as designed — it was the
measurement system that couldn’t see it.
Destroyed Credibility: When operators know the gage
is unreliable, they develop workarounds. They re-measure parts until
they get a passing result. They apply “operator judgment” instead of
following the procedure. They stop trusting the quality system, and once
that trust is gone, no amount of calibration can restore it.
MSA in Practice: A
Practical Framework
Implementing effective MSA requires more than occasional studies. It
requires a systematic approach embedded in your quality management
system.
Phase 1: Inventory and
Prioritize
List every measurement system in your organization. For each one,
assess the risk: What decisions does this measurement support? What is
the consequence of a wrong measurement? How critical is the
characteristic being measured?
Prioritize your MSA efforts based on risk. A final dimensional
inspection on a safety-critical part deserves far more attention than an
in-process check on a non-critical aesthetic feature.
Phase 2: Establish Baselines
Conduct proper MSA studies on your prioritized measurement systems.
Use the AIAG MSA manual methodology or equivalent. Document the results
honestly — even when they’re uncomfortable.
Phase 3: Fix What’s Broken
When a study reveals an inadequate measurement system, address it
immediately. Common fixes include:
- Improved fixtures that reduce part seating
variation - Better operator training with standardized
technique - Instrument upgrades when the fundamental capability
is insufficient - Environmental controls when temperature, humidity,
or vibration affect results - Measurement strategy changes — moving from a noisy
measurement to a more capable one
Phase 4: Monitor Continuously
MSA is not a one-time event. Measurement systems degrade over time.
Establish a schedule for ongoing monitoring — periodic linearity checks,
stability studies, and abbreviated Gage R&R studies to confirm that
your measurement systems remain adequate as conditions change.
Phase 5:
Design Measurement Systems Into New Processes
The most cost-effective time to address measurement capability is
during process design. When APQP is done properly, measurement systems
are evaluated during the planning phase, before production begins.
Measurement strategy is not an afterthought — it is an integral part of
process design.
The Connection to Everything
MSA is not an isolated tool. It is the lens through which every other
quality tool operates. Understanding this connection transforms how you
think about measurement:
SPC depends on MSA. Control charts are only as
reliable as the data feeding them. If your measurement system variation
exceeds the control limit spread, your charts are monitoring measurement
noise, not process behavior.
Process capability depends on MSA. Your Cpk value is
a ratio of specification width to process spread. If measurement
variation is inflating your observed process spread, your Cpk is
artificially low. If measurement bias is shifting your observed process
mean, your Cpk may be misleading you about both centering and
capability.
FMEA depends on MSA. When you assess detection
capability in your FMEA, you’re assessing your measurement system’s
ability to detect failures. A poor measurement system means poor
detection, which means higher risk ratings — which should trigger
different design and control strategies.
PPAP depends on MSA. Every production part approval
is based on measurement data. If your measurement system is inadequate,
your PPAP submission is a work of fiction — however
well-intentioned.
Customer satisfaction depends on MSA. Your
customer’s experience is determined by whether the parts they receive
conform to their requirements. If your measurement system cannot
reliably assess conformance, you are flying blind.
The Leadership Challenge
MSA is one of the most under-resourced areas in quality management.
The reasons are understandable: it’s technical, it’s unglamorous, and
its value is invisible until something goes wrong. No executive ever
toured a plant and said, “Show me your measurement system analysis
program.”
But leaders who understand systems know that the quality of decisions
is bounded by the quality of information. You can have the best people,
the best processes, and the best intentions — but if your measurement
system is feeding you distorted data, your best people will make the
wrong decisions, your best processes will appear to be failing, and your
best intentions will produce the worst outcomes.
Investing in measurement capability is not a cost. It is the
prerequisite for every other quality investment you will ever make. It
is the foundation. And foundations, by definition, are the last thing
you want to discover is cracked.
The organizations that take MSA seriously are the ones that make
decisions with confidence. The ones that don’t are the ones that are
always surprised — by customer rejections, by process behavior they
can’t explain, by capability indices that don’t match reality.
Your measurement system is the interface between your process and
your understanding of it. Make sure it’s telling you the truth.
Peter Stasko is a Quality Architect with 25+ years
of experience transforming organizations across automotive, aerospace,
and pharmaceutical industries. He has led measurement system improvement
programs that have recovered millions in false-reject costs and
prevented customer-facing quality failures across three continents. His
approach to MSA treats measurement capability not as a compliance
requirement, but as the foundation of every quality decision an
organization will ever make.
