Most manufacturing plants conduct Gage R&R because their customers demand it. They run bias studies because the math is simple. But gage linearity remains the most frequently ignored characteristic in the AIAG MSA manual. It falls between the cracks because it requires specialised reference standards and produces a regression line rather than a single pass-or-fail percentage.
I have audited plants where a micrometer passed calibration with excellent repeatability, yet systematically measured 3 micrometres high at the bottom of its range. On a component with a ±10 micrometre tolerance, that error is the difference between a functional assembly and a field failure. A single-point calibration cannot catch this drift. Only a linearity study maps how bias changes across the entire operating range.
Gage linearity is the study of how the systematic error of a measurement system changes in relation to the magnitude of the measured value. It exposes a critical flaw in conventional thinking: the assumption that an instrument calibrated accurately at one point is equally accurate everywhere. In reality, mechanical wear, thread pitch errors, and electronic drift create uneven bias distributions that distort measurements at the extremes of the scale.
The Mechanics of Linearity Failure
Consider a transmission plant manufacturing precision shafts with a nominal diameter of 12.000 mm and a tolerance of ±0.010 mm. The inspection plan uses a 0–25 mm digital micrometer. The Gage R&R study yields excellent results under 5% study variation. Calibration records are current and clean.
The problem emerges when operators measure parts across the lower and upper specification limits. A linearity study reveals that at 10 mm, the micrometer systematically adds 2–3 micrometres. At 14 mm, it subtracts 1–2 micrometres. The instrument is perfectly accurate only at the midpoint of its range.
This slope creates a dangerous blind spot. Parts measured at the lower end are actually closer to the lower specification limit than the display indicates. Parts at the higher end are larger than recorded. The plant unknowingly accepts non-conforming parts because the measurement error systematically masks the true process variation at the edges of the tolerance band.
Executing the AIAG Linearity Study
The AIAG MSA manual outlines a strict methodology for quantifying this error. You cannot rely on a single master standard. You must select a minimum of five calibrated reference values, distributed evenly across the operating range of the instrument. For a 0–25 mm micrometer, standards at 2, 7, 12, 17, and 22 mm provide the necessary spread.
Each reference standard must possess a calibration uncertainty at least ten times tighter than the measured feature's tolerance. A single operator measures each standard repeatedly, typically 10 to 12 times per standard, in randomised order. Randomisation eliminates operator memory effects and prevents the systematic biases that sequential measuring introduces.
You calculate the bias for each reference standard by subtracting the reference value from the mean of the repeated measurements. Plotting these bias values on the y-axis against the reference values on the x-axis generates a scatter plot. A best-fit linear regression line through these points reveals the instrument's linearity.
AIAG MSA Linearity Study Procedure
- 01Select 5+ Reference StandardsDistribute evenly across the instrument's operating range with calibration uncertainty 10x tighter than tolerance.
- 02Conduct Randomised TrialsOne operator measures each standard 10–12 times in randomised order to eliminate memory effects.
- 03Calculate Point BiasSubtract the reference value from the mean of repeated measurements for each standard.
- 04Plot Regression LineGraph bias against reference values. The slope of the best-fit line indicates the magnitude of linearity error.
- 05Evaluate % LinearityCompare the slope against the process range and tolerance. Results exceeding 5–10% require system correction.
Industry-Specific Failure Modes
Linearity failures manifest differently depending on the gauge type and application. In automotive transmission plants, technicians often measure gear backlash using a dial indicator across a range of 0.020 mm to 0.150 mm. A Gage R&R result under 10% looks acceptable, but a linearity study frequently reveals the indicator over-measures by 30% at values below 0.050 mm.
This systematic error causes engineers to set regulatory valve clearance too loosely, resulting in elevated transmission noise and customer complaints. In foundries, Brinell hardness testers covering 80 to 250 HBW often show optimistic bias above 200 HBW. The result is softer, non-conforming castings passing inspection and entering the supply chain.
Medical device manufacturers face similar risks when measuring implant rods with tolerances of ±5 micrometres. Laser micrometres can develop waviness in their linearity, causing the bias to oscillate depending on where the part sits within the measurement range. Without mapping this behaviour, manufacturers cannot trust their own inspection data.
Calculating and Interpreting the Results
The slope of the regression line quantifies the problem. To determine severity, calculate % Linearity using the formula: (absolute value of the slope multiplied by the process range) divided by the tolerance, expressed as a percentage. A result exceeding 5–10% renders the measurement system unfit for that application.
Standard MSA Linearity Thresholds
This calculation forces a hard decision. If the linearity fails, a single-point calibration offset will not save you. Applying a constant correction factor based on the midpoint simply shifts the entire regression line, potentially worsening the error at the extremes while creating a false sense of security at the nominal value.
A single-point calibration cannot catch drift. Only a linearity study maps how bias behaves across the entire operating range.
Software like Minitab automates the regression analysis and visualisation, but the interpretation requires human judgement. You must understand whether the failure stems from mechanical wear, an inherent design limitation of the gauge, or an environmental factor affecting the sensor. The software provides the mathematics; the engineer provides the diagnosis.
Corrective Actions for Failed Linearity
When a study reveals unacceptable linearity, you have four immediate options. The most direct is repairing or replacing the instrument. Mechanical micrometres often fail linearity due to worn guides, bent spindles, or degraded threads. A thorough mechanical rebuild can restore the geometric integrity required for linear measurements.
If the instrument performs adequately in a specific band, restrict its qualified operating range. A 0–25 mm micrometer with poor low-end linearity can be officially limited to an 8–18 mm range within your control plan. This removes the non-linear regions from your quality system without requiring capital expenditure.
Modern digital instruments and coordinate measuring machines (CMMs) support software compensation. You can programme the system to apply a mathematical correction based on the known bias curve. However, this compensation must be validated periodically, as the underlying mechanical or electronic drift will continue to progress over time.
Audit Requirements and the Cost of Neglect
In the automotive sector, IATF 16949 and the German VDA 5 standard explicitly require linearity studies as part of the measurement system analysis process. Ford, GM, and Stellantis customer-specific requirements dictate exactly which instruments need this validation. ISO 9001 does not explicitly demand it, but any Tier 1 supplier operating under IATF rules faces immediate non-conformance findings during third-party audits if linearity data is missing.
The financial impact of ignoring linearity is measurable. I have reviewed cases where plants discovered that 2% of their monthly production—roughly 1,200 parts—shipped with latent defects because the measurement system masked the non-conformance at the edge of the tolerance band. Warranty claims and customer returns traced back to that single unchecked measurement error exceeded €380,000 over three years.
A linearity study takes two days and costs approximately €500 in calibration and labour. The return on that investment is immediate. Once you map the bias across the entire operating range, you eliminate the blind spot. Your process capability indices reflect reality, your inspection data becomes trustworthy, and your customer receives parts that genuinely conform to specification.