A tier-1 automotive supplier receives a warning letter: burrs on a machined hole exceed the tolerance limit. The default reaction on the shop floor is to walk to the machine and start twisting dials. The engineer increases the cutting speed, reduces the feed rate, changes the tool, and adjusts the coolant. Every shift tries a different combination, hoping the defect simply disappears.
Two weeks later, the team has logged twenty unstructured trials. Surface finish might have improved, but nobody knows exactly why. When the defect resurfaces a month later on a new batch of material, they are back at square one without a working standard.
I have walked into this exact scenario at a fabrication plant. It is the classic failure of One-Factor-At-A-Time (OFAT) testing. Design of Experiments (DOE) is the systematic alternative. It is a structured method for planning, running, and analysing trials so you can quantify the effect of multiple variables simultaneously, find their interactions, and lock in an optimal, defensible process window.
Why One-Factor-At-A-Time Testing Fails
OFAT testing isolates variables. You hold everything steady, change one parameter, and measure the output. The flaw is that industrial processes rarely behave in isolation. Changing the cutting speed fundamentally alters how the feed rate interacts with the material.
If you test speed and feed independently, you will never see the combined effect. You end up with local optimisation—finding a setting that works for today's batch—while remaining blind to the actual process dynamics. DOE solves this by testing variables in deliberate, orthogonal combinations.
Consider a milling operation aiming to minimise surface roughness (Ra). We have three variables: cutting speed, feed per tooth, and depth of cut. Using a full factorial DOE (a 2-level, 3-factor design), we need exactly 8 runs to map the entire process space.
| Run | Speed (m/min) | Feed (mm/z) | Depth (mm) | Result: Ra (μm) |
|---|---|---|---|---|
| 1 | 100 | 0.05 | 0.5 | 0.8 |
| 2 | 150 | 0.05 | 0.5 | 0.6 |
| 3 | 100 | 0.10 | 0.5 | 1.4 |
| 4 | 150 | 0.10 | 0.5 | 1.1 |
| 5 | 100 | 0.05 | 1.0 | 1.0 |
| 6 | 150 | 0.05 | 1.0 | 0.7 |
| 7 | 100 | 0.10 | 1.0 | 1.8 |
| 8 | 150 | 0.10 | 1.0 | 1.5 |
From these 8 rows, statistical software can separate the noise from the signal. You quantify the main effects to see which factor dominates the variance. You map the interactions to see if high speed mitigates a high feed rate. You pinpoint the exact combination that delivers the lowest Ra value.
Structuring the Experiment
Before touching a CNC machine, define the problem in measurable terms. 'Improve surface finish' is an opinion. 'Achieve Ra < 1.0 μm on bore A45 of part GB-782' is a measurable response variable suitable for statistical analysis.
Select a response variable that is highly sensitive to process changes and directly relevant to the customer's drawing specifications. Most importantly, verify the measurement system first. If your Gage R&R is above 30%, any DOE you run will only measure measurement error.
Next, identify the factors. Gather the operators, tooling engineers, maintenance, and quality team. Map out every potential variable. Filter this list down to controllable factors that are safe and viable to test, then assign them high and low levels based on engineering knowledge and equipment limits.
The DOE Execution Sequence
- 01Screening DesignUse a fractional factorial (e.g., Plackett-Burman) to filter 6-10 factors down to the vital few.
- 02Full FactorialTest all combinations of the 2-4 remaining critical factors to map exact interactions.
- 03Response SurfaceAdd centre points to model curvature and find the precise mathematical optimum.
- 04Confirmation RunProduce a batch at the predicted optimal settings to verify the statistical model.
If you start with too many factors, the math becomes unmanageable. A full factorial with 5 factors requires 32 runs. With 10 factors, you need 1,024 runs. Always start with a fractional factorial screening design to eliminate the trivial many before committing resources to optimisation.
Execution Discipline on the Shop Floor
The execution phase is where most DOEs fail. Running an experiment on a production machine introduces uncontrolled variables: tool wear, ambient temperature shifts, and material lot variations. Discipline during the run is non-negotiable.
You must randomise the run order. Do not execute the trials in the neat sequence printed on your worksheet. Running them sequentially means that tool degradation over time will correlate with your factor levels, completely destroying the statistical validity of your data.
Hold everything that is not a tested factor constant. Use material from a single lot, the same operator, and the same measurement gauge. Document everything, even factors you assume are irrelevant, like ambient humidity or machine warm-up time. These notes save analyses when the data looks contradictory.
Analysis and Statistical Significance
Modern software like Minitab or JMP handles the heavy matrix algebra, but the quality engineer must interpret the outputs. Analysis of Variance (ANOVA) is the primary engine. P-values below 0.05 flag which factors and interactions are statistically significant.
Key DOE Validation Thresholds
Look at the Pareto chart of effects to instantly see which variables drive the variance. An interaction plot is the most valuable diagnostic tool available—if the lines are parallel, there is no interaction. If the lines cross, the effect of one variable depends entirely on the level of another.
Software calculates the matrix, but engineering knowledge defines the factors and interprets the physics behind the data.
Finally, check the residual plots. If the residuals are not normally distributed and randomly scattered, your model does not adequately explain the process. You are missing a variable, or the measurement system is flawed. Do not proceed until the model is statistically sound.
Implementation and Standardisation
Finding the optimum is useless if you do not implement it. Once the confirmation run proves the model correct, the real work begins. You must translate the statistical findings into operational standards.
Update the PFMEA to reflect the newly quantified cause-and-effect relationships. Revise the Control Plan to monitor the critical parameters identified by the ANOVA. Change the CNC program, train the operators on the new setpoints, and lock the machine parameters to prevent unauthorised adjustments.
At the automotive plant in Slovakia, implementing this exact methodology turned two weeks of OFAT chaos into a three-day systematic study. A fractional factorial screening design proved that three of the six suspected factors—the feed rate, tool type, and their interaction—controlled 85% of the burr formation variance.
We ran an 8-run full factorial followed by centre points, identified the optimal parameters, and dropped the burr height from 0.3 mm to 0.05 mm—well below the OEM tolerance. The solution held permanently because it was based on mathematical proof, not guesswork.
Compliance with IATF 16949 and VDA 6.3
In automotive and aerospace quality systems, DOE is not optional. IATF 16949, clause 8.5.1.3, explicitly requires the use of multi-variable DOE or equivalent methodologies to identify and optimise process parameters during Advanced Product Quality Planning (APQP).
The German VDA 6.3 process audit, specifically question P6.2.1, asks whether process parameters are systematically identified and optimised. Walking into a customer audit or a VDA 6.3 assessment with a documented DOE protocol demonstrates profound process understanding.
Major OEMs like BMW, VW, and Stellantis increasingly demand DOE evidence for critical process characteristics before approving a Production Part Approval Process (PPAP) submission. Relying on legacy machine settings or 'operator feel' no longer passes third-party scrutiny.
Ultimately, Design of Experiments shifts an organisation from reactive troubleshooting to proactive process design. It replaces tribal knowledge with empirical data, ensuring that the process standard reflects the physical optimum of the machine, the tool, and the material.