The Most
Expensive Ignorance in Manufacturing
Walk into any manufacturing plant struggling with a quality problem
and you will find the same scene: engineers huddled around a conference
table, changing one parameter at a time, running trial after trial,
convinced that systematic investigation means adjusting a single
variable while holding everything else constant. They are conducting
experiments in the most expensive, most inefficient, and most misleading
way possible — and they do not even know it.
Design of Experiments, or DOE, is arguably the most powerful
statistical tool available to manufacturing engineers. It is also the
most systematically misused, underused, and misunderstood tool in the
quality profession’s arsenal. Not because the mathematics are impossibly
complex — modern software handles that — but because the thinking it
requires runs counter to almost every instinct that manufacturing people
have developed through years of troubleshooting.
The result is a manufacturing landscape where organizations spend
fortunes on experimentation that yields fragments of understanding while
the real drivers of their quality problems — the interactions, the
nonlinearities, the factors they never thought to test — remain hidden
in plain sight, driving defects day after day, shift after shift, batch
after batch.
What Design of
Experiments Actually Does
At its core, DOE is a structured method for deliberately varying
multiple factors simultaneously in a planned pattern so that you can
extract maximum information from minimum experimental effort. Instead of
the traditional one-factor-at-a-time approach — change temperature, hold
everything else constant, see what happens — DOE changes several factors
together according to a mathematical matrix that lets you untangle not
just the individual effects of each factor but also how they interact
with each other.
Consider a welding process. You suspect that weld quality depends on
voltage, wire feed speed, travel speed, and gas flow rate. The
traditional approach would test each factor individually: vary voltage
while keeping everything else fixed, then vary wire feed speed, then
travel speed, then gas flow. Four factors, maybe three levels each,
twelve experiments minimum, and at the end you would know the effect of
each factor in isolation — but you would know nothing about how voltage
and wire feed speed interact, or how travel speed and gas flow combine
to create effects that neither produces alone.
DOE, by contrast, can test all four factors simultaneously in as few
as eight or sixteen runs, and at the end you would have a complete map
of not just the main effects but every interaction between every pair of
factors, and potentially even three-factor interactions. You get
dramatically more information from dramatically fewer experiments.
This is not a marginal improvement. This is the difference between
stumbling around in a dark room with a flashlight and turning on the
overhead lights.
The One-Factor-at-a-Time
Disaster
The most common experimental approach in manufacturing is also the
worst. One-factor-at-a-time experimentation, or OFAT, is what happens
when engineers apply their intuition to experimental design. It feels
logical: control everything, change one thing, observe the result. It
feels scientific. It is neither.
The fundamental problem with OFAT is that it cannot detect
interactions. And in manufacturing, interactions are everywhere. The
effect of temperature on a chemical process depends on pressure. The
effect of cutting speed on surface finish depends on tool geometry. The
effect of humidity on coating adhesion depends on the substrate
preparation method. These are not edge cases or theoretical curiosities
— they are the norm in complex manufacturing processes.
When you test factors one at a time, you build a mental model of your
process that assumes every factor operates independently. This model is
almost always wrong. And because it is wrong, the “optimal” settings you
derive from it are not optimal at all. They are the settings that look
best under a simplifying assumption that does not hold in reality.
I once consulted at a plastics injection molding plant that had spent
six months optimizing a warpage problem using OFAT. They had tested mold
temperature, melt temperature, injection speed, packing pressure, and
cooling time individually. Each experiment gave them a “best” setting
for that factor. They combined all the best settings and the warpage was
worse than when they started. The reason was that the optimal mold
temperature depended on the melt temperature, and the optimal packing
pressure depended on the cooling time. The factors interacted, and their
OFAT approach had systematically missed every interaction.
Six months of production time, thousands of dollars in scrap, and the
answer was not just incomplete — it was actively misleading. A properly
designed fractional factorial experiment could have mapped the entire
process space in two weeks.
The Factorial Revolution
The insight that powers DOE is surprisingly old. Sir Ronald Fisher
developed the principles of factorial experimentation in the 1920s for
agricultural research. Genichi Taguchi adapted them for manufacturing in
the 1950s. George Box expanded and refined them throughout the latter
half of the twentieth century. The mathematics have been available for a
century. The software has been accessible for decades. And yet, the
majority of manufacturing organizations still default to OFAT when faced
with a process problem.
A full factorial experiment tests every combination of every level of
every factor. Two factors at two levels each requires four runs. Three
factors at two levels requires eight runs. Four factors at two levels
requires sixteen runs. This exponential growth is the objection most
engineers raise: “We cannot afford to run that many trials.” But
fractional factorial designs allow you to strategically omit certain
combinations while still capturing the most important effects and
interactions. A half-fraction of a four-factor, two-level design cuts
sixteen runs to eight while still resolving all main effects and most
two-factor interactions.
The key word is “strategically.” DOE is not about running fewer
experiments randomly. It is about running the right experiments — the
minimum set that extracts maximum information. Every run in a designed
experiment does double, triple, or quadruple duty because every data
point contributes to the estimation of multiple effects simultaneously.
In OFAT, each run tells you about one factor at one level. In DOE, each
run tells you about every factor and their interactions.
Screening, Optimization,
and Robustness
DOE in manufacturing typically follows a three-stage strategy, and
organizations that skip stages almost always regret it.
Screening experiments come first. When you have a
process with many potential factors — material lot, machine number,
operator, temperature, humidity, speed, pressure, time, and a dozen more
— you cannot test them all at every level. Screening designs, like
Plackett-Burman or Resolution III fractional factorials, test many
factors at two levels in very few runs to identify the vital few factors
that actually matter. A twelve-run Plackett-Burman design can screen
eleven factors. If only three or four of those eleven turn out to be
statistically significant, you have just eliminated seven or eight
factors from further investigation — and you did it in twelve runs
instead of the hundreds that OFAT would have required.
Optimization experiments come second. Once you know
which factors matter, you explore them in more detail — more levels,
narrower ranges, full factorial or Response Surface Methodology designs
— to find the combination of settings that optimizes your response.
Central Composite Designs and Box-Behnken Designs are the workhorses
here, fitting a mathematical model to your process that can predict the
response at any combination of factor settings within the experimental
range.
Robustness experiments come third, and this is where
Taguchi’s contribution shines. A robustness experiment deliberately
introduces noise — the variation you cannot control in normal
production, like raw material batch variation, ambient humidity
fluctuations, or operator differences — and seeks factor settings that
minimize the effect of that noise on the response. The goal is not just
to find optimal settings but to find settings that stay optimal even
when conditions change. This is the difference between a process that
performs beautifully in a controlled laboratory and a process that
performs reliably on a noisy production floor.
Organizations that skip screening and jump straight to optimization
waste resources optimizing factors that do not matter. Organizations
that skip robustness deliver processes that perform well in development
but fall apart in production. The three-stage approach is not
bureaucratic overhead — it is the fastest path to genuine
understanding.
The Interaction Trap
Let me return to interactions, because they are the single most
important reason to use DOE, and the single most common reason
organizations resist it.
An interaction means that the effect of one factor depends on the
level of another factor. In mathematical terms, the effect of factor A
is different when factor B is at its high level versus its low level. In
practical terms, the knob you have been turning to adjust your process
does not do what you think it does — its effect changes depending on the
state of some other knob you are not watching.
Interactions are not rare in manufacturing. They are ubiquitous.
Chemical reactions depend on temperature and pressure simultaneously.
Mechanical properties depend on material composition and processing
conditions simultaneously. Surface finishes depend on tool geometry and
cutting parameters simultaneously. Every manufacturing process is a
system of interacting variables, and any experimental approach that
treats them as independent will produce incomplete and often incorrect
conclusions.
I have seen organizations spend years optimizing a process using
OFAT, achieve what they believe is the best possible performance, and
then watch a DOE-trained engineer come in and find settings that improve
performance by twenty or thirty percent in a matter of weeks — not
because the engineer is smarter, but because the engineer tested
interactions that the OFAT approach could not see.
The resistance to DOE often comes from a place of misplaced
confidence. Engineers who have spent years working with a process feel
that they understand it intuitively. And they do understand aspects of
it — the main effects, the obvious relationships, the factors that
matter most in isolation. But intuition is remarkably bad at detecting
interactions. The human brain does not naturally think in terms of joint
effects and conditional dependencies. DOE forces you to test what your
intuition cannot imagine.
Common Failures in DOE
Implementation
When organizations do attempt DOE, they often stumble in predictable
ways.
Choosing the wrong factors. The most carefully
designed experiment is worthless if you are testing the wrong variables.
Brainstorming sessions that generate factor lists often include factors
that are convenient to measure rather than factors that are likely to
matter. The screening stage exists partly to address this, but screening
designs only work if the vital few factors are among the many you chose
to test. Garbage in, garbage out applies to experimental design just as
much as it applies to data analysis.
Setting factor ranges too narrow. If the difference
between the high and low levels of a factor is smaller than the natural
variation in the process, the experiment will not be able to detect that
factor’s effect. Conversely, if ranges are set so wide that the process
produces scrap at the extremes, the experiment tells you where not to
operate but not how to optimize within the viable region. Selecting
appropriate factor ranges requires process knowledge and judgment.
Ignoring measurement system error. If your
measurement system cannot reliably distinguish between good and bad
parts, your DOE results will be dominated by measurement noise rather
than real process effects. This is why Measurement System Analysis
should precede DOE — a point that is obvious in hindsight but routinely
overlooked in practice.
Confounding factors you care about. Fractional
factorial designs deliberately confound some effects with others to
reduce the number of runs. If you confound a main effect with an
interaction that is actually significant, you will misattribute the
effect and draw incorrect conclusions. Understanding resolution and
confounding patterns is essential for choosing the right fractional
design.
Running experiments without randomization. If you
run all the low-temperature trials in the morning and all the
high-temperature trials in the afternoon, any ambient temperature drift
over the course of the day will be confounded with your temperature
effect. Randomizing run order distributes lurking variables across all
factor levels and prevents systematic bias.
Overinterpreting results from too few replicates. A
single replicate of a fractional factorial design can identify large
effects but provides no estimate of experimental error, which means no
statistical significance tests. Repeating the entire design or adding
center points provides the error estimate needed to distinguish real
effects from noise.
The Cultural Barrier
The biggest obstacle to DOE adoption is not mathematical complexity
or software availability. It is cultural. DOE requires a fundamentally
different mindset than the troubleshooting approach that most
manufacturing organizations reward.
Troubleshooting is reactive and local: find the problem, fix the
problem, move on. It values speed and decisiveness. DOE is proactive and
systematic: understand the process, map the relationships, predict the
outcomes. It values thoroughness and rigor. These are not complementary
mindsets in most organizations — they are competing philosophies, and
troubleshooting almost always wins.
The engineer who runs an OFAT experiment and “fixes” the problem by
Friday is praised. The engineer who proposes a two-week DOE to properly
understand the process is questioned about the cost, the production
time, the complexity, and whether all this statistical rigor is really
necessary when we just need to get the line running again.
This cultural bias creates a vicious cycle. Problems that are “fixed”
by OFAT tend to recur because the fix addressed a symptom or a main
effect while missing the interaction that drives the root cause. Each
recurrence is treated as a new problem, addressed with a new OFAT
experiment, which produces a new incomplete fix, which leads to another
recurrence. The organization never escapes the cycle because it never
invests in the kind of systematic understanding that DOE provides.
Breaking this cycle requires leadership that values prevention over
reaction, understanding over expediency, and long-term capability over
short-term firefighting. It requires recognizing that the cost of a
well-designed experiment is almost always less than the cost of the
repeated failures that result from not running one.
The ROI of Doing It Right
The return on investment for DOE, when done properly, is difficult to
overstate. A single well-designed screening experiment can eliminate
months of trial-and-error investigation. A single optimization
experiment can find process settings that reduce defect rates by orders
of magnitude. A single robustness experiment can create a process that
withstands the variation that previously caused chronic quality
problems.
I have seen cases where a two-week DOE program delivered more process
understanding than two years of OFAT experimentation had produced. Not
because the DOE engineers were smarter but because the methodology
extracted dramatically more information from dramatically fewer
experimental runs. The mathematics guarantee it: a factorial design uses
every data point to estimate every effect simultaneously, while OFAT
uses each data point to estimate only one effect, wasting most of the
information each run contains.
The economic case is straightforward. If each production run costs
money — material, labor, machine time, overhead — then every run that
does not contribute to understanding your process is money spent without
learning. DOE minimizes those uninformative runs. It is the most
efficient possible way to convert experimental investment into process
knowledge.
Software, Training, and
the Path Forward
Modern DOE software has eliminated the mathematical barrier. Minitab,
JMP, Design-Expert, and even Excel add-ins can generate experimental
designs, analyze results, and produce contour plots and response
surfaces with a few clicks. The mathematics of factorial design, ANOVA,
and regression modeling are handled transparently. You do not need a
statistics degree to use DOE effectively — you need process knowledge,
good judgment about factor selection and range setting, and the
discipline to follow the methodology.
What you do need is training. Not in the mathematics, but in the
thinking. Engineers need to understand why interactions matter, why
randomization matters, why screening before optimization matters, and
why OFAT fails. This is conceptual training, not computational training,
and it can be delivered effectively in a focused workshop format.
The path forward for any manufacturing organization is the same:
start small. Pick a chronic process problem that OFAT has failed to
solve. Assemble a cross-functional team to identify the likely factors.
Run a screening experiment. Follow up with optimization. Document the
results. Let the success speak for itself. Then scale.
DOE is not a tool you adopt once and use forever. It is a capability
you build gradually, experiment by experiment, success by success, until
it becomes part of how your organization thinks about process
understanding. The organizations that reach this point do not just solve
problems faster — they prevent problems that other organizations never
see coming.
The Bottom Line
Every manufacturing process is a system of interacting variables.
Understanding that system requires an experimental approach that can
detect and quantify interactions. One-factor-at-a-time experimentation
cannot do this. It never could. Design of Experiments can, has been able
to for a century, and is more accessible today than at any point in
history.
The question is not whether your organization can afford to invest in
DOE. The question is whether it can afford not to. Every day that your
engineers troubleshoot with OFAT is a day they are spending more money
to learn less about your processes than a properly designed experiment
would reveal in a fraction of the time. The defects you are fighting
today are hiding in the interactions you have never tested. DOE is how
you find them.
Peter Stasko is a Quality Architect with over 25 years of
experience in manufacturing quality management, statistical process
control, and continuous improvement. He has implemented DOE programs
across automotive, aerospace, electronics, and medical device
manufacturing, helping organizations move from reactive troubleshooting
to systematic process understanding.
