This became one of my favorite engineering sayings. Not because customers are misleading, but because evaluation samples rarely represent reality.
The typical situation
A customer sends:
- Their best samples
- Their worst samples
What is almost always missing:
- Normal production variation - the boring middle that makes up 99% of real output.
The result
The evaluation looks excellent. Then production performs differently, because production is mostly the variation that was never in the sample box.
My current questions
Before trusting any sample set, I ask:
- How many suppliers?
- How many production lines?
- How many years of process history?
- What recently changed?