There is no universal pilot sample size: it depends on the risk being tested, expected defect rate, desired precision and number of distinct file populations. A small operational trial can reveal workflow problems without proving that rare material errors are acceptably controlled.
NIST describes exact binomial confidence limits and cautions that normal approximations can be unsuitable when sample sizes or failure counts are small. (itl.nist.gov)
For illustration, under an independent, representative binomial sampling model, with zero observed material defects, the one-sided 95% upper confidence limit is:
Upper defect-rate limit = 1 − 0.05^(1/n).
Calculated examples:
| Independently reviewed files | Material defects observed | Approximate upper 95% bound |
|---|---|---|
| 100 | 0 | 2.95% |
| 300 | 0 | 0.99% |
These calculations are not recommended acceptance thresholds. Zero observed defects in 100 files does not demonstrate a zero-defect process.
The interpretation weakens if files share the same narrow template, inspector, event or production worker, or if reviewers miss defects. Separate loss types and materially different workflows may need their own evidence.
If the objective is to prove an improvement against a baseline rather than bound a defect rate, obtain a sample-size design for that comparison. A sample sufficient for workflow debugging may be insufficient for supplier selection or broad rollout.
Sources cited on this page
- itl.nist.govitl.nist.gov