Failure Mode Detection Effectiveness

A product line has a check designed to catch a missing safety label, and the team wants to know whether that check actually catches the failure before shipment. Failure mode detection effectiveness tests a defined control against known or simulated defects and records what escaped.

ASQ describes detection in FMEA as the ability of controls to find a failure mode before effects occur, and its root-cause material supports testing controls against the mechanism, while a local measured catch rate is distinct from an FMEA ordinal detection score. Define the failure mode, since "wrong label" may include missing, swapped, outdated and illegible labels and you must decide which the control should catch.

Map the point of detection, because a check before packing, before dispatch and after a customer complaint have very different value. Identify the control by naming the scanner, test, inspection or reconciliation and its actual operating rules.

Design safe tests, because known-defect samples or controlled simulations can check detection without sending harmful goods to customers. Record true positives by counting defects the control correctly identified under the test and noting how they were handled, and record misses too, since a known defect not flagged is a false negative, so preserve the evidence and fix the control before relying on it.

Track false alarms as well, because a control that flags many acceptable items can slow work and encourage bypass, so report specificity alongside catch rate. Use representative samples, since a test of only obvious errors may overstate real performance and should include subtle and varied failure modes.

Check volume, because a sample of five tests cannot prove a 99% detection rate, so show counts and uncertainty, and verify configuration, since a scanner may work in the lab but use an older rule in one production line and each relevant deployment needs testing. Check human steps and bypasses as well: inspectors may need clear instructions, lighting and time, a theoretical detection capability is not actual operating effectiveness, and supervisors may override a warning during a rush, so log and review exceptions under an approval rule.

Inspect handoffs, because a detected defect needs quarantine and correction and an alert ignored or lost in a queue is not an effective escape-prevention control. Define the denominator, since known defective test cases differ from all units inspected and a raw flagged-unit count is not a detection rate, and compare to customer escapes, because complaints and returns can reveal failure types the test set did not cover.

Use a risk lens: a rare severe failure may need multiple independent controls even when a simple sample catch rate looks good, and better detection may reduce the chance of escape but does not lower severity, because the possible consequence of a defect is unchanged. Separate prevention from detection, since a poka-yoke that prevents the wrong part from being assembled differs from a final check that finds mistakes later, and review changes because new packaging, software or materials can defeat an old inspection, so retest after meaningful process changes.

Measure response time, since a control that finds an error days later may limit neither shipment nor harm, reconcile test logs by keeping date, defect type, result, system version and reviewer (missing results should not be counted as passes), and avoid score confusion, because in some FMEA scales a high detection number means detection is poor while in a catch-rate metric a high percentage is good, so label both. After tuning a scanner, validate corrections by repeating known-defect and false-alarm tests before resuming normal reliance; for an owner, detection effectiveness asks whether the control actually finds the failures it promises to find early enough to matter, and tested evidence beats a checklist stating that a control exists.