Sample matters
February 21, 2010 by Thuy
Sample choices for production tests
Our Newsletter #2 2010 has just been sent to subscribers. You can also download it here in pdf format and register to receive the next instalment directly in your mailbox, it is free.
Our latest two newsletters were devoted to the identification of the right suppliers: how to spot quality laggards, and how to pick the right factory.
We now turn to no less vital matters: choices to make when defining quality policy. This part of the quality manual is sometimes misunderstood. We set the record straight.
1. Sampling methods for inspection
Our customers frequently face difficulties in their definition of quality inspection criteria. The most common practice is to sample units from the production lot. It is most important of course that sampling be done randomly from a full lot of ready products to ensure true representation of the full lot. Units must be picked independently from the factory and especially from the factory production and quality teams.
- Single unit sampling
A single unit is picked and examined. This would catch lot-wide mistakes or omissions in a production batch but is not statistically significant.
- Sampling methods based on statistical representation
The idea is to pick a sample big enough to draw conclusions on the full lot, with a certain degree of confidence in the results. Statistical methods are used.
Back in the days when pocket calculators were the size of a building the US government conveniently produced a series of tables for quick reference. They are known as Acceptable Quality Level (AQL) definition tables and are the most widely used statistical method.
-
- AQL levels 1 (light control), 2 (normal) and 3 (tight control) are most commonly used. Level 2 is typically used, with 1 used in case of high trust and 3 in case of low trust.
- Special AQL levels S1 to S4: These levels are based on very small sampling population, and used when products are complex, with many checkpoints.
AQL's strongest point is its acceptance by most factory Quality departments across Asia. This means no misunderstandings. Another good statistical method is the Lot Tolerance Percent Defective (LTPD) method, but being more recent it is much less familiar to factories.
- Percentage of lot
Mostly used by European clients (influence of the metric system?) and not strictly statistically significant as representativity varies depending on lot size, which a simple percentage fails to capture. The sample is simply defined as a percentage of the full lot.
- Full lot inspection (100% inspection)
For small production runs up to a few hundred pieces. Mostly used in cases when there is no factory QC, or its reliability is very patchy, or each piece is to some extent handmade and subject to wide variations.
The advantage of 100% check is complete visibility. It allows piece by piece decision and immediate action since all defects are identified in the full lot by the end of inspection.
2- Adaptability is key
- New production
Typically a few pre-production units are first released, and they are all checked. Then for a period of time after the pp units are authorized, a tight sampling method will be used in order to find and root out any deviations. Once defects levels are down and production is stable, sampling levels can be reduced.
- New supplier
Same as above, with production of an existing product at a new supplier, or a new product at a new supplier.
- Ongoing production
Stable ongoing production requires less intense sampling, and can be lightened over time.
If a quality alert is found, beyond the escalation for this alert (see next article) one of the possible impacts is that sampling levels for all productions will go up if there is a worry of instability.
Some clients also use this as a symbol for the factory to know that it is on notice. It is a marker to the factory's management that trust is being lost, before commercial action is taken.
Also prepare to handle the results !
When preparing for inspection results, you need to consider :
1. Defects must be defined, categorised. Tolerances are to be set.
- Product defect levels are defined as:
-
- Critical: usually defined as resulting in physical harm. This category can of course be extended by the customer.
-
- Major: causing a major product malfunction, ie product is non-functioning as advertised or unsellable at its advertised price.
-
- Minor: all other defects
In practice the customer will pre-define categories for most likely defects in the quality manual, so that lot sample inspection results are meaningfully interpreted
- Tolerance: a number of defects is usually tolerated before a lot is deemed to fail testing. This number depends on:
-
- Lot and sample size
-
- Defect(s) combination and severity. Critical defect tolerance is usually set at zero, while major defect tolerance while not zero is always a number smaller than minor defect tolerance.
-
- Customer defined certainty requirement. Within the AQL standard for example these thresholds are defined in the second table, for each given %.
Some customers sum major + minor defects to have only one category aside from critical. This is not standard practice (it is statistically less relevant) but at the end of the day what matters is what makes sense to the customer, and that whatever method used is followed consistently in order to allow statistical analysis and variation root cause analysis.
2. There will be defects. Be ready for it.
A pre-defined escalation process is key to a successful conclusion.
If inspecting 100% of a product lot corrective action is easy to organize: each single defect is pointed out by the inspection team to the factory and corrected by the factory. A new inspection takes place to make sure the corrections were done correctly, and once all is OK the lot ships.
But when inspecting less than 100% there is a complication: failed tested products will of course be pointed out and repaired, but what about the other defects, lurking in the untested remainder of the lot?
Assume that a lot shows critical, major or minor defects above the preset tolerances.
Below are some standard escalation processes :
- Reject the lot
Most common, and for example defined for AQL. If a lot is found above defect tolerance level it is rejected. The factory is to inspect all products in the full lot, identify all defects, fix them, then resubmit the full lot to random sampling and inspection. Repeat as long as the lot cannot be cleared.
(Note: it is common practice that any re-inspection time is billed to the factory, as re-inspection is due to factory-caused defects or negligence)
- Inspect additional units to make lot rejection decision on a broader sample
In case of doubt about the relevance of the sample taken (for example if using a non statistical sampling method), taking a new sample of same or different size can help decide on further action. This is typically not necessary when using AQL or LTPD.
- Inspect 100% of lot
If lot size is not too large, moving the failed lot to 100% inspection is useful as it takes inspection outside of sampling, and into certainty. It allows for the reasonably quick identification of all defects on all pieces in the full lot, and their correction as soon as possible.
This course of action is very useful for small to medium sized lots of valuable items with fairly short test time, when time is tight and factory rework must be done as quickly as possible.
Summary:
We hope these few pointers were useful, and will be happy to refine your quality manual and inspection processes with you.