Just as processes that produce a product may vary, similarly quality inspection of a product by different QA’s may present different results. It’s because of their inapt understanding of quality, which in a garment is a degree of adherence to given specifications, Measurement Systems Analysis (MSA) ensures that the decision on a product or a process is not a reflection of the error in measurement system. MSA evaluates the method, instruments, and the entire process of obtaining measurements to ensure the integrity of data used for analysis, usually quality analysis. MSA is an important element of Six Sigma methodology and of other quality management systems. Rakhi Handa, who is now Managing Director, Product Safety & Quality Assurance at Target, had earlier worked for Unisource Worldwide Quality Assurance, and was its Global Head, discusses how MSA ensures the successful implementation of Six Sigma.
I once inspected a shipment of 6,000 linen patchwork skirts for Polo Ralph Lauren which failed for insecure stitches at the bottom hem. I was sure the Production Manager was going to be in serious trouble. I was amazed to find that he received no reprimand and the QA team, comprising 17 people, was made to work overnight, to 100% re-check the shipment and re-offer it for inspection the next day.
Without going further into the debate of who should be responsible for quality, there can be no arguing the significance of the role a QA plays in the process of assuring quality.
Now ask yourself this – if you are a manufacturer of apparel, or if you work for such an organization, how much time and effort does your organization spend in training those QA and monitoring them to ensure that they are familiar with quality standards, requirements and procedures?
The answer, in most cases, would be too brief or none.
Most factories attach newly hired QA to older QA employees for a duration of a couple of days to a couple of weeks and from thereon, the QAs are inducted into a line, a production floor or a particular buyer, depending on the size and the structure of the manufacturer. There aren’t too many organizations that provide ongoing formal training and monitoring once the initial “training” has been completed.
My first exposure to a structured QA training program came during my tenure at Polo Ralph Lauren, where a QA Trainer monitored not just the Polo QA performance, but also that of factory QAs and certified auditors. The trainer’s single most important function was to minimize the variation in the way different QAs viewed product and also to ensure repeatability and reproducibility of results between the different QAs and within the QAs themselves. It wasn’t until I underwent my own Six Sigma training that I recognized within the training programs of Polo, the rudimentary elements of the concept of Measurement System Analysis.
Measurement System Analysis or MSA is a very interesting tool used in Six Sigma. While it is not unique to the Six Sigma tool box, it is a method of evaluating the accuracy and precision of that which we use to measure our process, whether an instrument, a machine or, as in the case of the Apparel Industry, a group of human beings (QAs).
Measurement System Analysis is an experimental and mathematical method of determining how much the variation within the measurement process contributes to overall process variability.
At the risk of sounding excessively simplistic, there are 3 basic concepts that I would like to outline in order to explain what exactly I mean by a measurement system and the variation in that measurement system:
• Every process produces a “product”. • Every product possesses qualities or “features”. • Every quality feature can be measured.
Total variation = Product variation + Measurement variation.
Some variation is inherent in the system/product design.
Some variation is due to a faulty performance of the measurement system.
In general, we want the gauge (tool or the equipment used for measurement) to be able to give us the same measurement each time, and we want the gauge to be able to distinguish one item from another in our range of interest. It is to verify the ability of our measurement system to do this, which is the reason for conducting an MSA.
So, what do I mean by accuracy, precision and resolution?
Accuracy denotes the bias, linearity, stability and correlation of the gauge, where bias is the difference between the observed average of measurements and the agreed upon standard value. Linearity and stability refer to the changes in those measurement values across the scale and over the time, respectively.
Precision indicates the reproducibility and repeatability of the measurement system or the ability of the gauge or measurement tool to accurately repeat its results on the same part, while reproducibility is the ability of two or more gauges to accurately repeat their results on the same part.
Resolution, although not really relevant to the Apparel Industry, really implies the smallest dimension a scale can measure. For instance, on a measurement tape, the resolution would be 1/16th of an inch.
The slide above is a graphical representation of what exactly I mean by precision and accuracy.
There are two types of MSA:
1. Gauge R&R (repeatability and reproducibility) which is essentially to verify the consistency of a measurement system that uses machinery, equipment or gauges such as thermometers, blood pressure gauge, heart rate monitors, etc.
2. Attribute R&R which is a process to measure subjective or attribute data such as Chromatic standards and Visual inspection (pass/fail).
It is this last, the visual inspection, which the Apparel Industry uses to measure the quality of the product being manufactured.
I find it intriguing that, despite the fact that the measurement system in the Apparel Industry is the QA/QC in the factory (together with his or her measuring tape), and therefore, potentially filled with subjectivity and inconsistency, there is little or no effort by organizations to analyze the effectiveness of this measurement system. And by effectiveness I mean:
• Do all QAs have the same understanding of quality standards? • Do all QAs clearly understand the different types of defects? • Do all QAs understand the customer’s requirements? • Do all QAs measure and check a garment following the same method?
The purpose of conducting an MSA is, therefore:
1. To assess your inspection standards against your customer’s requirements. 2. To determine if inspections across all shifts, machines, etc. use the same criteria to determine good from bad. 3. To quantify the ability of inspectors to accurately repeat their inspection decisions. 4. To identify how well these inspectors are conforming to a “known master”. 5. To discover where training is needed, procedures are lacking and standards are not defined. Before you set out to fix any measurement system, you need to assess its capability first, to analyze where it stands today. And this, in essence is what MSA aims at doing – to assess the current capability of your measurement system.
8 easy steps on how to conduct an MSA:
1. Select 30 samples of the same style/colour. 2. Select inspectors – fully trained and qualified. 3. Each inspector inspects the garments in random order. 4. Each inspector repeats the inspection. 5. Data is entered into spreadsheet. 6. Document the results. 7. Implement appropriate actions to fix the process, if necessary. 8. Re-run the study to verify the effectiveness of the fixes.
To maintain the integrity of the MSA, it is important that the inspectors should inspect the garments in a random order the second time.
Once the results are documented, the Appraiser Score, Effective Screen Score and the Screen Effectiveness are calculated. And what are these?
Appraiser Score (in percentage) – The fraction of times the appraiser agrees with himself during an attribute R&R, this is the checker’s repeatability.
Effective Screen Score (in percentage) – All operators agree within and between themselves during an attribute R&R. This indicates the reproducibility of the operators.
Screen Effectiveness – The ability of the attribute measurement system to discriminate good from bad.
There must be a high percentage of agreement of more than 80% in the Screen Effective Score vs. Attribute in order for you to be able to conclude that the measurement system is effective and that it actually works.
The above example is an actual example of an MSA conducted at a factory in 2005. It demonstrates quite clearly the inconsistencies that can arise in a measurement system when checkers and QCs are not properly trained, or do not have a clear understanding of what the buyer expects.
The Standard (Std.) in this case, was the buyer QA’s assessment of the garments checked, the assumption being that the buyer QA would have the best understanding of what the buyer actually wants.
Conclusion
1. Within Appraiser Score was poor, ranging from 50% to 59.37% indicating appraiser’s repeatability was roughly half. In other words, when the checker checked the same garment a second time, his findings were different, roughly half the time.
2. Between Appraiser Score was extremely poor at 15.63% indicating very low level of reproducibility. That is, the checkers were inconsistent amongst themselves and could not provide the same result twice.
What was really scary in this case, was that the factory QC’s and the buyer QA’s findings were consistent only 12.5% of the time. Clearly, the factory was relying on a QA system that was weak and one which was unable to ensure good garments being packed the first time, resulting in a high level of final inspection failure by the buyer QA.
The same experiment could also be conducted to verify the consistency of how different people measure garments. In this case, a smaller sample size of 5 garments could be taken and 4-5 main points of measurements measured by 3 different QA’s. You would be surprised at the findings. A difference of 1/8th of an inch in handling should be allowed as the maximum tolerance from person to person. Anything beyond that should be considered as inability to repeat or reproduce, as the case may be.
But there’s another thing, although the MSA stipulates a minimum sample size of 30 when conducting the experiment, it doesn’t really have to be so. I remember conducting an MSA once where quality was particularly bad. I was in the factory for a meeting, together with the Production Manager, QA Manager, Finishing Manager and PPC Manager. To prove a point of what an issue ‘Quality’ was, I randomly picked up just 5 garments from finishing, checked them myself (making my pass/fail results on these garments as the standard) and then requested all the 4 Managers to check the same garments. Just the checking of those 5 garments by 4 different people gave such inconsistent and disastrous results, as to clearly prove to the factory staff that the measurement system or the method used to check garments was in itself at fault. The factory subsequently launched into a stringent training and monitoring program for all checking, QC and finishing teams.
Because the Measurement System depends entirely on human beings and because no two human beings view the same defect in exactly the same manner, it becomes all the more necessary to focus on first measuring how effective the measurement system is, and then fixing the problems by simply providing training and awareness. There is nothing more critical than ensuring standardization within the measurement system, for this is the tool with which you gauge the quality of your organization.
In most of the Six Sigma projects I have found that by conducting an MSA and then strengthening the measurement system, most of the problems in the sewing line get minimized automatically. But there is another reason to strengthen the measurement system – it is when the quality assurance department becomes pro-active and starts finding defects at their inception or at their source, rather than at the end of line, that quality issues start getting highlighted. As QA’s and QCs become increasingly aware of quality (and as the measurement system becomes strengthened), they focus on the production line more closely and therefore, usually are able to find defects at source, so that the production or manufacturing team is forced to take corrective measures during production.
In the absence of the concept of the Toyota system of self-check on quality, having an effective measurement system becomes all the more important.
Post a Comment