DMAIC ^^^ CHAPTER 10 Measurement Systems Analysis R&R STUDIES FOR CONTINUOUS DATA Discrimination, stability, bias, repeatability, reproducibility, and linearity Modern measurement system analysis goes well beyond calibration. A gage can be perfectly accurate when checking a standard and still be entirely unacceptable for measuring a product or controlling a process. This section illustrates techniques for quantifying discrimination, stability, bias, repeatability, reproducibility and variation for a measurement system. We also show how to express measurement error relative to the product tolerance or the process variation. For the most part, the methods shown here use control charts. Control charts provide graphical portrayals of the measurement processes that enable the analyst to detect special causes that numerical methods alone would not detect. MEASUREMENT SYSTEM DISCRIMINATION Discrimination, sometimes called resolution, refers to the ability of the measurement system to divide measurements into ‘‘data categories.’’ All parts within a particular data category will measure the same. For example, 326 MEASUREMENT SYSTEMS ANALYSIS if a measurement system has a resolution of 0.001 inches, then items measuring 1.0002, 1.0003, 0.9997 would all be placed in the data category 1.000, i.e., they would all measure 1.000 inches with this particular measurement system. A measurement system’s discrimination should enable it to divide the region of interest into many data categories. In Six Sigma, the region of interest is the smaller of the tolerance (the high specification minus the low specification) or six standard deviations. A measurement system should be able to divide the region of interest into at least five data categories. For example, if a process was capable (i.e., Six Sigma is less than the tolerance) and s ¼ 0:0005, then a gage with a discrimination of 0.0005 would be acceptable (six data categories), but one with a discrimination of 0.001 would not (three data categories). When unacceptable discrimination exists, the range chart shows discrete ‘‘jumps’’ or ‘‘steps.’’ This situation is illustrated in Figure 10.1. Figure 10.1. Inadequate gage discrimination on a control chart. R&R studies for continuous data 327 Note that on the control charts shown in Figure 10.1, the data plotted are the same, except that the data on the bottom two charts were rounded to the nearest 25. The effect is most easily seen on the R chart, which appears highly stratified. As sometimes happens (but not always), the result is to make the X-bar chart go out of control, even though the process is in control, as shown by the control charts with unrounded data. The remedy is to use a measurement system capable of additional discrimination, i.e., add more significant digits. If this cannot be done, it is possible to adjust the control limits for the round-off error by using a more involved method of computing the control limits, see Pyzdek (1992a, pp. 37^42) for details. STABILITY Measurement system stability is the change in bias over time when using a measurement system to measure a given master part or standard. Statistical stability is a broader term that refers to the overall consistency of measurements over time, including variation from all causes, including bias, repeatability, reproducibility, etc. A system’s statistical stability is determined through the use of control charts. Averages and range charts are typically plotted on measurements of a standard or a master part. The standard is measured repeatedly over a short time, say an hour; then the measurements are repeated at predetermined intervals, say weekly. Subject matter expertise is needed to determine the subgroup size, sampling intervals and measurement procedures to be followed. Control charts are then constructed and evaluated. A (statistically) stable system will show no out-of-control signals on an X-control chart of the averages’ readings. No ‘‘stability number’’ is calculated for statistical stability; the system either is or is not statistically stable. Once statistical stability has been achieved, but not before, measurement system stability can be determined. One measure is the process standard deviation based on the R or s chart. R chart method: ^ ¼ R d2 ^ ¼ s c4 s chart method: The values d2 and c4 are constants from Table 11 in the Appendix. 328 MEASUREMENT SYSTEMS ANALYSIS BIAS Bias is the difference between an observed average measurement result and a reference value. Estimating bias involves identifying a standard to represent the reference value, then obtaining multiple measurements on the standard. The standard might be a master part whose value has been determined by a measurement system with much less error than the system under study, or by a standard traceable to NIST. Since parts and processes vary over a range, bias is measured at a point within the range. If the gage is non-linear, bias will not be the same at each point in the range (see the definition of linearity above). Bias can be determined by selecting a single appraiser and a single reference part or standard. The appraiser then obtains a number of repeated measurements on the reference part. Bias is then estimated as the difference between the average of the repeated measurement and the known value of the reference part or standard. Example of computing bias A standard with a known value of 25.4 mm is checked 10 times by one mechanical inspector using a dial caliper with a resolution of 0.025 mm. The readings obtained are: 25.425 25.400 25.425 25.425 25.400 25.400 25.400 25.425 25.375 25.375 The average is found by adding the 10 measurements together and dividing by 10, 254:051 X ¼ ¼ 25:4051 mm 10 The bias is the average minus the reference value, i.e., bias ¼ average
reference value ¼ 25:4051 mm
25:400 mm ¼ 0:0051 mm The bias of the measurement system can be stated as a percentage of the tolerance or as a percentage of the process variation. For example, if this measurement system were to be used on a process with a tolerance of  0.25 mm then % bias ¼ 100  jbiasj=tolerance ¼ 100  0:0051=0:5 ¼ 1% R&R studies for continuous data 329 This is interpreted as follows: this measurement system will, on average, produce results that are 0.0051 mm larger than the actual value. This difference represents 1% of the allowable product variation. The situation is illustrated in Figure 10.2. Figure 10.2. Bias example illustrated. REPEATABILITY A measurement system is repeatable if its variability is consistent. Consistent variability is operationalized by constructing a range or sigma chart based on repeated measurements of parts that cover a significant portion of the process variation or the tolerance, whichever is greater. If the range or sigma chart is out of control, then special causes are making the measurement system inconsistent. If the range or sigma chart is in control then repeatability can be estimated by finding the standard deviation based on either the average range or the average standard deviation. The equations used to estimate sigma are shown in Chapter 9. Example of estimating repeatability The data in Table 10.1 are from a measurement study involving two inspectors. Each inspector checked the surface finish of five parts, each part was checked twice by each inspector. The gage records the surface roughness in minches (micro-inches). The gage has a resolution of 0.1 m-inches. 330 MEASUREMENT SYSTEMS ANALYSIS Table 10.1. Measurement system repeatability study data. PART READING #1 READING #2 AVERAGE RANGE INSPECTOR #1 1 111.9 112.3 112.10 0.4 2 108.1 108.1 108.10 0.0 3 124.9 124.6 124.75 0.3 4 118.6 118.7 118.65 0.1 5 130.0 130.7 130.35 0.7 INSPECTOR #2 1 111.4 112.9 112.15 1.5 2 107.7 108.4 108.05 0.7 3 124.6 124.2 124.40 0.4 4 120.0 119.3 119.65 0.7 5 130.4 130.1 130.25 0.3 We compute: Ranges chart R ¼ 0:51 UCL ¼ D4 R ¼ 3:267  0:51 ¼ 1:67 Averages chart X ¼ 118:85 LCL ¼ X
A2 R ¼ 118:85
1:88  0:109 ¼ 118:65 UCL ¼ X þ A R ¼ 118:85 þ 1:88  0:109 ¼ 119:05 2 R&R studies for continuous data 331 The data and control limits are displayed in Figure 10.3. The R chart analysis shows that all of the R values are less than the upper control limit. This indicates that the measurement system’s variability is consistent, i.e., there are no special causes of variation. Figure 10.3. Repeatability control charts. Note that many of the averages are outside of the control limits. This is the way it should be! Consider that the spread of the X-bar chart’s control limits is based on the average range, which is based on the repeatability error. If the averages were within the control limits it would mean that the part-to-part variation was less than the variation due to gage repeatability error, an undesirable situation. Because the R chart is in control we can now estimate the standard deviation for repeatability or gage variation: e ¼ R d2 ð10:1Þ where d2 is obtained from Table 13 in the Appendix. Note that we are using d2 and not d2 . The d2 values are adjusted for the small number of subgroups typically involved in gage R&R studies. Table 13 is indexed by two values: m is the number of repeat readings taken (m ¼ 2 for the example), and g is the number of parts times the number of inspectors (g ¼ 5  2 ¼ 10 for the example). This gives, for our example e ¼ 0:51 R ¼ 0:44  ¼ d2 1:16 332 MEASUREMENT SYSTEMS ANALYSIS The repeatability from this study is calculated by 5:15e ¼ 5:15 0:44 ¼ 2:26. The value 5.15 is the Z ordinate which includes 99% of a standard normal distribution. REPRODUCIBILITY A measurement system is reproducible when different appraisers produce consistent results. Appraiser-to-appraiser variation represents a bias due to appraisers. The appraiser bias, or reproducibility, can be estimated by comparing each appraiser’s average with that of the other appraisers. The standard deviation of reproducibility (o ) is estimated by finding the range between appraisers (Ro) and dividing by d2 . Reproducibility is then computed as 5.15o . Reproducibility example (AIAG method) Using the data shown in the previous example, each inspector’s average is computed and we find: Inspector #1 average ¼ 118:79 -inches Inspector #2 average ¼ 118:90 -inches Range ¼ Ro ¼ 0:11 -inches Looking in Table 13 in the Appendix for one subgroup of two appraisers we find d2 ¼ 1:41 ðm ¼ 2, g ¼ 1), since there is only one range calculation g ¼ 1. Using these results we find Ro =d2 ¼ 0:11=1:41 ¼ 0:078. This estimate involves averaging the results for each inspector over all of the readings for that inspector. However, since each inspector checked each part repeatedly, this reproducibility estimate includes variation due to repeatability error. The reproducibility estimate can be adjusted using the following equation: ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi s s ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi   Ro 2 ð5:15e Þ2 0:11 2 ð5:15  0:44Þ2
5:15 
¼ 5:15  1:41 d2 nr 52 pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ¼ 0:16
0:51 ¼ 0 As sometimes happens, the estimated variance from reproducibility exceeds the estimated variance of repeatability + reproducibility. When this occurs the estimated reproducibility is set equal to zero, since negative variances are theoretically impossible. Thus, we estimate that the reproducibility is zero. 333 R&R studies for continuous data The measurement system standard deviation is pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi m ¼ e2 þ o2 ¼ ð0:44Þ2 þ 0 ¼ 0:44 ð10:2Þ and the measurement system variation, or gage R&R, is 5.15m . For our data gage R&R ¼ 5:15  0:44 ¼ 2:27. Reproducibility example (alternative method) One problem with the above method of evaluating reproducibility error is that it does not produce a control chart to assist the analyst with the evaluation. The method presented here does this. This method begins by rearranging the data in Table 10.1 so that all readings for any given part become a single row. This is shown in Table 10.2. Table 10.2. Measurement error data for reproducibility evaluation. INSPECTOR #1 INSPECTOR #2 Part Reading 1 Reading 2 Reading 1 Reading 2 X bar 1 111.9 112.3 111.4 112.9 112.125 1.5 2 108.1 108.1 107.7 108.4 108.075 0.7 3 124.9 124.6 124.6 124.2 124.575 0.7 4 118.6 118.7 120 119.3 119.15 1.4 5 130 130.7 130.4 130.1 130.3 0.7 118.845 1 Averages ! R Observe that when the data are arranged in this way, the R value measures the combined range of repeat readings plus appraisers. For example, the smallest reading for part #3 was from inspector #2 (124.2) and the largest was from inspector #1 (124.9). Thus, R represents two sources of measurement error: repeatability and reproducibility. 334 MEASUREMENT SYSTEMS ANALYSIS The control limits are calculated as follows: Ranges chart R ¼ 1:00 UCL ¼ D4 R ¼ 2:282  1:00 ¼ 2:282 Note that the subgroup size is 4. Averages chart X ¼ 118:85 LCL ¼ X
A2 R ¼ 118:85
0:729  1 ¼ 118:12 UCL ¼ X þ A R ¼ 118:85 þ 0:729  1 ¼ 119:58 2 The data and control limits are displayed in Figure 10.4. The R chart analysis shows that all of the R values are less than the upper control limit. This indicates that the measurement system’s variability due to the combination of repeatability and reproducibility is consistent, i.e., there are no special causes of variation. Figure 10.4. Reproducibility control charts. Using this method, we can also estimate the standard deviation of reproducibility plus repeatability, as we can find o ¼ Ro =d2 ¼ 1=2:08 ¼ 0:48. Now we know that variances are additive, so 2 2 2 ¼ repeatability þ reproducibility repeatabilityþreproducibility ð10:3Þ R&R studies for continuous data 335 which implies that reproducibility ¼ qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 2 2 repeatabilityþreproducibility
repeatability In a previous example, we computed repeatability ¼ 0:44. Substituting these values gives qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 2 2 reproducibility ¼ repeatabilityþreproducibility
repeatability qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ¼ ð0:48Þ2
ð0:44Þ2 ¼ 0:19 Using this we estimate reproducibility as 5:15  0:19 ¼ 1:00. PART-TO-PART VARIATION The X-bar charts show the part-to-part variation. To repeat, if the measurement system is adequate, most of the parts will fall outside of the X -bar chart control limits. If fewer than half of the parts are beyond the control limits, then the measurement system is not capable of detecting normal part-to-part variation for this process. Part-to-part variation can be estimated once the measurement process is shown to have adequate discrimination and to be stable, accurate, linear (see below), and consistent with respect to repeatability and reproducibility. If the part-to-part standard deviation is to be estimated from the measurement system study data, the following procedures are followed: 1. Plot the average for each part (across all appraisers) on an averages control chart, as shown in the reproducibility error alternate method. 2. Con¢rm that at least 50% of the averages fall outside the control limits. If not, ¢nd a better measurement system for this process. 3. Find the range of the part averages, Rp. 4. Compute p ¼ Rp =d2 , the part-to-part standard deviation. The value of d2 is found in Table 13 in the Appendix using m ¼ the number of parts and g ¼ 1, since there is only one R calculation. 5. The 99% spread due to part-to-part variation (PV) is found as 5.15 p. Once the above calculations have been made, the overall measurement sysqffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi tem can be evaluated. 1. The total process standard deviation is found as t ¼ m2 þ p2 . Where m ¼ the standard deviation due to measurement error. 2. Total variability (TV) is 5.15t . 3. The percent repeatability and reproducibility (R&R) is 100  ðm =t Þ%. 336 MEASUREMENT SYSTEMS ANALYSIS 4. The number of distinct data categories that can be created with this measurement system is 1.41  (PV/R&R). EXAMPLE OF MEASUREMENT SYSTEM ANALYSIS SUMMARY 1. 2. 3. 4. 5. Plot the average for each part (across all appraisers) on an averages control chart, as shown in the reproducibility error alternate method. Done above, see Figure 10.3. Con¢rm that at least 50% of the averages fall outside the control limits. If not, ¢nd a better measurement system for this process. 4 of the 5 part averages, or 80%, are outside of the control limits. Thus, the measurement system error is acceptable. Find the range of the part averages, Rp. Rp ¼ 130:3
108:075 ¼ 22:23. Compute p ¼ Rp =d2 , the part-to-part standard deviation. The value of d2 is found in Table 13 in the Appendix using m ¼ the number of parts and g ¼ 1, since there is only one R calculation. m ¼ 5, g ¼ 1, d2 ¼ 2:48, p ¼ 22:23=2:48 ¼ 8:96. The 99% spread due to part-to-part variation (PV) is found as 5.15p . 5:15  8:96 ¼ PV ¼ 46:15. Once the above calculations have been made, the overall measurement sysqffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi tem can be evaluated. 1. The total process standard deviation is found as t ¼ m2 þ p2 qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi pffiffiffiffiffiffiffiffi t ¼ m2 þ p2 ¼ ð0:44Þ2 þ ð8:96Þ2 ¼ 80:5 ¼ 8:97 2. Total variability (TV) is 5.15t . 5:15  8:97 ¼ 46:20 3. The percent R&R is 100  ðm =t Þ% 100 4. m 0:44 ¼ 4:91% % ¼ 100 8:97 t The number of distinct data categories that can be created with this measurement system is 1:41  ðPV=R&RÞ. 1:41  46:15 ¼ 28:67 ¼ 28 2:27 R&R studies for continuous data 337 Since the minimum number of categories is five, the analysis indicates that this measurement system is more than adequate for process analysis or process control. Gage R&R analysis using Minitab Minitab has a built-in capability to perform gage repeatability and reproducibility studies. To illustrate these capabilities, the previous analysis will be repeated using Minitab. To begin, the data must be rearranged into the format expected by Minitab (Figure 10.5). For reference purposes, columns C1^C4 contain the data in our original format and columns C5^C8 contain the same data in Minitab’s preferred format. Figure 10.5. Data formatted for Minitab input. Minitab offers two different methods for performing gage R&R studies: crossed and nested. Use gage R&R nested when each part can be measured by only one operator, as with destructive testing. Otherwise, choose gage R&R crossed. To do this, select Stat > Quality Tools > Gage R&R Study (Crossed) to reach the Minitab dialog box for our analysis (Figure 10.6). In addition to choosing whether the study is crossed or nested, Minitab also offers both the 338 MEASUREMENT SYSTEMS ANALYSIS Figure 10.6. Minitab gage R&R (crossed) dialog box. ANOVA and the X-bar and R methods. You must choose the ANOVA option to obtain a breakdown of reproducibility by operator and operator by part. If the ANOVA method is selected, Minitab still displays the X-bar and R charts so you won’t lose the information contained in the graphics. We will use ANOVA in this example. Note that the results of the calculations will differ slightly from those we obtained using the X-bar and R methods. There is an option in gage R&R to include the process tolerance. This will provide comparisons of gage variation with respect to the specifications in addition to the variability with respect to process variation. This is useful information if the gage is to be used to make product acceptance decisions. If the process is ‘‘capable’’ in the sense that the total variability is less than the tolerance, then any gage that meets the criteria for checking the process can also be used for product acceptance. However, if the process is not capable, then its output will need to be sorted and the gage used for sorting may need more discriminatory power than the gage used for process control. For example, a gage capable of 5 distinct data categories for the process may have 4 or fewer for the product. For the purposes of illustration, we entered a value of 40 in the process tolerance box in the Minitab options dialog box (Figure 10.7). Output Minitab produces copious output, including six separate graphs, multiple tables, etc. Much of the output is identical to what has been discussed earlier in this chapter and won’t be shown here. R&R studies for continuous data 339 Figure 10.7. Minitab gage R&R (crossed) options dialog box. Table 10.3 shows the analysis of variance for the R&R study. In the ANOVA the MS for repeatability (0.212) is used as the denominator or error term for calculating the F-ratio of the Operator*PartNum interaction; 0.269/0.212 = 1.27. The F-ratio for the Operator effect is found by using the Operator*PartNum interaction MS term as the denominator, 0.061/0.269 = 0.22. The F-ratios are used to compute the P values, which show the probability that the observed variation for the source row might be due to chance. By convention, a P value less than 0.05 is the critical value for deciding that a source of variation is ‘‘signifiTable 10.3. Two-way ANOVA table with interaction. Source DF SS MS F P PartNum 4 1301.18 325.294 1208.15 0 Operator 1 0.06 0.061 0.22 0.6602 Operator*PartNum 4 1.08 0.269 1.27 0.34317 Repeatability 10 2.12 0.212 Total 19 1304.43 340 MEASUREMENT SYSTEMS ANALYSIS cant,’’ i.e., greater than zero. For example, the P value for the PartNum row is 0, indicating that the part-to-part variation is almost certainly not zero. The P values for Operator (0.66) and the Operator*PartNum interaction (0.34) are greater than 0.05 so we conclude that the differences accounted for by these sources might be zero. If the Operator term was significant (P < 0.05) we would conclude that there were statistically significant differences between operators, prompting an investigation into underlying causes. If the interaction term was significant, we would conclude that one operator has obtained different results with some, but not all, parts. Minitab’s next output is shown in Table 10.4. This analysis has removed the interaction term from the model, thereby gaining 4 degrees of freedom for the error term and making the test more sensitive. In some cases this might identify a significant effect that was missed by the larger model, but for this example the conclusions are unchanged. Table 10.4. Two-way ANOVA table without interaction. Source DF SS MS F PartNum 4 1301.18 325.294 1426.73 Operator 1 0.06 0.061 0.27 Repeatability 14 3.19 0.228 Total 19 1304.43 P 0 0.6145 Minitab also decomposes the total variance into components, as shown in Table 10.5. The VarComp column shows the variance attributed to each source, while the % of VarComp shows the percentage of the total variance accounted for by each source. The analysis indicates that nearly all of the variation is between parts. The variance analysis shown in Table 10.5, while accurate, is not in original units. (Variances are the squares of measurements.) Technically, this is the correct way to analyze information on dispersion because variances are additive, while dispersion measurements expressed in original units are not. However, there is a natural interest in seeing an analysis of dispersion in the original units so Minitab provides this. Table 10.6 shows the spread attributable to the R&R studies for continuous data 341 Table 10.5. Components of variance analysis. Source VarComp % of VarComp Total gage R&R 0.228 0.28 Repeatability 0.228 0.28 Reproducibility 0 0 Operator 0 0 Part-to-Part 81.267 Total Variation 81.495 99.72 100 different sources. The StdDev column is the standard deviation, or the square root of the VarComp column in Table 10.5. The Study Var column shows the 99% confidence interval using the StdDev. The % Study Var column is the Study Var column divided by the total variation due to all sources. And the % Tolerance is the Study Var column divided by the tolerance. It is interesting that the % Tolerance column total is greater than 100%. This indicates that the measured process spread exceeds the tolerance. Although this isn’t a process capability analysis, the data do indicate a possible problem meeting tolerances. The information in Table 10.6 is presented graphically in Figure 10.8. Linearity Linearity can be determined by choosing parts or standards that cover all or most of the operating range of the measurement instrument. Bias is determined at each point in the range and a linear regression analysis is performed. Linearity is defined as the slope times the process variance or the slope times the tolerance, whichever is greater. A scatter diagram should also be plotted from the data. LINEARITY EXAMPLE The following example is taken from Measurement Systems Analysis, published by the Automotive Industry Action Group. 342 MEASUREMENT SYSTEMS ANALYSIS Table 10.6. Analysis of spreads. Source StdDev Study Var (5.15*SD) Total gage R&R 0.47749 2.4591 5.29 6.15 Repeatability 0.47749 2.4591 5.29 6.15 Reproducibility 0 0 0 0 Operator 0 0 0 0 Part-to-Part 9.0148 46.4262 Total Variation 9.02743 46.4913 % Study Var (%SV) 99.86 100 % Tolerance (SV/Toler) 116.07 116.23 Figure 10.8. Graphical analysis of components of variation.