Demonstrating a State of Control

Operators of analytical laboratories in a regulated environment are always concerned about being in a demonstrable state of control. One important aspect of this ‘state of control’ is the proper use of certified reference materials in the calibration of instruments and the performance of procedures. This article looks at a structured approach to decision rules for user measurements with certified reference materials to guard against bias.

Decision rules
Asking if one’s systems and procedures are in a state of control might seem to be an easily answerable question, especially if one used a traceable certified reference material (CRM) from the National Institute for Science and Technology (NIST), for example. After all, the CRM comes with a certified value together with an expanded measurement uncertainty doesn’t it? Actually, the methodology required to answer this question is not as well known as it should be. The question becomes: How do you construct the acceptance limits for user measurements of a CRM?

Let’s look first at an example of an instrument calibration for absorbance accuracy of a spectrophotometer. The following is a quotation from a NIST CRM, which NIST calls a standard reference material (SRM), certificate for glass filters used for the calibration of the absorbance scale of a spectrophotometer (1): “An acceptable level of agreement between the user’s measurements and the certified value and its expanded uncertainty overlaps any part of the user’s tolerance band defined by the measured mean and the user-defined level of acceptability.”

NIST appears to propose that one needs just two elements to construct an acceptable level of agreement with the certified value and the user’s requirements. The first part is simple. Take the value of the certified standard value from the certificate, x_o, and its expanded uncertainty U. Hence the uncertainty interval at 95.45% confidence (k=2) is x_o± U.

NIST then introduces the concept of user acceptability based upon the measurement capability of the instrument or the laboratory, A. So if one simply adds the SRM expanded uncertainty to the spectrophotometer manufacturer’s tolerance to the measurement capability, A, then these form the acceptance limits for satisfactory calibration or method performance, i.e., x_o ± (U + A).

In addition, this certificate states that the measurement process should be “to demonstrate that a user’s measurements are traceable within acceptable limits to the accuracy defined by SRM 1930 the user must first determine the required tolerances or acceptable uncertainty for the application in question. It is recommended that a number of replicate measurements be made for each filter and wavelength, with removal and replacement of the filter between replicate measurements.”

Assume that, at a particular wavelength, the certified value is 0.4326 absorbance units and the uncertainty is ±0.0029 absorbance units (U). Also assume that the mean value measured on the user’s spectrometer is 0.4251 and that the spectrometer has a specification for absorbance accuracy of ±0.0035 absorbance units (A).

Therefore x_o ± (U + A) becomes the acceptance limits (AL)0.4326 ± (0.0029 + 0.0035) or 0.4262 to 04390. As the mean measured value found was 0.4251, which falls outside this range, the spectrometer is considered inaccurate. One can clearly confirm the effectiveness of this decision rule by running a Monte Carlo simulation (MCS)model using the SRM values and setting the acceptance limits as above. The result is shown in Figure 1.

Figure 1: Monte Carlo simulation (MCS) model in Minitab Devize for standard reference materials certified value of 0.4326 absorbance units and s=0.075 (N=50,000) with acceptance limits (U+A). LSL is lower specification limit. USL is upper specification limit. [Courtesy of author]

The simple approach presented, however, is not sufficient, especially as the certificate method mentions replicate measurements and a mean value. Therefore, one needs to include the metrological uncertainty of the user’s measurement in the acceptance limits.

The SRM certificate references NIST Special Publication 829 (2) as the source document for acceptance limits and, in section E3a, the statistically correct calculation for the AL required is defined as Equation 1:

Equation 1

This is because the above simpler approach does not take into account the metrological uncertainty of the mean measured value, which depends on the standard error of the mean of n measurements. If one takes six measurements and calculates the standard deviation, s, of 0.0015 absorbance units (Equation 2),

Equation 2

then the ALs become AL = ±0.0016 + (U + A) = ±0.0080 giving an acceptance range of 0.4246 to 0.4406.
After taking into account the metrological uncertainty, the mean measured value, 0.4251, now falls within this range, so the spectrometer is considered sufficiently accurate. This enhanced decision rule is confirmed once again by MCS modeling Figure 2.

Figure 2: Monte Carlo simulation (MCS) model in Minitab Devize for standard reference materials certified value of 0.4326 absorbance units and s=0.075 (N=50,000) with acceptance limits based on metrological uncertainty+(U+A). [Courtesy of author]

For those readers wishing for more statistical background and excellent discourse, please read Churchill Eisenhart’s classic paper of 1963 on “Realistic evaluation of precision and accuracy of instrument calibration systems” (3), which is freely available from the NIST archive as is NIST Special Publication 829.

Standard control
If this acceptance limit approach works for calibration, will it work for standards and methods? Yes, it will if one has a certified and traceable CRM with a measurement uncertainty.

Assume one has a primary CRM with a certified value of 99.46%, with an expanded uncertainty of 0.15 (i.e., x_o  Â± U . If one has generated a working standard that has been assayed six times against this primary standard, and finds a mean value of 99.23 with a sample standard deviation of 0.062, from an analytical viewpoint, the laboratory has decided that it is acceptable to have a bias between the standards of ±0.05%.

Using the full equation for the acceptance range, ∆_c, and including the metrological uncertainty, this becomes Equation 3:

The difference between the certified value and the mean value is

therefore, at 95% confidence, there is no evidence that the working standard does not lie within the combined acceptance limits set by the laboratory requirement, the metrological uncertainty, and the measurement uncertainty of the CRM.

Using MCS (Figure 3), it is clear that the combined acceptance limits encompass the large majority of the distribution of the CRM values and that less than 0.03% of the values is likely to lie outside these limits.

Figure 3: Monte Carlo simulation (MCS) model in Minitab Devize for certified reference materials certified value of 99.46% and s=0.075 (N=50,000) with acceptance limits based on metrological uncertainty+(U+A). [Courtesy of author]

Conclusion
The method for calculating acceptance limits for calibrations based on user measurements of certified reference materials has been described and the NIST basis for it discussed. The approach has been confirmed using Monte Carlo simulations. If laboratories adopt this approach, they will have a sound statistical basis for deciding if an instrument or method is ‘fit for purpose’ or not.

References
1. NIST, SRM 1930 Glass Filters Certificate.
2. D. Becker et al., “Use of NIST Standard Reference Materials for Decisions on Performance of Analytical Chemical Methods and Laboratories,” NIST Special Publication 829 (1992).
3. C. Eisenhart, J. Res. Natl. Bur. Stand., 67C, 161 (1963).

Article DetailsPharmaceutical Technology
Vol. 40, No. 6
Pages: 47–48, 54

Citation
When referring to this article, please cite it as C. Burgess, "Demonstrating a State of Control," Pharmaceutical Technology 40 (6) 2016.