Measurement Uncertainty without the Math

One of the barriers to be overcome in the wider application of measurement uncertainty (MU) to reportable values is the need to perform calculations that can be daunting to many in the analytical laboratory. Performing these calculations need not be so daunting, however. An earlier column showed how the basics of simple error budget calculations based on the Eurachem/CITAC guideline (QUAM) could be performed (1, 2). A new guide was published in 2015 extending MU concepts into decision making and setting target measurement uncertainty (3). Decision rules involving compliance with specification using guard band principles requiring MU have been published as well (4, 5). The importance of being able to calculate MU easily, therefore, is increasing. A new cloud-based software tool (Minitab DEVIZE) has become commercially available that makes the process of calculating MU simpler and much quicker (6). This article looks at some examples comparing the results from the traditional QUAM model with Monte Carlo Simulation (MCS) using this tool.

Monte Carlo simulation modeling
In the original QUAM guidance, MCS modeling was discussed briefly and a technical methodology given in Appendix E3 (2), which is quite mathematical intensively and employs Microsoft Excel.

The following principles behind MCS for MU, however, are simple:
 

• Identify the sources of error and their data shape (distribution).

• Decide how they are connected and related. (Both these steps are the same as the QUAM model.)

• Select, at random, a value from each of the error sources and calculate a value for the measurand in accordance with the defined interrelationships.

• Repeat the calculation N times (usually N is 10000 or more).

• Calculate the overall mean and standard deviation of the measurand from all N simulated values.

Example one: Making a standard solution of a certified reference material (CRM).
Suppose that one has a CRM with a certified value of 99.46% and a MU of 0.16 (k=2). A mass of 101.32 mg of this CRM is dissolved in water, transferred quantitatively to a 100.0 mL Grade A volumetric flask, and the solution is made up to the calibration mark with water. The measurand, the concentration in mg/L, is directly given by Equation 1.

Equation 1

However, the process involved in making the solution is subject to uncertainties in both the actual amount of the CRM and the volume of diluent. If these uncertainties are quantified, one will be able to estimate the overall measurement uncertainty of the solution. This process with its error sources is represented diagrammatically in Figure 1.

Figure 1: Process for making the CRM standard solution with identified sources of uncertainty. MU is measurement uncertainty. [Courtesy of the author]

The error budget constituting the overall uncertainty for the CRM concentration is constructed from the interrelation of the uncertainty sources represented by the individual bricks in Figure 2.

Figure 2: Error budget for the measurement uncertainty (MU) of the standard solution of the certified reference material (CRM) [Courtesy of author].

On the basis of Figure 1 and Figure 2, the inputs for the MCS model can be defined. The inputs may be fixed (e.g., the mass of the CRM and the coefficients of expansion of water and borosilicate glass). Variables are specified with the appropriate distributional model (e.g., the laboratory temperature is assumed to follow a uniform distribution of between 18 °C and 22 °C, while the repeatability of filling the volumetric flask to the calibration mark is normally distributed with a standard deviation of 0.02 mL). The completed MCS input model is shown in Figure 3.

Figure 3: Inputs to Minitab Devize Monte Carlo Simulation (MCS) model for the certified reference material (CRM) standard solution. [Courtesy of author]

Based upon these inputs, the relevant outputs and the MCS model can be defined as:

• The amount of CRM corrected for the uncertainty of the balance and the uncertainty of the CRM purity

• The volume correction of the flask itself with respect to temperature and calibration

• The final diluent volume taking into account the uncertainty of filling to the calibration mark

• The final concentration of the CRM solution.

These outputs to be calculated by the MCS model are shown in Figure 4. The interrelationships between the inputs and outputs are illustrated in Figure 5.

Figure 4: Outputs to be calculated using the Minitab Devize Monte Carlo Simulation (MCS) model for the certified reference material (CRM) standard solution.

Figure 5: Full Monte Carlo Simulation (MCS) model used in the MCS model for the certified reference material (CRM) standard solution. [Courtesy of author]

An advantage of the MCS model over the conventional QUAM method becomes apparent when the analytical system becomes more complex.

Now it is possible to run the model N times to generate the data, in this instance, 50,000 times, and see the output frequency distribution for CRM concentration (Figure 6). Even 50,000 iterations take only a few seconds in the cloud using DEVIZE.

Figure 6: Distribution of concentration for the certified reference material (CRM) standard solution from the Monte Carlo Simulation (MCS) model. [Courtesy of author]

When the QUAM calculation for MU is performed, the mean values are essentially identical, but the value for s is a little larger at 1.27. In this example, it was assumed in the MCS model that the specification for the final concentration had to lie within the range 990 to 1010 mg/L. Inspection of Figure 6 clearly indicates that the model predicts that a proportion (approximately 1.2%) of the concentration data will lie outside the top limit.

An advantage of the MCS model over the conventional QUAM method becomes apparent when the analytical system becomes more complex. To illustrate this advantage, the MCS model is applied to a complex standard analytical method (7) used in the determination of leachable cadmium from ceramic and glass cooking ware from a food safety perspective. The example was chosen not because of its detailed application but because of its complexity and the fact that the QUAM gives as worked example A5 (2)i thereby allowing a direct comparison between the QUAM methodology and that of the MCS model.

Example two: Leachable cadmium (Cd) determination by atomic absorption spectrometry (AAS)
In this example, more factors are involved than in the standard solution example. The procedure involves the following:

• A complex sample preparation involving acid concentration, time, and temperature factors

• A standard solution CRM preparation

• An AAS calibration for cadmium requiring a linear least squares regression model

• Measurement of the cadmium concentration in the sample solution

• Calculation of the measurand, which is the amount of cadmium released in mg dm-2.

Full details are given in the QUAM guideline, which requires seven pages to describe the process and calculations necessary to calculate the overall combined uncertainty (standard deviation) (see Equation 2).

Equation 2

A process overview is shown in Figure 7. The inputs required for MCS model for this example are shown in Figure 8.

Figure 7: Process overview of leachable cadmium determination by atomic absorption spectrometry (AAS). [Courtesy of author]

Figure 8: Inputs for the Monte Carlo Simulation (MCS) model for leachable cadmium determination by atomic absorption spectrometry (AAS). [Courtesy of author]

The following two assumptions were made:

• The area variation for non-circularity of the sample can be calculated from a triangular distribution of the diameter ±0.01 from the measurement error and assuming the area of an ellipse.

• As the absorbance value of the Cd sample was not given, it was calculated from the concentration in the QUAM and assigned a s value of 0.001 based on pooling the linear least squares (LLSQ) calibration replicates for 0.1 and 0.3 mg L-1 .

The output calculations for the MCS model and the interrelationships are shown in Figure 9 and Figure 10, respectively. Note that Cd conc is called C₀ and Cd released is called r in Figure 7 and the QUAM example.

Figure 9: Output calculations for the Monte Carlo Simulation (MCS) model for leachable cadmium (Cd) determination by atomic absorption spectrometry (AAS). [Courtesy of author]

Figure 10: Output interrelationships for the Monte Carlo Simulation (MCS) model for leachable cadmium determination by atomic absorption spectrometry (AAS). [Courtesy of author]

The MCS model was run 50,000 times, and the results for the mean Cd released (r) and its associated standard deviation calculated from Equation 3.

Equation 3

The excellent agreement between the MCS model and the QUAM model is shown in Figure 11. In addition, the DEVIZE software provides an option to show a sensitivity analysis. This effectively shows a ranking of the various factor effects (see Figure 12).

Figure 11: Leachable cadmium results from the Monte Carlo Simulation (MCS) model and comparison with QUAM values. [Courtesy of author]

Figure 12: Sensitivity analysis of the results from the leachable cadmium (Cd) determination by atomic absorption spectrometry (AAS). [Courtesy of author]

Hence, of the 12 factors listed, the three major contributors to the overall standard deviation of the measurand are the temperature factor, the intercept of the calibration curve, and the measuring cylinder. This conclusion is consistent with that from the uncertainty budget in Figure A5.8 from the QUAM.

Conclusion
In the original QUAM guidance, MCS modeling was discussed in a quite mathematical manner using an Excel spread sheet. In this column, the MCS tool from Minitab, DEVIZE, has been shown to produce similar results without the need for individual calculations in a QUAM uncertainty budget in a more user-friendly graphical format.

The more accessible and simpler approach of MCS might encourage the calculation of MUs in more complex analytical processes on a routine basis.

References
1. C. Burgess, Pharmaceutical Technology 37 (9) pp. 62-73 (September 2013).
2. S L R Ellison and A Williams (Eds), Eurachem/CITAC guide: Quantifying Uncertainty in Analytical Measurement, Third edition (2012), www.eurachem.org/index.php/publications/guides/quam
3. R. Bettencourt da Silva, A. Williams (Eds), Eurachem/CITAC Guide: Setting and Using Target Uncertainty in Chemical Measurement (1st ed., 2015), www.eurachem.org/index.php/publications/guides/gd-stmu
4. C. Burgess, Pharmaceutical Technology 37 (7) (July 2013).
5. C. Burgess, Pharmaceutical Technology 38 (10) (October 2014).
6. Minitab Devize, www.minitab.com/en-us/products/devize/?WT.srch=1&WT.mc_id=SE022648&url_id=160744988&gclid=CPWs9Ln88sMCFSOy2wodNQsAHQ.
7. British Standard BS 6748:1986, Specification for limits of metal release from ceramic ware, glassware, glass ceramic ware and vitreous enamel ware.

Article DetailsPharmaceutical Technology
Vol. 40, No. 2
Pages: 36–40

Citation:
When refering to this article, please cite it as C. Burgess, "Measurement Uncertainty without the Math," Pharmaceutical Technology 40 (2) 2016.