Methods for Identifying Out-of-Trend Results in Ongoing Stability Data

It is important to distinguish between out-of- specification (OOS) and out-of-trend (OOT) results in stability studies. The authors discuss three methods for identification of OOT results—the regression-control-chart method, the by-time-point method, and the slope-control-chart method—and further compare the z-score method and the tolerance interval in OOT analysis. The results highlight the need for issuing a regulatory confirmed guideline for identification of OOT results for ongoing stability data.

The two terms out-of-trend (OOT) and out-of-specification (OOS) results are in many cases confused by pharmaceutical companies and regulatory agencies. OOT results are defined as a stability result that does not follow the expected trend, either in comparison with other stability batches or with respect to previous results collected during a stability study (1). OOT results are not necessarily OOS, but they do not look like a typical data point. Although OOT results are a serious problem, the scientific literature and regulatory guidelines do not fully address this issue.

According to FDA's Guidance for Industry: Investigating Out-Of-Specification (OOS) Test Results for Pharmaceutical Production (2), OOT results should be limited and scientifically justified. The guideline, however, does not define the process for identification of OOT results in stability data. The CMC Statistics and Stability Expert Teams of the Pharmaceutical Research and Manufacturers of America made an attempt to address this problem by suggesting several statistical methods for the identification of OOT results (3). The proposed statistical methods were redesigned and analyzed for the purposes of this study.

The aim of this study was to make a statistical confirmation of the statistical methods, which will prove their functionality in identification of OOT results in ongoing stability data within a batch or data among batches. In addition, a comparison was made between the z-score method and the tolerance interval (TI) in terms of defining the limits for identification of the present OOT result.

Materials and methods

For the purpose of this study, data from ongoing stability studies of a final drug product with a shelf life of 36 months were used. The ongoing studies were conducted on 10 batches of Product X. Product X is manufactured in a tablet dosage form and consisted of one active substance with defined strength of 10 mg and packaged in a primary aluminium–polyvinyl chloride (Al–PVC) blister and a secondary package. The ongoing studies were conducted for 36 months in stability chambers at a constant temperature of 25 °C ± 2 °C and relative humidity of 60% ± 5% in accordance with the ICH guideline Q1A(R2) (4).

The reported data are single data results for the assay attribute, calculated as a percentage of the declared active substance concentration. The assay attribute was analyzed in accordance to the validated internal method of the manufacturer at the time points of 0, 3, 6, 9, 12, 18, 24, and 36 months in all of the tested batches.

The first nine batches were used as historical data for the purposes of the by-time-point method and the slope- control-chart-method in addition to which the tenth batch was compared and analyzed. The historical data were used to define the limits for identification of present OOT results in the tenth batch; the regression-control-chart-method analysis was conducted only on the tenth batch.

In addition, simulated data also were implemented. The simulated data were comprised of eight test time points for each of the 10 simulated batches. Unlike the experiment, in the simulation, the 10 batches were tested using the regression-control-chart method. In the by-time-point method and the slope-control-chart method, however, the historical data of the real batches were used to individually analyze the 10 randomly generated batches.

Regression-control-chart-method. The regression-control-chart method is used to compare the results within the batch and detect present OOT results. For the purpose of this method, the tenth batch was examined. Several least-square regression lines were fit to the suitable data (5). The first regression line was constructed from the three results for assay at the first three time points (0, 3, and 6 months). With extrapolation of that regression line, the expected values for Y and the Y residuals were calculated (see Figure 1) (6). The procedure was then repeated by gradually adding all the other consecutive time points.

Figure 1: Least-square line method for the time period of 0–9 months. (ALL FIGURES COURTESY OF AUTHORS)

The next step was to calculate the mean and standard deviation (σ) of the Y residuals of the regression line. As a result, a sum of five means and standard deviations corresponding to the time periods (0–6, 0–9, 0–12, 0–18, and 0–24 months) were constructed. To identify the present OOT result, the z-score test was used to calculate the z-value for each Y residual at each time point. The z-value is based on the means and standard deviations of the defined time periods (see Table I). The z-value was limited to -3 < z <+3, where 99.73% of the future results are expected to enter the interval within these limits.

Table I: Regression-control chart for the tenth batch.

Testing the precision of the TI in comparison to the z-score test, the TI for the same five time periods was calculated according to the suitable equation with defined certainty (α=0.027) and confidence (γ=0.95) (7). To compare the two methods for each TI value, a corresponding z-value was calculated (see Table II).

Table II: The regression-control chart method limits from the tolerance interval (TI) and z-values for the corresponding TI values.

By-time-point method. The by-time-point method is used to determine whether a result is within expectations on the basis of experiences from other batches measured at the same stability time point. To minimize the α and β error (6), batches that comprise the historical data (Batches 1 to 9) were individually tested for present OOT results. In addition, calculations for the mean and standard deviation of the values for the tested attribute were made from the historical data for each time point individually (see Figure 2).

Figure 2: Representation of the historical data with the use of the by-time-point method.

To identify the present OOT result, the z-score test was used to calculate the z-value for each value of Y (the assay result) of the tenth batch. Analysis was made at each time point using the mean and the standard deviation from the historical data corresponding to the tested time point (see Table III). The value of z was again limited to -3 < z < +3.

Table III: Mean value and standard deviation of the historical data and z-value for Batch 10.

The TI was also calculated with α=0.027 and γ=0.95 for each time point. To compare the two methods for each TI value, a corresponding z-value was calculated (see Table IV).

Table IV: The by-time-point method limits from the tolerance interval (TI) and z-values for the corresponding TI values.

Slope-control-chart method. The slope-control-chart method is commonly used when it is necessary to compare the results between several tested batches or between the currently tested batch and other batches from the historical database. For the purpose of this experiment, control limits were defined from the historical data and then used to test Batch 10 for the presence of OOT results. For each time point, a least-squares regression line that includes all data up to that time point was constructed. The regression line was constructed from the data of 0, 3, and 6 months, and the slope of that line was calculated. The procedure was repeated until the last tested time point of the ongoing stability study. The mean and standard deviation of the slopes for the given time intervals were calculated. The slope-control chart was then constructed (see Table V).

Table V: Slope-control chart and the z-values for Batch 10.

To identify the present OOT result, the z-score test was used to calculate the z value for the slope at each time period of Batch 10. The value of z was limited to -2 < z <+2, provided that 95.45% of the future values will enter the interval of these limits. Unlike the previous two methods for identification of OOT results, where the absolute value of the result was analyzed, in this method, the authors analyzed the values for the slope. Because small changes in the slope value cause a significant change in the regression line (and in this case it would mean the kinetics of degradation), for this model, narrower limits for the z-value were chosen.

Additionally, the TI also was calculated from the slope values at each time interval. In this case, however, the TI was calculated with defined α=0.045 and γ=0.95, for proper comparison with the limits determined by the z-score test (7). In order to compare the two methods for each TI value, a corresponding z-value was calculated (see Table VI).

Table VI: The slope-control chart method limits from tolerance intervals (TI) and z values for the corresponding TI values.

Results and discussion

The simulation gave the same results as the experiment. Therefore, this study was focused only on elaborating the experiment on its own. It must be noted that the obtained limits in this experiment will only apply to this final product in the given dosage form, strength, and primary and secondary packaging.

With the use of the regression-control-chart method, three OOT results were detected in the time points of 9, 18, and 36 months (see Table I). The result in the 9-month time point deviates approximately 0.19%. The result in the 18-month time point deviated by 2%, and the result in the 36-month time point deviated by 3% from the expected value according to the regression line. Taking into consideration that in the time point of 9 months, the regression line was constructed of only three points; the result was falsely identified as an OOT result, and it was not investigated further. In terms of the control limits, the z-score test provides a constant limit of 3σ standard deviations throughout the whole regression line unlike the TI that limits the results within 15σ for the time point of 6 months to 5σ for the time interval of 24 months.

The by-time-point method identified two OOT results in the time points of 18 and 24 months (see Table III). Compared with the results from the historical data for the appropriate time points, the results of the tested batch deviated approximately by 2%. According to the z-value the results deviated 5σ from the average value of the historical data at those time points. In this method, the z-score test provided limits of 3σ, and the TI constant limit of 5.4σ (see Table IV).

The slope-control-chart method analysis resulted in identifying two OOT results (see Table V). The present OOT result for the time point of 18 months deviated 2.05σ and for 24 months deviated 5.97σ from the average value for the slope, according to the z-value. The TI, on the other hand, provided limits of 3.6σ, which were wider than the limits comprised from the z-score test. Ultimately, each manufacturer is responsible for choosing its own control limits, suitable to the analysis of the corresponding final product with its own strength and primary and secondary packaging.

This study provided a thorough explanation of the proposed methods for identification of OOT results. The methods were redesigned and improved to achieve proper evaluation of the tested stability data. The experiment revealed the positive and negative features of the proposed methods, thereby defining their appropriate use.

The regression-control-chart method allowed analysis of the results within a batch, which was achieved by comparing the absolute values of the results and the predicted values that were obtained by extrapolation of the regression line. The main disadvantage of this method was the necessity of having results for each time point due to the fact that the construction of the regression line was based on gradually adding the values in each subsequent time point. For the time period of 0–9 months, the regression line was constructed only from three points; therefore, the calculations for the predicted values were prone to an error. This method, however, is suitable for identification of present OOT results in cases where there is no historical stability data.

The by-time-point method provides analysis of the results in each time point individually, and no assumptions about the shape of the degradation curve are needed. The main advantage of the method was that the absence of having a result in any time point did not affect the analysis of the previous or next time point result. In conclusion, this method is more appropriate for analysis of the results of the first four time points. The main disadvantage is that a large history of data is preferred for proper use of this method. This method, therefore, is not suitable for analysis of ongoing stability data at the beginning of the production of the final product.

By measuring the slope of the regression line, the slope-control-chart method provided analysis of each time point individually by analyzing the influence of each time-point result on the regression line. Any small change in the value of the tested attribute from point to point was precisely recorded in the slope value of each time point. It is advised, therefore, to establish slightly narrower limits in this method in comparison to the first two methods. The main disadvantage of this method is that if the test of the attribute were omitted in any time point for various reasons, the limits of that time point may not be appropriate.

In terms of the limits, the z-score method produced limits that remain constant around all of the time points in all of the methods for OOT results identification. The dependence of the TI on the number of samples included in the calculation was a major drawback for its use in the methods for OOT results identification. The TI requires a large number of results, which is difficult to meet in everyday practice within the pharmaceutical industry. The freedom of choosing the z-score limits remains a decision of each manufacturer, and it is determined according to its own requirements.

Conclusion

The pharmaceutical industry still lacks having a proper guideline for the identification of present OOT results among ongoing stability data. As a result, many pharmaceutical companies are not harmonized in the way they conduct this type of analysis.

In this study, three methods for identification of OOT results in ongoing stability data were proposed: the regression-control-chart method, the by-time-point method, and the slope-control-chart method. To obtain more accurate identification of existing OOT results, simultaneous use of all three methods is advised, which will result in getting a visual image of the results of the analyzed batches. The use of the z-score method for defining the limits for the OOT results is preferred. Lastly, the study highlighted the necessity of issuing a regulatory confirmed guideline for identification of OOT results within ongoing stability data.

Adrijana Torbovska* is an analyst in the Quality Control Department of ReplekPharm, Kozle 188, 1000 Skopje, Macedonia, adrijana.torbovska@replek.com.mk Suzana Trajkovic-Jolevska, PhD, is a professor in the Drug Quality Control Department, Faculty of Pharmacy, Ss Cyril and Methodius University, Skopje, Macedonia.

* To whom all correspondence should be addressed.

References

1. MHRA, Guidance for Out Of Secification Investigation, online presentation, (2010), www.mhra.gov.uk/home/groups/comms-con/documents/websiteresources/con088215.pdf, accessed May 13, 2013.

2. FDA, Guidance for Industry: Investigating Out-Of-Specification (OOS) Test Results for Pharmaceutical Production (Rockville, MD, 2006).

3. PhRMA CMC Statistics and Stability Expert Teams, Pharm. Technol. 27 (4), 38-52 (2003).

4. ICH, Q1A (R2) Stability Testing of New Drug Substances and Products (Feb. 2003).

5. P.Rowe, Essential Statistics for the Pharmaceutical Sciences (John Wiley & Sons, West Sussex, UK, 2007), pp. 169-194.

6. W.W. Daniel, Biostatistics- A Foundation for Analysis in the Health Sciences (John Wiley & Sons, Hoboken, NJ, 9th ed., 2009), pp. 93-131, 215-304, and 409-484.

7. S. Bolton and C. Bon, Pharmaceutical Statistics–Practical and Clinical Applications (Marcel Dekker Inc., Monticello, NY, Vol. 135, 4th ed., 2004), pp. 96-150.