OOS, OOT, OOC, and OOSC

The pharmaceutical industry is not immune to rampant proliferation of acronyms. To alleviate confusion about being out-of-control (OOC), this column proposes concepts and definitions for the industry.

Lynn D. Torbeck

No plant manager wants to hear the words, "Your process is out-of-control." The process was probably designed, developed, nursed, nurtured, fine-tuned, tested, and coddled until it met the definition of a validated process (1). Now, someone who was never involved in process development declares it is as OOC. Not a welcome message and it begs the question, "out-of-control for what?" Modifiers and definitions are needed.

There are several ways that the process can be described as OOC. There could be equipment failures or changes in the raw materials used. The process could be financially OOC (i.e., losing money), or out-of-regulatory-control with cGMP violations. Perhaps the operators are creating excessive deviations and confusion by not following the standard operating procedures because of a dysfunctional work culture; managerial malfeasance has resulted in the demise of several companies in the past. If we say it is OOC, we are obligated to specify how.

The most notorious lack of control is being out-of-specification (OOS, 2). An OOS situation exists when a reportable result collected at a single point in time exceeds a predetermined specification (3).

This event precipitates a cascade of activity as described in the OOS guidance. This has been discussed extensively for years and is well understood by the industry.

Out-of-trend

A more recent concern is being out-of-trend (OOT). An OOT situation exists when one or more reportable results, collected over a period of time, are compared with a historical data set or with a statistical model and determined to be different from a practical significance standpoint.

The cause behind OOT occurrences can take several forms. The process could experience a sudden change in the average or have an outlying data point. The standard deviation of the data could increase or decrease. The general direction of the data could be gradually upward or downward, which is often called process drift. Finally, the process could cycle up and down. Cycling can occur if the operators over control for the critical process parameters (also known as independent variables or factors) and actually induce additional variability into the critical quality attributes (also known as dependent variables or responses). Note that OOT is usually perceived as a negative event but it could be a positive one. If the variability decreases or if the average moves towards the target, that would be a beneficial trend. If stability data does not change over the course of the study, that would be a neutral trend. These changes need not reach the level of statistical significance to raise an alert and initiate a response.

This concept is supported by the recent Supreme Court case for Matrixx (4, see page 118 for the full report). The court held that a change or difference does not have to be statistically significant to warrant a preliminary examination of the data or even an investigation. Thus, OOT is seen as a practical significance issue and not a statistical one.

The comparison of practical significance and statistical significance was discussed in the March 2010 Statistical Solutions column on the role of statistical significance tests (5). The goal of a statistical significance test is to prevent us from deciding something is important when it is just due to chance. In fact, if a difference or change is not of any practical importance or significance, being statistically significant is frankly not of much interest.

Western Electric 8

The application of statistical concepts to manufacturing in the United States is often traced to 1924 when Dr. Walter Shewhart wrote his famous memo describing for the first time the statistical control chart (or Shewhart chart) while he was working at Western Electric in Cicero, Illinois.

At that time, the term "control" was innocuous enough and didn't raise questions when he and others spoke of control limits and of being OOC. In the context of statistical control charts, if the pattern of values exceeds one or more the Western Electric 8 (WE8) rules, the data set is said to be "out-of-control" (6). (Note that most companies only use one or two, and no more than three, of the eight rules to avoid over-control.) But we need an additional modifier here as well. It is clearer to say the data set or process is "out-of-statistical-control" (OOSC).

This has been the case since 1924, and there is no reason to change such an old habit now. If someone speaks of being out-of-control in the context of a statistical control chart we assume it is out-of-statistical-control. It should be pointed out that it is possible to be out-of-statistical-control and still always meet the specifications.

Not all processes, for example biotechnology, will easily exist in a consistent state of statistical control. Such a situation is often called being in a "state of engineering control." In the chemical and refining industries, engineering control is more narrowly defined as the application of proportional, integral, or derivative tuners for process control.

cGMP perspective

That said, the cGMPs still call for companies to have firm control of their processes, statistical or otherwise. Being out-of-statistical-control, but within specifications is not acceptable from a cGMP perspective because there is a higher probability of manufacturing defective product. Companies are required to show continuing due diligence, provide data and analysis to prove that the process is being brought into a state of statistical control, or provide a scientific and statistical study to explain why it cannot be controlled.

To summarize, for control charts, practical significance and OOT go together, and statistical significance and statistical control go together. There are four possible combinations, the control charts for which can be viewed together at PharmTech.com/ controlcharts:

1. Not OOT and not OOSC: the ideal situation (see Figure 1)

Figure 1: Not out-of-trend and not out-of-statistical-control.

2. Not OOT but OOSC: cannot yet see a practical change, but failed one or more WE8 rules (see Figure 2)

Figure 2: Out-of-statistical-control but not out-of-trend. Fails Western Electric rule number 2 (7 points on the same side of the center line).

3. OOT but not OOSC: can see a practical change that is not yet statistically significant (see Figure 3)

Figure 3: Not out-of-statistical-control but is out-of-trend. Passes all 8 Western Electric rules.

4. OOT and OOSC: can see a practical change that is statistically significant (see Figure 4).

Figure 4: Out-of-trend and out-of-statistical-control.

Finally, any of these four combinations can be OOS or not OOS as the specification concept is independent of trend (practical significance) and statistical significance.

Lynn D. Torbeck is a statistician at Torbeck and Assoc., 2000 Dempster Plaza, Evanston, IL 60202, tel. 847.424.1314, Lynn@Torbeck.org, www.torbeck.org.

References

1. FDA, Guidance for Industry: Process Validation: General Principles and Practices (Rockville, MD, Nov. 2008)

2. FDA, Guidance for Industry: Investigating Out-of-Specification (OOS) Test Results for Pharmaceutical Production (Rockville, MD, Oct. 2006)

3. L.D. Torbeck, Pharm. Technol. 34 (3), 21–23 (1999)

4. Matrixx Initiatives, Inc. v. Siracusano, 131 S. Ct. 1309, (2011)

5. L.D. Torbeck, Pharm. Technol. 34 (3), 76 (2010).