The Promise and Threat of Quality Risk Management

As a tool for the appropriate prioritization of resources, quality risk management (QRM) holds great promise for patients, government, and industry. Just as great, however, is the potential for QRM to degenerate into a non-value added exercise of identifying noncritical, improbable, low-risk scenarios indefinitely. The key to which way it goes is understanding a typical distribution of uncontrolled systems, that is, the Pareto Plot.

Jason J. Orloff

As a statistician, I have never been comfortable with subjective risk analysis. The process is fundamentally imperfect because it cannot anticipate the unknown. Furthermore, it lacks the rigor of actuarial risk analysis, which is beyond all but the most critical factors related to safety and efficacy. After conducting tremendous research and development to turn data into process knowledge, it seemed a disappointing end to boil all the information down to a human judgment call. Translating that effort into a subjective scale of 1 to 10 for severity, probability, and detectability left me wanting more than a notional approximation. However, after three days of discussion with the authors of the International Conference on Harmonization's quality guidelines, Q8, Q9, and Q10, at an ICH workshop in Washington, DC, last fall^*, I've gained a new understanding of risk assessment's value.

Risk assessment enables subject-matter experts to say to the best of their ability, "This is important, and that is not." Risk assessment is not a perfect tool by which analysts can anticipate all dangers—known and unknown—but it is valuable precisely because we cannot anticipate all danger. Risk assessment provides a framework within which to capture the knowledge upon which we have made risk-control decisions. Within this framework, learning can be fed back for capture and future review. The QRM process then enables management to establish priorities and move a project forward from the laboratory to manufacture with an understanding of, diligent control of, and conscious acceptance of risk.

Because risk assessment and control fundamentally rely on hypothesis, judgment, and expert opinion, it is open to endless attack and argument. The resolution of which must be the test of reason. The goal is to draw a line between the "vital few and trivial many" scenarios. This pattern was first recognized by Dr. Joseph M. Juran in 1951 when he coined the Pareto Concept of Quality, giving us a powerful conceptual and visual tool.

As an hypothetical example, Figure 1 shows a Pareto Plot of process parameters for the Sakura Tablet case study, which was last revised in March 2009 by the Japan National Institutes of Health. The figure illustrates the Pareto concept also known as the 80/20 rule. Many factors in a system are trivial and only a few factors are vital. As a rule of thumb, about 80% of the problems come from roughly 20% of the factors identified. The plot provides insight into several aspects of risk management.

Figure 1: The Pareto Plot of Risk Priority Number (RPN), where risk=severity * probability * detectability. The RPN informs the control strategy. As a process improves the probability or detectability changes, and the RPN adjusts accordingly.

First, it can be seen that the factors have been ranked by the magnitude of their risk. As one moves away from the origin, the effect of subsequent factors decays logarithmically. Some factors clearly cause more risk. "The level of effort... should be commensurate with the level of risk," according to the ICH Q9 guideline. These vital factors require the greater investment of resources.

The second point is that, following a logarithmic distribution, the identification of low risk, noncritical, and improbable factors extends infinitely while their risk approaches zero. But where do we draw the line between the vital few and the trivial many? Ultimately, that is a judgment call for negotiation between industry and regulatory authority. However it is a judgment call to be made by experts backed with an in-depth understanding of the underlying science and a common covenant to work on what is vital. The Pareto Plot does not provide hard lines of priority but can allow the negotiators to see the magnitude between what is vital, what is perhaps important, and what is neither.

Risk analysis provides a starting point for continual improvement. It is the best tool we have today for recognizing our imperfect understanding, prioritizing the work before us, and committing ourselves to the iterative process of improvement. In the pre-Q8, Q9, and Q10 world, the inability to admit that our understanding was incomplete and that our systems were imperfect meant a tremendous investment in maintaining a perception to the contrary and generated a culture of mutual distrust. If applied correctly, the post-Q10 world could enable industry to move beyond a philosophy where every batch of a product is expected to be a replicate of the validation runs.

Instead, we should set out with the intention to change our processes. We should be able to change them for the better. The key is to recognize risk assessment as an ongoing process that combines both objective science and subjective judgment to appropriately prioritize the allocation of resources. As the risk-control strategy is refined, risk is reduced, and new priorities emerge. Our effort must be applied to the vital few things that matter.

*Note: The ideas in this article were generated during the October 2010 "Integrated Implementation Training Workshops for ICH Q8, Q9 & Q10," which took place in Bethesda, MD.

Jason J. Orloff is a statistical and engineering consultant at PharmStat, 2000 Dempster Plaza, Evanston, IL 60202, tel. 847.424.1314, www.PharmStat.com.