Reducing the Risk in Risk Management

Has risk management gone mainstream? One might think so given the June 13, 2011 article in the Chicago Tribune on using the game of poker to train stockbrokers in the nature of risk (1).

Lynn D. Torbeck

In the pharmaceutical industry, the reception of International Conference on Harmonization Guidelines Q8, Q9, and Q10 has led to widespread discussion, if not implementation, of the Quality Trio guidelines, with ICH Q9 specifically addressing risk management.

While it is perhaps too early to say that risk management has gone mainstream, it has certainly had its fair share of news coverage of late. Due to a rare 9.0 magnitude earthquake and an unprecedented 45-ft tsunami that washed over the nuclear power plant in Fukushima, three out of six reactor cores melted down—the worst possible outcome. It will take decades and billions of yen to clean up.

The world was stunned by that event. If a highly-regulated industry run by a detail-oriented and meticulous culture such as Japan made this mistake, what chance do the rest of us have at risk management? Which raises the questions, what are the risks in risk management and can we reduce them? First it is important to define terms.

In ICH Q9, risk to a patient equals probability times severity. What is probability? We can do little better than this observation from Bernoulli in 1713:

"For it should be presumed that a particular thing will occur or not occur in the future as many times as it has been observed, in similar circumstances, to have occurred or not occurred in the past." —D. Bernouilli

This definition has been refined since then. Here are the three common definitions given:

• An undefined subjective measure of belief from personal estimate, gut feeling or intuition

• The observed frequency of outcomes, as a fraction, percentage or a proportion

• A theoretical characteristic of the situation as determined by the limiting frequency of an infinite random series.

I note with concern and dismay that ICH Q9 does not include probability in the list of definitions. Perhaps the authors assume that everyone has the same universal understanding of this complex topic, so a definition is not needed.

Severity is the second component of risk and is defined in ICH Q9 as "a measure of the possible consequences of a hazard" (i.e., a meltdown). Now, let's turn to key steps for mitigating the risk in risk management.

Include top management. They can delegate work but not the responsibility of a mission critical process. Note that risk management is a not an event but an ongoing reevaluation. Management must fight the danger of complacency, distraction, and stagnation. Risk analysis cannot simply become window dressing for the regulators. Risk reduction is costly, and management reflects the importance of the task with support, finances, and resources.

At least one person in the risk team must understand probability well enough to teach it to the other members. Few people have studied probability and even fewer have practical experience of using probability. Real life probability estimation is much more difficult than textbook exercises. Yet, probability theory is the core of determining risk.

Risk assessment cannot be seen as a check-the-box activity; it is a serious issue and requires serious effort and commitment. The success of the assessment is a direct result of the skill, education, and experience of the company's team members. Technical experts are absolutely necessary even if external experts must be called in. All stakeholders must work to agreement.

Industry in general is less interested in the risk of an everyday occurrence, such as the risk of failing the moisture test for the next production run. What is more important is estimating risks such as a recall or another Heparin type tragedy. The difficulty in doing so is compounded by the lack of information and actionable data. In the absence of historical data, we are left with a gut-feeling, which can often underestimate the risk. The best experts must be consulted.

"For every complex and difficult problem, there is a solution that is simple, intuitive, easy to understand—and completely wrong." —Anonymous

Estimating probability and severity on scales of low, medium, and high (or on a scale of 0–4) are so simplistic as to be almost worthless; like using third grade English to study a college subject. Formal training in probability is needed to advance beyond this simplistic approach.

Finally, focus on the patient, build for the future, collect relevant data, and store it for easy access and analysis. Expand the knowledge base of the team and the company. Learn from others and learn advanced probability theory. Continue to refine and improve. Only through our best efforts can industry say with confidence that it understands the risks facing patients.

Lynn D. Torbeck is a statistician at Torbeck and Assoc., 2000 Dempster Plaza, Evanston, IL 60202, tel. 847.424.1314, Lynn@Torbeck.org, www.torbeck.org.

Reference

1. N. Popper, Chicago Tribune, June 13, 2011.