Defining Quality Metrics to Ease Drug Shortages

There are two topics in today’s pharmaceutical landscape commanding the attention of both the industry and the regulators: drug shortages and quality metrics. It is increasingly difficult to discuss one of these topics without discussing the other. Establishing, maintaining, and interpreting quality metrics to measure the suitability of pharmaceutical products and the capability of the manufacturer to provide these products consistently and without delay has become a high priority for the industry and FDA.

Preventing drug shortages
The first piece of this puzzle emerged in 2012 when Congress passed the Food Drug Administration Safety and Innovation Act (FDASIA) enhancing FDA’s capability to proactively react to, prevent, and alleviate drug shortages. This direction is codified in the language contained in Title VII—Drug Supply Chain and Title X—Drug Shortages. Specifically, Title VII Section 705 of the Act states FDA “shall inspect establishments described in paragraph [1] that are engaged in the manufacture, preparation, propagation, compounding, or processing of a drug or drugs (referred to in this subsection as ‘drug establishments’) in accordance with a risk-based schedule established by the Secretary” (1). Additionally, this section also describes the risk factors to be considered in establishing the inspection schedule. The last risk factor listed in this section is “Any other criteria deemed necessary and appropriate by the Secretary for purposes of allocating inspection resources.” Section 706 of the same act allows FDA to request certain information from companies in advance of or in lieu of inspections by stating “Any records or other information that the Secretary may inspect under this section from a person that owns or operates an establishment that is engaged in the manufacture, preparation, propagation, compounding, or processing of a drug shall, upon the request of the Secretary, be provided to the Secretary by such person, in advance of or in lieu of an inspection…”

The next piece of this puzzle is found in Title X section 506C-1 (Annual Reporting on Drug Shortages). Title X section 506C-1 requires FDA to annually provide Congress “a report on drug shortages…”

The third piece of the puzzle fell into place in a February 12, 2013 Federal Register Notice. FDA asked the industry to “assist the Food and Drug Administration in drafting a strategic plan on drug shortages as required by the Food and Drug Administration Safety and Innovation Act…” This notice asked a series of thought-provoking questions including “What metrics do manufacturers currently use to monitor production quality?” and “How frequently would such metrics need to be updated to be meaningful?” (2).

The last piece of the puzzle revolves around the industry reaction to this information. Many trade organizations responded to the questions in the Federal Register. Some prepared white papers while others held meetings to discuss the issue with their members. The general consensus, and the easy part from this activity, was that industry needed to actively engage with the agency in defining suitable quality metrics to provide information to the agency supporting their efforts to eliminate drug shortages and determine the appropriate quality metrics to be used in establishing a risk-based approach to inspections.

PDA proposes metrics
Companies currently use a variety of metrics to measure performance including, but not limited to, corrective action and preventive action (CAPA), out of specification (OOS), batch rejection rate, complaints, field alerts, recalls, batch yield, training effectiveness, and environmental monitoring excursions. The goal is to review and define which of these metrics can be used to measure performance based on product quality and suitability as opposed to performance based on compliance standards. The hard part is trying to define them so they are not subject to interpretation: they are simple but comprehensive and applicable to traditional, biotech, virtual, contract manufacturers, drug substance manufacturers, etc. The industry needs to ensure the metrics chosen provide meaningful data while avoiding unintended consequences. The metrics need to be simple so they do not divert or dilute a company’s resources from daily activities associated with producing and delivering high quality medicines.

So, what might these metrics be? The Parenteral Drug Association (PDA) held their Annual Meeting in San Antonio in April 2014 (3). During this meeting, there was an open member session where PDA proposed seven potential metrics for consideration. For each metric proposed, there was a definition to consider, a calculation, a recommendation on how to report the metric to FDA, and a discussion of potential unintended consequences that might result from the implementation of the metric. The metrics proposed by PDA were:

• Complaints by product

• Batch reject rate by product

• Batch reject rate by site

• OOS rate by product

• OOS rate by site

• Recalls by product

• Recalls by site.
  

The PDA recommendations included some guidelines and principles clarifying the reporting of these metrics. Product metrics were defined as items with the same formula regardless of their final packaging configuration. Under this model, drug substance manufacturers would consider each drug substance batch as a product batch. Site metrics would be a compilation of the product metrics for the site and would combine drug-product and drug-substance metrics for sites that manufacturer both. The recommendation for reporting was to collect the data monthly and report annually, perhaps using annual product reviews or annual report dates to determine the cycle.

Quality culture
The metrics proposed by PDA seem reasonable but there is a missing piece to this puzzle. The most important metric used to determine a company’s well-being is a measure of their quality culture. The culture of a company dictates the veracity of their metrics. Achieving a quality culture requires management and employees to establish an environment where responsibility, accountability, and reliability are paramount, and to understand the role each person performs in delivering a high-quality product to the customer and sustaining that performance on a continual basis. Management must educate employees and provide the tools and environment where they can perform their functions in an atmosphere that encourages excellence and continuous improvement.

The trouble with using quality culture as a metric is determining how to measure it. An unhealthy quality culture is easy to identify. People in a poor culture do not understand their job and its importance to the business. They often appear stressed, and they hide their mistakes or blame others for their errors. There is no evidence of teamwork. People work in silos and rarely, if ever, seek input or advice from others. The metrics that could potentially be used to measure a poor culture include a large employee turnover, an overabundance of deviations attributed to human error, and lack of pride in the performance of their jobs. In contrast, a robust, healthy quality culture can be evidenced by alignment of goals between quality and operations, self-sustained work teams that focus on continual improvement, and employees who incorporate quality into their jobs on a daily basis. They are not afraid to speak up and offer suggestions for improvement to their colleagues. People understand the importance of their jobs and respect each other and their management. This culture welcomes inspections and views these inspections as another tool to use in their continual improvement initiatives.

The establishment of simple quality metrics that not only measure the quality of the product but also reflect the quality culture of an organization is required to assist FDA in establishing a risk-based audit program.

References
1. FDASIA.
2. FDA, Federal Register, Vol. 78, No. 29 (Feb. 12, 2013) 9928-9929.
3. PDA Annual Meeting, Apr. 7-11, 2014, Session titled Quality Metrics Update and Proposed Definitions.

About the Author
Susan J. Schniepp is vice-president of quality and regulatory affairs at Allergy Laboratories.