Quality Metrics and Best Practices for Working with Contract Labs

JGI/Jamie Grill/Getty Images

As pharmaceutical manufacturers continue to outsource more manufacturing and quality functions to contract partners, the question of who owns final product quality has become more complex. In the end, the company marketing the product is responsible, but achieving that quality is increasingly becoming a joint responsibility involving contract partners.

In 2015, FDA released draft guidance (1) that attempts to guide both sponsor companies and their partners in understanding the most important product quality metrics.  The goal behind the guidance is to help the agency schedule plant inspections based on risk, so that companies with the most rigorous quality-assurance (QA) metrics would be inspected less frequently than those with weaker quality systems. Another goal is to help FDA inspectors better understand, and take steps to help address, potential product shortages.

The guidance would assess certain measurements as a baseline and consider data from both sponsor and contract labs and manufacturing organizations including the number of out-of-specification (OOS) results, the number of lot-release and stability tests, and whether annual product reviews are published on a timely basis.  

FDA’s draft would also assess corporate commitment to quality by requiring that senior quality executives review and sign off on documentation, evaluating the effectiveness of the company’s corrective actions and preventive actions (CAPA) programs, and checking on how well it is using process capability analysis to look at critical product quality attributes.  

Industry’s reaction to the draft guidance was strong and immediate, with some companies and trade groups voicing criticism and calling for revisions (2). For contract laboratories and sponsors alike, the draft guidance reaffirms basic benchmarks and offers new reasons to assess programs and interactions, and establish best practices.  

Pharmaceutical Technology interviewed quality assurance consultant Susan Schniepp, chair of the Parenteral Drug Association’s (PDA) regulatory affairs and quality advisory board, as well as Eurofins Lancaster Laboratories’ quality assurance specialists Art Pezzica, senior director of quality assurance, and Brittany Cloud, senior quality assurance specialist, for their views on best practices, as well as industry reaction to the draft guidance.  

Quality metrics and OOS resultsPharmTech: In its draft guidance on quality metrics, FDA seems to be focusing on the basics, such as OOS.  Without having sponsors micromanage their partners, what do you feel should be reported for optimal results and to justify a sponsor’s comfort with a contract laboratory?

Pezzica (Eurofins Lancaster Labs): For contract laboratories, FDA is only looking for unconfirmed out-of-spec results, but there are other metrics that laboratories monitor that can provide a certain comfort level for sponsors. These might include operator errors, unconfirmed deviations, and misreported data. Data quality is very important. It’s our laboratory’s product.

Cloud (Eurofins Lancaster Labs): Quantitative data, such as invalidated OOS rates and CAPAs, are very important, but so is qualitative information. For instance, does your contract partner have any Six Sigma or Lean projects underway? These can speak volumes as to how far the company is willing to go to achieve continued excellence or success.  

If a contract partner is making positive changes, how is it handling change control? Is it investing in improving infrastructure or adding new employees?  These questions provide insights into the overall quality culture of the firm, and go beyond the compliance statistics to give a picture of what is going on.

CAPA and quality metricsPharmTech: CAPA has been a compliance challenge for many companies. Complexity would seem to increase with the number of functions being outsourced and the number of contract partners involved. How should concerns about CAPA influence reporting metrics?

Cloud (Eurofins Lancaster Labs): The key is having a dynamic and transparent relationship in which both partner and sponsor can speak openly. This involves not only reporting on open or completed CAPA items, but also looking at effectiveness trends to see whether any special programs and initiatives that the lab has established are actually working.

It is also important to hold regular quality meetings with the client to review CAPA. These meetings allow the contract partner to develop best practices to address client issues, but also to learn from clients. In some cases, we’ve been able to implement some of our clients’ best internal and external audit practices to improve our overall quality assurance.

Communication should be dynamic, and should go beyond merely reporting metrics.  Ideally, it becomes a fluent discussion that improves not just the way you operate your lab, but the overall relationship as well as the quality of the product that eventually reaches the consumer.

Schniepp (PDA): With CAPAs, clients should share more information on their investigations with contract partners.  As a contract lab, you are typically just one part of a CAPA investigation. You run your tests and report results to the client, but you often don’t learn what the final results were. That information could be very helpful in improving your understanding of the client’s process and what can affect it. It would be helpful to have that information loop back, to help you improve your systems.

Data integrityPharmTech: Data integrity has been a crucial issue for manufacturers and their contract partners, and many contract partners are falling short of FDA’s expectations. How should contract labs and sponsors work together to ensure that requirements are being met?

Pezzica (Eurofins Lancaster Labs): It’s crucial to have a strong validation program in place, and to focus on ethics and training. We know that we have good systems, but, due to the recent cases of insufficient data integrity affecting the industry, we recently formed a team to reassess our electronic systems. This team has implemented a risk-assessment program that is reviewing all our electronic data systems to identify any potential risks that might affect data integrity. Once the analysis is complete, we plan to focus on the top 20 potential risk areas and assess each electronic system for any potential risk to data integrity.

We also have an ethics program in place and emphasize the importance of training. We make sure that our employees are fully aware that, while human error may occur, any attempt at falsification would be grounds for immediate dismissal.  It is essential for any contract lab to ensure that its employees divulge errors and strive to maintain the highest quality data results.

PharmTech: Do you find that you have to work with clients to help them better understand data integrity requirements from global regulatory agencies, and the need to assess risks?

Cloud (Eurofins Lancaster Labs): I routinely host audits and have found the level of expertise varies among sponsors. Some clients are experts in this area and can share best practices with us. Others may know very little about regulators’ expectations for data integrity, while other clients are in between. Our job is to work with companies of all types to ensure that they understand requirements and can be comfortable with their data.

When we work on product risk assessments with clients, we always share our standard operating procedures (SOPs) when that is requested, so that clients will know that we are following our procedures and policies.

In some cases, we will even visit client sites to discuss our processes and how we incorporate quality into our data integrity systems. We have to take a holistic approach to meet the needs of all our clients, since they come from so many different areas of the business

PharmTech: Do you see a lack of understanding of what documentation processes should be?

Cloud (Eurofins Lancaster Labs): Some clients may not understand differences between various regulatory agencies’ expectations. They may, for example, only want to follow FDA or European Medicines Agency (EMA) or, in the UK, the Medicines and Health Regulatory Agency (MHRA) guidance. In these cases, we must impress upon them the need to be ready on all fronts, and be ready to defend that position and our practices.

Whenever a discrepancy comes up with a client, we need to triage it first and ask two crucial questions:  

• Could performing this function this way cause us to violate a regulatory requirement?  

• Would we be violating any of our own SOPs?

If we can confidently say that we’re not failing on either front, then we are meeting the goals of that intended practice and procedure, and there is no need to change it.

Pezzica (Eurofins Lancaster Labs): Clients should have a clear understanding of contract lab realities. A process that may work at a client’s manufacturing facility that is producing 10 or 20 products may be difficult to manage at a facility where thousands of different products are being tested using different analytical methods and test procedures.

We always try to accommodate client requests, but we must also ask some hard questions:  Can we implement that process here and do it correctly all the time?  Does the change in process increase, even slightly, the potential of creating more errors? If so, we discuss our concerns with our customers.

If the procedure or process will add value, but also add costs, is it a cost that all clients are willing to absorb, or will it be a single client-specific request? In the end, it is really a balancing act. We always need to assess the potential impact on our operations and whether it will add value. Managing client-specific requests will always make a project more difficult and add costs to our operation.

Managing client expectationsPharmTech: How can both sides balance the need for maximum transparency with realistic expectations?

Pezzica (Eurofins Lancaster Labs): We meet twice a year with some of our larger customers, and always include both of our quality departments, to discuss any issues that come up. For some clients, we’ll follow their OOS procedures instead of our own, but their request has to be reasonable, and involve something that we can be successful doing. Whenever differences of opinion come up, we discuss and resolve them.

Schniepp (PDA): As a contractor, you’re often in a tough spot. This is especially evident with early reporting. You want to tell clients when numbers start trending in a certain way, to warn them, like Paul Revere, that something may start to go wrong.  

However, in reality, when you do that, clients can overreact and take over, to a point where they can get in the way of root cause investigations and other lab work. Over time, if this keeps happening, a contract laboratory might not want to tell its clients about anything too early, before they have something concrete to report. But then, clients may be angry that they weren’t given advanced warning. It is definitely a difficult balancing act.

You’re between the proverbial rock and a hard place.  I used to address this by calling clients and managing their expectations and fears with a ‘Don’t panic yet.’

Cloud (Eurofins Lancaster Labs): Transparency is so important to any contract lab-sponsor relationship. If a contractor is transparent and builds quality into its relationships with clients up front, clients tend to trust them. Having quality assurance involved, on both sides of the team, is very important, and this involvement is needed as early as possible.

I do a lot of onboarding, and we tend to find that, when the quality assurance department is involved early and clients can put a face to a name, trust will be built and will make any challenges easier to manage if and when they come up. In this type of relationship, you can call or email each other at any time, even for things that may seem relatively trivial. Proactive communication can help avoid problems, and also clarify boundaries and prevent micromanagement.

Pezzica (Eurofins Lancaster Labs): For any investigation that we do, we notify our clients within 24 hours. At that point, we are open to discussion, and we ask them to share additional inside knowledge about the product. We find that most clients are comfortable with this approach. At times, some have made unreasonable requests, but we’ve been able to rely on our knowledge of FDA and other regulatory agency requirements to enable open communication.

PharmTech: Can you give any examples of unreasonable scope requests?

Pezzica (Eurofins Lancaster Labs): Sometimes, we get test results that may not be what was expected, but still within specifications. The company may want us to invalidate the result, even though results were within specifications, and all method requirements were met.  In the end, we may agree that the result might be considered out of trend, but the data do not support invalidating the result.

Cloud (Eurofins Lancaster Labs): Some companies may try to set unreasonable expectations through quality agreements. For example, a company might require monthly metrics reporting of many different metrics, or dedicated QA personnel just for their investigations, as well as daily status reports on any investigations and CAPAs. Excessive levels of reporting and custom requests would have a negative impact on our operations without adding value for our clients.

Data reportingPharmTech: Are more pharmaceutical companies building IT connections to facilitate data reporting?

Cloud (Eurofins Lancaster Labs): With some larger companies, we record data and results directly into portions of their laboratory information management systems (LIMS).  A few companies may have special data acquisitions systems that they want us to use, so we’ll bring them onsite and work with them. We’ve even had clients recommend certain systems for instruments and equipment that we’ve brought online.  

As a contract partner, flexibility in adapting to client needs is important to sustaining the relationship. We’re open to working with different data and IT systems.

Pezzica (Eurofins Lancaster Labs): We offer our clients secure access to their data through our online portal, LabAccess.com, which provides a window into our LIMs, and scans and links raw data so that clients can see how all the data support the final COA report.  They receive a full data package, including any investigations related to the group of samples that was submitted for testing. Each client can only see its specific data. The software also allows us to control who receives the data, for instance, in cases where we may be working on multiple projects with different divisions of the same company, those data could be restricted to specific individuals from the different divisions.

Some clients review all of the raw data. We use this system to provide companies the assurance that we’re showing them everything that we did. This is important for data integrity and compliance.

Operational excellencePharmTech: Are you becoming more involved in customer operational excellence programs?  Do you have one in place at your company?

Cloud (Eurofins Lancaster Labs): Throughout the years, we’ve had a process improvement program, where anyone in the company can suggest lean or process improvement projects, which are taken on not just by management but by quality, data review, lab, document control, and other departments. Anyone can be involved in this program throughout the lifespan of a project.

In one of our most effective recent projects, we renovated a raw material wet chemistry lab to reduce human error significantly. We like to share this information with our clients, because we feel that it enhances our relationship and improves the quality of the final products overall. Numbers alone don’t measure quality.

PharmTech: Do you find that more of your pharma and biopharma customers have continuous improvement programs in place?

Cloud (Eurofins Lancaster Labs): A few of the larger companies do, but most do not, apart from executive-level programs. By definition, such programs don’t go into the trenches and ask the people working in the labs to suggest improvements.  

Pezzica (Eurofins Lancaster Labs): You can’t manage what you don’t measure. Our QA department leaders meet with all department directors and managers to review all quality metrics.

We ask them to come prepared to explain trends, whether positive or negative. This allows us to assess issues as well as solutions so that we can improve our processes.

Currently, we do this quarterly for 18 different departments, analyzing a series of metrics such as OOS, report reprints, client complaints, and various other quality indicators. We find that the results are worth the investment in time and effort that this approach requires.  

We have also just implemented a new quality program that we call ‘Integrated QA.’ We’ve hired and transferred QA people who will be physically located in technical departments so that they can observe operations and eliminate potential problems or errors.

In the traditional approach, QA staff is in the office.  They perform audits and problems or issues are generally brought to them. We want to reverse this, so that QA goes to problems to help identify and prevent them. We also want to give our analysts guidance on how to document issues that might occur. In two years, when somebody looks at these data, they need to tell the entire story of how they were generated and any decisions that were made based on them. They need to be able to stand on their own.

References
1. FDA, Request for Quality Metrics Guidance For Industry, Draft Guidance (CDER, July 2015), accessed Jan. 19, 2015.
2. R. Hernandez, “Overwhelming Response from Industry on FDA’s Guidance on Quality Metrics,” December 7, 2015, BioPharmInternational.com, online, accessed Jan. 19, 2015.

Article DetailsPartnerships in Outsourcing Supplement 
Vol. 40, No. 13
Pages: 06–13

Citation:
When referring to this article, please cite it as A. Shanley, “Quality Metrics and Best Practices for Working with Contract Labs," Pharmaceutical Technology Partnerships in Outsourcing Supplement 40 (13) 2016.