Trends in On-Site Supplier Audits

Robyn Mackenzie/shutterstock.com

Derived from the Latin word audire (to hear), the term “audit” dates back to the Middle Ages, when independent examiners would have a company’s bookkeepers read account books out loud to them, to ensure that no theft or sloppy accounting was happening.

While the term can strike fear into the unprepared, the audit can be a huge benefit to any company, allowing for a systematic, independent examination of processes. In the pharmaceutical industry today, quality systems are usually audited by a trained and knowledgeable team. The audit’s outcome should always be a written report. 
In a recent research project, audit reports written by in-house and independent auditors were randomly chosen for two five-year periods (from 2002 through 2006 and 2012 through 2016). The number of observations made, the severity of those observations (i.e., whether deficiencies were “critical,” “major,” or “minor”), and the auditors’ recommendations were assessed and results were then compared.

Findings showed that the observations of the earlier audits were more critical than those made a decade later.

One possible reason could be that suppliers have optimized their processes and have become more accustomed to hosting on-site audits.

However, the earlier audits lasted longer than the later ones (as measured by audit person-days, derived by multiplying the number of days required for the audit by the number of auditors performing it). Fewer days available for on-site audits means that each auditor has less time to dive deeper into more complex processes. The result is fewer, and possibly more superficial, observations. As a result, the audit reports may be less meaningful, and of less benefit to both the audited company and its pharmaceutical industry customer. 
 

Material and methods

Reports from audits of suppliers and service providers (see Figure 1) were randomly chosen and compared. The total number of reports reviewed was 129-39 reports for the first five-year period (2002–2006) and 90 reports for the second five-year period (2012–2016), which included audits performed by third-party audit providers. The numbers of observations (including recommendations) and their severity, whether “critical,” “major,” or “minor” within both time periods were compared and the differences obtained were highlighted. 

Figure 1: Suppliers and service providers audited on site.

For the second five-year period (2012–2016), more than 40% of the reports from on-site audits issued by third-party audit providers were available. Such reports were not available during the earlier period, however. In addition, there were questions of whether there were differences between the audits made by inhouse auditors and those done by independent third parties, in terms of the quantity and criticality of observations.  

Results

As seen in Table I, the mean number of audit observations decreased by approximately 38% from the earlier to the later period, with 11 observations made per audit between 2002 and 2006, and eight between 2012 and 2016.

Table I: Comparison of number and criticality of audit observations within two time periods.

When comparing in-house versus outside auditor results, the number of observations was more or less comparable in both cases. To calculate the mean “severity” of observations, the total number of “critical” observations was multiplied by a factor of 5; “major” observations were multiplied by a factor of 2, and “minor” observations, by 0.5.  The number of recommendations issued was multiplied by a factor of 0.1 (1). The resulting figures were added, and the resulting sum was divided by the total number of audits executed. Results showed that the number and severity of observations fell from 6.5 to 5.85 during the 10 years separating the two audit periods (see Table I and Figure 2). 

Figure 2: Differences in audit observation criticalities within two five-year periods.

What led to these differences? During the earlier period, two out of three audits lasted approximately one day and one out of three took longer than one day (see Figure 3). In contrast, more than 90% of the audits performed during the later period lasted one day, with the balance lasting more than one day.

Figure 3: Influence of decreased audit (person days) on audit results (2002-2006 in comparison to 2012-2016).

When audit person-days were compared within these two time periods, the differences were even more striking (see Figure 3). During the earlier period, more than 90% of the audits lasted more than one person-day, whereas only 78% of all audits conducted during the later period lasted more than one person-day.

Correlating the reduction in audit person-days with the decrease in number of audit findings and lower criticality of observations made: The average audit duration was 2.4 audit person-days in 2002-2006 and 1.9 audit person-days in 2012–2016 (down by 26%), and this reduction in audit duration resulted in 38% fewer observations and an 11% drop in critical observations (see Figure 4).

Figure 4: Correlation between less audit time and number and criticality of audit observations (2002-2006=100%).

Discussion 

On-site audits are part of the supplier qualification process. Although supplier qualification is more than on-site auditing, planned audits should be of sufficient duration (2) if they are to provide an appropriate level of confidence that suppliers and vendors are able to supply consistent quality of materials, components, and services to customers in accordance with regulatory requirements.

How long should an audit last? Suggested lengths range from two to eight person-days (3, 4) up to 18 person-days, depending on the supplier’s size and the complexity of its products and manufacturing processes (5). Most sources refer to the duration of regulatory inspections, for which official recommendations exist.

The recommended duration of supplier audits by customers is not stated, but some sources refer to “narrow time schedules” for customer audits (6) without quantifying what “narrow” really means. Narrow audit schedules might influence audit quality, because the number of observations will decrease due to restricted time frames. 

Because the auditor does not have sufficient time to audit complex processes intensively, weak points that might negatively affect product quality and consequently harm patients may not be detected. In such cases, superficial audit findings, which are often only formal deviations from GxP requirements, would not reflect severe systemic quality problems.

The gap between expectations and reality

Ironically, between the earlier and later periods studied, regulatory requirements have increased audit requirements, mandating not only that customer audits identify any and all of the supplier’s GxP deficiencies, but that the entire supply chain be fully mapped out to ensure integrity and transparency (7). 

The gap between regulatory expectations and audit reality is growing, with regulators calling for greater stringency and suppliers becoming less willing to host on-site audits for more than one day. The result has been less meaningful audits with less findings, and thus, fewer suggestions on how suppliers might improve (see Figure 4). Such reports are more surrogates than meaningful reports, yet regulators rely on them when they inspect manufacturers’ supplier qualification records and quality systems (8). 

The number of findings made during the 2012-2016 period has decreased by more than one third compared to that during the 2002–2006 period. The severity of findings has also decreased, by approximately 11% (because audit reports during the earlier period did not always distinguish between “major” and “minor” deviations, a percentage comparison of observations had to be estimated for that period). Between the two periods being studied, the percentage of critical observations dropped from more than 5% to 1% (see Figure 2).

Experience has shown that suppliers have become less willing to host on-site customer audits, as shown in Table II, which provides supplier responses to customer requests for on-site audit appointments. This leaves pharmaceutical manufacturers in a difficult position, because regulators are demanding more stringent audits at a time when suppliers are less willing to host them. So what can customers do? 

Table II: Some supplier responses on on-site audit requests (from the author's experiences).

Table III: How to handle the necessity of on-site audits effectively.

One thing that is of vital importance is to plan and prepare very carefully for the audit, so that the exercise is benefitial for both the auditor and the auditee. In this context, it is essential to be aware of the audit “environment” (e.g., whether GxP or ISO), and whether the audit focuses more on systems or on products. Suppliers are more willing to devote time to on-site customer audits for larger customers, so it might be helpful to consolidate business with strategic suppliers, rather than divide work among many smaller suppliers. In addition, it might be worth working with a third-party audit provider that audits supplier facilities for more than one customer who buys the same product or service (see Table III).

Table I shows that the results of audits performed by a company’s own auditors are comparable with those made by third parties, based on the number of observations made (in both cases, approximately eight observations). The criticality of observations made by the two groups is only slightly different: 6.04 compared to 5.58 when calculated according to reference 1. However, as seen in Figure 5, the percentage distribution of observations made by third-party auditors compared to in-house auditors did not differ from each other; therefore, it can be said that  the results of on-site audits being executed by either party are more or less comparable. 

Figure 5: Percentage distribution of audit observations (2012-2016).

The fact that more than 40% of audits in the earlier period were carried out by independent third parties had no influence on audit quantity and quality.  What seemed to have the most impact was the fact that suppliers are less willing to grant, or devote sufficient time to, onsite audits by their pharmaceutical industry customers.

Conclusion

Suppliers and service providers today are less willing to host onsite customer audits for more than one day than they were ten years ago. At the same time, however, global regulatory authorities expect thorough on-site audits as part of the supplier qualification process, given the increasing emphasis on supply-chain transparency and integrity. The unwillingness of suppliers/service providers to host audits longer than one person-day results in audits with fewer, less critical observations, compared to audits performed 10 to 15 years ago.

Figure 6: Decision matrix-own audits, audits by third parties, or reports (proposals).

If audit reports are to be meaningful, customers should make the audit as beneficial as possible for their suppliers, while ensuring that it meets regulator’s expectations. The following strategies (see Figure 6) can help achieve both goals:

• Well-prepared, risk-based audits that makes the experience a “win/win” for both customer and supplier 

• Qualification of strategic suppliers and service providers

• Increased use of third-party auditors, who audit suppliers for several customers at a time, reducing the number of audits required.

References

1. M. Pfeiffer, Pharm. Ind. 75 (12), 1924-1928 (2013).
2. European Commission, EudraLex, Volume 4, Good Manufacturing Practice Guidelines, Chapter 5 (Production).
3. Certqua, Newsletter 10/2013, Wichtige Fristen für die Re-zertifizierung
4. C. Johner, Wie lange Ihr Auditor bleiben darf .
5. ISO/IEC 27006:2015, Requirements for Bodies Providing Audits, DAkkS, Deutsche Akkreditierungsstelle, Übergang, as of Apr. 11, 2016 (revision 1).
6. M. Hiob, Vorbereitung auf Audits, Logfile No. 33/2016 .
7. E. Norton, “Around the World in 80 Ways,” MHRA Inspectorate, Feb. 27, 2017.
8. GMP News, Welche externen Auditberichte darf man verwenden.
9. M. Pfeiffer and H. Scheidecker, Pharm. Ind. 72 (3) 540-545 (2010). 

Article Details

Pharmaceutical Technology
Supplement: Outsourcing Resources
Vol. 41
August 2017
Pages: s10-s15

Citation

When referring to this article, please cite it as M. Pfeiffer, "Trends in On-Site Supplier Audits," Pharmaceutical TechnologyOutsourcing Resources Supplement (August 2017).