The End of the 21 CFR Part 11 Controversy and Confusion?

After more than 6 years of debate and varying interpretations, the US Food and Drug Administration has issued new draft guidance on the intended scope and application of 21 CFR Part 11. This article examines the reasons for the Part 11 controversy and confusion within the pharmaceutical industry, and discusses the open questions raised by the new draft guidance and the remaining challenges for FDA and the industry.

More than 6 years have elapsed since the US Food and Drug Administration's (FDA's) 21 CFR Part 11 regulations regarding the use of electronic records and electronic signatures came into effect.1 In February 2003, FDA issued new draft guidance concerning the scope and application of Part 11, which describes how the agency intends to interpret and enforce the requirements during its ongoing re-examination of the regulations.2 Many people in the pharmaceutical industry have welcomed this new guidance and see it as a positive development that will lead to a simplified FDA approach to Part 11 and a significant reduction in the industry's compliance burden.

Will this shift in FDA's interpretation and its intended use of enforcement discretion put an end to the controversy and confusion around Part 11? This article examines the basic reasons for the disconnects between FDA and the industry, and discusses some of the new questions and issues that will require further clarification to achieve a more sensible approach.

The root causes

Many factors have been suggested as reasons for the controversy and confusion that have surrounded Part 11 within the pharmaceutical industry. These discussions have usually focussed on FDA's efforts and the industry's arguments that the agency has failed to provide timely and sufficiently specific guidance to fill in all the details that lurk beneath the high-level principles outlined in the regulations. The root causes of the problems, however, point to both FDA and the industry.

From the outset, in its efforts to develop Part 11, FDA fell into a classic trap of failing to recognize a vast difference between its perception and reality. FDA did not realize that there was a substantial gap between what it perceived or presumed to be the existing level of good software and systems development practice being employed in the pharmaceutical industry and the reality of the industry's practices. Whereas there may have been individual companies that did not fit into this broad generalization, the truth is that the industry had not fully embraced and internalized many of the basic, fundamental concepts of good practice in this area. As a consequence, the common level of practice was far below what FDA perceived as the industry's starting point.

This "reality gap" set the stage for all of the controversy and confusion that followed the issuance of the final rule in 1997. It also helps to explain why and how FDA so grossly underestimated the economic impact of the new regulations and why the industry has struggled to understand and consistently comply with some of the key requirements ever since. Even though the industry has narrowed the reality gap through its remediation efforts during the past few years, a significant gap still remains.

To understand why this reality gap exists, we need to look at where the industry was in the early 1990s, when the first discussions regarding the issues that culminated in Part 11 were taking place. During the 1980s, both the industry and the regulatory authorities focussed heavily on the burgeoning concepts and requirements for process validation. As a result, the pharmaceutical industry worked through and adopted a new approach to managing quality and control, and eventually shifted its focus and culture from being product-oriented to process-oriented. Since that time, the industry has developed tremendous knowledge and experience in dealing with process-based issues.

Today, pharmaceutical companies do not question the need to properly validate production processes and it has become second nature for this to occur in a planned, structured, controlled and carefully documented way. The industry readily accepts and understands the notion that you cannot test quality and integrity into a product. This understanding also extends to the concept that product quality and integrity result from following a good, scientifically challenged, well-controlled process against which defined, confirmatory spot-checks are performed through quality control (QC) testing. What the industry did not appreciate in the early 1990s and still struggles to accept is that the very same concepts apply to the quality and integrity of computerized systems, as these attributes in the electronic context also result from following a planned, rational, controlled and documented process. For a variety of reasons, the industry has generally not successfully transferred its understanding of these process-oriented concepts into the electronic environment. This non-transference, combined with the poor level of existing software and systems development practice, helped to create (and continues to feed) the industry's share of the root causes of Part 11 issues.

Shifting the culture paradigm

The differences in views and understanding between the industry and FDA regarding good practice in this area have resulted in numerous adverse inspectional findings and the pain and expense of regulatory enforcement actions. Part 11 has been an effective driver in this process and a leading cause in triggering the beginnings of a significant but grudgingly accepted culture change in the industry. The pharmaceutical industry is undergoing, but still resisting, a major paradigm shift in terms of its approach to software and systems development.

Why is such a culture change necessary? Because even though there are some areas of good practice, it is not yet embedded in the industry's culture. There is still a widespread lack of understanding regarding the logical and scientific basis for some of the key requirements, and this continues to fuel companies' and individuals' resistance to adopting practices that will routinely meet those requirements. The Part 11 requirement for automatic, computer-generated, time-stamped audit trails is a prime example. Many people in the industry still view this as an additional, unnecessary requirement that is being imposed by the regulators for purposes of facilitating inspections, and deterring and being able to detect fraud. This mindset is misguided, as an audit trail is a rational, basic component of good practice, the primary purpose of which is for the company's benefit, not the regulators'.

An accurate and complete audit trail is critically important to the pharmaceutical company; for example, when a manufactured product fails to meet its final release specifications and must be rejected. When this occurs, the company has to be able to conduct a detailed failure investigation to determine what went wrong and when. Smart companies recognize that this is as much a matter of good business practice as it is a clear requirement under the good manufacturing practice regulations. If an accurate and complete audit trail is not available, the company may not be able to pinpoint a definitive root cause of the failure and thus cannot assure itself or the regulatory authorities that it can continue to manufacture without running into the same unacceptable variation in the finished product.

As another example, experienced systems developers in the pharmaceutical industry will usually seek to defend their standard practices as sufficient and "fit for purpose," even if they do not always document their work or adhere to other stated regulatory expectations. In this regard, they often express views that the requirements of Part 11 are too cumbersome and technically unnecessary. However, when asked if they would feel comfortable if their standard practices were used to develop the guidance system for a commercial airliner, the typical reaction is "absolutely not." They then become visibly uncomfortable when they are reminded that if an airliner's guidance system fails to work as intended, hundreds of people could die; but, in contrast, if some of the systems they are developing do not work properly, drug products could be inappropriately released and thousands, if not hundreds of thousands, could be harmed. The fact that this is not clearly and consistently understood as a driving factor for improving the level of software and systems development practice in the industry further underscores the need for a far-reaching culture change.

The core problems

Following the issuance of Part 11, piecemeal guidance flowed out of FDA in both official (but draft) guidance documents and unofficially in numerous statements and opinions provided as "podium policy" by agency officials at various meetings and conferences. Many of the industry's complaints here are well-founded. Indeed, FDA's interpretation of the requirements did creep with time, and resulted in some irrational and unnecessary extremes. The positions set forth in FDA's draft guidance on electronic copies of electronic records provide a good example.3 In that draft guidance, FDA sought to impose additional requirements that went far beyond the letter and original spirit of the regulations; for example, when FDA for the first time "suggested" that inspected firms "should" convert original electronic records into another format to accommodate the agency's technical capabilities.4

FDA's inconsistent and occasionally over-reaching guidance certainly did contribute to the pharmaceutical industry's difficulties, but it was the underlying state of the industry's practices that truly presented the core problems. Because the industry did not fully understand or adhere to basic good software and systems development practices, companies expected (and, in many cases, needed) FDA to be much more explicit, prescriptive, and definitive in its Part 11 guidance. It was unreasonable and unrealistic to expect that FDA would or even could teach the industry about software and systems development, particularly since the concepts and standards were not new and the industry had much greater technical capabilities than the regulatory authorities could apply to these issues. The more it appeared that FDA was making up the rules as it went along, the more the industry struggled and became entangled in its own internal debates regarding the basic expectations and how to effectively meet them.

Two core problems within the pharmaceutical industry have served to fuel the issues. As noted earlier, the industry's approach to software and systems development was (and to some degree still is) not fully consistent with the same basic requirements of good practice that a college student would learn. In addition, most companies have employed an organizational structure in which the development function and the business (that is, system use and maintenance) function are separated in terms of both their operational responsibility and management of budgets.

Looking at the pharmaceutical industry's typical approach to software and systems development, for the most part, the industry has relied very heavily on vendor-developed and vendor-maintained software and systems. Although most companies have performed some level of in-house development or customization, the vast majority of the industry's software and systems have been procured from outside vendors. Two key (and probably related) observations generally apply in this regard. First, the industry historically has not applied the same degree of rigour and control to its management and qualification of software vendors that it has routinely applied to suppliers of ingredients or other components of the manufacturing process. Second, there has been a tendency for the industry to have a tremendous amount of blind (or at least very nearsighted) faith in its vendors and both its in-house developed and purchased systems. As an example, a company that had developed a system in-house nearly 20 years earlier had been relying on an outside vendor to maintain the system for several years. Because the firm had been using the system without any apparent problems for a long period of time, the company was very confident about its integrity and reliability and proudly defended the system when FDA challenged its validation. Much to its surprise, when the firm was forced to take a hard look at the system, it discovered that a fundamental design flaw had been built into the way the software rounded numbers. As a result, many years of QC laboratory analyses and results were called into question, and it required an extraordinary, time-consuming and costly effort to reassess the system's data and reassure and convince FDA that no further regulatory action was necessary with regard to the company's US-marketed products.

Given this background and the typical standard of practice in the industry, and despite FDA's guidance and enforcement efforts to date, it is still fairly common to find that the basic requirements and their perceived costs are seen by many in the industry as unnecessary and unreasonably burdensome. Much of this continues to be driven by critical and widely held misunderstandings. For example, validation is often viewed as an externally imposed add-on, rather than being clearly understood as a key fundamental component of good software and systems development practice. There are numerous other examples of FDA's inspectional criticism of companies' practices regarding basic issues such as code review and traceability. The industry ordinarily perceives these kinds of observations as bureaucratic regulatory nitpicking, instead of recognizing that FDA is focussed on and concerned about such issues because they are the basic components of good practice.

The typical corporate structure employed in the industry has also been a core problem in fuelling companies' difficulties. In general, information systems (IS) or information technology (IT) functions have not been approached as a core competency in the pharmaceutical industry. In most companies, IS/IT departments are established and managed primarily as a service function to the rest of the business. The IS/IT organization is ordinarily considered as an administrative or infrastructure necessity that is there to respond to and support the company's more critical core business demands in research and development, manufacturing and commercial operations. As a result, the IS/IT departments' objectives and goals have typically been focussed on and driven more by system delivery deadlines and budgets than by quality and good practice. To add to the problem, the budgets for IS/IT development and the use and maintenance phases of systems are typically managed in separate parts of the company. This separation of fiscal responsibility leads to a tendency to develop systems and pass them to the business function, which must then deal with the consequences of any development shortcomings. With this approach, the development organization is considered to have met its objectives if it delivers a new system on time and under its operating budget, even if decisions were made to skip key development steps (such as integration or system testing) to meet the delivery deadline. Despite its obviously shortsighted nature, this approach is not uncommon, and companies often incur change and maintenance expenses that far outstrip the costs to properly develop the system in the first place.

FDA's new draft guidance

On 20 February 2003, FDA issued a new draft guidance regarding the scope and application of Part 11.5 In this draft guidance, FDA described its "narrowed" interpretation of the regulations and explained how it intends to apply its enforcement discretion during its ongoing re-examination of Part 11 as part of its new "science and risk-based approach" to good manufacturing practices (GMPs) for pharmaceuticals. The agency also withdrew all its previous draft Part 11 guidance and its 1999 Compliance Policy Guide regarding the enforcement of Part 11.6 In summary, the new guidance document more explicitly states what FDA has been saying for some time - that until it decides how it intends to go forward with Part 11 in the context of the new GMP initiative, it is not going to enforce a number of the Part 11 requirements per se, but will fully enforce the requirements of the predicate rules. For some of the FDA-regulated industries, this shifted enforcement focus will probably result in a significantly reduced impact of Part 11 requirements on their businesses as a practical matter. This will not likely be the case for the pharmaceutical, medical device and biologics industries; however, because many of the requirements of Part 11 are also clearly required under the existing predicate rules for manufacturing, clinical, laboratory and other regulatory practices.

Has the guidance helped?

There is a question whether FDA's most recent draft guidance will really help the pharmaceutical industry and at the same time alleviate the agency's enforcement burden. At the outset, FDA may be creating more issues than it is aiming to solve.

Some of the language in the draft guidance is contrary to long-standing FDA positions and some of it certainly seems to be unrealistic in light of the industry's practices. For example, in the draft guidance, FDA suggests that some systems may not need to be validated.7 For decades, FDA has consistently taken the sensible scientific position that all systems should be validated, according to their intended use. This new approach undermines that position, and potentially opens the floodgates for undefined risk-based decisions to not validate certain systems. In fact, some companies are already (and incorrectly) interpreting the draft guidance as supporting their using "a justified and documented risk assessment" to avoid validating systems that clearly fall within the validation requirements of the predicate rules.

In the absence of clear guidelines and requirements from FDA, these decisions are likely to be made in ways that are contrary to the agency's expectations and not consistent with good software and systems development practices. As another example, FDA suggests that when a computer system is "merely incidental" to the creation and use of paper records, and only the paper records are used to make regulatory decisions, the computer system would not be subject to Part 11 requirements.4 This is another loophole that may potentially be subject to abuse and one that, in the absence of 100% data verification, is clearly inconsistent with the need to ensure the integrity of what is printed on the paper records and thus used for regulatory purposes. The "merely incidental" use of a computer system may also be contrary to the reality of corporate practices and basic human nature. Experience has shown that if people have access to a computer system, they are going to use it, even if their company's procedures require them to rely exclusively on the paper records. Otherwise, the company and its employees will not be taking advantage of the powerful technology and added productivity that electronic systems can provide (one of the very reasons that the industry has used in its arguments for easing the requirements of Part 11).

There are several other points raised in the draft guidance that will require further definition and clarification from FDA to establish a clear baseline for the industry and to foster a shared understanding of the agency's expectations for Part 11 compliance. Another example comes from FDA's discussion of its intended application of enforcement discretion regarding legacy systems. In the draft guidance, the agency states that it "will not normally take regulatory action to enforce" Part 11 "with regard to legacy systems that otherwise met predicate rule requirements prior to August 20, 1997."6 Trade associations representing both the pharmaceutical and medical device industries have already submitted comments to FDA and raised significant questions concerning the intended meaning of this language, as most older (pre-1997) systems have undergone changes in some form after Part 11 came into effect. Will such systems still qualify for the enforcement discretion described by FDA? To date, there is no clear answer. The issues here are further complicated by the "otherwise met predicate rule requirements" language, as it is rare to find older systems that were fully compliant in August 1997, given the nature of the industry's practices concerning basic predicate requirements, such as validation.

Risk assessment

In the latest draft guidance, FDA recommends that regulated companies base their approaches and decisions on a "justified and documented risk assessment" in three key areas - the need for and extent of validation, the application of audit trails, and the retention of records.7 The agency, however, does not go on to explain what it means or expects to see in this regard. This core risk-based concept is likely to be a source of great challenges if FDA does not address the issue in greater detail.

FDA's suggestion for the development and use of risk assessments related to computerized systems makes sense from both a scientific and a regulatory perspective. It is also consistent with standard engineering approaches to hazard control. The difficulty is that the agency's conceptual rationale once again seems to be heading for another manifestation of the "reality gap" problem. The concepts and methods of hazard control and risk mitigation are not well-understood or routinely employed for software development in the pharmaceutical industry. Unlike the knowledge and capabilities that have been developed in the medical device industry regarding the understanding and management (that is, appropriate mitigation) of risk, the pharmaceutical industry generally has much more limited experience in the formalized application of hazard control methodologies. This has been reflected in a number of problematic inspections, where internal decisions and boundaries as to what was done or should have been done (for example, with regard to the scope of testing) was not grounded in good science and logic or based on a thorough understanding of the design and intended functionality of the software and system. Without the scientific and logical foundation provided by the rigorous application of a hazard control methodology, this type of approach appears to be arbitrary and becomes very difficult for the inspected company to defend.

Conclusion

FDA's recent draft guidance raises a number of new fundamental questions, and sets the stage for more controversy and confusion regarding Part 11 and how it will be interpreted and enforced. Given this reality, one thing seems clear - there is more collaborative work to be done if FDA and the pharmaceutical industry are going to succeed in reducing their respective compliance and enforcement burdens.

The application of risk assessment and hazard control concepts and methods is perhaps the most critical area requiring further clarity. If FDA does not address these issues with much greater specificity, and if the agency's Center for Drug Evaluation and Research (CDER) does not take full advantage of the knowledge already developed in the Center for Devices and Radiological Health (CDRH) in this regard, the pharmaceutical industry is only likely to receive further guidance through enforcement. This would be an inefficient and frustrating approach for both the regulators and the industry, and it would not be much different from the experience with Part 11 during the last few years.

The recent efforts by FDA and the industry to reach a more sensible and scientifically rational approach to Part 11 are a well-intentioned start in the right direction. To achieve that laudable and rational goal, both FDA and the industry will need to work together to openly confront what they each need to better understand, and to define a sustainable and less controversial way forward.

References

1.

www.fda.gov/ora/compliance_ref/part11/FRs/background/pt11finr.pdf

2. www.fda.gov/cder/guidance/5505dft.PDF

3. www.fda.gov/ohrms/dockets/dockets/OOd1540/OOd-1540-c000001-vol1.pdf

4. www.fda.gov/ora/compliance-ref/cpg/cpggenl/cpg160-850.htm

5. www.fda.gov/cber/gdlns/esigglos.htm

6. www.fda.gov/ora

7. www.fda.gov