In the second half of this two-part series, the author suggests that to qualify and validate a pharmaceutical manufacturing facility, one must coordinate protocol and SOP development, scheduling and implementation, turnover package preparation, and the management and resolution of deviations and discrepancies. In combination with the programs described in Part I, these activities will help deliver projects on schedule, at estimated cost, and with quality assured.
All pharmaceutical validation projects are labor and capital intensive, and each must be planned and managed carefully. Numerous tasks and activities must be identified early and then scheduled to support the project completion date. Stakeholders such as the Quality Assurance (QA) and Calibration–Metrology departments must be alerted to impending increased workloads under compressed time frames. Standard operating procedures (SOPs) and protocol formats must be developed, test equipment must be purchased or rented, and contractors must be evaluated and hired. Managers must decide whether the US Food and Drug Administration will participate in the design review process, and if so, what will be the agency's exact involvement and participation. Considering the set of activities and programs that require timely completion, pharmaceutical validation projects must be carefully organized, managed, and monitored.
Part I of this article covered the following four critical components common to all successful validation projects: design review to ensure GMP compliance, project scope definition, project labor and cost estimating, and validation master plan development (1). Part II of this article introduces three additional programs, thereby providing validation project managers and participants the knowledge to plan and execute a project properly, no matter how difficult or complex. This final article also examines activities that are initiated well after project inception and often continue to project completion and operations:
This article does not present a detailed discussion of system-specific validation technologies because preparation and field execution of protocols are usually highly customized by system and are beyond the article's scope. General guidelines that are useful for protocol development and execution are described, however.
The concept of turnover packages (TOPs) as applied to pharmaceutical projects also are discussed. TOPs for biopharmaceutical projects were first described in the late 1980s and have evolved into systems used by many engineering firms and manufacturers. Finally, this article reviews the management and resolution of protocol deviations. Deviations are commonplace and tend to increase in frequency with increasing equipment complexity. These three programs, when combined with those described in the preceding article, can assist project managers to deliver a validated facility on time and at budget. Equally important, quality will be evident, the transition time to operation will be shortened, and regulatory expectations will be satisfied.
Protocol and SOP development, scheduling, and implementation
Protocols and SOPs are fundamental to all validation projects, and are required by all regulatory agencies for compliant facilities. Protocol and SOP development usually makes up 30–40% of all labor on a typical validation project, and therefore, document quality and efficient preparation cannot be understated. In general, approved protocols and draft SOPs must be delivered to coincide with certain milestones in the facility construction schedule, and therefore, careful planning and scheduling are essential.
One overriding principle that is central to all validation projects is that construction will not wait for protocol preparation, review, and approval. The validation team must deliver field-ready protocols that are suitable for implementation when mechanical systems are installed and started. The construction schedule must be interpreted and fully understood. Late protocol delivery often means that verifications and inspections will become more complicated, or even missed entirely. System access is reduced or eliminated, and the personnel responsible for installation and start-up may have left the site. Unfortunately, failure to deliver protocols on time usually extends the validation schedule far beyond the established completion date, thus adding cost and creating conflicts for resources and personnel. Neither of these outcomes are desirable, but they can be avoided with proper planning and management.
Very early in the project, protocol and SOP formats must be standardized and approved for use. In general, the SOP format is usually well established at project inception; therefore, companies rarely need to modify document templates for project purposes. Protocol format, however, is often a different matter. Historically, protocol content and arrangement have varied widely within the industry, although some harmonization is now evident. Employing different contractors on projects, each with different standard offerings, can lead to format inconsistency. Many equipment vendors now supply very detailed protocols that differ considerably from those prepared in-house or by consultants. And within companies, preferences of individual departments and divisions often cause protocol layout and content to deviate from the accepted standard. Fortunately, however, industry protocol format has evolved in such a way that many once obscure features are now standard from company to company and from consultant to consultant. Although no regulatory expectations or requirements exist for protocol format, all protocols must provide the documentation that proves the system or equipment was installed and operates according to design. Consequently, the following guidelines and lessons learned are noteworthy and should be considered during protocol development.
Preparing protocol forms. By definition, validation is the establishment of documented evidence that provides a high degree of assurance that a specific process will consistently produce a product that meets its predetermined specifications and quality attributes (2). Emphasis has been added, because nowhere in the regulations is it stated that the validation of equipment and systems must be absolute. Absolute assurance is impossible to attain and attempts to reach this level of confidence come at high cost, both financial and labor related (3). Applying this basic principle to protocol preparation, documents should be prepared with the following considerations.
For installation qualification (IQ), carefully manage the breadth and depth of equipment and system inspection and verification. For example, a typical piping and instrumentation diagram (P&ID) for a sterilizer or water-for-injection (WFI) system literally contains hundreds of individual pieces of field-verifiable information. These pieces can range from the obvious (e.g., pump manufacturer and model number) to less critical items (e.g., the size of electrical conduit) to the least important (e.g., type of covering on pipe insulation). Although confirming that the proper centrifugal pump is installed will ensure product quality, an exhaustive inspection of pump subcomponents will not provide the same benefit. It must be remembered that most pharmaceutical equipment is supplied and installed by reputable vendors and contractors (4). Therefore, the opportunity to detect and correct errors occurs long before the item is shipped and installed. The tendency and temptation in validation is to over-specify and over-inspect. This should be avoided because all inspections must be meaningful and contribute to the high degree of assurance described previously.
One way to manage IQ tests and verifications is through the careful design of protocol forms and attachments. In the early history of pharmaceutical validation, engineers and scientists often inspected equipment and systems with design specifications in hand. Although this method always achieved IQ objectives, validation personnel were frequently hindered by the large number of individual specifications contained in these design documents. Many specified features were not verifiable or were verifiable only with much difficulty and expense (e.g., pump impeller material of construction).
Figure 1: Suggested form for installation qualification verification. Companies should create a form for each major piece of equipment, instrument, or utility. In this example, the form applies to a large central station air-handling unit. Shading indicates that the specification is unavailable.
To avoid this dilemma, protocols have evolved to contain forms, similar to those in batch records, that force the comparison between actual and design information. These forms satisfy regulatory requirements and provide a concise and organized document when implemented. In general, validation personnel should create a form (see Figures 1 and 2) for each major piece of equipment, instrument, or utility. Protocols for complex systems (e.g., WFI) will contain dozens of forms. Major components requiring inspection can be identified on drawings by the assignment of unique numbers (e.g., Transmitter PIT-XXX, Fan EF-XXX, Pump P-XXX). Minor components with an assigned tag number (e.g., hand valves, heating, ventilation, and air-conditioning volume dampers, drains) can usually be omitted. Forms also may be correlated with critical components and subsystems identified in the system TOP.
Figure 2: An example of an installation qualification verification form. This form is for an air-handling unit chilled water supply and return. Shading indicates that the specification is unavailable.
The first column of each form (see Figures 1 and 2) lists attributes that require inspection and verification. The next column contains specifications and data. The third column, "Actual," is used to record actual data. Contents of each cell in columns 1 and 2 are determined by carefully reviewing design documents, specifications, and submittals. Again, the attributes chosen for inspection are those that provide high assurance of delivery and installation according to design. If a specification is unavailable, the corresponding cell is shaded and actual data are entered for record purposes during execution.
By carefully designing forms in this manner, the breadth and depth of system inspection are controlled, protocols can be easily duplicated and customized for similar systems, and implementation is organized and expedited. Furthermore, IQ acceptance criteria are easily stated and completely unambiguous: Information contained in actual column corresponds with information contained in specified column.
Separating IQ, OQ, and PQ documents. Many companies combine IQ and operational qualification (OQ), or OQ and performance qualification (PQ), into one document. Although this approach is sometimes practical, it is not always advisable for several reasons:
Omitting QA approval of IQ. Validation is a systematic and multidisciplinary undertaking comprising two predominating fields: engineering and science. Usually, IQ is the engineer's responsibility, and PQ is the scientist's specialty. OQ serves as a transition between IQ and PQ and often has both engineering and scientific components. Because IQ is largely an engineering function, some companies omit the QA approval of IQ protocols entirely. Furthermore, acceptable OQ and PQ outcomes will often reinforce that systems were installed as designed. The regulatory risk associated with the exclusion of QA approval is low and will often expedite IQ review and approval.
Eliminating unnecessary or complicated tests. As described previously, all validation inspection and testing should be meaningful and demonstrate system or equipment compliance with an approved design. Labor- and resource-intensive inspections and tests that yield only a marginal increase in quality assurance should be avoided. This can include triplicate testing of alarms and operational sequences when only a single verification is sufficient. Other marginal tests include autoclave-jacket temperature uniformity (unless used for diagnostic purposes) and the operation of hot WFI systems at subprocess temperatures (<70 °C) for extended periods of time. In addition, do not risk damage to systems and equipment to test an alarm. Running pumps dry to activate low-pressure or low-flow alarms is not a good practice. Changing setpoints or equivalent methods can often simulate these conditions. Above all, avoid specifying inspections and tests that are difficult to perform or implement, unless there is a regulatory requirement or other compelling, beneficial need.
Preparing acceptance criteria. All acceptance criteria must be based on system and equipment design, not arbitrarily assigned or determined. Designs are based on both quantitative (numerical) and qualitative user requirements and industry standards. If a certain quality or output is expected from an engineered system, the system is designed accordingly. Common examples include:
Acceptance criteria should be set equal to, and in some cases broader than, the underlying specification. It is a serious, but common, mistake to assign acceptance criteria to a system that exceeds the system's capability or intent. Such an error will lead to test failures and excessive, wasteful testing. Examples include particulate counting in compressed gas systems for which filters are nonexistent or particle-additive, and using high population, high D-value bioindicators when low F0 sterilization cycles (liquids) are validated. When design specifications may be unknown (e.g., for legacy systems), testing for information and baseline development (for future validation needs) are the preferred acceptance criteria and avoid the problems and deviations described previously.
All successful validation projects are characterized by careful planning and scheduling of project activities. Early on, validation deliverables must be identified and coordinated with the facility construction schedule. This task is not easy. Validation commonly deals with mechanically complete systems, while system construction proceeds in stages and by building sections. Manufacturing facilities should be turned over to the owner system by system, but this usually happens only on fast-track projects. The construction manager should be encouraged to turn over each system in the order needed by validation (e.g., supporting systems first, followed by process equipment). Rarely are individual systems and equipment installed in their entirety and started. Usually, a section of a building is constructed, with each building trade being scheduled to perform a specific job. For example, riggers will set a pump skid while pipefitters concurrently fabricate associated piping. Then, this work is followed by electricians, who install conduit and pull conductors, followed by insulators and the instrumentation and calibration teams. The outcome is that qualification, particularly IQ, is discontinuous with many starts and stops. Although 60 hours may have been allocated to implement a system's IQ and OQ, the entire duration could be weeks or months, depending on the construction schedule (see Figure 3). It is the validation project manager's responsibility to carefully review the construction schedule to determine when systems will be mechanically complete, started, and turned over to the owner. On the basis of this review, a validation schedule can be prepared that may be synchronized with the construction schedule.
Figure 3: A partial validation schedule integrated with a facility construction schedule.
Foremost, validation deals with complete systems, not sections or portions, and the validation schedule must be arranged as such. System completion dates must be projected from the construction schedule and added to the validation schedule. Project management software such as Microsoft Project (Microsoft Corporation, Redmond, WA) is commonly used to develop and maintain this schedule, although other software is available and equally useful. Dependencies between construction and validation must be created in the validation schedule to allow for construction delays or improvements. When preparing a validation schedule from the underlying construction schedule, the following points should be considered:
This section has addressed the preparation, scheduling, implementation, and verification of protocols and SOPs. For an average facility, it is common to have 30 or more qualified and validated systems, each with its own required documentation. Clearly, careful upfront planning is required to ensure that all validation activities are coordinated with the facility construction schedule. Because most construction projects are fixed price, it is unlikely that the construction team will delay fabrication, installation, or start-up because protocol development and approval are late. It is essential to deliver approved documents on time for implementation. Any delay results in opportunity lost, added expense, and a prolonged schedule.
Equally important is the development of forms for recording data and information in IQ protocols. All protocols must be based on design information. Because specifications can be seemingly limitless for complex sanitary and process systems, the breadth and depth of inspection and testing must be carefully managed. Acceptance criteria also must be based on design, not arbitrarily established when no specifications exist. Incorrect acceptance criteria often lead to test failures, unnecessary investigations, and needless expenditure of resources. Validation personnel should remember that validation provides a high degree of assurance—not absolute assurance—that a system was installed and operates according to design. Assurance approaching absolute comes at substantial cost, usually manifested by unnecessary, irrelevant testing, numerous deviations, and schedule delays.
Turnover package (TOP) preparation
TOPs for pharmaceutical and biopharmaceutical facilities is an optional, but important, program that was first described formally around 1987 (5), although the concept originated several years before then. In the late 1980s, the TOP program was instituted at several new facilities, mainly biopharmaceutical, by one large engineering firm specializing in biotech facility design, construction, and validation. Other companies quickly adopted the program. The objective then, as it still remains today, is to shorten the duration required to completely validate a manufacturing facility. More specifically, TOPs were designed to replace~90% of typical IQ inspections and verifications recorded in protocols, while still achieving the same degree of regulatory compliance. When a TOP is properly planned and implemented, these objectives are accomplished, with the added benefit of consolidated system documentation available to the maintenance team at project completion. In this respect, the TOP is the precursor for much of the current industry effort to commission, rather than validate, pharmaceutical facilities today. Nonetheless, the TOP is still relevant, because it provides a definitive system baseline for all validation activities, facility operations, and maintenance requirements that follow.
TOP planning and development should begin early in the facility design process, preferably when P&IDs and airflow diagrams (AFDs) are approved. The TOP is a prevalidation activity that continues until a system or piece of equipment is transferred to the owner. The construction manager must be made aware of TOP requirements during the bidding process, not after, because the TOP may add to construction cost and schedule. Before beginning a comprehensive and organized TOP program, the following elements must be addressed or compiled:
It is advisable that TOP implementation be integrated with and managed as part of the validation program, if possible. There are several reasons for this. TOPs directly benefit the validation team because each TOP contains system drawings, specifications, and manuals that are useful for protocol development and execution. Furthermore, the TOP contributes to overall facility GMP compliance. Most validation personnel have QA backgrounds, with the attendant attention to detail and knowledge of GMP.
Figure 4: Boundary between two validated systems: purified water system and fluidized-bed dryer make up air unit.
A three-ring binder is allocated for each system or piece of equipment scheduled for TOP development. Multiple binders may be required for large or complex systems with many subsystems and components. Then, system boundaries are established for each identified system (see Figure 4). Boundaries are usually designated by highlighting system drawings (i.e., P&IDs and AFDs) to demarcate systems. Because utilities interface with or support other process systems and equipment, it is important to identify components, instruments, piping, and so forth, included in one system but excluded from the other. The following suggestions may be useful when establishing system boundaries:
It is the TOP coordinator's responsibility, in consultation with other project team members, to define system and equipment boundaries. Once boundary drawings have been prepared, copies are transmitted to the construction manager for reference purposes and the validation team to support protocol preparation. The original highlighted drawing is inserted into the corresponding TOP binder.
Concurrent with system boundary development, a list of required forms, reports, certifications, drawings, and submittals for TOP inclusion is prepared by the coordinator and forwarded to responsible parties, including equipment vendors, Instrumentation–Calibration Departments, and the construction manager and subcontractors. Project participants may be allowed to use their own standard documents (e.g., pressure test forms, cleaning certifications, calibration data sheets), provided these forms supply the detail and authentications required by GMPs. All vendor and contractor standard forms should be reviewed in advance, if possible. If these documents are unacceptable, test forms that meet both project and GMP requirements should be prepared and offered as replacements.
After boundary drawings and the document list have been distributed, a matrix is prepared for each system or piece of equipment where TOP is required. A spreadsheet is commonly used to organize the matrix. The left column of the spreadsheet is subdivided into the following areas and disciplines:
Each area and discipline is further subdivided by the document type required to prove acceptable system installation and precommissioning. Using process piping as an example, documents to be assembled and filed with the TOP may include:
Similar documents that provide proof of satisfactory installation and operation also are identified for each of the disciplines (e.g., general and equipment) described previously.
The matrix is completed by reviewing each boundary drawing and then recording equipment numbers, instrument numbers, or line numbers in the cells in the top row (one number per cell) (see Figure 5). Only major, numbered items should be recorded (e.g., pumps, tanks, control valves, fans, instruments, and filters). Hand-operated valves and other minor components should be omitted. With the basic matrix completed, the project team then must decide which documents, certifications, and reports are required to substantiate satisfactory installation. Some document types will be required for some systems but not others. Ordinarily, cells in the body of a matrix are crossed or shaded to indicate where a specific document is required. Later, as documents are received and added to the TOP, the corresponding cell can be darkened or crossed again to indicate that the requirement has been fulfilled. In this way, the matrix acts as a punchlist, so that missing items and other important information can be determined and identified by simple review.
Figure 5: Typical turnover package (TOP) matrix (1 of 6 total pages).
After matrices have been prepared for all selected systems and equipment, each matrix must be approved by the construction manager, facility owner, TOP coordinator, and QA department. Approval demonstrates the importance of TOP as a partial replacement for IQ and helps reinforce the value of TOP for expediting project completion.
Each matrix is placed into the corresponding system binder along with the boundary drawing. It is now the TOP coordinator's responsibility to monitor construction progress and to collect and file the documents identified in each matrix. Some of these documents relate to common inspections such as motor–pump alignment, piping pressure testing, and air balancing. In many instances, the TOP coordinator, along with the owner and a QA specialist, should witness field testing as performed. The product of this testing (e.g., reports, certifications, and test forms) should be reviewed, approved, and added to the TOP binder. It is important to get construction manager agreement early that tests and verifications will be witnessed because this may cause a momentary delay in the construction schedule while individuals are assembled to witness the test (5).
In addition to test forms and certifications, equipment specifications, manuals, factory reports, and so forth should be included in each system TOP. Copies of red-lined drawings also should be added until redrawn as-built drawings become available. Because system TOPs may be consulted numerous times while construction is underway, control of incomplete binders is essential. Validation staff often refer to information (e.g., manuals) in each TOP to prepare protocols and SOPs. Original documents and test reports must not be accidentally removed and lost.
When all TOP matrix requirements have been satisfied for a particular system or piece of equipment, the binders are audited for completeness. When properly planned and executed, the TOP binder is essentially complete when the system is turned over to the owner for OQ and PQ. Individuals who approved the matrix initially approve the completed TOP. Once approved, it is common to prepare three copies of each binder—one for maintenance, one for operations, and a third for archive purposes. IQ completion then becomes an academic exercise that is basically limited to certifying TOP approval and documentation of system SOPs. Finally, as systems undergo change control and are modified and altered, the corresponding TOP is updated accordingly.
Deviation and discrepancy management
Deviations from protocol test methods and acceptance criteria are commonplace and inevitable when protocols are implemented. As systems and equipment become more complex, the number of deviations and discrepancies per system tend to increase. In addition, deviations occur more frequently as the breadth and depth of inspection and testing increases. Generally, there are three sources of protocol deviations:
All deviations require careful management, evaluation, and scientific justification. If a system or piece of equipment exhibits multiple deviations, one should question whether the system is complete and suitable for validation. Within the past 15 years, protocols have begun to include pages for documenting and explaining deviations as standard features. This has partly arisen because validation concepts today are much better understood, particularly the relationship of design conformance to product quality and therapeutic response. One fact is absolutely clear about system and equipment deviations: There should be no anticipated effect or alteration on product quality beyond official or recognized standards (3).
Deviations—be they protocol errors, test failures, or discrepancies from test methodology—must be fully described on protocol deviation pages and in validation final reports. All remedial actions required to restore the equipment or system to an operational and compliant state should be fully explained. If properly implemented, commissioning will serve to reduce deviations to negligible levels. Only validation-ready systems will be available for inspection and testing. For multimillion-dollar projects with numerous systems and pieces of process equipment, a centralized deviation management function may be beneficial. Centralized management ensures that each deviation is processed uniformly and will remove the burden of follow-up and closeout from the validation engineer and scientist. Furthermore, a central program is well suited for computerization, perhaps using any of the CAPA-style programs currently in use today.
Beneficial, or "positive" deviations, also should be acknowledged and explained. Positive deviations occur infrequently, and are usually limited to improved materials of construction (e.g., 304L stainless steel specified, 316L delivered), expanded operating ranges and additional, unspecified features. These deviations should at least be recognized as such, although there is rarely any need to perform an in-depth investigation or analysis.
Conclusion
This article completes a two-part series about the essentials of effective validation project management. Every project is unique in size and complexity, but common features are found in all projects. These include the requirement for design review to ensure compliance with good manufacturing practices, labor and cost estimating, and master plan development. Turnover packages in the style described in this article may be unfamiliar but should be considered for all projects, regardless of size, duration, or urgency of completion. All projects use protocols to prove and document the "high degree of assurance" required by regulatory authorities. Nonetheless, it is important that protocols be carefully designed to manage the degree of inspection and testing, to avoid low value or needless verifications, and to arrive on time approved and ready for implementation. Unfortunately, deviations are often too familiar and probably unavoidable but can be lessened by design control, system commissioning, and careful document review.
Successful validation project completion is never guaranteed. All projects confront uncertainty, conflicts in schedule and limitation of resources. The concepts and details of validation are sometimes misapplied and frequently misunderstood. Validation is multidimensional. It requires special individuals who are scientifically literate, mechanically oriented, and very practical. Validation is also very systematic. Failure of a precursor (e.g., installation qualification or operational qualification) often translates into suboptimal performance of the successor (e.g., operational qualification and performance qualification).
Considering that many validation projects are performed once in a career or corporate life cycle, it is no surprise that projects frequently extend beyond completion dates and budgeted costs. Properly planned and managed projects increase the likelihood of success. Poor results usually are the outcome of incomplete planning and inadequate execution. By following the guidelines and principles presented here, project goals (e.g., completion on time, at budget, with quality) become more attainable. Considering the cost to build new facilities and the added cost to validate them, well-managed validation projects are self-compensating. Rework, operational inefficiencies, and compliance concerns are reduced or eliminated. Clearly, these benefits, both economic and regulatory, should never be overlooked or minimized.
William Garvey is a senior advisor at Pfizer Global Research and Development, Eastern Point Road, Groton, CT 06340, tel. 860.715. 2277, fax 860.715.7806, william.garvey@pfizer.com.
References
1. W. Garvey, "Essentials of Validation Project Management, Part I," Pharm. Technol. 29 (12), 68–76 (2005).
2. US Food and Drug Administration,"Guideline on General Principles of Process Validation," (FDA, Rockville, MD, May 1987), p. 2.
3. W. Garvey, "Effective Validation Project Management," oral presentation at Interphex Conference 2005, New York, NY, April 26–28 2005.
4. W. Garvey, "Integrated Validation Programs for Solid Dosage Facilities–Part 2," Amer. Pharm. Rev. 2 (3), 17 (1999).
5. M. Chin, "TOP: A Rational Approach For Ensuring Proper Biopharmaceutical Plant Construction," in Proceedings from PharmTech Conference '87 (Aster Publishing Corporation, Eugene, OR, 1987).