Application of Validation Principles to the Training Quality System: Part 9, Quality Risk Management Hazard Identification

The training function in a pharmaceutical facility is vital to site operations. The objective of training is straightforward; training is responsible for employee learning. Site training programs must be effective to accomplish employee learning. Effective training is difficult; the broad scope of topics, trainer communication skills, trainee motivation, logistics, documentation, maintenance of training quality, and other considerations may challenge effective training. Training is a cost; training in some organizations may be restricted by budget, time, facilities, conflicting priorities and criticalities, or other limitations. Identifying problematic aspects in training and appropriately addressing these considerations is crucial to accomplishing effective employee learning.

Discussion topics

This manuscript discusses the application of quality risk management (QRM) concepts within a site training program. QRM application helps to identify potential problematic aspects in training; these potential issues may then be addressed by following the QRM process. ICH Q9(R1) is the fundamental document describing QRM (1). Hazard identification (i.e., potential problem identification) is the first step in the QRM process. This paper focuses on problem identification within a comprehensive training process. Specific application of QRM to training and education is identified in ICH Q9(R1).

QRM application in training provides a systematic approach that complements traditional training—a thought process distinct from usual training. Typical training involves information, actions, and procedures (i.e., how to do things); training should also address understanding (i.e., why are things done). QRM identifies potential problems during training—QRM addresses what might go wrong. QRM thus improves training and makes training content more relevant. QRM application facilitates poka-yoke, mistake-proofing, defensive design, or other approaches intended to prevent problems—QRM proactively facilitates problem prevention. Adult learners in pharma are practical in orientation; QRM problem identification is harmonious with adult learning (2). Examples of potential training problems are presented throughout the following discussion:

Training organizations in pharma. Training functions within pharma may be organized in different ways. Training may be the responsibility of each technical function (e.g., engineers train other engineers, chemists train other chemists, etc., within function departments); each function accomplishes their individual personnel training. Centralized training with varying levels of activities is also possible; some centralized functions may be responsible for minimal administrative activities (e.g., personnel training records); others may manage training requests, training content documents, schedule sessions, and periodically monitor training quality—a much more comprehensive responsibility. Some training organizations may be linked to corporate human resource (HR) systems to integrate government-mandated training within employee files. The following discussion addressing QRM in training utilizes a centralized function with responsibility for individualized training and administrative activities. The point of this discussion is the identification of potential problems adjunctive to typical training regardless of organizational structure.

Training management. The role of training management is critical to training program success. Training management is responsible for actual training as well as training function management. Quality systems guidance documents (3–5) identify expectations of management; QRM considerations are also mentioned in these documents. Training management may need to mediate conflicting training time/costs vs. site criticalities with other site functions. Other management training responsibilities are also crucial for success. Site training programs often rely on personnel in other departments to accomplish training objectives; training management must ensure that these individuals know and understand their responsibilities supporting training. Training management also provides an unspoken message to trainees during training; conscientious effort in training with thorough and well-designed training content conveys a message of quality and compliance to employees. A common thread in deficient training is a lack of visible management involvement; one cannot demonstrate commitment by staying in the office and going to meetings. Training management must be “on the floor” when and where training happens (i.e., management by walking around [MBWA]) (6). Current workplace trends in Zoom management exacerbate management lack of visibility. An ongoing commitment to training excellence demonstrated by management visibility helps to develop a quality culture in the organization.

Training and process validation. This discussion is structured according to the FDA lifecycle approach to process validation (7). The process validation model provides a comprehensive approach to training. Process validation is designed, executed, and continually maintained; training activities can likewise follow this same approach. Table I describes three stages in a training lifecycle akin to three stages in process validation. Training stage 1 describes the design of the training topic including content, presentation, and associated elements; stage 2 comprises actual training performance by trainers and trainees; stage 3 addresses the ongoing maintenance of training (i.e., effectiveness, content accuracy, and compliant performance). A basic tenet of process validation is the identification and control of variation; training problems may include performance variation by trainers, uncontrolled logistics, and other causes. Training management should develop a training master plan akin to a validation master plan containing training strategies, approaches, procedures, documentation, and commitments. The following continues previous discussions correlating training with process validation including validation application overview, training content development, trainer qualification, trainee prerequisites, the training process, performance monitoring, management overview, and documentation (8–15).

QRM terminology

QRM terminology may often confuse readers based on common language within their pharma settings—specifically the words ‘hazard’ and ‘risk’. Hazard and risk are often assumed to describe dangerous workplace situations (e.g., fall, slipping, electric shock, overhead machinery, others). Per ICH Q9(R1), ‘hazard’ and ‘risk’ are not synonymous (16,17). Hazard is a source of harm; after a hazard is identified, its risk of causing harm (high, medium, low, or descriptive numeric) is assessed using risk assessment tools or subjective methods. Risk assessment describes the probability of harm occurrence and the level of harm severity; some common risk analysis methods include failure mode and effect analysis (FMEA), preliminary hazard assessment (PHA), and fault tree analysis (FTA); these tools are identified in ICH Q9(R1) Annex I. FMEA integrates frequency, severity, and detection in its Risk Priority Number (RPN), which quantifies risk. Table II more completely defines terminology as explained in Q9(R1) training modules and used in the following discussion. Q9(R1) changed the initial step in the risk assessment process to ‘hazard identification’ from ‘risk identification’. Although this change enabled consistency with the definition of hazard in other regulatory documents, it may cause confusion in common word usage. Hazard identification in training is the primary objective of the following discussion.

Stage 1: training design

Stage 1 comprises the design, planning, and development of training to be executed in actual employee training. Training design must comprehensively address the topic to be trained. Stage 1 design addresses new training as well as evaluating and integrating training changes identified during subsequent training stages. When the training process is designed, conscious thought about what might go wrong is integrated with the how and why being trained. Training design is the most important activity in the training process and may be complex depending on the subject topic. The following briefly describes elements of training designed in stage 1:

Content, presentation, testing, and process. Numerous content categories (e.g., policies, procedures, equipment operation, analytical, etc.) are required for pharma employee training. Content presentation describes the method of delivering content; computer delivery (e.g., one-way delivery), virtual reality, Zoom interactions, classroom training, and skill delivery (i.e., on-the-job [OJT] training with a subject matter expert [SME]) are general categories. Testing to verify effective training is related to content complexity; “read and sign,” multiple choice, and skills demonstration to an SME are options.

Supporting elements. Supporting elements in training design include the trainer, trainee, and logistics. Background, technical expertise, and communication skills of trainers must be consistent with training complexity. Trainees must be capable of competent performance post-training. Training logistic considerations such as facilities and equipment, number of trainees per session, scheduling, internet access, and other relevant elements are specified in training design.

Documentation—Training Content Record. All of the above must be recorded in an approved controlled document for use in future actual training. This document serves as an ongoing record of training subject matter. Document changes are controlled via change management; content is modified as experience is gained, and risks are evaluated. Supplementary information used in training to emphasize key content is appended.

Management. Training management is ultimately responsible for training design and related elements. Establishment of training standards for content, presentation, testing, trainers, trainees, logistics, and related elements is required. Standards must be documented. Standards are the backbone of the training effort that helps to establish a quality culture in the organization. Training compliance with standards should provide a quality message to trainees. A Training Master Plan listing training standards, approved Training Content Records, and other function information is recommended.

Stage 1 hazards. Stage 1 hazards are identified during the design and development of the training content by the training author. Hazards are identified concurrently with the ‘what’ and ‘why’ of training; authors address ‘what might go wrong’ along with the what and why in subject matter. Addressing potential hazards within training leads to more effective content, presentation, and testing. Design personnel must anticipate potential problems during training design. Illustrations, videos, checklists, and other aids addressing problems should be incorporated into training design. Trainers must verbally emphasize critical content in their training presentation. Testing should emphasize questions on critical content. Addressing training problems through different types of learning (e.g., visual, auditory, reading/writing, kinesthetic, etc.) is recommended; employees learn in different ways. The greater the anticipated risk to product, process, or organization, the greater the potential problems, and the greater the effort that is required in training—a principle of ICH QRM. Table III provides examples of stage 1 hazards based on experiences and observations.

Case study #1: deficient training design—machine setup. Customer complaints and returned product were received on a new prescription sedative product. Distinct white spots on the blue tablet surface were observed. Recall letters were sent and product was back-ordered with availability, financial, and regulatory ramifications. Engineers determined that milling equipment was incorrectly set up causing large white-spot particulate. Performance by manufacturing technicians was the hazard in this case; incorrect machine-part assembly was directly correlated to ineffective training. Improvements in training content and illustrations highlighting correct setup were developed. Trainees were required to demonstrate correct equipment assembly to an SME to successfully pass training.

Stage 2: training performance

Stage 2 training performance comprises the executed training of a trainee by a trainer and review of the training process. Stage 2 activities confirm the stage 1 design and identify necessary modifications—actual stage 2 training experiences between trainer and trainee may differ from stage 1 expectations. Stage 2 observation also affords opportunity to confirm trainer–trainee focus on critical aspects of the training subject matter:

Training performance. The actual training process executes the planned training designed in stage 1.

Training observation and review. Stage 2 training is evaluated; trainer, trainee, training management, training designer, and other interested individuals are involved. Overlooked or unanticipated problems are noted. Trainer emphasis on critical parameters is reviewed. Trainee engagement is evaluated.

Documentation—Personnel Training Record. The Personnel Training Record documents completion of training for employees. When an auditor asks to see the training record for a specific employee, this document will be provided. Training documentation contains dates and times of training sessions; these must be consistent with training design. If training is not clearly documented, it didn’t happen.

Management. Observation and evaluation of the training process by management provides an opportunity to enhance training design and performance. If changes are needed, the stage 1 Training Content Record is modified through the site change management process.

Stage 2 hazards. Stage 2 hazards that were not anticipated in stage 1 design but become apparent during actual training are identified. New hazards noted in stage 2 are evaluated per ICH Q9(R1) methods and integrated per change management. Actual performance variation in trainers and trainees not anticipated in stage 1 design is noted. Training content, testing, logistics, and other considerations are modified in the Training Content Record via change management. Table IV and case studies provide examples of potential stage 2 hazards.

Case study #2a: training logistics—training in the auditorium. Good manufacturing practice (GMP) training for more than 200 site employees was conducted in response to a regulatory audit observation. The site auditorium was reserved, and a two-hour session was scheduled for 3 pm to include first and second shift employees. The session discussed data integrity, original data recording, error corrections, signature/date verification, “white-out” use, “post-dating,” and numerous other documentation topics. A few days after the training was completed, a site engineer was observed recording original data on scrap paper for later transfer to his notebook. Other technicians were observed writing data on “Post-it” notes instead of batch records. Continuing “white-out” use was also observed. Many key topics addressed in the training were not understood—training was an obvious failure. Poor training logistics was the fundamental hazard that caused ineffective training. Failure modes included too many people in the same room, a lengthy session at the end of the workday (i.e., sleeping employees), uncomfortable room temperature, smartphone distractions, and too many topics without adequate discussion. Trainer communication was also identified as a hazard; supplementary illustrations, lack of hardcopy handouts; and no Q&A with trainer were deficiencies. Training was poorly designed—-an efficient training session became more important than employee learning. Training was disingenuous; management just wanted it done to claim accomplishment of the regulatory commitment. Training was not given appropriate time, effort, and priority to accomplish employee learning. New sessions with smaller groups and improved presentation of content were developed and rescheduled. Large group auditorium training sessions were eliminated in future training.

Case study #2b: training logistics—computer fatigue. A site training program for “read and sign” training was managed through a training computer system. Employees were assigned training materials to be read; computer links were provided. Employees were allowed a 30-day time limit to complete the training; completion was self-paced. Employees would routinely give training a low priority and delay training until the deadline. This approach was not acceptable for new employees assigned an excessive number of training modules; more than 50 documents were often “read and sign” in one day—training was ineffective. Trained performance was the hazard identified in this case. The training computer system was modified to allow completion of not more than five training modules in one day.

Case study #3: training by observation. A senior SME operator was the designated training person for a non-sterile manufacturing area. He always accomplished training of new personnel in one demonstration session. Some procedures required several hours of demonstration. The trainee and trainer then signed documentation that the trainee was qualified to perform the task without actual trainee demonstration of competency. When the newly qualified trainee was assigned manufacturing tasks, the senior person would be nearby to help as needed. Training deficiencies became apparent when the trainee was working alone. When the trainer was asked about his training methods, he responded that his manager told him to complete training as quickly as possible so that the trainee can begin working. Deficient technician performance was the hazard in this case. The trainer should have adjusted training repetition and emphasis based on trainee understanding and complexity of training content. After adequate repetition, the trainee should have been required to independently demonstrate trained skills. Training could not be firmly standardized due to the variation in trainee skills; experienced operators could be adequately trained by observation; inexperienced operators required additional repetition. Management did not give the priority to training due to production criticalities; production criticalities were more important than training. The manager had no incentive to make training a priority vs. his manufacturing responsibility; what the manager did undermined or undercut training. Future training required complete skills demonstration to an independent SME.

Stage 3: continued training verification

Continued training verification comprises the periodic review of training content and performance. Activities in stage 3 enable continuous improvement in training and maintenance of training quality. Training periodic reviews are specified in a Training Master Plan or other scheduling commitments:

Training effectiveness. Training effectiveness is evaluated by review of experiential documents (e.g., deviations, non-conformances, investigations, corrective and preventive action [CAPA]), stakeholder interviews, trainer/trainee discussions, and other reviews that critically evaluate training performance. Investigations citing training deficiencies as causes of process deficiencies indicate the need for training improvements. Problem trends should be noted and addressed.

Content and ancillary documentation. Training content in the Training Content Record is reviewed for accuracy. Changes in equipment, processes, cleaning methods, and other variables may have occurred without concurrent changes in training. Supportive documents (e.g., flow diagrams, piping and instrumentation diagram [P&ID], sampling protocols, etc.) relevant to training may also be obsolete, inaccurate, or incomplete.

Procedures. Performance of trained procedures may also have been modified in response to equipment changes or improved efficiency without concurrent changes in training. Employee shortcuts in performing trained procedures are a common occurrence. Non-compliance with procedures are among the highest frequency of FDA-483 citations (18)

Documentation—Training Periodic Review. Performance of the Training Periodic Review must be documented.

Management. Completion of periodic training reviews as identified in the Training Master Plan or other commitment is the responsibility of management. If new hazards are identified, these must be submitted for evaluation through the QRM process and change management for implementation if appropriate.

Stage 3 hazards. Stage 3 hazards may be identified in periodic review activities for a variety of reasons. New hazard identification may occur years after training was successfully completed. Stage 3 periodic review provides the opportunity to review ongoing routine performance of trained activities vs. approved training content, supporting materials, and procedures. Changes may have occurred in any of the above, making training inconsistent with aforementioned materials. For example, utilities may have been modified, supporting sampling diagrams may have changed, procedures may have changed, or practices may have changed without concurrent changes in training. Stage 3 hazards are also evaluated and integrated in stage 1 design per change management. Table V and case studies provide examples, experiences, and observations of potential stage 3 problems.

Case study #4: low tablet target potency. Periodic review of a tablet compressing process was conducted. The manufacturing technician was an excellent machine operator (“Best Technician Award”). His manufactured products always met acceptance specifications and product yields were always above average. Further evaluation of product potency data made under his responsibility indicated a slightly low average potency (97–98%) compared to lots made by other personnel. The manufacturing supervisor observed his machine setup and operation; tablet weight setup was within specifications, but always at a theoretical weight of less than 100%. When questioned why tablet weight was not adjusted to 100%, the technician responded that he always set a slightly low weight so that he would have a high product yield—management wants high yield and more profit—this is why he was “Best Technician.” Technician performance was the hazard in this case. GMP allows product release within specification (usually 95-105%), but product must be formulated to contain 100% unless otherwise justified. The technician was retrained on GMP requirements. Training documentation was revised to emphasize 100% formulation target requirements. Recognition procedures were modified to eliminate product yield as criterion for employee performance.

Case study #5: supporting documentation. Periodic review of site utility systems was conducted. Documents reviewed included procedures, supporting documents, and associated training. Supportive drawings and training documents were incomplete and not updated. Hazard in this case was inaccurate documentation. Investigation of discrepancies between documents indicated that final changes on drawings and training were not completed due to competing work priorities; responsible personnel were assigned other projects with higher priority. Review of other systems (e.g., compressed air, ethanol, and other utilities that had been recently modified) had similar inconsistencies—engineers were never allowed to completely finish a project before being reassigned by management to higher priority work.

Case study #6a: training culture—cleaning process review. Internal review of a product cleaning process was conducted in advance of a regulatory audit. Technical, Manufacturing, and quality assurance (QA) supervisors observed technician performance. Several discrepancies compared to approved procedures were noted. The amount of cleaning agent was modified; specific steps were changed to increase efficiency; an unspecified organic solvent was used to remove stains. When questioned about discrepancies, the manufacturing technician expressed site reality—the approved cleaning procedure was ineffective. Management advised to fix the cleaning problems without involving the technical group. The site culture was identified as the hazard in this case. The above occurrence demonstrated problems with interpersonal relationships, interactions between groups, function silos, and general disrespect for approved policies and procedures; technicians proudly stated, “We do whatever it takes to get the job done.” Management leadership and attitude were also identified as contributing to this hazard. The site quality culture begins with management (19). Several corrective actions were initiated within various functions at the site. Personnel learned to follow procedures and not “to do whatever it takes”.

Case study #6b: training culture—aseptic compounding. A sterile compounding facility required technician training and annual re-qualification on aseptic techniques. Testing required handwashing, garbing, work area sanitization, technique demonstrations, aseptic transfer of sterile media between vials, finger impressions testing, and related activities. Personnel observing testing noted unusually high frequencies of hand sanitization and surface sanitization by technicians; supervisors were continually reminding trainees to sanitize their hands. Most troubling was hand sanitization immediately prior to finger impression testing. The donning of new sterile gloves or spraying 70% alcohol immediately before finger-impression testing does not challenge aseptic technique and does not represent actual work performance. The site culture was identified as the hazard in this case. Technicians explained that the department manager told them, “There shall be no problems with qualification testing—do whatever necessary to prevent failures.” A deficient quality culture at the site translated into substandard employee behavior..

Summary

This paper discusses applications of QRM principles in a site training quality system. QRM applications enable several significant benefits to training.

QRM in training content. QRM in training focuses on identification of training problems. QRM application provides a distinct and comprehensive approach that complements traditional training. Training fundamentally comprises prospective activities (i.e., how to do things and why things are done). QRM provides additional relevant considerations to training—what can go wrong. QRM thus helps to describe training reality in the workplace. Hazard identification facilitates problem evaluation and subsequent action as needed.

QRM in training lifecycle integration. This discussion is structured according to the lifecycle approach as used in process validation. A lifecycle approach to training is described that affords opportunities to review and continually improve the training process. Three stages of QRM activities are identified, and QRM hazards are identified in each stage:

Stage 1 training design comprises planning and relates to hazards that are associated with training content, trainers, trainees, and the training process; stage 1 hazards are identified when new training is originally developed.

Stage 2 training performance relates to hazards associated with actual training activities when trainers present content to trainees; observation of stage 2 training may identify new hazards and necessitate modifications to content, trainer, or other designed elements.

Stage 3 continuous training monitoring (i.e., training periodic review) relates to hazards associated with the ongoing performance of trained procedures—training effectiveness, content accuracy, employee performance, consistency with related documentation, and compliance with approved procedures. Observation of training content and/or performance in stage 3 may also necessitate modifications to training content.

Figure 1 illustrates the respective stages of the training lifecycle. Stages of the training lifecycle (black print) anchor the overall training process; hazards are originally identified in stage 1 when training is developed. If new hazards are identified in stage 2 or stage 3 (red print), these are forwarded to stage 1 for evaluation and integration with training content, presentation, testing, and other elements; training quality is maintained and continually improved by this process.

QRM in training management. Training management has significant responsibility and influence on performance of site training as follows:

Responsibilities. Training management is responsible for all training activities in all stages as well as for the general training function administration. Training is a cost; training management must balance accomplishing training objectives within site budgetary constraints. Numerous QRM hazards may be associated with management responsibilities. Management must integrate QRM considerations to maintain and improve training content. Management in this context includes training management as well as other site function management involved in training.

Associated training personnel. A site training program often utilizes individuals from other site functions as trainers, SMEs, training content authors, or other contributors. These individuals also have direct responsibility for employee training. Training management must ensure these contributors know and understand their responsibilities, including QRM considerations, and that they are accountable for successful employee learning.

Quality culture. Training management significantly influences the attitudes and performance of trained employees. Employee training is among the first activities of new employees; this relationship continues throughout the employee career with additional training, re-training, and other interactions. Management commitment to training excellence, including QRM considerations, must be visible and sets the tone for employee performance. Management actions influence the development of a quality culture in the organization.

Acknowledgments

Helpful discussions with Kevin O’Donnell, PhD, Health Products Regulatory Authority (HPRA) Ireland; Alan M. Mancini, RPh; and Richard Poska, RPh, managing director, Flexo CMC Consulting, Nokomis, Fla., are greatly appreciated.

References

1. ICH. Q9(R1) Quality Risk Management, Final version (January 2023).
2. Pappas, C. Eight Important Characteristics of Adult Learners. elearningindustry.com, May 8, 2013.
3. ICH. Q10 Pharmaceutical Quality System, Step 4 version (June 2008).
4. FDA. Guidance for Industry, Quality Systems Approach to Pharmaceutical CGMP Regulations (CDER, CBER, September 2006).
5. FDA. CDER Quality Management Maturity. fda.gov (accessed July 24, 2024)
6. Belyh, A. Management by Walking Around (MBWA)—The Essential Guide. cleverism.com, July 28, 2020.
7. FDA. Guidance for Industry, Process Validation: General Principles and Practices (CDER. CBER, January 2011).
8. Pluta, P. L. Application of Validation Principles to the Training Quality System. Part 1—Overview. Journal of GXP Compliance 2018, 22 (6).
9. Pluta, P. L. Application of Validation Principles to the Training Quality System. Part 2—Technical Training Modules. Journal of GXP Compliance 2019, 23 (1).
10. Pluta, P. L. Application of Validation Principles to the Training Quality System. Part 3—Trainers. Journal of GXP Compliance 2019, 23 (3).
11. Pluta, P. L. Application of Validation Principles to the Training Quality System. Part 4—Trainees. Journal of GXP Compliance 2019, 23 (4).
12. Pluta, P. L. Application of Validation Principles to the Training Quality System. Part 5—Training Process. Journal of GXP Compliance 2020, 24 (1).
13. Pluta, P. L. Application of Validation Principles to the Training Quality System. Part 6 —Continued Training Verification. Journal of GXP Compliance 2020, 24 (6).
14. Pluta, P. L. Application of Validation Principles to the Training Quality System. Part 7—Training Management Overview. Journal of GXP Compliance 2021, 25 (4).
15. Pluta, P. L. Application of Validation Principles to the Training Quality System. Part 8—Training Process Documentation. Journal of GXP Compliance 2021, 25 (6).
16. ICH. Quality Risk Management, ICH Q9(R1) Training Slides Hazard Identification—Part 1. ich.org/page/training-library (accessed July 25, 2024).
17. ICH. Quality Risk Management, ICH Q9(R1) Training Slides Hazard Identification—Part 2. Consideration in Relation to Identification of Failure Modes, in the Pharmaceutical Environment. ich.org/page/training-library (accessed July 25, 2024).
18. Hoare, A. The Most Common FDA 483 Observations. ideagen.com, April 18, 2023.
19. Friedman, R. Establishing a Culture of Quality. fda.gov, September 2021.

About the author

Paul L. Pluta, PhD, RPh, is a pharmaceutical scientist with experience in pharma industry, university academia, pharmaceutical journal editorship, and community and hospital pharmacy practice.

Article details

Pharmaceutical Technology®
Vol. 48, No. 11
November 2024
Pages: 20–26

Citation

When referring to this article, please cite it as Pluta, P. L. Application of Validation Principles to the Training Quality System: Part 9, Quality Risk Management Hazard Identification. Pharmaceutical Technology 2024, 48 (11), 20–26.