Are all your SMEs Qualified Trainers?

I got a phone call from my Lead SME’s boss one morning. “How many more sessions do you need”, I asked him. I had already delivered 4 back-to-back workshops with class sizes of 25-30 SMEs; which was beyond optimal. So I asked him why. I needed to find out what was driving the surge in identified Qualified Trainers (QTs). I learned that a retrospective qualification needed to take place in order to close out an inspection observation. The total number of SMEs needing “proper paperwork” was well over 700. Since the redesigned training system was now in effect, these undocumented SMEs as Trainers would have to follow the new procedure. Or would they? Our discussion shifted to what type of training these SMEs will be delivering.

I then shared a related story with him. Several years prior, I got entangled with a “CAPA crisis” that involved QTs. No sooner did we launched the QT program and put the new procedure into effect, the CAPA quality system temporarily shut down shipping over a weekend. Upon return to the site, I was summoned to an emergency meeting from the security gate. Amazingly, a new practice/rule that only a Qualified Trainer can conduct training evolved from “only OJT QT’s can deliver OJT and perform Qualification Events” as per the SOP! This was clearly a case of misunderstood scope.

Does every SME need to be qualified as a Trainer?

Trainers Grid for determining QTs Scope of Training

In the Life Sciences arena, there are 5 recurring situations that require training: Self, Corrective Actions, Classroom (ILT), Structured OJT, and Qualification Events (Final Performance Demos).

Self can be achieved by the individual reading the procedure and signing the training record. This is also known as Read & Understand (R & U) for SOPs. I personally don’t think of it as training, it is reading. Yet, in some situations, reading is all that is required to gather the SOP information.  If on the other hand, you need to execute the steps of the SOP and complete required forms, then additional training with the SOP Author or a QT is the appropriate next level of training.

Deviations/ Corrective Actions stemming from a Corrective Action Preventive Action incident. Minimally an SME or the SOP Author is needed to ensure the credibility of the content. These types of training sessions have become known as Corrective Actions “Awareness” Training.  And more and more SMEs are now being required to deliver this training in a classroom setting.  They need to be qualified to deliver classroom sessions especially if the event is related to a significant CAPA or regulatory inspection observation.

Classroom (Instructor-Led Training) is preferred for knowledge-based content affecting a wide range of employees. The skillset needed is facilitation / managing the classroom and delivering content as designed by the instructional designer. Think of GMP Refresher sessions in the Training Room.

Years ago, it was a lot clearer to distinguish between classroom trainers and SMEs as OJT Trainers.  OJT was delivered 1-1 by “following Joe/Jane” around.  Classroom Trainers delivered their content in a classroom of many learners using slides, flipcharts, and handouts.  They were usually full-time dedicated training staff.  Instructor-led training requires training in learning theory design and practice in what used to be referred to as platform skills.  Today, it is more commonly known as “Running a Classroom” or “Basic Facilitation Skills”.  

Many of today’s OJT QTs are also being requested to deliver “Group Training” sessions on content found within their SOPs.  While the target audience may be the same set of peers, the scope, objectives, and tools used to deliver instructor-led training is vastly different from the OJT train the trainer course.

Group Training vs. OJT
Are your QT’s becoming Duo-Purposed?

Structured OJT is On the Job Training delivered by a Qualified OJT SME using the approved OJT Methodology. OJT QT’s attend the Qualified Trainers Workshop which focuses on the OJT Steps Model, how to perform the equipment, and complex SOPs via hands-on and the challenges of Life as a Trainer.  Should every seasoned employee become a QT based on their seniority and subject matter expertise? No, not necessarily.  Because there are some SMEs that don’t want to share their knowledge and therefore, may not make an effective OJT Trainer.  Establishing a set of nominating criteria provides an objective rationale for additional interpersonal qualities that help define a more well- rounded SME. 

Qualification Events (the Final Performance Demonstration) are formally documented observations of learners performing the procedure/task at hand in front of a Qualified OJT SME using an approved SOJT Checklist or rubric.  It is these events that set apart a Technical SME from a Qualified Trainer.  The QT workshop includes a dedicated lesson on what to look for during Q-Events and what the QT signature means for the integrity of the Employee Qualification Program.

Can having too many QTs be a problem?

It can be when there is no one else to train; to deliver OJT steps. While many of you may be wishing for this situation, it can eventually happen if staffing levels are adequate, shifts are normalized and SOPs revisions are managed via R & U only with the LMS.   How do you keep your QTs engaged and fresh if there are no opportunities for OJT sessions? I have some ideas for you to explore.

Re-examine the practice of online R & U only for SOP revisions.  I bet some of those revisions were significant enough for a face to face discussion (aka Group Training)  and there is probably at least one SOP revision in the past year that should have required a demonstration of task for optimum transfer of learning back on the job.  *Just because all employees are now qualified, doesn’t mean the program sits in hiatus waiting for new hires to join the company

When you have too many QTs who may be underutilized, it is also an appropriate time to administer the Trainer Mojo Assessment.  Based on the QTs scores, it might be time to say thank you for a job well done for the low scoring QTs.  You may be pleasantly surprised by who is ready to walk away from the training role?  Or you may have a cadre of QTs who legitimately need more training and hence, the need for some new modules is now justified.  Many of your excess SMEs were identified long before criteria was put into a place or the SOP was established.  If the Trainer Mojo Assessment doesn’t bring any discussion, perhaps it’s time to “re-nominate” them using the criteria within the SOP and offer a refresher series on the QT Workshop content. Or arrange for developmental assignments that expand their subject matter expertise or advances their training repertoire into a classroom facilitators? 

What is exciting for me is that many OJT-QTs are stepping up and volunteering to attend the SMEs as Classroom Facilitators workshop as part of expanding their QT’s toolkit.  Many of them want to learn more about teaching peers and working with adults.  A few have now become promoted to full-time trainer for L&D /QA departments.  Which of your OJT QTS are ready to step up and move into the classroom?  It’s time to find out and be part of the current trend.  -VB

ready to have a chat with Vivian?

Who is Vivian Bringslimark?

The Trainer MOJO Assessment can be yours. Sign up for the mailing list and be sure to mention the Trainer MOJO Assessment.

(c) HPIS Consulting, Inc.

How committed to SOJT is your organization?

Are you looking for management support for the Qualified Trainers and the time needed to deliver SOJT?  If only we were required to have a procedure for that!  It may not be an SOP or even a policy document, but industry guidance documents provide a lot of references to management involvement.

ICH Q10 – Pharmaceutical Quality System

While not mandatory, management needs to seriously take notice of ICHQ10 guidance document released in April 2009 (1).  In particular to the following:

  • MANAGEMENT RESPONSIBILITY 2.3 Quality Planning 

“(d) Management should provide the appropriate resources and training to achieve the quality objectives”.

  • CONTINUAL IMPROVEMENT OF THE PHARMACEUTICAL QUALITY SYSTEM

4.3 Outcomes of Management Review and Monitoring

The outcome of management review of the pharmaceutical quality system and monitoring of internal and external factors can include:

(b) Allocation or reallocation of resources and/ or personnel training.”

Under a Quality System: Managers Expectations for Training

Referencing the Sept. 2006 issue of Guidance Document for Quality Systems: IV. Management Responsibilities | B. Resources (2), we see alignment with the training CGMPs for continued training so “as to remain proficient in their operational functions and in their understanding of CGMP regulations.”  “Typical quality systems training should address the policies, processes, procedures, and written instructions related to operational activities, the product/service, the quality system, and the desired work culture (e.g., team building, communication, change, behavior).”

And my personal favorite, “When operating in a robust quality system environment, it is important that managers verify that skills gained from training are implemented in day-to-day performance.”  The responsibility for training under a quality system is not assigned to just one person or one function.  It is a shared responsibility across the entire organization.

Goals of the Train-the-Trainer Program (TTT) vs. OJT Program vs. Employee Qualification Program

In order to have qualified employees, they need to receive structured on the job training delivered by a qualified trainer who is content qualified and training process qualified via a Qualified Trainers’ workshop.   Each of the three “programs” has defined outcomes that are dependent upon each other.  Unfortunately, the term program has been a bit overused throughout the years and can have a variety of meanings for folks.  For purposes of this blog, TTT program means OJT Qualified Trainers workshop, OJT Program means OJT methodology/procedure and Qualified Employee program means having a robust quality training system beyond the newest Learning Management System (LMS).

As you can see from the diagram, Qualified Trainers are at the core of all three “programs”.  The OJT QT Workshop is designed to prepare the QT’s for the realities of life as a Qualified Trainer. It includes the pre-work of familiarizing themselves with the Quality Training System procedures and then in class: exploring basic learning theory and committing to what the QT signature means. The OJT Program is about delivering structuring on the job training consistently following the approved methodology. An Employee Qualification Program is the validation of training effectiveness of the OJT methodology and the clarity of the underlying procedures.

Readiness Factors for SOJT and an Employee Qualification Program

Let’s start with a written purpose statement for having qualified employees beyond it’s required.  What is the company’s philosophy on achieving qualified status?  Is there agreement among the leadership for the level of rigor required to demonstrate performance and achieve a pass rating?  Where are the SOJT program goals written?  Does a schedule exist for SOJT and qualification events other than an LMS printout with required due dates.  That is not a schedule for SOJT.  Do you have clearly defined objectives for the QT workshop captured in a document or perhaps a procedure?  Is there a single owner for all three programs or is responsibility and accountability assigned accordingly?

Ronald Jacobs and Michael Jones, in their 1995 ground breaking book, Structuring on-the-Job Training, inform us that SOJT as a system functions within a larger context, namely the organization.  SOJT is not a standalone program.  Conflicts, competing priorities and mixed messages can influence the success of your SOJT program.  What else is going on in the organization that will compete for the same set of QTs? Remember they are also your most experienced and technical subject matter experts.  How is the overall Employee Qualification program aligned with the other quality systems? 

Recognition for QTs and Qualified Status

Most QTs are not fully dedicated to delivering training for departments.   There are pros and cons for this decision.  For now, I will leave them out.  Suffice it to say, they are tasked with both their “day” job and the responsibility for delivering training when needed.  They are at times, doing two jobs.  Whether or not they are compensated additionally for delivering SOJT, acknowledging their contribution to the department and the organization is part of management support.   It takes more than “you are doing a good job, keep it up”. 

Often supervisors and managers don’t know what else they are supposed to do to show their support, other than allow them to attend the QT workshop.  The interested ones will “pop” in during lunch and chat with their direct reports.  Others will show up at the end for the poster activity (equivalent to a written test) and some will come to learn about the parking lot issues that need follow up.  The energy in the room when this happens is amazing. 

“When operating in a robust quality system environment, it is important that managers verify that skills gained from training are implemented in day-to-day performance.

Guidance Document for Quality Systems, Sept 2006

To help ease the knowledge gap between a manager and their now Qualified Trainer, I started delivering the Leadership Briefing module prior to the QT workshop delivery.  The purpose is to provide an overview of the content highlights, alignment with initiatives / CAPAs/ agency commitments and more importantly to secure agreement for the following:

  • criteria for nominating a QT
  • roles and responsibilities of QT
  • scope of work QT’s can be assigned
  • expectations for QT’s post launch
  • what happens day one after workshop is done
  • what is the status of the SOJT checklists
  • scheduling and budget concerns.

If the organization says they support the qualification program, then what happens when employees achieve qualification status?  Nothing?  A non-event? Or is it announced in newsletters, plasma screens and other company announcements?  Is it a big deal to be able to perform independently and free up a much-needed QT for another learner?  I keep hearing over and over again about how there aren’t enough QT’s to deliver SOJT the right way.  One would think qualification status on SOPs, especially big complex processes deserves SOME kind of recognition.   Just how committed are the managers and supervisors?  QTs and employees draw their own conclusions about the level of real management support for the programs.  

Supporting QTs is more than participation in QT Workshop

If truth be told, after launching the QT workshop, many supervisors privately don’t support the program.  They lose their top performers during the workshop and the hours it takes to train someone. Forget about giving QTs adequate time to complete the paperwork properly! And then leadership wonders why good documentation practice (GDP) issues continue to be a problem? The non-distracted performance observations that QT’s are expected to conduct for the qualification demonstration drive supervisors and line managers crazy the most – what, they can’t do anything else but observe?  Hence, many QT’s are asked to multi-task just to get the work done: not enough resources they are told.  For supervisors, productivity and the workload will always trump SOJT and qualification events, until their bonuses include completion of training and qualification events.

What Real Support Is Supposed To Look Like

My key take away message is that attending the TTT program/ QT workshop is not the end of the OJT program or the Employee Qualification Program but rather the launching point.  Management support needs to go beyond just nominating QTs and allowing them to participate in the workshop.  The real support is in the alignment of goals, clarifying expectations continuously, allocating resources for training and budgeting time to deliver OJT using an approved OJT methodology that includes qualification events.  This commitment of time and sponsorship for qualified employees is a culture shift for many line managers and site leaders.  But actions do speak louder than words.  -VB

(1) Guidance for Industry Q10 Pharmaceutical Quality System | US Department of HHS | FDA
| CDER | CBER April2009 ICH

(2) Guidance for Industry Quality Systems Approach to Pharmaceutical CGMP Regulations U.S.Department of HHS | FDA | CDER | CBER | CVM | ORA| September 2006 Pharmaceutical CGMPs

Jacobs RL, Jones MJ. Structured on-the-job training: Unleashing employee expertise in the workplace.  San Francisco: Berrett-Koehler: 1995.

How can you tell if your site is ready for SOJT?

HPISC has created a 2 part checklist of questions and attributes to explore. The checklist is available gratis when you become a HPISC mailing list member. Just be sure to include that are you interested in receiving the SOJT Readiness Checklist.

Mailing List Subscribers automatically receive newest assets and tools directly into the inbox.

Who is Vivian Bringslimark?

(c) HPIS Consulting, Inc.

Employee Qualification is the Ultimate Level 3 Training Evaluation

On the Job Training is as old as some of the original apprentice-style forms of learning and ranges from very informal like follow Joe around to structured OJT that is formally documented and includes a qualification event observed by a Qualified Trainer.  While OJT means on the job training, the steps for OJT can also vary from trainer to trainer and from company to company unless the methodology is captured in an approved written procedure. 

Multiple Performance Demonstrations Occur

One of the first instances of a demonstration occurs from the trainer himself.  S/he shows the learner how to perform the technique, task, or process.  The learner observes and asks questions.  Then the roles reserve and the learner performs a mimicked rendition of what s/he observed.  The trainer provides feedback and sometimes will ask questions intended to assess the knowledge gained as well. 

Is one demonstration enough to determine OJT is done?  Sometimes it is.  When the task is simple, one time is all that most learners need.  When the task or process is complicated, it will take more than one demonstration to get the SOP steps right.  The nature of the SOP or the complexity of the task at hand determines this. 

But, how do I proceduralize that, you ask?  It starts by not arbitrarily picking the magic number 3. I have engaged in countless discussions regarding the exhaustive list of exceptions to forcing the rule of 3 times to practice.  And some QT’s will argue for more than 3 sessions especially when the procedure is so infrequently performed.  It’s not strictly about the number of times. We recognize that multiple sessions become practice sessions when the learner is still demonstrating the procedure under the supervision of his/her trainer.  But documenting the number of demonstrations and/or practice sessions is still a challenge for the Life Sciences Industry. 

At what point, is the learner going to be qualified to perform independently? As an industry, there is no standard number of times.  There are no standard learners either.  There is a range of “quick studys” to typical to slow learners.  The caveat to this is monitoring both the quick study and the slow learner.  In the QT workshops, this topic is explored using scenarios with tips and techniques that are shared during the debriefings.  Qualified Trainers know what is typical and they are empowered to evaluate the outcome of the learner’s demonstration(s).  Is the procedure being performed according to the SOP or is the learner still a bit hesitant about the next step? Is s/he relying on the QT for the assurance that the step is right?  While the steps may be performed correctly, is it also the confidence of both the QT and learner that we are assessing as well.

How many times is enough? Until both the learner and the QT are confident that s/he is not going to have an operator error deviation a week after going solo.  The QT is ultimately the one who has to assess progress and determine that “with a few more sessions”, my learner will get this or no, s/he may never get it and it’s time to have a discussion with the manager.

BTW, what does “Qualified Employee” mean?

Being SOP Qualified is the demonstrated ability of an employee to accurately perform a task or SOP independent of his OJT Qualified Trainer with consistency to meet acceptable quality standards. It satisfies the CFR ξ 211.25 (c ) regulation, “there shall be an adequate number of qualified employees to perform”.

Don’t be tempted to take the Performance Demo short-cut!

The end goal of OJT and the Qualification Event is for the employee to perform independently of his/her QT.  In order to be “released to task”, a final performance demonstration is scheduled, observed, and documented by an OJT Qualified Trainer. But don’t be fooled into taking the performance demo short cut!  The last step in the training portion of OJT is a performance demonstration to show the OJT-QT that the employee can perform the steps AND perform at the same level of proficiency as his/her peer group. If s/he can’t perform at this level, then the learner is not ready to “go solo”.

He may need more encouragement to build up confidence, correct paperwork documentation errors, and time to become proficient with his/her speed while maintaining accuracy.  That’s what practice sessions are for; time to master confidence with the steps and increase speed.  When his/her performance is on par with “business as usual” performance levels, then the employee is ready to perform the final demonstration aka the Qualification Event.  While the “readiness indicator” may not be documented, the (Q-Event) must be formally captured, assessed with the outcome being documented and communicated to both the learner and his/her supervision.  It is a separate event from the OJT demonstrations

Final Performance Demo = Qualification Event

During the final performance demonstration, the QT observes the learner’s performance.  When feedback is provided, it is evaluative and the rating result is formally documented.  Granted, when someone is watching us, we tend to follow the rules.  With enough repeated practice sessions,  learners tend to perform procedures as “business as usual”.  It’s how they learn the ebb and flow from their peers.  This is the optimum moment to determine if s/he is truly ready to perform without coaching or supervision from his QT.  If a QT has to interrupt to correct a misstep or remind the employee that his step is out of sequence, the event is terminated and documented as requires more review. 

More training practice is then scheduled until readiness is once again achieved.  And this also means the learner cannot sign for his work without his trainer’s co-signature or initials.  Do not misinterpret this as signing for the verification entry aka the second check.  In this situation, the Qualified Trainer cannot be both the co-signer and the second check person verification/reviewer.  You will need three sets of initials to properly document the supervision of a learner requiring more practice.  Otherwise you violate data integrity rules around independent verification.

Qualification events are not intended to be a rushed get ‘er done / one and done paperwork exercise.  Sufficient time for proficiency and expected department productivity levels is required to ensure knowledge has been retained and skill can be accurately repeated.  OJT demonstrations are not to be misused as the Q-Event.  This distinction is critical to ensuring a successful qualification event and the confidence of consistently performing the SOP tomorrow, next week, etc.  And not creating a deviation one day or one week after declaring the learner qualified.

It happens when QT’s are urged to “get’em done” by impatient or overly anxious supervisors consumed with productivity and not quality metrics.  With the qualification event being so recent, the QT will most certainly be interviewed as part of the investigation.  The checklist will also be examinedThis tool is supposed to help the QT be as objective as possible and consistently evaluate performance as demonstrated.  But typically, the checklist used to qualify individuals shows all Yeses; otherwise, they wouldn’t be qualified status.  And that, of course, depends on how well the critical steps and behaviors are captured in the OJT Checklist.  Yes, he was able to demonstrate the step, critical task, and/or behavior, but what we don’t know is how well?  Are we to assume that No means “No, not at all” and Yes means performed “Well” or it is “As Expected” or “Adequate” or maybe, in this case, it was “Sort of”?  The comments column would have been the ideal place to record observations and enter comments.

Validating Your SOP Effectiveness

Meeting FDA expectations for qualified employees is paramount.  But the “100% Trained on Curricula Requirements” printouts aren’t winning favor with FDA.   In the March 2015 article, Moving Beyond Read & Understand SOP Training”, I asserted that the current 100% trained reports and SOP quizzes would not be enough to satisfy the performance challenge for training effectiveness.  Are your employees qualified? How do you know? has become the training effectiveness question asked at every inspection.  The use of “100% completed” reports is a metric for completeness only; a commonly used data point from the LMS.  It does not address the transfer of learning into performance back on the job.  Neither does the 5-question multiple-choice “SOP Quiz”. The true measure of effective OJT is an observed performance demonstration of the SOP; aka the qualification event.

Employee Qualification is the ultimate Level 3 Training Effectiveness Strategy

Level 3 Behavior Change –> Transfer of Training/Learning

The focus of Employee Qualification is about the employee’s ability to apply knowledge and skill learned during OJT back on the job or in the workplace setting.  I call this Transfer of Training.  Others in the training industry refer to this as Level 3 – Behavior Change.   Actual performance is the ultimate assessment of learning transfer.   If an employee is performing the job task correctly during the final  performance demonstration (Q-Event), his performance meets the expectation for successful “OJT Required SOP”.

Yet, according to the 2009 ATD research study “The Value of Evaluation”, only 54.6% of respondents indicated that their organization conducts Level 3 evaluations.   The top technique used is follow-up surveys with participants (31%), while observation of the job was fourth (23.9%).    

If on the job assessment is the “ultimate” measure of transfer, then why isn’t it being used more frequently? “Post-training” assessments are time and labor-intensive.   But for organizations that have to meet compliance requirements (46.9% of survey respondents), documenting training effectiveness is now on FDA performance radar.

Not all SOPs require a Qualification Event

SOPs generally fall into two categories: FYI-type and OJT Required.  The more complex an SOP is, the more likely errors will occur.  Observing “critical to quality” steps is a key focus during the final performance demonstration.  However, a 1-1 documentation path for every OJT related SOP may not be needed.  Instead, batch SOPs a/o multiple SOPs of similar processes can be grouped into a “module” with documentation supporting similarity.  Where there are differences in these SOPs, then the Q Event would also require observation of these unique CTQ differences.

Two Types of SOPs
Two Types of SOPs: Only Critical Task-Based SOPs Required OJT and Qualification Events

An active Employee Qualification Program also verifies that the training content in this case the SOP, accurately describes how to execute the steps for the task at hand.  If the SOP is not correct or the qualifying documentation (checklist) is too confusing, a cause analysis needs to be conducted. Successful qualification events also validate the OJT methodology is effective. That Qualified OJT Trainers are consistently delivering OJT sessions for “OJT Required SOPs”. 

What does “Qualified Employee” mean for a company?

Qualified Employee status is not only a compliance imperative but a business driver as well. A qualified workforce means a team of well-trained employees who know how to execute their tasks accurately and with compliance in mind, own, and document their work properly.  When anyone in the organization can emphatically answer “Yes, my employees are qualified and yes, I have the OJT checklists to back that up”, then the Employee Qualification Program is not only working but is also effective at producing approved products or devices fit for use. The bonus is a renewed level of confidence in the ability of employees to deliver on performance outcomes for an organization.

*The Value of Evaluation: Making Training Evaluations More Effective. An ASTD Research Study, 2009, ASTD.

What happens when the performance demonstration becomes more of a "this is how I do it discussion" instead of an actual demonstration? Read the Impact Story - I've Fired My Vendor - to learn more.

Who is Vivian Bringslimark?

(c) HPIS Consulting, Inc.