On the Job Training is as old as some of the original apprentice-style forms of learning and ranges from very informal like follow Joe around to structured OJT that is formally documented and includes a qualification event observed by a Qualified Trainer. While OJT means on the job training, the steps for OJT can also vary from trainer to trainer and from company to company unless the methodology is captured in an approved written procedure.
Multiple Performance Demonstrations Occur
One of the first instances of a demonstration occurs from the trainer himself. S/he shows the learner how to perform the technique, task, or process. The learner observes and asks questions. Then the roles reserve and the learner performs a mimicked rendition of what s/he observed. The trainer provides feedback and sometimes will ask questions intended to assess the knowledge gained as well.
Is one demonstration enough to determine OJT is done? Sometimes it is. When the task is simple, one time is all that most learners need. When the task or process is complicated, it will take more than one demonstration to get the SOP steps right. The nature of the SOP or the complexity of the task at hand determines this.
But, how do I proceduralize that, you ask? It starts by not arbitrarily picking the magic number 3. I have engaged in countless discussions regarding the exhaustive list of exceptions to forcing the rule of 3 times to practice. And some QT’s will argue for more than 3 sessions especially when the procedure is so infrequently performed. It’s not strictly about the number of times. We recognize that multiple sessions become practice sessions when the learner is still demonstrating the procedure under the supervision of his/her trainer. But documenting the number of demonstrations and/or practice sessions is still a challenge for the Life Sciences Industry.
At what point, is the learner going to be qualified to perform independently? As an industry, there is no standard number of times. There are no standard learners either. There is a range of “quick studys” to typical to slow learners. The caveat to this is monitoring both the quick study and the slow learner. In the QT workshops, this topic is explored using scenarios with tips and techniques that are shared during the debriefings. Qualified Trainers know what is typical and they are empowered to evaluate the outcome of the learner’s demonstration(s). Is the procedure being performed according to the SOP or is the learner still a bit hesitant about the next step? Is s/he relying on the QT for the assurance that the step is right? While the steps may be performed correctly, is it also the confidence of both the QT and learner that we are assessing as well.
How many times is enough? Until both the learner and the QT are confident that s/he is not going to have an operator error deviation a week after going solo. The QT is ultimately the one who has to assess progress and determine that “with a few more sessions”, my learner will get this or no, s/he may never get it and it’s time to have a discussion with the manager.
BTW, what does “Qualified Employee” mean?
Being SOP Qualified is the demonstrated ability of an employee to accurately perform a task or SOP independent of his OJT Qualified Trainer with consistency to meet acceptable quality standards. It satisfies the CFR ξ 211.25 (c ) regulation, “there shall be an adequate number of qualified employees to perform”.
Don’t be tempted to take the Performance Demo short-cut!
The end goal of OJT and the Qualification Event is for the employee to perform independently of his/her QT. In order to be “released to task”, a final performance demonstration is scheduled, observed, and documented by an OJT Qualified Trainer. But don’t be fooled into taking the performance demo short cut! The last step in the training portion of OJT is a performance demonstration to show the OJT-QT that the employee can perform the steps AND perform at the same level of proficiency as his/her peer group. If s/he can’t perform at this level, then the learner is not ready to “go solo”.
He may need more encouragement to build up confidence, correct paperwork documentation errors, and time to become proficient with his/her speed while maintaining accuracy. That’s what practice sessions are for; time to master confidence with the steps and increase speed. When his/her performance is on par with “business as usual” performance levels, then the employee is ready to perform the final demonstration aka the Qualification Event. While the “readiness indicator” may not be documented, the (Q-Event) must be formally captured, assessed with the outcome being documented and communicated to both the learner and his/her supervision. It is a separate event from the OJT demonstrations.
Final Performance Demo = Qualification Event
During the final performance demonstration, the QT observes the learner’s performance. When feedback is provided, it is evaluative and the rating result is formally documented. Granted, when someone is watching us, we tend to follow the rules. With enough repeated practice sessions, learners tend to perform procedures as “business as usual”. It’s how they learn the ebb and flow from their peers. This is the optimum moment to determine if s/he is truly ready to perform without coaching or supervision from his QT. If a QT has to interrupt to correct a misstep or remind the employee that his step is out of sequence, the event is terminated and documented as requires more review.
More training practice is then scheduled until readiness is once again achieved. And this also means the learner cannot sign for his work without his trainer’s co-signature or initials. Do not misinterpret this as signing for the verification entry aka the second check. In this situation, the Qualified Trainer cannot be both the co-signer and the second check person verification/reviewer. You will need three sets of initials to properly document the supervision of a learner requiring more practice. Otherwise you violate data integrity rules around independent verification.
Qualification events are not intended to be a rushed get ‘er done / one and done paperwork exercise. Sufficient time for proficiency and expected department productivity levels is required to ensure knowledge has been retained and skill can be accurately repeated. OJT demonstrations are not to be misused as the Q-Event. This distinction is critical to ensuring a successful qualification event and the confidence of consistently performing the SOP tomorrow, next week, etc. And not creating a deviation one day or one week after declaring the learner qualified.
It happens when QT’s are urged to “get’em done” by impatient or overly anxious supervisors consumed with productivity and not quality metrics. With the qualification event being so recent, the QT will most certainly be interviewed as part of the investigation. The checklist will also be examined. This tool is supposed to help the QT be as objective as possible and consistently evaluate performance as demonstrated.
But typically, the checklist used to qualify individuals shows all Yeses; otherwise, they wouldn’t be qualified status. And that, of course, depends on how well the critical steps and behaviors are captured in the OJT Checklist. Yes, he was able to demonstrate the step, critical task, and/or behavior, but what we don’t know is how well? Are we to assume that No means “No, not at all” and Yes means performed “Well” or it is “As Expected” or “Adequate” or maybe, in this case, it was “Sort of”? The comments column would have been the ideal place to record observations and enter comments.
Validating Your SOP Effectiveness
Meeting FDA expectations for qualified employees is paramount. But the “100% Trained on Curricula Requirements” printouts aren’t winning favor with FDA. In the March 2015 article, “Moving Beyond Read & Understand SOP Training”, I asserted that the current 100% trained reports and SOP quizzes would not be enough to satisfy the performance challenge for training effectiveness. Are your employees qualified? How do you know? has become the training effectiveness question asked at every inspection.
The use of “100% completed” reports is a metric for completeness only; a commonly used data point from the LMS. It does not address the transfer of learning into performance back on the job. Neither does the 5-question multiple-choice “SOP Quiz”. The true measure of effective OJT is an observed performance demonstration of the SOP; aka the qualification event.
Employee Qualification is the ultimate Level 3 Training Effectiveness Strategy
The focus of Employee Qualification is about the employee’s ability to apply knowledge and skill learned during OJT back on the job or in the workplace setting. I call this Transfer of Training. Others in the training industry refer to this as Level 3 – Behavior Change. Actual performance is the ultimate assessment of learning transfer. If an employee is performing the job task correctly during the final performance demonstration (Q-Event), his performance meets the expectation for successful “OJT Required SOP”.
Yet, according to the 2009 ATD research study “The Value of Evaluation”, only 54.6% of respondents indicated that their organization conducts Level 3 evaluations. The top technique used is follow-up surveys with participants (31%), while observation of the job was fourth (23.9%).
If on the job assessment is the “ultimate” measure of transfer, then why isn’t it being used more frequently? “Post-training” assessments are time and labor-intensive. But for organizations that have to meet compliance requirements (46.9% of survey respondents), documenting training effectiveness is now on FDA performance radar.
Not all SOPs require a Qualification Event
SOPs generally fall into two categories: FYI-type and OJT Required. The more complex an SOP is, the more likely errors will occur. Observing “critical to quality” steps is a key focus during the final performance demonstration. However, a 1-1 documentation path for every OJT related SOP may not be needed. Instead, batch SOPs a/o multiple SOPs of similar processes can be grouped into a “module” with documentation supporting similarity. Where there are differences in these SOPs, then the Q Event would also require observation of these unique CTQ differences.
An active Employee Qualification Program also verifies that the training content in this case the SOP, accurately describes how to execute the steps for the task at hand. If the SOP is not correct or the qualifying documentation (checklist) is too confusing, a cause analysis needs to be conducted. Successful qualification events also validate the OJT methodology is effective. That Qualified OJT Trainers are consistently delivering OJT sessions for “OJT Required SOPs”.
What does “Qualified Employee” mean for a company?
Qualified Employee status is not only a compliance imperative but a business driver as well. A qualified workforce means a team of well-trained employees who know how to execute their tasks accurately and with compliance in mind, own, and document their work properly. When anyone in the organization can emphatically answer “Yes, my employees are qualified and yes, I have the OJT checklists to back that up”, then the Employee Qualification Program is not only working but is also effective at producing approved products or devices fit for use. The bonus is a renewed level of confidence in the ability of employees to deliver on performance outcomes for an organization.
*The Value of Evaluation: Making Training Evaluations More Effective. An ASTD Research Study, 2009, ASTD.
What happens when the performance demonstration becomes more of a "this is how I do it discussion" instead of an actual demonstration? Read the Impact Story - I've Fired My Vendor - to learn more.
Who is Vivian Bringslimark?
(c) HPIS Consulting, Inc.