What happens next for new hires who achieve 100% complete for R & U SOPs?

After weeks if not months of waiting for your new hire, she is finally here, finishing up 1st-day orientation. Day 2, she’s all yours. Are you excited or anxious? The LMS (learning management system) printout of training requirements is overwhelming; even for you. Bottom line question running through your mind — when can s/he start carrying their weight for the department workload?

If your department onboarding is like most in the Life Sciences industry, your next milestone is the completion of all the required SOPs to be done Day 1, Week 1, and quite possibly Month 1.  After that, the learning journeys for new hires follow different paths depending on whether they will operate equipment or generate controlled documents and review reports.

For folks who run laboratory instruments or operate manufacturing equipment, a significant portion of the LMS printout of training requirements is to be delivered via On the Job Training (OJT); SOJT more specifically. Structured on the job training (SOJT) is an organized and planned step by step approach for completing hands-on training requirements. The SOJT approach uses some variation of the following OJT steps: Read, Observe, Practice, Qualify. And is managed by a qualified trainer who is monitoring the new employee’s progress until s/he is fully qualified.

Pay Now or Pay Later

Yet for many line managers, they want their trainees now. Ironically, the faster you “push” trainees through their training matrix, the slower the learning curve. This proactive vs. reactive dilemma is not new. Traditional OJT aka “follow Joe around” looks like a win – win for everyone on the surface. The new hire gets OJT experience, a SME is “supervising” for mistakes, and supervisors are keeping up with the production schedule. So what’s wrong, you ask? This hurry up and get ‘em done push often leads to more errors, deviations, and quite possibly CAPA investigations for numerous training incidents. It’s a classic case of pay now or pay later.

After 6 months or so, the trainee isn’t new anymore and everyone “expects” the new employee to be fully qualified by then with no performance issues and no deviations resulting from human error. Assumptions have been made about what he/she has learned based on time in the department. Unfortunately, this is the very reason for “getting the SOPs all out of the way” the first 30 days.  What is needed, is the ability to schedule when training requirements are needed based on the number of days from the start date. Really good LMSes offer smart curricula; when the first curricula requirements are satisfied the next curricula is then assigned with realistic due dates and so on.

Is SOJT Applicable Beyond Manufacturing and the Lab?

What about the new hire who is assigned to Quality Assurance?  Surely their role and job responsibilities are critical to the organization.  They may not operate complex equipment or run highly specialized testing instruments, but the documents they generate and approve using modern Quality Management System software are just as important for compliance.  Could the SOJT methodology be used here as well?  Perhaps for the e-doc processes.  And in many organizations, first round super users receive training from the vendor. 

But what about all the other non-equipment functions within the organization that are compliance-focused?  What happens after new hires reach “100% complete” with their training requirements? What is their version of On-the-Job Training?  If we require them to use the SOJT methodology, we bump into challenges.  For example, the following table highlights where the nature of their tasks do not fit neatly into the steps and certainly would not be consistently executed across the rest of the organization.

Unfortunately, the only “plan” for these employees is the list of required SOPs with due dates and possibly a few required courses.   The SOPs are listed for them via curricula and can either be found in an e-doc system, shared drive, or a similar controlled share point repository.  How to manage this list between now and the due date is really up to the employee.  After all, we hire adult professionals and if they are experienced, they know what they need to do; get their curricula done.  It is up to them to manage their own path. 

And in many ways, this individualized approach looks a lot like informal learning.  Jay Cross in his TD article, “Not Without Purpose”, tells us that informal learning respects workers.  Moreover, he believes that “employees thrive when given the freedom to decide how they will do what is asked of them.” 

When the formal curricula of R & U for SOPs is completed, the new hire begins his/her on the job experience.  And this is where non-linear learning takes place. It’s where learning and performing intersect. Where the demands of the job function require the learner to put into practice what he read and “understood” about the procedure.  Now the procedures come to life as daily problem solving gives rise to real learning opportunities.  In reality though, the retention rate for those early R & U SOPs is pretty dismal.  The best we can hope for is that they remember a procedure exists and they know how to search for it.  They are in essence repeating that exercise making the 1st round a waste of time and effort.

Yet, on the job experiences are the biggest source of learning for employees.  It is what ultimately qualifies them for their job.  But we are not formally capturing it.  At best, we remind our folks to update their CVs once a year.  In the SOJT model, it’s called practice and at least one time we document it. Then we formally evaluate the performance demonstration and capture it as the Qualification Event.  But in this other approach, gaps in performance go unchecked despite having 100% trained reports.  What documents their current qualified status, the original CV that was used at the time of hire?

“Unfortunately, the only ‘plan’ for these employees is the list of required SOPs with due dates and possibly a few required courses.”  Vivian Bringslimark, HPIS Consulting, Inc.

Training, Education and Experience or Any Combination Thereof

Through our recruiting and hiring process we hire experienced people or at least that is the intention.  For clinical operations, it is paramount to successful recruiting.  When filling clinical operations roles, potential hires need to have solid compliance foundations and previous experiences.  But it’s their expertise that gets them hired.  So, you would think they don’t need much new training.  And yet, many of the SOPs, protocols and trial studies are new and evolving so they need to know what’s in these documents.  When high performance includes compliance achievement, it is a must have for these roles.

The good news is that these new hires are highly intelligent, have a strong desire and work ethic.  They are conditioned to facilitate their own learning to transfer their knowledge and clinical skills.  After receiving their curricula assignments, their learning journey resembles more like the organic nature of informal learning where they experience natural learning discoveries on their own.  It is not uncommon then, that Clinical Operations departments tend to operate independently from the rest of the CMC organization with their own set of SOPs and unfortunately form independent silos.

Expanding QA L & D Scope

More recently, QA L&D has begun to partner with ClinOps to better meet the needs of this population and support their role in the future of the organization.  At first glance, inserting ClinOps into the Quality Training System may sound like the easy solution.  But when 300 must-read / need-to-know documents show up in an LMS printout, “it just does not make sense” remarked a Director of Quality Systems.

So rather than structuring their on the job experiences as in the SOJT approach, these folks need an On-the-Job Learning (OJL) methodology that curates their documents and information they need while learning in the flow of their work assignments; not at all once at the beginning of their journey.  Employees want to learn their procedures when they need it the most.  It saves time and reinforces what they read because now they are applying what they just read. 

The effectiveness check is truly their work product.  We can evaluate how well they followed the procedure to produce the output of their responsibility.  This is a far better measure than a 5- question multiple-choice knowledge check for the procedure during week 1 of onboarding. Some would argue that productivity is hindered because the task is suspended while the learner is searching for the procedure.  Which is more lost productivity, reading the SOP for the second time or reading it the first time and executing the task correctly immediately after?

Ken Taylor, President and Editor in chief for Training Industry, Inc., believes that formalization allows us to recognize the impact that this self-directed learning approach has on their performance.  The formal part is the list of documents “that they need to know”. What’s missing is their “final performance demonstration” when we can declare the employee is qualified! When is their Qualification Event captured?

REAPS for COJL

If we revisit the SOJT Model above with the intention to modify for C-OJL, Curated On the Job Learning, it might look like the following:

Read: REAPS

While 300 documents is overwhelming, the better approach is to ask, “what will they do with the information they read”?  And then ask, “when do they need to know this” rather than assign it too far in advance.  Believe it or not, there are times when reading may be acceptable.  When the outcome is awareness only or for information only and no effectiveness check is required or a performance outcome affected by the application of the information, then read can be used.  But please do not call it training.

The initial onboarding should not just be a hyper focus to get the list done.  “As L&D Professionals, we need to provide context over content to our audiences”, says Mark D’Aquin.  Context can be achieved when the curricula requirements are presented as an intended curated list of “content” for the individual’s role and how carefully the requirements have been mapped to performance-based outcomes. Not only does it add a layer of personalization, it highlights just how important the job functions and the outcomes are to the organization.  These discussions with the supervisor and possibly the mentor(s), involves not only the role, but timelines, expected work products and “reports” that will be peer reviewed for compliance and company standards.

Engage: REAPS

Assigning a buddy or pairing them with a peer mentor provides just enough structure to allow the new hire to build positive relationships with peers and begin to build psychological safety where they feel safe to contribute or challenge the status quo without fear of being rejected or isolated. For example, many professionals bring with them their previous employer’s procedures and practices. 

And while it may be the same industry, even the same role, invariably the new company will require differences in their procedures. This means new hires have to unlearn their previous knowledge and relearn new terms and methods rather rapidly.  Mentors can facilitate this process by meeting with new hires to help them debrief and reflect on these changes, ensuring that the new hire leaves the conversation with the right summary of steps. 

“Debriefing is a critical conversation to reframe the context of a situation to clarify perspectives and assumptions, both subjectively and objectively”. “The process of coming to know why an action was taken reveals the knowledge, assumptions, values, beliefs, and feelings behind the action and attaches meaning to information. Critical reflection bridges past learning within the context of a new situation.”  NLN in collaboration with INACSL, June 2015.

In their TIQ May June 2020 article, Emily Blancato & Shelley Stanley, insist that “there must be structure not only around the task but who the new hire should be partnered with to complete it”.  They recommend the following:

  • Identify what your existing employees need to support your new hires in these sessions.
  • Ensure the existing staff will have the time and balance necessary to provide a positive experience during their time with the new hire.
  • Confirm the staff understands the expectations of the program.
  • Prioritize the most common tasks first to allow both the new hire and the team to see immediate productivity gained.

When compared to the SOJT approach, this looks similar at first glance.  The difference is that SOJT Qualified Trainer, is the primary driver in completing OJT indicating curricula requirements.  In the COJL approach, the learner is the driver while the mentor is an appointed resource to foster connections with others to get clarity on content at the time of need and provide recommendations for additional resources.

Apply: REAPS

When the work products start to become deliverables, this is the biggest opportunity to check for effectiveness of the training requirements and the balance of formalness vs. autonomy. It is not only performance oriented, there are feedback loops to have more powerful conversations focused on supporting the new hire’s learning experiences and growth. 

“If you are not taking the time to be specific in your feedback and connecting that feedback to learning, your direct reports won’t be successfully competent or autonomous – or motivated”,  Annemarie Spadafore, 2020, p.28.

It is not intended as control, but as specifics for areas to improve, stay accountable and therefore grow.  For example, if Mentors are not skilled in facilitating debriefing discussions, supervisors can ask new hires to self-report their own learning debriefs and share them with their direct supervisors as a means to guide these periodic check in conversations.

“Jay Cross believes that ‘employees thrive when given the freedom to decide how they will do what is asked of them.’”

Sign-Off: REAPS

Ideally this would happen with the direct supervisor, but it is acceptable for the mentors and designated peers to perform this formal qualification event.  What can be used as evidence?  The final work product of the new hires as they successfully demonstrate their job responsibilities and functions as described in their Job Description.  These Q-Events formally connect the dots of the new hire’s CV to Position Requirements to Job Function and back to Job Description, documenting the full cycle.

“By focusing on the complete journey of the learner and adding structure to that journey, we will be able to better understand the impact of all elements of the learning experience”, Ken Taylor, 2019, p.3.

Let’s go back to your new hire that you are anxiously waiting for.  When was the last time you reviewed her training requirements?  Have priorities changed?  Does she truly need to read the first 100 SOPs by the end of the month?  Will you entertain the idea of letting her read and UNDERSTAND them in the flow of her work rather than by some arbitrary due date?  Who would you assign as her mentor or peer buddy to help her become more engaged and integrated into the company’s culture?  Are you at all interested in her post-orientation onboarding experience? Most importantly, will she prematurely leave the company because of a subpar learning experience?  – VB

Curated On the Job Learning (COJL) StepCompliance Focused Role Based Training
Read – curated curricula documentsPresent the list of documents as a curated mapping for the individual in their role.
Engage – in discussions with mentor Create a vehicle for intentional social sharing exchanges.
Apply – what you read to your taskSchedule debriefing check-ins with supervisor.
Peer –  review of finished work product – Does it meet approved document expectations?Provide meaningful feedback to increase competence and professional growth.
Sign Off – Qualification EventFormally capture documented evidence for each identified job responsibility check point as listed in the Job Description.
  • Adelman, P. “Developing the Next Generation of Physician Leaders”, CLO, MayJune, 2020, 45 -47, 52.
  • (Argyris, 1992; Argyris & Schon, 1974; Brookfield, 1986,1990, 1993, 1995, 2000; Freire, 1970/2000; Mezirow, 1978, 1990, 2000; Schon, 1983, 1987; Tennyson, 1990, 1992; Tennyson & Breuer, 1997; Tennyson & Rasch, 1988). In National League of Nursing, “Debriefing Across the Curriculum”, International Nursing Association for Clinical Simulation and Learning, June 2015.
  • Blancato, E., Stanley, S. “5 Ways to Maximize Your On-the-Job Training Program”, TIQ, MayJune, 2020,40-42.
  • Cross, J. “Not Without Purpose”, T&D, June, 2006, 42-44.
  • D’Aquin, M. “5 Steps to Formally Make the Informal Part of Your Content Strategy”, TIQ SeptOct, 2019, 36-39.
  • Jacobs, RL, Jones, MJ. Structured On – The – Job Training: Unleashing Employee Expertise In The Workplace. San Francisco: Berrett – Koehler,1995.
  • Kirkpatrick, J, Kirkpatrick, WK. “Harnessing Informal Learning”,  TIQ, MayJune, 2020, 28-31.
  • Spadafore, A. “Back to Work”, TD August, 2020, 26-31.
  • Stashak, S. “Non-Linear Learning: A must have for higher productivity”, OREILLY Blog, June 28, 2019.
  • Taylor, K. “Thinking More Broadly About How Adults Learn”, TIQ, SeptOct, 2019, p.3.

Who is the Author, Vivian Bringslimark?

(c) HPIS Consulting, Inc.

Why we love and hate Training Curricula and our LMS (Learning Management System)

Let’s begin with why we have curricula in the first place.  So that folks are trained in the “particular operations that the employee performs” (§21CFR 211.25).  Having this “list” is a huge improvement from the previous practice years ago. 

Remember when SOPs sat in binders on a shelf?

I was once told, “If an SOP is sitting in a binder in my department, then everybody is going to train on it”.  He had the entire SOP binder collection sitting on a shelf in his office.  He routinely required that all his direct reports sign the training form to back up his statement.  I congratulated him on documenting the READ part.  But when I asked him if his folks were trained on how to conduct the preventative maintenance procedure for cleaning the HVAC air ducts on the roof, he extolled, “that’s not our responsibility!” “Yes, you are right, but you all signed for the UNDERSTOOD part of those SOPs as well!” I replied.  He didn’t take my comment very well. 

The quality system regulation for training is quite clear regarding a procedure for also identifying training needs.  And “each manufacturer shall establish procedures for identifying training needs and ensure that all personnel are trained to adequately perform their assigned responsibilities” (§21CFR820.25). To meet that expectation, industry practice is to have a process for creating curricula, maintaining the accuracy and completeness of curricula requirements. 

Once I explained the purpose of curricula and how this manager could determine which procedures his folks should train on rather than using the SOP Index, he stopped the unrequired R & U for SOPs that were outside of their responsibilities.  Then I jokingly said, “Now you have time to send them to Annual GMP Training!” He frowned and walked away.

§820.25 also states “Training shall be documented”.  Yes, it feels like a lot of paperwork.  Thank goodness today’s LMSes can perform reconciliation of training completion against curricula requirements.  So, I’m grateful that this task can be executed with a click for simple reports. And with just a few key search words, more advanced reports can be compiled. No more excel spreadsheets filled with matrix like set ups cross referencing SOPs, attendance and employees! But there’s more to training curricula than setting them up and earning 100% Training Complete.

Who is Really Responsible for Employee Training?

Almost every training SOP I read will include a statement that Department Managers are responsible for training of their direct reports.  Managing SOP changes and ensuring affected employees are up to date is a constant challenge for everyone. Especially for companies whose weekly revision churn rate is anywhere from 10 to 150+ revisions.

First, we rely on the LMS to automatically revise curricula or at least expect the LMS Admin to click [Revised] function.  Then we wait for the notifications either automatically from the LMS or the Admin.  And then we begin the huddle brow beating to “get ‘em done” before or by the effective date. The end goal for most people is the proverbial check mark (√) and lots of line items claiming “Read and Understood” the revised procedure.  This is when we hate the LMS because the notifications won’t go away until we sign off.  Btw, this isn’t training at all when the real outcome is a rushed end-user who has less than 15 minutes to “train on the changes”.

What Do They Really Need to Know About the Revised SOP?

Training is about closing gaps in knowledge and skills. Consider analyzing what precipitated the SOP change. Was is it a significant CAPA issue or FDA commitment that requires a solution more than “Read and Understood”/ e-sign for the revised SOP? How significant is the change in the procedure?

The SOP Author has listed what changed. Call him/her and ask why the change was made. Perhaps it is related to a deviation or audit observation that other employees need to know about. Find out when the changes will go into effect and how will folks be notified. Take advantage of the revision history page if available. And then incorporate the what, the where and the when into the training session:

  • Will the changes be upon an effective date?
  • Or a trigger condition that now has a different decision path?
  • Or is it a completely overhauled FORM to be used?

Include these notes as part of your preparation for end-users to recognize when to make the changes in their tasks.  PLEASE NOTE: Revision training is not to be used as a substitute for planned On Job Training or new hire SOP Training.

SOP Revisions Needing Real OJT: Will Management Agree with the Recommendation?

But what if the SOP revision requires more than a “tell and forget” discussion?  Are the changes significant enough to require a demonstration?  Will a one-time demo by the Qualified Trainer (QT) be enough to call it OJT?   Does the revision require some practice to unlearn the old way and learn the new steps?  How confident are the Qualified Trainers in redirecting the discussion with management regarding the most effective method to use, despite that it will take longer to satisfy the LMS requirement?

First and foremost, the QT signature means that employees were trained and assessed as having learned the content of what was delivered.  Can the QT confidently say their colleagues are trained in the revision and can perform the revision without errors?    Can the department / site / organization incur waste as in mistakes, scrap work, delayed release of product, and/or re-work as a result of “lack of proper training for the revision”? It comes down to how critical to quality is the new change.

So, Who Owns Curricula In Your Company? 

In those same Training SOPs that I mentioned earlier, I usually find a responsibility statement about determining training requirements as well.  Who has the responsibility to ensure that curricula are accurate and up to date as well as no overdue training for SOP revisions? Is it QA L&D because they manage the LMS or is it Dept Management as per the Training SOP?

What is the frequency for curricula review in your procedure? I realize that curricula requirements work is tedious even painful at times.  That’s why donuts show up for meetings scheduled in the morning, pizza bribes if it’s lunchtime and quite possibly even cookies for a late afternoon discussion.  So, I get it when folks don’t want to look at their curricula again or even have a conversation about them. 

Once a year curricula upkeep?

It’s like having a fichus hedge on your property.  If you keep it trimmed, pruning is easier than hacking off the major overgrowth that’s gone awry a year later.  And yet, I continue to get push back when I recommend quarterly curricula updates.  Even semi-annual intervals are met with disdain.  In the end we settle for once a year and I cringe on the inside.  Why? 

Because once a year review can be like starting all over again. If your LMS is mature enough to manage revisions via auto-update all affected curricula, then once a year may be the right time span for your company.  Others in our industry don’t have that functionality within their training database.  For these administrators, revisions mean manual creation into the “course catalog” each time with a deactivation/retirement of the previous version; some may be able to perform batch uploads with a confirmation activity post submission.  And then, the manual search for all curricula so that the old SOP number can be removed and replaced with the next revision.  Followed by a manual notification to all employees assigned to either that SOP or to the curricula depending on how the database is configured.  I’m exhausted just thinking about this workload.  

What about the periodic review of SOPs, which in most companies is two years.  What is the impact of SOP’s that become deleted as a result of the review?  Can your LMS / training database search for affected curricula and automatically remove these SOPs without intervention from the LMS administrator as well? 

And then there’s the corrective actions that have resulted in major SOP revisions that require a new OJT session and quite possibly a new qualification event?  What impact do all these changes have on the accuracy of your curricula? Can your administrator click the revision button for these as well?

Interestingly enough, I have seen companies who get cited with training observations, often have outdated and inaccurate curricula!  Their documentation for curricula frequently shows reviews overdue by 2 – 3 years or not performed since original creation, and in some places, no specialized curricula at all! 

  • “They were set up wrong.” 
  • “The system doesn’t allow us to differentiate enough.” 
  • “Oh, we were in the process of redoing them, but then the project was put on the back burner.” 

Are you waiting to be cited by an agency investigator during biennial GMP inspection or Pre-Approval Inspection? The longer we wait to conduct a curricula review, the bigger the training gap can become.  And that gap snowballs into missing training requirements, which leads to employees performing duties without being trained and qualified.  Next thing you know, you have a bunch of Training CAPA notifications sitting in your inbox.  Not to mention a FD-483 inspection report and quite possibly a warning letter.  So why does QA L&D continue to experience pushback for this required activity? Maybe folks really like morning bagels or the pizza bribes?

Why do Curricula Owners pushback on curricula reviews?

Will once a year result in a “light trim” of curricula requirements or a “hack job” of removing outdated requirements and inaccurate revision numbers? Will you be rebuilding curricula all over again?  It’s a self-fulfilling prophecy. Can we please change this already? You just might be surprised at how many SOPs are no longer required and that you may reduce the time needed to train on SOPs.  In the meantime, better bring the donuts and coffee!  -VB

Tired of bloated and over-allocated curricula?

Who is Vivian Bringslimark?

(c) HPIS Consulting, Inc.

Will You Please Just Sign the Training Form!

For companies who are required to have standard operating procedures (SOPs), managing changes and ensuring affected employees are up to date is a constant challenge. Especially for companies whose weekly revision churn rate is from 10 to 150+ revisions. The standard approach is to “get ‘em done” in the fastest amount of time. The end goal for most people is the proverbial (√) and lots of records claiming “read and understood” the procedure.

This blog has been merged into blog – “Why We Love and Hate Training Curricula and our LMS”.

Who is Vivian Bringslimark?

(c) HPIS Consulting, Inc. All rights reserved.

Why Knowledge Checks are Measuring the Wrong Thing

When I taught middle school math, tests were used to assess knowledge comprehension and some application with word problems and a few complex questions requiring logic proofs.  Results were captured via a score; an indication as to how well you answered the questions.  Whether you call it a quiz, a knowledge check or any other name it is still assessing some form of knowledge comprehension.

Life sciences companies are required by law to conduct annual regulations training (GXP Refreshers) so as to remain current in the operations being performed.  Given the design and delivery of most refresher training sessions, it is heavy with content that is delivered to a passive learner.  Listening to a speaker or reading slide after slide and clicking the next button is not active engagement.  It’s boring, mind-numbing and, in some cases, downright painful for learners.  As soon as the training is over, it’s very easy to move past the experience.  That also means most of the content is also soon forgotten.  Regulatory agencies are no stranger to this style of compliance training and delivery.  But when departures from SOPs and other investigations reveal a lack of GXP knowledge, they will question the effectiveness of the GXP training program. 

So why are we still using tests?

In our quest for effective compliance training, we have borrowed the idea of testing someone’s knowledge as a measure of effectiveness.  This implies that a corporate classroom mirrors an educational classroom and testing means the same thing – a measure of knowledge comprehension.  However, professors, colleges, universities and academic institutions are not held to the same results standard.  In the Life Sciences, two very common situations occur where knowledge checks and “the quiz” are heavily used: Annual GXP Refresher and the Read & Understand Approach for SOPs. 

Typical Knowledge Check

But what is the knowledge assessment measuring?  Is it mapped to the course objectives or are the questions so general that they can be answered correctly without having to attend the sessions?  Or worse yet, are the questions being recycled from year to year / event-to-event?  What does it mean for the employee to pass the knowledge check or receive 80% or better? When does s/he learn of the results? In most sessions, there is no more time left to debrief the answers. This is a lost opportunity to leverage feedback into a learning activity.  How do employees know if they are leaving the session with the “correct information”?

The other common practice is to include a multiple-choice quiz consisting of 5 questions as the knowledge check for SOPs that are “Read & Understood” especially for revisions.  What does it mean if employees get all 5 questions right?  That they will not make a mistake?  That the R & U method of SOP training is effective?  The search function in most e-doc systems is really good at finding the answers.  It doesn’t mean that they read the entire procedure and retained the information correctly.  What does it mean for the organization if human errors and deviations from procedures are still occurring?  Does it really mean “the training” is ineffective?

Conditions must be the same for evaluation as it for performance and training

What should we be measuring?

The conditions under which employees are expected to perform need to be the same conditions under which we “test” them.  So, it makes sense to train them under those same conditions as well.  What do you want/need your employees (learners) to do after the instruction is finished? What do you want them to remember and use from the instruction in the heat of their work moments?  Both the design and assessment need to mirror these expectations. 

Ask yourself, when in their day to day activities will employees need to use this GMP concept?  Or, where in the employees’ workflow will this procedure change need to be applied?  Isn’t this what we are training them for?  Your knowledge checks need to ensure that employees have the knowledge, confidence and capability to perform as trained.  It’s time to re-think what knowledge checks are supposed to do for you. And that means developing objectives that guide the instruction and form the basis of the assessment.

Objectives drive the design and the assessment

Instructional Design 101 begins with well-developed objective statements for the course, event, or program.  These statements aka objectives determine the content and they also drive the assessment.  For example, a quiz or a knowledge check is typically used for classroom sessions that ask questions about the content.  In order for learners to be successful, the course must include the content whether delivered in class or as pre-work. But what are the assessments really measuring?  How much of the content they remember and maybe how much of the content they anticipate applying when they return to work? 

When the desired outcomes are compliance back on the job, knowledge checks need to shift to become more application-oriented (performance-based) such as “what if situations” and real scenarios that will require an employee to analyze the situation, propose a workaround and then evaluate if the problem-solving idea meets the stated course objectives.

Performance objectives drive a higher level of course design

Training effectiveness is really an evaluation of whether we achieved the desired outcome.  So, I ask you, what is the desired outcome for your GXP training and SOP Training: to gain knowledge (new content) and/or to use the content correctly back in the workplace? The objectives need to reflect the desired outcomes in order to determine the effectiveness of training; not just knowledge retention.

What is the desired outcome?

When you begin with the end in mind namely, the desired performance outcomes, the objective statements truly describe what the learners are expected to accomplish.  While the content may be the same or very similar, how we determine whether employees are able to execute what they learned post training requires more thought about the accuracy of the assessment.  It must be developed from the performance objectives in order for it to be a valid “instrument”.  The learner must perform (do something observable) so that it is evident s/he can carry out the task according to the real workplace conditions.

Accuracy of the assessment tools

The tools associated with the 4 Levels of Evaluation can be effective when used for the right type of assessment.  For example, Level 1 (Reaction) surveys are very helpful for Formative Assessments.  See below – Formative vs. Summative Distinction.  Level 2 (Learning) knowledge assessments are effective in measuring retention and minimum comprehension and go hand in hand with learning-based objectives.  But when the desired outcomes are actually performance-based, Level 3 (Behavior) checklists can be developed for the performance of skills demonstrations and samples of finished work products. Note: Level 4 is Results and is business impact-focused.

Difference between formative and summative assessments

Trainers are left out of the loop

Today’s trainers don’t always have the instructional design skill set developed.  They do the best they can with the resources given including reading books and scouring the Internet.  For the most part, their training courses are decent and the assessments reflect passing scores.  But when it comes to Level 4 (Results) impact questions from leadership, it becomes evident that trainers are left out of the business analysis loop and therefore are missing the business performance expectations.  This is where the gap exists.  Trainers build courses based on knowledge/content instead and develop learning objectives that determine what learners should learn.  They create assessments to determine whether attendees have learned the content, but this does not automatically confirm learners can apply the content back on the job in various situations under authentic conditions.

Levels of objectives, who knew?

Many training mangers have become familiar with the 4 Levels of Evaluation over the course of their time in training, but less are acquainted with Bloom’s Taxonomy of Objectives.  Yes, objectives have levels of increasing complexity resulting in higher levels of performance.  Revised in 2001, the levels were renamed for better description of what’s required of the learner to be successful in meeting the objective. 

Bloom’s Revised Taxonomy of Objectives

Take note, remembering and understanding are the lowest levels of cognitive load while applying and analyzing are mid-range.  Evaluating and creating are at the highest levels.  If your end in mind is knowledge gained ONLY, continue to use the lower level objectives.  If; however, your desired outcome is to improve performance or apply a compliant workaround in the heat of a GMP moment, your objectives need to shift to a higher level of reasoning in order to be effective with the training design and meet the stated performance outcomes. Fortunately, much has been written about writing effective objective statements and resources are available to help today’s trainers.

Did we succeed as intended? Was the training effective in achieving the desired outcomes?

To ensure learner success via the chosen assessment, the training activities must also be aligned with the level of the objectives.  This requires the design of the training event to shift from passive lecture to active engagement intended to prepare learners to transfer back in their workspace what they experienced in the event.   The training design also needs to include practice sessions where making mistakes is an additional learning opportunity and teach learners how to recognize a deviation is occurring.  Michael Allen refers to this as “building an authentic performance environment”. 

Our training outcomes need to be both knowledge gained, and performance based.  Now, the agency is expecting us to document that our learners have the knowledge AND can apply it successfully in order to follow SOPs and comply with the regulations. Thus, trainers and subject matter experts will need to upgrade their instructional design skills if you really want to succeed with training as intended.  Are you willing to step up and do what it takes to ensure training is truly effective? – VB

Allen,M. Design Better Design Backward, Training Industry Quarterly, Content Development, Special Issue, 2017, p.17.

Who is Vivian Bringslimark, the Author?

Need to write better Knowledge Check questions?
Tips for Writing KCs
Find out the Do’s and Don’ts for Writing Assessment Questions

Want to see the Table of Contents before you request it? No problem!

(c) HPIS Consulting, Inc.

Did we succeed as intended? Was the training effective?

When you think about evaluating training, what comes to mind? It’s usually a “smile sheet”/ feedback survey about the course, the instructor and what you found useful. As a presenter/instructor, I find the results from these surveys very helpful, so thank you for completing them. I can make changes to the course objectives, modify content or tweak activities based on the comments. I can even pay attention to my platform skills where noted. But does this information help us evaluate if the course was successful?

This blog has been merged with Why Knowledge Checks are Measuring the Wrong Thing.

(c) HPIS Consulting, Inc.

Moving From Traditional OJT to SOJT: What does it take to make OJT structured?

Hint: It’s more than 100% complete with your curricula requirements!

The Evolution

First there was “Just go follow Joe around” training …

And then came “and it shall be documented” …

Next the follow up question:  “Are they trained in everything they need to know?”

So, line managers used the SOP Binder Index and “Read and Understand SOP” became a training method…

But alas, they complained that it was much too much training and errors were still occurring …

So, training requirements were created, and curricula were born.

Soon afterwards, LMS vendors showed up in our lobbies and promised us with a click and a report, we could have a training system!

But upper management called forth for METRICS! So, dashboards became a visible tool. Leader boards helped create friendly competition among colleagues while “walls of shame” made folks hang their heads and ask for leniency, exemptions and extensions  … 

Why Traditional OJT Takes So Long

Experienced SMEs are not always the best OJT Trainers
Experienced SMEs are not always the best Qualified Trainers

While the introduction to this blog poked fun at how we’ve evolved over the last 30 + years, allow me to step back in time for a brief refresher about On the Job Training.  Traditional OJT (TOJT) is not planned so for the new hire it comes across as informal almost incidental.  Monikers like “follow Joe or Jane around” or “Sink or Swim” accurately described how TOJT occurred.  My personal favorite is “trial by fire”.  It can certainly feel like that when the “trainer” albeit a SME, is not qualified to train resulting in inaccurate steps and possible bad habits like unsafe short cuts.  It causes anxiety for the new hire especially when the daily schedule dictates what will be “taught” by the technician assigned to that task. Sometimes, over-anxious new hires quit before on boarding is finished because of their chaotic OJT sessions and lack of confidence in their trainers. 

Structured OJT Defined (SOJT)

SOJT is the process in which a qualified employee (trainer) passes job knowledge and skills to another employee in an organized, personalized and thoughtfully planned manner.  It involves both learning and doing at the same time. 

For first time managers, SOJT appears to take a lot of work and time.  They would rather focus on the production schedule convincing themselves that new hires are being trained while shadowing their training buddy.  Managers are evaluated on how productive they are, not on how many new hires are being trained effectively.  They also tell themselves, that it takes 6 – 12 months to get fully up to speed, so why spend even more time planning and scheduling a new hire’s learning journey.  In spite of that impression, the Quality of OJT directly impacts:

  • time it takes to “bring someone up to speed”
  • how Employees perceive the training they receive
  • how satisfied Employees feel about their jobs and working conditions
  • how quickly Employees can be re-trained on new SOP revisions.

But Managers Find the Time to Train Twice!

Difference between TOJT and SOJT
Difference between TOJT and SOJT

Yet that is exactly what happens when the root cause of a deviation or a mistake is deemed as Human/Operator Error. The corrective action is often “repeat training”.  This training involves unlearning the first sessions and then relearning the proper way, causing more strain on the new hire’s space for learning capacity (cognitive overload).  This is why TOJT takes so long. If, SOJT had been executed instead, the error more than likely would not have happened.  It’s an argument that I have been defending for over 30 years with Senior Leaders.  Why punish the New Hire for management’s decision to make production more important than effective training?

DOES HAVING CURRICULA MAKE OJT STRUCTURED?

Just having curricula doesn’t necessarily make OJT structured.  SOJT includes a deliberate review of regulatory, departmental and positional /functional requirements.  It also includes a TRAINING SCHEDULE for the curricula requirements and an individualized learning plan for new hires; not just a matrix with their name highlighted in yellow marker. 

Isn’t The Training Matrix Enough?

Just ask an OJT Qualified Trainer that question and watch the reaction you get. Maybe a shrug and a half smile if you’re lucky. The matrix is only a listing of the requirements usually generated from the LMS or an excel spreadsheet for those who haven’t migrated to an LMS yet. In more sophisticated systems, it will include due dates. But most printouts don’t include the Qualified Trainers assigned to deliver the training or any additional information about the learner.

why isn't the training matrix enough?
QTs need more than a matrix to manage Learners training curricula

From this list, QTs are expected to manage the completion of the requirements, deliver effective training sessions, provide feedback in a safe and nurturing learning environment and qualify learners with qualification events / performance assessments.  All that from a training matrix? Wow.   

During the HPISC Qualified Trainers workshop, I present the difference between TOJT (traditional) and SOJT (structured).  When I ask the QT’s where they feel their organization is, most of them will say still in the TOJT box but closer to the middle of the range.  Why I ask?  Invariably, they’ll tell me OJT is not scheduled.  “Just because I have the list of curricula requirements doesn’t mean the training gets scheduled or that qualification events get conducted”.  But rather, it happens when someone makes it a priority, an inspection is coming or a CAPA includes it as part of corrective actions.

There Is Still A Huge Misunderstanding Regarding R&U For Sops

For years, we have been documenting that we’ve read and understood our required SOPs. I can still recall my Employee Blue Card that listed every Monthly GMP and SOP meeting/discussion I attended. When I was asked about logging every SOP that I read during an interview, I responded that it was not a practice where I worked at the time. After I accepted the new position, I realized why that was such a watershed moment for me. 

Oh, But Now We Have Curricula!

As an industry, we became obsessed about signing for our SOPs.  Then LMSes were developed to help us not only record our R & U SOP dates but to give us a mechanism for tracking what SOPs were assigned to us, reporting 100% Trained metrics and overdue requirements.  And for a short while those records were sufficient. 

But over the years, FDA Investigators began seeing a disconnect between the R & U training records and the actual performance of employees who signed that they understood the SOP.  Upon the FDA site tour, they observed departures from the written procedures.  They uncovered deviations involving Operator /Human Error, repeat deviations and even CAPAs for training fixes.  So, is it falsification of records?  Is it poorly written SOPs? Is the training ineffective? 

FDA stepped up their expectations and began asking deeper questions.  How do you know if your training is effective?  This question applies to both GxP Training sessions as well as SOPs. 

  • Who is qualified to train employees and how do they get qualified?
  • Are your employees qualified to perform their job functions and how do you know? 
  • What does that documentation look like?

SOPs Are Now Online, But It’s Still Read and Understood for SOP!

Well, I’m told we are now more compliant with ensuring that only the most current version of the SOP is used for training. LMSes and eDoc platforms have given us efficient mechanisms to document that employees have completed their training curricula. But do they understand their procedures enough to perform them correctly back on the job?

It’s still Read and Understand SOPs! Whether we access our SOPs through an eDoc system or the LMS portal, we are still only reading them. To call this eLearning is a bit of a stretch especially when compared to the design of today’s eLearning courses. Nonetheless, some new hires are still being provided with a long list of required SOPs and trained on where to find them in the LMS. “Oh, we’ve made it easier for them to manage their SOP list. They’re online now!”

I’m excited that as an industry, we are evolving with our training practices to keep up with regulators’ expectations regarding GMP and SOP Training.  But have we changed the training culture yet?  Are we just replacing attendance forms for e-signatures or are we delivering effective training?  Can we confidently say, “Yes, our employees are qualified prior to release to task”?

FD-483 Observation: “Your site has numerous instances of R & U for SOPs”

So, when there is a large number of documented R & U events for SOPs, very little OJT documentation and they are still finding repeat errors, HE/OE deviations and a host of CAPAs, FDA Investigators are going to examine what is your training process and how effective is it.

SO, WHAT ELSE MAKES OJT STRUCTURED?

SOJT is formal and it’s documented. Without the use of approved training materials, department SMEs often use different methods each time they deliver training and the training content can vary from “trainer to trainer”.  This causes confusion for the new hire during his/her onboarding phase which can then lead to Operator Error or non- compliant performance later.   

During an on-site response to an urgent performance problem, the Head of Operations expressed deep concerns about inconsistent OJT being delivered by his trainers.  During an FDA inspection, he was shown numerous examples from the Investigator, that his SMEs were teaching techniques for a critical process procedure that

  • were not written down nor were they approved (aka their secret sauce) and
  • were not at all consistent with each other. 

Naturally, this led to a FD-483 observation, a high visibility corrective action project with global impact.  As part of the CAPA (Corrective Action Preventive Action) investigation, trainers were interviewed. Their responses revealed the use of varied content; despite having an OJT checklist, the actual procedure, and knowledge of the site training SOPs. 

QT’s need to use “Quality Control Unit” approved written procedures (aka SOPs) as the main document to train with and the proper documentation to record an OJT session.  But documenting OJT sessions has been a bit of a challenge for Trainers, LMS Administrators, and QA Doc Control Staff. 

What Is Considered An OJT Session?

The hardest question to resolve isAre we required to capture every OJT session or just one?”  My favorite lament is – “Do you know what that will do to our database not to mention the amount of paperwork that would create!” A workaround to all these questions and concerns is to capture at least one session along the progression of each OJT step as per your OJT Steps Model, thus documenting adherence to the procedure. If we keep it simple and document that our learners have experienced each step, then we are complying with our OJT process and minimally documenting their OJT progression.

Even more challenging is adding the required SOJT steps to curricula.  Unfortunately, it generates almost double the requirements. So for companies who focus solely on the number of requirements, this is daunting.  What helps is differentiating between R & U step and completion of the actual OJT events.  Some organizations go one step further and also add the Qualification Event as a final requirement.

SOJT IS ALSO DELIVERED BY A DEPARTMENT SME QUALIFIED TO DELIVER OJT.  The old adage for selecting department trainers based documented R & U SOP paperwork no longer meets regulatory expectations.  The expectation today is to have a QA approved process for qualifying SMEs. I’m not talking about a mere mention in your training policy.  But a standalone process that depicts steps from start to finish.  Yes, it is necessary if you want to answer “Yes, I have a procedure for Qualifying SMEs as Department Trainers”.  Being able to present a SOP that addresses how your SMEs are qualified to sign the training records is priceless in the middle of an ongoing regulatory inspection.

What Belongs In The Qualified Trainer SOP?

Ideally, the procedure needs to include these 4 parts:

I. Eligibility

II. QT Workshop/TTT Equivalent 

III. Evaluation

IV. Performance Demonstration

I. Eligibility

In some organizations there is no difference between SMEs and Trainers.  And this is precisely why FDA has asked for clarification.  Can anyone be a Qualified Trainer (QT)?  Establishing criteria is the best way to reduce favoritism and the tendency to pick the most senior person.  SMEs need to become QTs through a formal process.   The selection criteria should be part of the qualifying documentation along with any supporting statements for eligibility selection (for example, nominated by supervisor or responded to the call for volunteers).

In this definition, everyone who is a QT is an SME in some specified area.  All nominees need to be content qualified on the subject matter they will be teaching.  This means being able to produce proper documentation confirming nominee’s eligibility. This is not always the case.  I have seen this assumption backfire horribly and cause major ripple effects on project milestones.  Find out now before an inspection, please.

II. QT Workshop/ TTT Equivalent

Sometimes known as Train-the-Trainer.  The course needs to include learning theory, training and coaching adult learning peers and agreeing to use the proper documentation.  Another key component of the OJT TTT workshop is exploring the challenges of “Life as a Qualifed Trainer”. Learning how to facilitate a live classroom event that can come later.  In fact, many QT’s are stepping up and want to expand their trainer’s toolkit for “Basic Facilitation Skills / Running a Live Classroom Event”.

III. QT Workshop Evaluation:

Simply attending this course isn’t enough.  But whether a written test is the best measure to use is open for debate.  Let’s start with the question: Why do we need an evaluation in the first place?  If you’re tempted to say because FDA wants it, I suggest re-reading CFR x211.25 again.  If you are anticipating the training effectiveness question, then you are in sync with industry practice.  But it is less about “the test” and more about how you determine the effectiveness of the training event.  Benchmarking from other certification courses, a written test usually follows the course.  So, having a written evaluation is not unreasonable.  The debate is about what format you use.

The Written Test

Most folks are familiar with taking a written test.  When informed upfront, QT’s expect the test to come at the end of the workshop.  But what is the outcome of test?  What does it really measure?  Is it a measure of their retention or comprehension?  Since SOPs are not supposed to be memorized, how can we dictate memorization of the course content? Open book is allowed in some organizations.  What then does the paper-n-pencil test accomplish?  Having the knowledge doesn’t mean that they will use the concepts “in the moment of choice”.

Consider the Action Planner

If learning transfer is what we really want and expect to achieve, then wouldn’t some kind of post workshop action planner be a more appropriate measure of effectiveness?  “Oh, but we can’t control what happens after the workshop”, you say?  “Using an action planner requires buy-in from their managers.  And commitment to follow through to host the 1-1 follow up meetings with their QTs” is what you might be thinking right now.

Let us not lose focus on the ultimate goal of the workshop.  It’s a 3-way partnership between QTs, L&D/QA Training, and Managers.  The real work happens AFTER the workshop is over.  What better way to use classroom time to discuss strategies for barriers and challenges and document their commitment for applying the concepts and procedures than a post workshop action planner?  Can the written test do all this?

IV. Performance Demonstrations

This is after all a qualification of SMEs as Trainers program and therefore, a performance demonstration is required.  What type depends on where your site is with respect to employee qualifications.  The two types are demonstrating a live OJT session in the classroom and conducting an employee qualification at the workstation. 

RECAP: WHAT IT TAKES TO MAKE OJT STRUCTURED

  1. Rigorous Curricula that includes SOJT and Qualification Events not just SOPs
  2. Methodology for OJT Steps that includes documentation
  3. Qualified Trainers who deliver SOJT curricula requirements
  4. Individualized Learning Plans and schedules for New Hire’s Learning Journey

Is that all?  No!

How Committed Is Your Organization To SOJT?

Ronald Jacobs and Michael Jones, in their 1995 groundbreaking book, Structuring On-the-Job Training, inform us that SOJT as a system functions within a larger context, namely the organization.  SOJT is not a standalone program.  Conflicts, competing priorities and mixed messages can influence the success of your SOJT program.

Do these sound familiar?

  • Business priorities and organizational change initiatives
  • Upper managements real perception of the value of OJT
  • Union shop and potential violation to use SMEs as Qualified Trainers
  • Alignment of goals for training and goals for other quality systems
  • Willingness of line leaders and staff functions to manage and maintain SOJT after launch

The Training Quality System, in my opinion, is THE most cross functional system.  It has to harmonize with other quality systems AND organizational systems in order to deliver performance improvement. 

Moving from TOJT to SOJT what it takes
Management Support is needed to make SOJT work

So, management support has to be more than lip service.  The real support is in the alignment of goals, clarifying expectations, allocating resources and budgeting time to deliver OJT using an approved OJT methodology.  This is a culture shift for many organizations but well worth the effort if management really believes in SOJT.    The “proof is in the pudding”.  Are your leaders “walking their talk”? – VB

Who is Vivian Bringslimark?

got curricula great! Now move forward with structuring your OJT methodology

(c) HPIS Consulting, Inc.

Is Your Quality Training SOP Sufficient?

Is your response, “Yes!” or is it “Yea, I think so. How can I be sure?”

  • Are you trying to explain what you do for training or describe how your site handles the Quality Training System?
  • Who is your audience?
  • Who are the procedures being written for?

There shall be written procedures even for training!

Step back and take a big view of training from start to finish. There are three distinct segments: (1) preparation, (2) delivery, and (3) measuring effectiveness. To some leaders, the idea of creating more than one training procedure for these three segments is baffling.  Training isn’t that hard. Why are you making this so complicated, they ask?  The simpler the process is, the easier it is to follow and administer, is what they are really thinking.

Yet, training root cause analyses involving operator or human error often yields some contributing factors around how the individual was trained.  On the Job Training is not that simple after all and requires more than a check on someone’s to-do list.  Find out why in the e-Book, Training Root Cause Analysis. In spite of this, the level of detail in Training SOPs is always a concern for both SOP Author and Approver.    

But SMEs like KISS for Training

From an end-user perspective, I need enough detail so I can accomplish what I need to do without over-complicating my task, stay within compliance, and be effective as a qualified trainer. Since most Qualified Trainers are not full-time training professionals, the training procedures become their how-to tools not just a bunch of QA or HR rules about training. Without defining what your program includes and how to execute it, Qualified Trainers and other SMEs will conduct their training delivery assignments in a manner that is as simple as possible and may not be 100% compliant. Administrative procedures are not for them, they argue.

It takes 6 elements to make a training system robust

The big view of training may have 3 Big Blocks (preparation, delivery, and measuring effectiveness), but it takes 6 elements to execute it well.   They are:

  • GxP and Training Curricula
  • Planned OJT for Procedures
  • Use of Qualified Trainers
  • Employee Qualification and Training Effectiveness Measures
  • On-Going GxP Refresher Series
  • Training Documentation Process
6 Elements of a Robust Training System

For each of these elements, there needs to be a set of “how-to-execute” instructions. However, it is not enough to describe what these elements are. That leaves a lot of room for interpretation and inconsistent documentation or no documentation at all. For example:

  • Are you documenting on-the-job training; every time or just select sessions?
  • Does your organization complain about over-allocated curricula?
  • Do they understand the documentation requirements for your LMS?
  • Does your training document explain how to conduct OJT? You’d be surprised just how many different techniques SMEs have for conducting OJT on their watch.
  • Can anyone who has a signed training record be a department trainer? If yes, you need to upgrade your criteria and create a process around nominating and qualifying SMEs for OJT. Training is on the top 10 list for inspections.

So how is this not complicating Training?

The debate for the 1 All-Encompassing SOP (standard operating procedure) vs. multiple tailored work instructions will always bring comments after the latest version goes into effect. This includes non-QT end-user feedback criticizing how complicated and confusing the procedures are!  When pressed further for specifics, most admit that they just want to train themselves and not deal with the documentation requirements. So, addressing one set of concerns can actually create more complaints around the very changes. See the table below for pros and cons.

Screen Shot 2016-06-01 at 10.23.53 AM
Pros and Cons for One AP Training SOP vs. Smaller, Multiple Work Instructions

Remind me again who these procedures are for?

The real question is to what level of detail is appropriate.  And the answer lies in the culture of the department and the company’s history with the FDA.  I once worked in an organization where the procedures were stripped to bare essentials, even the definitions were removed!  A recent warning letter left battle scars for the General Manager.

But seriously, how much is enough? Another way to answer this question is by looking at your document control hierarchy of documents.  If your site differentiates between SOPs and Work Instructions, the need for detail can be relocated into a suitable work instruction that is designed to provide detailed how-to steps.  AKA Level 3 type documents.  Refer to the diagram below.  The SOP (Level 2) can then provide a broader high-level overview addressing “the what” without getting encumbered into the “how-to execute” steps.  Some document pyramids include a Level 4 in which much detail can be used for training purposes.

A screenshot of text

Description automatically generated
Referring to Document Pyramid determines level of appropriate detail

Back to the opening question

While addressing ease of navigation and use concerns is laudable, it is not always realistic to satisfy all users.  You need to explain what your Training “Program” contains, a high level policy that answers “the what”.  But you also need to describe how your training is conducted.  Ask yourself the following two questions:

  • Does the “Training SOP” contain many pages detailing steps for each of the 6 elements?  This is an indicator that you need to consider some Level 3 Work Instructions.
  • Do you have too many standalone task-focused SOPs (or Level 3 Work Instructions) that could be grouped into a few larger processes?  Recall the 3 Big Blocks of Training

From experience working with SME teams, three “how-to” procedures are usually sufficient: preparation process, delivery and documentation process, and measuring training effectiveness process.  So the next time you find yourself in the level of detail discussion, consider which of the levels (1,2,3,4) is the best place to park your “detailed” content. – VB

Who is Vivian Bringslimark?

Think your Training SOP could use a makeover? How about a document review?

Interested to see how the 3 Big Blocks of Training and the 6 Elements of a Robust Training System all interface with each other? Send your email request above.

(c) HPIS Consulting, Inc.

Are all your SMEs Qualified Trainers?

I got a phone call from my Lead SME’s boss one morning. “How many more sessions do you need”, I asked him. I had already delivered 4 back-to-back workshops with class sizes of 25-30 SMEs; which was beyond optimal. So I asked him why. I needed to find out what was driving the surge in identified Qualified Trainers (QTs). I learned that a retrospective qualification needed to take place in order to close out an inspection observation. The total number of SMEs needing “proper paperwork” was well over 700. Since the redesigned training system was now in effect, these undocumented SMEs as Trainers would have to follow the new procedure. Or would they? Our discussion shifted to what type of training these SMEs will be delivering.

I then shared a related story with him. Several years prior, I got entangled with a “CAPA crisis” that involved QTs. No sooner did we launched the QT program and put the new procedure into effect, the CAPA quality system temporarily shut down shipping over a weekend. Upon return to the site, I was summoned to an emergency meeting from the security gate. Amazingly, a new practice/rule that only a Qualified Trainer can conduct training evolved from “only OJT QT’s can deliver OJT and perform Qualification Events” as per the SOP! This was clearly a case of misunderstood scope.

Does every SME need to be qualified as a Trainer?

Trainers Grid for determining QTs Scope of Training

In the Life Sciences arena, there are 5 recurring situations that require training: Self, Corrective Actions, Classroom (ILT), Structured OJT, and Qualification Events (Final Performance Demos).

Self can be achieved by the individual reading the procedure and signing the training record. This is also known as Read & Understand (R & U) for SOPs. I personally don’t think of it as training, it is reading. Yet, in some situations, reading is all that is required to gather the SOP information.  If on the other hand, you need to execute the steps of the SOP and complete required forms, then additional training with the SOP Author or a QT is the appropriate next level of training.

Deviations/ Corrective Actions stemming from a Corrective Action Preventive Action incident. Minimally an SME or the SOP Author is needed to ensure the credibility of the content. These types of training sessions have become known as Corrective Actions “Awareness” Training.  And more and more SMEs are now being required to deliver this training in a classroom setting.  They need to be qualified to deliver classroom sessions especially if the event is related to a significant CAPA or regulatory inspection observation.

Classroom (Instructor-Led Training) is preferred for knowledge-based content affecting a wide range of employees. The skillset needed is facilitation / managing the classroom and delivering content as designed by the instructional designer. Think of GMP Refresher sessions in the Training Room.

Years ago, it was a lot clearer to distinguish between classroom trainers and SMEs as OJT Trainers.  OJT was delivered 1-1 by “following Joe/Jane” around.  Classroom Trainers delivered their content in a classroom of many learners using slides, flipcharts, and handouts.  They were usually full-time dedicated training staff.  Instructor-led training requires training in learning theory design and practice in what used to be referred to as platform skills.  Today, it is more commonly known as “Running a Classroom” or “Basic Facilitation Skills”.  

Many of today’s OJT QTs are also being requested to deliver “Group Training” sessions on content found within their SOPs.  While the target audience may be the same set of peers, the scope, objectives, and tools used to deliver instructor-led training is vastly different from the OJT train the trainer course.

Group Training vs. OJT
Are your QT’s becoming Duo-Purposed?

Structured OJT is On the Job Training delivered by a Qualified OJT SME using the approved OJT Methodology. OJT QT’s attend the Qualified Trainers Workshop which focuses on the OJT Steps Model, how to perform the equipment, and complex SOPs via hands-on and the challenges of Life as a Trainer.  Should every seasoned employee become a QT based on their seniority and subject matter expertise? No, not necessarily.  Because there are some SMEs that don’t want to share their knowledge and therefore, may not make an effective OJT Trainer.  Establishing a set of nominating criteria provides an objective rationale for additional interpersonal qualities that help define a more well- rounded SME. 

Qualification Events (the Final Performance Demonstration) are formally documented observations of learners performing the procedure/task at hand in front of a Qualified OJT SME using an approved SOJT Checklist or rubric.  It is these events that set apart a Technical SME from a Qualified Trainer.  The QT workshop includes a dedicated lesson on what to look for during Q-Events and what the QT signature means for the integrity of the Employee Qualification Program.

Can having too many QTs be a problem?

It can be when there is no one else to train; to deliver OJT steps. While many of you may be wishing for this situation, it can eventually happen if staffing levels are adequate, shifts are normalized and SOPs revisions are managed via R & U only with the LMS.   How do you keep your QTs engaged and fresh if there are no opportunities for OJT sessions? I have some ideas for you to explore.

Re-examine the practice of online R & U only for SOP revisions.  I bet some of those revisions were significant enough for a face to face discussion (aka Group Training)  and there is probably at least one SOP revision in the past year that should have required a demonstration of task for optimum transfer of learning back on the job.  *Just because all employees are now qualified, doesn’t mean the program sits in hiatus waiting for new hires to join the company

When you have too many QTs who may be underutilized, it is also an appropriate time to administer the Trainer Mojo Assessment.  Based on the QTs scores, it might be time to say thank you for a job well done for the low scoring QTs.  You may be pleasantly surprised by who is ready to walk away from the training role?  Or you may have a cadre of QTs who legitimately need more training and hence, the need for some new modules is now justified.  Many of your excess SMEs were identified long before criteria was put into a place or the SOP was established.  If the Trainer Mojo Assessment doesn’t bring any discussion, perhaps it’s time to “re-nominate” them using the criteria within the SOP and offer a refresher series on the QT Workshop content. Or arrange for developmental assignments that expand their subject matter expertise or advances their training repertoire into a classroom facilitators? 

What is exciting for me is that many OJT-QTs are stepping up and volunteering to attend the SMEs as Classroom Facilitators workshop as part of expanding their QT’s toolkit.  Many of them want to learn more about teaching peers and working with adults.  A few have now become promoted to full-time trainer for L&D /QA departments.  Which of your OJT QTS are ready to step up and move into the classroom?  It’s time to find out and be part of the current trend.  -VB

ready to have a chat with Vivian?

Who is Vivian Bringslimark?

The Trainer MOJO Assessment can be yours. Sign up for the mailing list and be sure to mention the Trainer MOJO Assessment.

(c) HPIS Consulting, Inc.

How committed to SOJT is your organization?

Are you looking for management support for the Qualified Trainers and the time needed to deliver SOJT?  If only we were required to have a procedure for that!  It may not be an SOP or even a policy document, but industry guidance documents provide a lot of references to management involvement.

ICH Q10 – Pharmaceutical Quality System

While not mandatory, management needs to seriously take notice of ICHQ10 guidance document released in April 2009 (1).  In particular to the following:

  • MANAGEMENT RESPONSIBILITY 2.3 Quality Planning 

“(d) Management should provide the appropriate resources and training to achieve the quality objectives”.

  • CONTINUAL IMPROVEMENT OF THE PHARMACEUTICAL QUALITY SYSTEM

4.3 Outcomes of Management Review and Monitoring

The outcome of management review of the pharmaceutical quality system and monitoring of internal and external factors can include:

(b) Allocation or reallocation of resources and/ or personnel training.”

Under a Quality System: Managers Expectations for Training

Referencing the Sept. 2006 issue of Guidance Document for Quality Systems: IV. Management Responsibilities | B. Resources (2), we see alignment with the training CGMPs for continued training so “as to remain proficient in their operational functions and in their understanding of CGMP regulations.”  “Typical quality systems training should address the policies, processes, procedures, and written instructions related to operational activities, the product/service, the quality system, and the desired work culture (e.g., team building, communication, change, behavior).”

And my personal favorite, “When operating in a robust quality system environment, it is important that managers verify that skills gained from training are implemented in day-to-day performance.”  The responsibility for training under a quality system is not assigned to just one person or one function.  It is a shared responsibility across the entire organization.

Goals of the Train-the-Trainer Program (TTT) vs. OJT Program vs. Employee Qualification Program

In order to have qualified employees, they need to receive structured on the job training delivered by a qualified trainer who is content qualified and training process qualified via a Qualified Trainers’ workshop.   Each of the three “programs” has defined outcomes that are dependent upon each other.  Unfortunately, the term program has been a bit overused throughout the years and can have a variety of meanings for folks.  For purposes of this blog, TTT program means OJT Qualified Trainers workshop, OJT Program means OJT methodology/procedure and Qualified Employee program means having a robust quality training system beyond the newest Learning Management System (LMS).

As you can see from the diagram, Qualified Trainers are at the core of all three “programs”.  The OJT QT Workshop is designed to prepare the QT’s for the realities of life as a Qualified Trainer. It includes the pre-work of familiarizing themselves with the Quality Training System procedures and then in class: exploring basic learning theory and committing to what the QT signature means. The OJT Program is about delivering structuring on the job training consistently following the approved methodology. An Employee Qualification Program is the validation of training effectiveness of the OJT methodology and the clarity of the underlying procedures.

Readiness Factors for SOJT and an Employee Qualification Program

Let’s start with a written purpose statement for having qualified employees beyond it’s required.  What is the company’s philosophy on achieving qualified status?  Is there agreement among the leadership for the level of rigor required to demonstrate performance and achieve a pass rating?  Where are the SOJT program goals written?  Does a schedule exist for SOJT and qualification events other than an LMS printout with required due dates.  That is not a schedule for SOJT.  Do you have clearly defined objectives for the QT workshop captured in a document or perhaps a procedure?  Is there a single owner for all three programs or is responsibility and accountability assigned accordingly?

Ronald Jacobs and Michael Jones, in their 1995 ground breaking book, Structuring on-the-Job Training, inform us that SOJT as a system functions within a larger context, namely the organization.  SOJT is not a standalone program.  Conflicts, competing priorities and mixed messages can influence the success of your SOJT program.  What else is going on in the organization that will compete for the same set of QTs? Remember they are also your most experienced and technical subject matter experts.  How is the overall Employee Qualification program aligned with the other quality systems? 

Recognition for QTs and Qualified Status

Most QTs are not fully dedicated to delivering training for departments.   There are pros and cons for this decision.  For now, I will leave them out.  Suffice it to say, they are tasked with both their “day” job and the responsibility for delivering training when needed.  They are at times, doing two jobs.  Whether or not they are compensated additionally for delivering SOJT, acknowledging their contribution to the department and the organization is part of management support.   It takes more than “you are doing a good job, keep it up”. 

Often supervisors and managers don’t know what else they are supposed to do to show their support, other than allow them to attend the QT workshop.  The interested ones will “pop” in during lunch and chat with their direct reports.  Others will show up at the end for the poster activity (equivalent to a written test) and some will come to learn about the parking lot issues that need follow up.  The energy in the room when this happens is amazing. 

“When operating in a robust quality system environment, it is important that managers verify that skills gained from training are implemented in day-to-day performance.

Guidance Document for Quality Systems, Sept 2006

To help ease the knowledge gap between a manager and their now Qualified Trainer, I started delivering the Leadership Briefing module prior to the QT workshop delivery.  The purpose is to provide an overview of the content highlights, alignment with initiatives / CAPAs/ agency commitments and more importantly to secure agreement for the following:

  • criteria for nominating a QT
  • roles and responsibilities of QT
  • scope of work QT’s can be assigned
  • expectations for QT’s post launch
  • what happens day one after workshop is done
  • what is the status of the SOJT checklists
  • scheduling and budget concerns.

If the organization says they support the qualification program, then what happens when employees achieve qualification status?  Nothing?  A non-event? Or is it announced in newsletters, plasma screens and other company announcements?  Is it a big deal to be able to perform independently and free up a much-needed QT for another learner?  I keep hearing over and over again about how there aren’t enough QT’s to deliver SOJT the right way.  One would think qualification status on SOPs, especially big complex processes deserves SOME kind of recognition.   Just how committed are the managers and supervisors?  QTs and employees draw their own conclusions about the level of real management support for the programs.  

Supporting QTs is more than participation in QT Workshop

If truth be told, after launching the QT workshop, many supervisors privately don’t support the program.  They lose their top performers during the workshop and the hours it takes to train someone. Forget about giving QTs adequate time to complete the paperwork properly! And then leadership wonders why good documentation practice (GDP) issues continue to be a problem? The non-distracted performance observations that QT’s are expected to conduct for the qualification demonstration drive supervisors and line managers crazy the most – what, they can’t do anything else but observe?  Hence, many QT’s are asked to multi-task just to get the work done: not enough resources they are told.  For supervisors, productivity and the workload will always trump SOJT and qualification events, until their bonuses include completion of training and qualification events.

What Real Support Is Supposed To Look Like

My key take away message is that attending the TTT program/ QT workshop is not the end of the OJT program or the Employee Qualification Program but rather the launching point.  Management support needs to go beyond just nominating QTs and allowing them to participate in the workshop.  The real support is in the alignment of goals, clarifying expectations continuously, allocating resources for training and budgeting time to deliver OJT using an approved OJT methodology that includes qualification events.  This commitment of time and sponsorship for qualified employees is a culture shift for many line managers and site leaders.  But actions do speak louder than words.  -VB

(1) Guidance for Industry Q10 Pharmaceutical Quality System | US Department of HHS | FDA
| CDER | CBER April2009 ICH

(2) Guidance for Industry Quality Systems Approach to Pharmaceutical CGMP Regulations U.S.Department of HHS | FDA | CDER | CBER | CVM | ORA| September 2006 Pharmaceutical CGMPs

Jacobs RL, Jones MJ. Structured on-the-job training: Unleashing employee expertise in the workplace.  San Francisco: Berrett-Koehler: 1995.

How can you tell if your site is ready for SOJT?

HPISC has created a 2 part checklist of questions and attributes to explore. The checklist is available gratis when you become a HPISC mailing list member. Just be sure to include that are you interested in receiving the SOJT Readiness Checklist.

Mailing List Subscribers automatically receive newest assets and tools directly into the inbox.

Who is Vivian Bringslimark?

(c) HPIS Consulting, Inc.