Batteries Not Included: Not All Trainers are Instructional Designers or Classroom Facilitators

When I left the manufacturing shop floor and moved into training, full-time trainers presented in the classroom using a host of techniques, tools and relied on their platform skills to present content.  Subject matter experts (or the most senior person) conducted technical training on the shop floor in front of a piece of equipment, at a laboratory station, or a workbench. 

For years, this distinction was clearly practiced where I worked.  Trainers were in the classroom and SMEs delivered OJT.  Occasionally a “fulltime” trainer would consult with an SME on content or request his/her presence in the room during delivery as a back-up or for the Q & A portion of a “presentation”.  It seemed that the boundaries at the time, were so well understood, that one could determine the type of training simply by where it was delivered.

Training boundaries are limitless today

Today, that’s all changed.  No longer confined to location or delivery methods, fulltime trainers can be found on the shop floor fully gowned delivering GMP (Good Manufacturing Practices) content for example. And SMEs are now in the classroom more each day with some of the very tools used by fulltime trainers!   What defines a fulltime trainer from an SME is less important, what is necessary however is what defines effective instruction.

Your title might have the word trainer in it.  One of your responsibilities might be a qualified trainer. And you know how to use PowerPoint (PPT). Does this make you an Instructional Designer as well?  Some say yes and others cry foul as they cling to their certificates and advanced degrees. So, forgive me when I say, not every Trainer or Training Manager has the skill set or ID competency embedded in his/her toolbox.   It’s analogous to the toy box on the shelf at Toys R Us – “NOTE: Batteries Not Included”.  Except in our case, the note may be missing from the resume, but definitely embedded into the job description if you are QA L&D or HR Training and Development.

Instructional Design is a recognized profession 

Instructional Design (ID) as a field of study has been offered by many prominent universities for quite some time and is now more known as Instructional Technology.  Underlying the design of a course or a learning event, is a methodology for “good” instructional design and really good instructional designers will confess that there is a bit of an art form to it as well.  Unfortunately, with shrinking budgets and downsized L&D staff, there are less resources available to develop traditional course materials of the past.  Not to mention, shrinking timelines for the deliverables.  So, it makes sense to tap SMEs for more training opportunities since many are already involved in training at their site.  But, pasting their expert content into a PPT slide deck is not instructional design. 

What is effective design? 

Basic Elements of Course Design

To me, effective design is when learners not only meet the learning objectives during training but also transfer that learning experience back on the job and achieve performance objectives / outcomes.  That’s a tall order for an SME, even for fulltime trainers who have not had course design training. 

The methodology a course designer follows be that ADDIE, Agile, SAM (Successive Approximation Model), Gagne’s 9 Conditions of Learning, etc., provides a process with steps for the design rationale and then development of content including implementation and evaluation of effectiveness.  It ensures that key elements are not unintentionally left out or forgotten about until after the fact like evaluation/ effectiveness or needs assessment.  In an attempt to expedite training, these methodology driven elements are easily skipped without fully understanding the impact of leaving them out can have on the overall training effectiveness.  There is a science to instructional design. 

PowerPoint Slides are only a visual tool

Using PowerPoint slides by themselves does not make the training successful.  It’s one of the main tools a trainer uses to meet the objectives of the learning event, albeit the main one. The “art form” occurs when a designer creates visually appealing slides / eLearning scenes as well as aligned activities and engaging exercises designed to provide exploration, practice, and proficiency for the performance task back on the job.  But there is a difference between a course that is created to help the Trainer achieve his/her agenda and one that successfully engages learners to participate, learn and then transfer their insights back home to the job where changed behavior improves the department’s metrics.

The more trainer/instructor driven the course is, the less participation is required from the learner. For example, the instructor makes all the decisions about the course objectives and content, develops the course, delivers the course, and conducts the assessment.

From passive to active to immersive

As you move along the Learner Participation Continuum, the learner is required to participate more, and the trainer does less “talking”. The learner acquires knowledge and skills through activities that s/he experiences with the assistance of a “facilitator”.  The facilitator is focused on helping the learners meet their needs and interests. It is through these firsthand experiences and facilitated dialogue with other learners that thoughtful analysis and interpretation can become the focus of the instruction. The end result is that learners take full responsibility for decisions, actions and consequences.

Moving from Presenter Controlled Training to Learner Focused Facilitation

Moving to a more Learner Focused approach shifts the effort of the design from “deliver this content” to facilitate learning transfer for performance back on the job; which is after all the end goal for a training event. The new design includes opportunities for group participation, utilization of participants’ expertise, and real-life problem solving; key principles of adult learning.

On the one end of the continuum is the lecture which is one-way communication and requires very little participation.  At the other end, we have experiential learning and now immersive learning environments with the introduction of 3D graphics, virtual simulations, and augmented reality.

Most Trainers and SMEs tend to suffer from the “curse of too much knowledge” and find it difficult to separate the need-to-know from the nice-to-know content.  As a result, it shows up in the slide deck with overburdened slides filled with a lot of “stuff”.  Training for them takes on a lecture-style format. The thought of facilitating an activity gives most SME a case of jitters and anxiety.

So, in the “SME as Facilitator” workshop, nominated SMEs as Facilitators are encouraged to step away from the podium and use their eyes, hands, and voice to engage with their audience. Easier said than done, yes. That’s why the course is designed to allow them to take small steps within the safety of a workshop environment.

But rather than trying to pull off a fully immersive session, SMEs as Facilitators are introduced to techniques that “liven up” the lecture. They are shown how to move back and forth from passive listening (sit, hear, see) to active involvement (write, construct, discuss, move, speak). This requires the ability to:

  • follow a well-organized design plan
  • capture and hold the attention of learners
  • use relevant examples and deviations if possible
  • show authentic enthusiasm
  • involve audience both directly and indirectly
  • respond to questions with patience and respect.

While lecture has its merits, today’s learners want engaging content; that is timely, relevant and meaningful.  And while virtual reality and simulations are engaging and very immersive, courses and learning events using these techniques rely on well-funded budgets.  Most Training Departments are not that fortunate.   In the middle of the range are “lively lectures” and alternate methods such as:

  • Shift one step to right to begin the move to active learningDemonstrations
  • Case Study
  • Guided Teaching
  • Group Inquiry
  • Read and Discuss
  • Information Search.

Take the 1st shift right.

It’s really about starting with the learners’ expectations and the current organizational culture and then moving one step to the right. If they are used to lectures from SMEs, then work on delivering effective lectures before experimenting with alternate training methods. The overnight shift may be too big of a change for the attendees to adjust to despite their desire for no more boring lectures. Small incremental steps are the key.

Shift to the right when ready for the upgrade

Moving from Lecture to Delivering an EFFECTIVE Lecture

Thoroughness in the preparation reflects care and thoughtfulness. Learners appreciate the personal desire to deliver a livelier lecture. Stepping away from the podium forces the Trainer/SME to take action and allow the learners to “get up close” with the SME as Facilitator. This in turn is reflected in the learner’s desire to respond to questions and dialogue during a facilitated discussion. The rule of thumb for lecturing is approximately 8-10 minutes max. For virtual sessions, the rule of thumb is approximately 5 minutes. 

Take the 2nd Shift: Cut Content to Add Interactivity

How is this done? Upfront in the design of the course materials. The course designers have spent time and budget to prepare a leader’s guide that captures their vision for delivering the course.  SMEs as Facilitators (Classroom SMEs) need to study the leader’s guide and pay attention to the icons and notes provided there. These cues indicate the differentiation from lecture, to an activity whether that be self, small group, or large group. While it may be tempting to skip exercises to make up for lost time, it is better for learner participation to skip lecture and modify an activity if possible.

“STOP TALKING and get learners engaged in some form of activity, practice or reflection exercise”, Vivian Bringslimark, HPIS Consulting, Inc. 

One of the benefits of shifting to this learner focused design is the opportunity for learners to process the content, to make it meaningful for themselves and then associate memory links to it for later recall when the moment of need is upon them.  This can’t happen while the trainer is lecturing.  It happens during activities and reflection exercises designed to generate their own ideas during small group interactions and link it back to the course content/objectives.  Learners are prompted to openly discuss issues and problems within a “learning lab” style environment. Trainers become empathetic listeners as they create a climate of trust and safety. They become a Facilitator.

Of course, this shift also requires that site leadership and local management not only support the facilitated learning lab concept but follow through on issues and concerns that surface. Failure to do so undermines not only the facilitator’s credibility but the entire training program.

Wow, won’t this take longer to design, you ask?  Yes, in the sense that the design is now from the learner’s point of view. This means that the designer will need to research examples, collect data, and might have to develop a story from an incident, a deviation or significant CAPA, etc. The reward is that the Trainer/ Classroom SME stops talking and gives employees more engaging learning sessions. So learners become more accountable for participating and guess what – the SME’s session is no longer a boring podium speech

References:

Silberman, M. (1990). Active Training: A Handbook of Techniques, Designs, Case Examples, and Tips.  Lexington Books, New York.

Who is the Author, Vivian Bringslimark?

HPISC Library has articles, impact stories and white papers.

SME Impact Story: The Real Meaning of TTT

White Paper: Step Away From the Podium

(c) HPIS Consulting, Inc.

Compliance Training: No More Passive Learning

Special Thanks to LTEN for Publishing this article!

Looking for inspiration to liven up a lecture? I have 10 ideas for you.

HPISC Library has articles, impact stories and white papers.

No More Boring GMP Lectures |Impact Story about how to introduce more interactivity into GMP Refresher sessions.

More GMP Resources Available here.

Available HPISC GMP Refresher Course | Looking for More Choices?

Who is Vivian Bringslimark?

(c) HPIS Consulting, Inc.

Moving from Presenter Controlled Training to Learner Focused Facilitation

The more trainer/instructor driven the course is, the less participation is required from the learner. For example, the instructor makes all the decisions about the course objectives and content, develops the course, delivers the course and conducts the assessment.

This blog has been merged with “Batteries Not Included: Not All Trainers are Instructional Designer or Classroom Facilitators”.

(c) HPIS Consulting, Inc.

Instructional Design: Not Just for Full Time Trainers Anymore

When I left the manufacturing shop floor and moved into training, full-time trainers presented in the classroom using a host of techniques, tools and relied on their platform skills to present content. Subject matter experts (or the most senior person) conducted technical training on the shop floor in front of a piece of equipment, at a laboratory station or a workbench.

This blog post has been merged with “Batteries Not Included: Not All Trainers are Instructional Designer or Classroom Facilitators.

(c) HPIS Consulting, Inc.

Moving from Lecture to Delivering an EFFECTIVE Lecture

While lecture has its merits, today’s learners want engaging content that is timely, relevant and meaningful. Yet, most SMEs tend to suffer from the “curse of too much knowledge” and find it difficult to separate the need-to- know from the nice-to-know content.

This blog has been merged with “Batteries Not Included: Not All Trainers are Instructional Designer or Classroom Facilitators”.

You might also want to get up to speed with the current trend for SMEs – check out the blog post – Are all your SMEs Qualified? Comments welcomed.

(c) HPIS Consulting, Inc.

Is an Awareness Training Session enough for successful User Adoption?

Note: This blog is part of an ongoing series. Blog # 7 – Go-Live Strategy introduces the 5 Steps.

Go-Live: Step 3 – Develop the Rollout Timeline and Training Schedules.

A redesign of the quality system SOPs more than likely resulted in significant changes in routine tasks.  What changed, what was removed, what was added that is truly new and what stayed the same? Simply reading the newest version in an e-document platform will not suffice as effective training.  Nor will reading the change history page or reviewing a marked-up version of SOP for the highlighted changes.  When your previously delivered change management sessions include this level of detail, then the content of your training session can focus more on the new process.  If successful user adoption is tied to the effectiveness check of your CAPAs, then the project team needs to discuss what the training rollout will look like.  

Identifying Critical Users

From the stakeholders’ analysis for Affected Users, consider who is directly affected and indirectly affected by the change in responsibilities. Which users are most critical to ensure success with adhering to the new steps and forms? I refer to them as the Primary Users who are directly affected. They are usually more functional in their responsibility rather by department titles or business units.  Another way to determine this is to review the responsibilities section of the new SOPs.  Who in your organization are these people? In this review, are there supporting and ancillary responsibilities with the steps and forms?  I call this group the Secondary Users that are indirectly affected.  Both sets of users need to be fully trained in their tasks and responsibilities in order to ensure that the new system will function per the SOPs. 

The Training Rollout

Training Roll Outs need to meet three different levels of Users needs.

The overarching question to address is whether or not everyone has to attend the training.  One awareness training session for both groups is extremely efficient but not nearly as effective if the training sessions were tailored based on the level of user need.  See figure at the right.  Within the indirectly affected group are the senior leadership team members. 

An executive briefing is more likely to be attended by these folks when it provides a summary of what they need to know only.  What does the general population need to know about these changes?   Keep this short and to the point. It’s the Primary Group of Users who need to not only be made aware of the changes but to also know how to execute the new forms.  Yes, this session is a bit longer in duration than Awareness Training and it should be.  These Users have more responsibilities for correct execution. 

Simply reading the newest version in an e-document platform will not suffice as effective training. Nor will reading the change history page or reviewing a marked-up version of SOP for the highlighted changes.

Vivian Bringslimark, HPIS Consulting, Inc.

Who is the Trainer?

Depending on the level of involvement of the Design Team members, the following minimal questions need to be addressed:

  • Should the Project Manager deliver any sessions?
  • Will we use Train the Trainer approach?
    • Where each design team member is assigned to deliver Awareness Training for their area of remit.
    • Do these members have platform skills to lead this session?
    • Will they be provided with a slide deck already prepared for them?
  • Solely the responsibility of the QA Training Department
    • Provided s/he was a member of the Design Team
  • What kind of Training schedule will we need?
    • Will we provide three different tiers to meet the needs of our Affected Users?

Did the Training Roll Out Meet the Learning Needs of Primary Users?

While the revised SOPs were in the change control queue, the design team for this client met to discuss the difference between Awareness Training and Primary Users Training. Briefly, the differences were:

Difference between Awareness Training and Primary Users Training Content

Awareness Training is more knowledge-based.  It tends to be information sharing and very passive until the Q & A session.  A knowledge check at the end is no assurance that there will not be any deviations.  Primary Users Training is intended to focus on the behavioral changes that will be needed for adoption back in the department. The session can be a workshop with real examples that are generated from the users as part of their concerns and questions. 

The design team concluded that the differences were significant enough to warrant two different classes based on the type of user.  The risk of deviations was too great and would send a negative message to the site leadership team about the new process design.  Early adopters were not at risk because they were already trained via their participation in the design team. 

The task of developing the Awareness Training and Primary Users materials was assigned to the instructional designer on the team.   Attending the Awareness Training would not be a substitute for participating in the Primary Users class.  However, attending the Primary Users class would automatically credit the Awareness Training requirement for users if they attended.

Given that condition, the Primary Users materials also included similar content from the Awareness Training and then expanded the level of detail to include the sequence of steps for executing associated new and revised forms.  The Primary Users class was designed to provide more in-depth discussion of the changes and to provide adequate time to become familiar enough with their responsibilities to minimize disruption on the day the procedures and forms go into effect. 

The system owner then scheduled all users to attend the Awareness Training.  He concluded that there would be too much confusion between which class to attend. Since Awareness Training was being delivered first due to a very short Go-Live window, it would be better that they received the same training or so he thought.  In addition, the system owner felt that all employees were actually Primary Users and would not attend the training session if it went past 60 minutes.  As a result, Primary Users were never identified, and no learner matrix was generated.   No one asked for more training until weeks after the SOPs and forms went into effect. 

A knowledge check at the end is no assurance that there will not be any deviations.

Vivian Bringslimark, HPIS Consulting, Inc.

But rather than schedule the Primary User class, Users who had questions or concerns stopped by the department for 1-1 help instead.  For weeks, the staff was interrupted from their daily tasks and was expected to conduct impromptu help sessions.  The intent of the Primary Users class was to provide a hands-on training workshop for their impacted documents and not have to stop and go find someone for help.  The slide deck for Primary Users was eventually uploaded to a shared drive.  When the department got tired of being interrupted, the system owner put out a general email with the link and redirected late adopters to the website link.  The slide deck was not designed to be a substitute manual. 

Had the design team followed through with identifying lead champions, the Primary Users training workshop would have been delivered to a small group of users who then could have fielded questions from their colleagues.  The original design team members did not agree to be change champions nor trainers for their departments.  They complained about their workload already being heavy and had no time to address implementation questions.  That was for the training department to deliver, they concluded.  And then reported back to management that their direct reports could not attend a second session on the revised procedures.

Stay tuned.  Next blog includes Steps 4 and 5 of the Go Live Strategy and wraps up this series.  Become a subscriber so you don’t miss any more blogs. 

Who is Vivian Bringslimark?

© HPIS Consulting, Inc.

Change Management and It’s Little Cousin Training

Training can be considered a change in what the learner knows now | can do now and what s/he knows and does afterward.  Training can close these gaps when knowledge and skill are lacking.  Thus, a trainer has a dual role: trainer and change agent. While on the surface, the actions a trainer takes looks like change management, it is a sub-part of the larger change management plan.  The scope is narrower.  It is focused only on the training content.  But the expectations for successful transfer back to the job and improved organizational results remain the same. One allocated and approved session designed and delivered by the training department is supposed to shift years of a mindset, alter behaviors and change the company’s culture.  Amazing!

This expectancy or shall I say management assumption is very real and prevalent in fast-moving start-up companies whose leaders claim to have a limited budget for “training” and no time to sit in frivolous meetings planning “the people side” of change.  I’m supposed to be grateful that I get an hour session.  What happens next also amazes me.  Training on the proposed system changes is reduced to an hour-long awareness session on the SOPs that have already gone into effect and folks are told – “Go with the flow. Change is part of our everyday life around here.  Get with it or get out”.  And here’s the rub for me, errors rise, deviations spike, users are annoyed, pissed off or disenfranchised and management blames the trainer for a failed change effort. Wow! Is this the management support that was promised to folks at the town hall meetings? 

The bigger the redesigned quality system changes are, the more you need to consider beyond just training awareness on the SOPs.  Assess the size of the change gap and the impact the new design will have on the culture; “the way we normally handle changes around here”.  Training rollout sessions are already time-crunched.  There is not enough time “allowable” to manage all the other non-training change issues like feelings, job security concerns, why the need for change retorts, lack of supervisory support post-training, etc. that actually get in the way of a successful learning transfer. And later create hurdles for improvement results.

Let’s look at the HPISC 5 Step Change Management Plan and apply it to quality system redesign projects.  (See the sidebar below.)

5 necessary parts of a Change Management Plan

QS Change Management Plan Considerations

1. Why is the change needed?

  • This may be really obvious when the site receives a Warning Letter.  But an explanation of how this became a driving force for the needed changes will do wonders for your employees to feel the urgency about the change.

2. What is really changing?

  • Will the changes be incremental or a huge transformational change such as “changing the quality culture”?
  • How are these SOP changes part of the GMP culture?

3. What are the pros and cons of the change?

  • Who benefits and who loses?
  • Are customers hurt or helped?
  • What are the stakeholder’s benefits from the change?
  • What about the benefits for the primary users?

4. What does success look like?

  • What will the outcomes of the change look like?
  • How will you and others know if the change has been successful?
  • What benchmarks will help you track progress?
  • When is the day or timepoint we get to declare success?

5. What other initiatives are we competing with

  • And how will adding the new change requirements impact already heavy workloads?

Will Awareness Training be the only vehicle for announcing the new changes?

Are the answers to the change management questions sitting with the trainer/training department or with the site leadership team?  Perhaps the answers can be found within the steering committee members?  Is the trainer supposed to address all of these questions in a 60-minute awareness session that also includes the SOP changes?  If successful user adoption is paramount to your strategic action plan, warning letter remediation plan, or CRL commitment response, you need to ensure that change management messages regarding these changes are included in the overall communication plan.  Don’t just rely on the design team members to deliver these messages casually at huddle updates.  That is not a communication plan. 

“The bigger the redesigned quality system changes are, the more you need to consider beyond just training awareness on the SOPs”.

Vivian Bringslimark, HPIS Consulting, Inc.

The design team with the aid of the project manager needs to schedule special change management sessions where the Affected Users are briefed on the status of the project and the answers to the questions listed above addressed.  Some leaders do not want to “waste time” on these sessions.  They are concerned that it will become a gripe session.  Instead, they think it’s better to just present the users with the revised procedures.  There’s less time to fret and grumble over it.

There is a false belief that once the Affected Users see the changes in a QA-locked down version, they will follow them “because it’s now in the approved SOP”.  Forced acceptance is not a change management strategy despite rampant practice in our industry. If awareness training will be the first time affected users are learning about significant system changes and the “Go-Live” date, be prepared to receive A LOT OF FEEDBACK FROM UPPER MANAGEMENT regarding how awful the awareness training sessions went. 

“Change is disturbing when it is done to us, exhilarating when it is done by us”.

Rosabeth Kanter, 1984, p. 64.

Who is Vivian Bringslimark?

This is 6th in the Redesigning Quality Systems series.

Camp, RR, Blanchard, PN. & Huszczo. Toward a More Organizationally Effective Training Strategy & Practice. New Jersey: Prentice-Hall, 1986.

Kanter, RM. The Change Masters. New York: Simon and Schuster, 1984, 64.

© HPIS Consulting, Inc.  All rights reserved.

Investigations 101: Welcome Newbies

So the event description is clarified and updated. The assigned investigator is up to speed on the details of “the story”.  What happens next? What is supposed to happen?  In most organizations, there is rush to find the root cause and get on with the investigation.  A novice investigator will be anxious to conduct the root cause analysis (RCA).  S/he can easily make early root cause mistakes like grabbing the first contributing factor as the root cause without being disciplined to explore all possible causes first.

Thus it makes sense to get the Investigators trained in root cause analysis. Unfortunately for many, this is the ONLY training they receive and it is not nearly enough. RCA is a subset of the investigation process and the training agenda is heavy on the tools, which is perfectly appropriate.  But when do they receive training on the rest of the investigation stages like determining CAPA significance and writing the report?  Given the amount of FD-483 observations and warning letter citations for inadequate investigations that continue to be captured, I’d say we need more training beyond RCA tools.  As a result, we are starting to see FDA “recommendations” for trained and QUALIFIED Investigators.  This means not only in how to conduct a root cause analysis, but also the Deviation and CAPA Process. 

This goes beyond e-sign the Read and Understand procedures in your LMS

E-Doc systems are a great repository for storing controlled documents.   Searching for SOPs has become very efficient.  In terms of documenting “I’ve read the procedure”, very proficient and there’s no lost paperwork anymore!  But learning isn’t complete if we’ve merely read through the steps.  We also need to remember it.  At best, we remember that we read it and we know where to find it when we need to look something up.  Does that translate to understood?  Maybe for some. 

To help us remember the actual steps, we need to do something with the knowledge gained.  This is where the responsibilities section of the procedure tells us who is to do what and when.  But the LMS doesn’t include structured and guided practice as part of the assigned curricula.  Unless your equipment and complex procedures are also flagged for Structured OJT and possible Qualification Events as in most Operations groups, practice happens incidentally as part of on the job experience.  Feedback is typically provided when there’s a discrepancy or a deviation.  This is reactionary learning and not deliberate practice. 

If we want Deviation Investigators to understand and remember their tasks (procedures) so they can conduct investigations and write reports that get approved quickly, then we need to design learning experiences that build those skills and ensures accurate execution of assigned roles and responsibilities for Deviations and CAPAs. They need an interactive facilitated learner centered qualification program.

More than just studying a set of procedures and filling out related forms

It’s about putting the learners; the assigned SMEs as Investigators and QA Reviewers, at the center of the whole learning experience.  It’s about empowering them to take charge of their own learning by enabling them to experience real work deviations / CAPA investigations and to deliberately practice new skills in a safe environment with the assistance of adult learning facilitator(s) and coaches.  Thereby bridging the “R & U Only Knowledge Gap”.

The look and feel of the program follows a Learn By Doing approach with customized learning content, using interactive techniques and offering more hands-on opportunities for them to engage with real work application that ensures learners are immediately using the knowledge and tools in class and for their homework assignments thus increasing the connections for knowledge transfer.

This requires a shift from the traditional mindset of a classroom course where the emphasis is on the expertise of the instructor and the content. The learners and their learning experience becomes the priority.  The instructor’s task isn’t to deliver the content, it’s to help their learners acquire knowledge and skill.

Shifting the priority to a more engaging Learning Experience

Qualifying SMEs as Deviation Investigators Program

This unique curriculum uses a variety of teaching methods fostering more balanced and meaningful instruction over the duration of the program.  It is not a single course or 2-day training event.  It is delivered in modules, with weekly “homework” assignments consisting of active deviations and open investigations.

“Spaced learning works, in part, because the brain needs resting time to process information, create pathways to related information, and finally place the new information into long-term memory – the main objective of learning.” (Singleton, Feb 2018, p.71). 

Each module revisits the Investigation Stages and builds on the prior lessons by reviewing and debriefing the homework.  Then, expanding on that content and including new lessons with increasing intensity of the activities and assignments.

By design, the program provides time and space to interact with the content as opposed to delivering content dumps and overwhelming the newbies; short-term memory gets maxed out and learning shuts down. The collaborative participation and contributions from the Investigators and Program Facilitator(s) result in better overall engagement. Everyone is focused on accomplishing the goal of the program; not just checking the box for root cause analysis tools.

The goal of the program is to prepare subject matter experts to conduct, write and defend investigations for deviations and CAPAs.  The program also includes QA reviewers who will review, provide consistent critique and approve deviations, investigations, CAPAs. Attending together establishes relationships with peers and mutual agreement of the content.  The learning objectives describe what the learners need from the Deviation and CAPA quality system procedures while the exercises and assignments verify comprehension and appropriate application.

“Learning happens when learners fire their neurons, not when the trainer gives a presentation or shows a set of Power-Point slides.” (Halls, Feb 2019, p.71).

Qualified, really? Isn’t the training enough?

Achieving “Qualified status is the ultimate measure of the training program effectiveness.  For newly assigned Investigators, it means the company is providing support with a program that builds their skills and confidence and possible optional career paths.   Being QUALIFIED means that Investigators have undergone the rigor of an intensely focused investigations curriculum that aligns with the task and site challenges.  That after completing additional qualification activities, Investigators have experienced a range of investigations and are now deemed competent to conduct proper investigations.

For the organization, this means two things.  Yes, someone gets to check the FDA commitment box.  And it also means strategically solving the issues.  Better investigations lead to CAPAs that don’t fail their effectiveness checks.  Now that’s significant performance improvement worthy of qualifying Investigators!  -VB

References:

  • Campos,J. The Learner Centered Classroom. TD@Work, August, 2014, Issue 1408.
  • Chopra,P. “give them what they WANT”, TD, May, 2016, p.36 – 40.
  • Halls,J. “Move Beyond Words to Experience”, TD, February, 2019, p. 69 – 72 DL.
  • Parker, A. “Built to Last: Interview with Mary Slaughter”, TD, May, 2016, p. 57.
  • Singleton, K. “Incorporating a Spiral Curriculum Into L&D”, TD, February, 2018, 70 – 71.

HPISC Coaching Brief available here.

What’s Your Training Effectiveness Strategy? It needs to be more than a survey or knowledge checks

When every training event is delivered using the same method, it’s easy to standardize the evaluation approach and the tool. Just answer these three questions:

  • What did they learn?
  • Did it transfer back to job?
  • Was the training effective?

In this day and age of personalized learning and engaging experiences, one-size training for all may be efficient for an organizational roll out but not the most effective for organizational impact or even change in behavior. The standard knowledge check can indicate how much they remembered. It might be able to predict what will be used back on the job. But be able to evaluate how effective the training was? That’s asking a lot from a 10 question multiple choice/ true false “quiz”. Given the level of complexity of the task or the significance of improvement for the organization such as addressing a consent decree or closing a warning letter, it would seem that allocating budget for proper training evaluation techniques would not be challenged.

Do you have a procedure for that?

Perhaps the sticking point is explaining to regulators how decisions are made using what criteria. Naturally documentation is expected and this also requires defining the process in a written procedure. It can be done. It means being in tune with training curricula, awareness of the types of training content being delivered and recognizing the implication of the evaluation results. And of course, following the execution plan as described in the SOP.   Three central components frame a Training Effectiveness Strategy: Focus, Timing and Tools.

TRAINING EFFECTIVENESS STRATEGY: Focus on Purpose

Our tendency is to look at the scope (the what) first. I ask that you pause long enough to consider your audience, identify your stakeholders; determine who wants to know what. This analysis shapes the span and level of your evaluation policy. For example, C-Suite stakeholders ask very different questions about training effectiveness than participants.

The all purpose standard evaluation tool weakens the results and disappoints most stakeholders. While it can provide interesting statistics, the real question is what will “they” do with the results? What are stakeholders prepared to do except cut training budget or stop sending employees to training? Identify what will be useful to whom by creating a stakeholder matrix.

Will your scope also include the training program (aka Training Quality System) especially if it is not included in the Internal Audit Quality System? Is the quality system designed efficiently to process feedback and make the necessary changes that result from the evaluation results? Assessing how efficiently the function performs is another opportunity to improve the workflow by reducing redundancies thus increasing form completion speed and humanizing the overall user experience. What is not in scope? Is it clearly articulated?

TRAINING EFFECTIVENESS STRATEGY: Timing is of course, everything

Your strategy needs to include when to administer your evaluation studies. With course feedback surveys, we are used to immediately after otherwise, the return rate drops significantly. For knowledge checks we also “test” at the end of the session. Logistically it’s easier to administer because participants are still in the event and we also increase the likelihood of higher “retention” scores.

But when does it make more sense to conduct the evaluation? Again, it depends on what the purpose is.

  • Will you be comparing before and after results? Then baseline data needs to be collected before the event begins. I.e. current set of Key Performing Indicators, Performance Metrics
  • How much time do the learners need to become proficient enough so that the evaluation is accurate? I.e. immediately after, 3 months or realistically 6 months after?
  • When are metrics calculated and reported? Quarterly?
  • When will they be expected to perform back on the job?

Measuring Training Transfer: 3, 6 and maybe 9 months later

We can observe whether a behavior occurs and record the number of people who are demonstrating the new set of expected behaviors on the job. We can evaluate the quality of a work product (such as a completed form or executed batch record) by recording the number of people whose work product satisfies the appropriate standard or target criteria. We can record the frequency with which target audience promotes the preferred behaviors in dialogue with peers and supervisors and in their observed actions.

It is possible to do this; however, the time, people and budget to design the tools and capture the incidents are at the core of management support for a more vigorous training effectiveness strategy. How important is it to the organization to determine if your training efforts are effectively transferring back to the job? How critical is it to mitigate the barriers that get in the way when the evaluation results show that performance improved only marginally? It is cheaper to criticize the training event(s) rather than address the real root cause(s). See Training Does Not Stand Alone (Transfer Failure Section).

TRAINING EFFECTIVENESS STRATEGY: Right tool for the right evaluation type

How will success be defined for each “training” event or category of training content? Are you using tools/techniques that meet your stakeholders’ expectations for training effectiveness? If performance improvement is the business goal, how are you going to measure it? What are the performance goals that “training” is supposed to support? Seek confirmation on what will be accepted as proof of learning, evidence of transfer to the workplace, and identification of leading indicators of organizational improvement. These become the criteria by which the evaluation has value for your stakeholders. Ideally, the choice of tool should be decided after the performance analysis is discussed and before content development begins.

Performance Analysis first; then possibly a training needs analysis

Starting with a performance analysis recognizes that performance occurs within organizational systems. The analysis provides a 3-tiered picture of what’s encouraging/blocking performance for the worker, work tasks, and/or the workplace and what must be in place for these same three levels in order to achieve sustained improvement. The “solutions” are tailored to the situation based on the collected data and not on an assumption that training is needed. Otherwise, you have a fragment of the solution with high expectations for solving “the problem” and relying on the evaluation tool to provide effective “training” results. Only when the cause analysis reveals a true lack of knowledge, will training be effective.

Why aren’t more Performance Analyses being conducted?
For starters, most managers want the quick fix of training because it’s a highly visible activity that everyone is familiar and comfortable with. The second possibility lies in the inherent nature of performance improvement work. Very often the recommended solution resides outside of the initiating department and requires the cooperation of others.   Would a request to fix someone else’s system go over well where you work? A third and most probable reason is that it takes time, resources, and a performance consulting skill set to identify the behaviors, decisions and “outputs” that are expected as a result of the solution. How important will it be for you to determine training effectiveness for strategic corrective actions?

You need an execution plan

Given the variety of training events and level of strategic importance occurring within your organization, one standard evaluation tool may no longer be suitable. Does every training event need to be evaluated at the same level of rigor? Generally speaking, the more strategic the focus is, the more tedious and timely the data collection will be. Again, review your purpose and scope for the evaluation. Refer to your stakeholder matrix and determine what evaluation tool(s) is better suited to meet their expectations.

For example, completing an after-training survey for every event is laudable; however, executive leadership values this data the least. According to Jack and Patricia Phillips (2010), they want to see business impact the most. Tools like balanced scorecards can be customized to capture and report on key performing indicators and meaningful metrics. Develop your plan wisely, generate a representative sample size initially and seek stakeholder agreement to conduct the evaluation study.

Life after the evaluation: What are you doing with the data collected?

Did performance improve? How will the evaluation results change future behavior and/or influence design decisions? Or perhaps the results will be used for budget justification, support for additional programs or even a corporate case study? Evaluation comes at the end but in reality, it is continuous throughout. Training effectiveness means evaluating the effectiveness of your training: your process, your content and your training quality system. It’s a continuous and cyclical process that doesn’t end when the training is over. – VB

 

Jack J. Phillips and Patricia P. Phillips, “How Executives View Learning Metrics”, CLO, December 2010.

Recommend Reading:

Jean-Simon Leclerc and Odette Mercier, “How to Make Training Evaluation a Useful Tool for Improving L &D”, Training Industry Quarterly, May-June, 2017.