I’m in love with my own content!

Many QA /HR Training Managers have the responsibility for providing a train-the-trainer course for their designated trainers.  While some companies send their folks to public workshop offerings, many chose to keep the program in-house.   And then an interesting phenomenon occurs.  The course content grows with an exciting and overwhelming list of learning objectives.

The supervisors of the SMEs struggle with the loss of productivity for the 2 – 3 day duration and quickly develop a “one and done” mindset.   Given the opening to “train” newly identified SMEs as Trainers, the instructional designer gets one opportunity to teach them how to be trainers.  So s/he tends to add “a lot of really cool stuff” to the course in the genuine spirit of sharing, all justifiable in the eyes of the designer.  However, there is no hope in breaking this adversarial cycle if the Training Manager doesn’t know how to cut content.

I used to deliver a two-day (16 hour) workshop for OJT Trainers. I included all my favorite topics.  Yes, the workshop was long.  Yes, I loved teaching these concepts.  I honestly believed that knowing these “extra” learning theory concepts would make my OJT Trainers better trainers.  Yes, I was in love with own my content.  And then one day, that all changed.

 

Do they really need to know Maslow’s Hierarchy of Needs?

During a rapid design session I was leading, I got questioned on the need to know Maslow’s Hierarchy of Needs.  As I began to deliver my auto-explanation, I stopped mid-sentence.  I had an epiphany.  My challenger was right.  Before I continued with my response, I feverishly racked my brain thinking about the training Standard Operating Procedures (SOPs) we revised, the forms we created, and reminded myself of the overall goal of the OJT Program.  I was searching for that one moment during an OJT session when Maslow was really needed.  When would an OJT Qualified Trainer use this information back on the job, if ever I asked myself?

It belongs in the Intermediate Qualified Trainers Workshop, I said out loud.  In that moment, that one question exercise was like a laser beam cutting out all nice-to-know content.  I eventually removed up to 50% of the content from the workshop.

 

Oh, but what content do we keep?

Begin with the overall goal of the training program: a defendable and reproducible methodology for OJT.  The process is captured in the redesigned SOPs and does not need to be repeated in the workshop.  See Have you flipped your OJT TTT Classroom yet?

Seek agreement with key stakeholders on what the OJT QTs are expected to do after the workshop is completed.  If these responsibilities are not strategic or high priority, then the course will not add any business value.  Participation remains simply a means to check the compliance box.  Capture these expectations as performance objectives.

How to align purpose of a course to business goals

Once there is agreement with the stated performance objectives, align the content to match these. Yes, there is still ample room in the course for learning theory, but it is tailored for the need to know only topics.

In essence, the learning objectives become evident.  When challenged to add certain topics, the instructional designer now refers to the performance objectives and ranks the consequences of not including the content in the workshop against the objectives and business goal for the overall program.

 

What is the value of the written assessment?

With the growing demand for training effectiveness, the addition of a written test was supposed to illustrate the commitment for compliance expectations around effectiveness and evaluation.  To meet this client need, I put on my former teacher hat and created a 10 question open book written assessment.  This proved to need additional time to execute and hence, more content was cut to accommodate the classroom duration.

My second epiphany occurred during the same rapid design project, albeit a few weeks later.   What is the purpose of the classroom written assessment when back on the job the OJT QTs are expected to deliver (perform) OJT; not just know it from memory? The true measure of effectiveness for the workshop is whether they can deliver OJT according to the methodology, not whether they retained 100% of the course content!   So I removed the knowledge test and created a qualification activity for the OJT QTs to demonstrate their retained knowledge in a simulated demonstration using their newly redesigned OJT checklist.  Now the OJT QT Workshop is value added and management keeps asking for another round of the workshop to be scheduled.  -VB

Are you ready to update your OJT TTT Course?

 

 

 

I’ve fired my [TTT] Vendor!  

Sustaining Qualified Trainer’s Momentum Post Launch

The Silver Bullet for Performance Problems Doesn’t Exist

Oh but if it did, life for a supervisor would be easier, right? Let’s face it, “people” problems are a big deal for management. Working with humans does present its challenges, such as miscommunications between staff, data entry errors, or rushing verification checks. Sometimes, the task at hand is so repetitive that the result is assumed to be okay and gets “a pass”.  Add constant interruptions to the list and it becomes even harder not to get distracted and lose focus or attention to the detail.

Actual behavior vs. performing as expected

In their book, Performance Consulting: Moving Beyond Training, Dana Gaines Robinson and James C. Robinson describe performance as what the performer should be able to do. A performance problem occurs when the actual behavior does not meet expectation (as in should have been able to do).   Why don’t employees perform as expected? Root cause analysis helps problem solvers and investigators uncover a myriad of possible reasons.   For Life Sciences companies, correcting mistakes and preventing them from occurring again is at the heart of CAPA systems (Corrective Actions Preventive Actions).

A closer look at performance gaps

Dana and James Robinson conducted research regarding performer actions and sorted their results into three categories of obstacles:

  • Conditions of performers
  • Conditions of the immediate managers
  • Conditions of the organization

A checklist for common Performance Causes  – scroll down for the Tool.

But, weren’t they trained and qualified?

Hopefully, employees are trained using an approved OJT (On the Job Training) Methodology in which they are shown how to execute the task and then given opportunities to practice multiple times to become proficient. During these sessions, they are coached by Qualified Trainers and given feedback on what’s right (as expected) and given specific instructions to correct what’s not right with suggestions for tweaking their performance so that their final performance demonstration is on par with their peer group. At the conclusion of the qualification event, employees must accept that they now own their deviations (mistakes) from this point forward. So what gets in the way of performing “as they should” or in compliance speak – according to the procedure?

Is it a lack of knowledge, skill or is it something else?

The Robinson’s explain that performance is more than the training event. It’s combination of the overall learning experience and the workplace environment that yields performance results. Breaking that down into a formula per se, they suggest the following: learning experience x workplace environment = performance results.

The root cause investigation will include a review of training and the qualification event as well as a discussion with the performer.

  • Is it a lack of frequency; not a task often performed?
  • Is it a lack of feedback or delayed feedback in which the deviation occurred without their awareness?
  • Is it task interference?

The work environment includes organizational systems and business unit processes that together enable the performer to produce the outcomes as “expected”.   These workplace factors don’t always work in perfect harmony resulting in obstacles that get in the way of “expected” performance:

  • Lack of authority – unclear roles, confusing responsibilities?
  • Lack of time – schedule conflicts; multi-tasking faux pas?
  • Lack of tools – reduced budgets?
  • Lack of poorly stored equipment/tools – lost time searching?

Isn’t it just human nature?

Once the root cause investigation takes on a human element attention, it’s easy to focus on the performer and stop there.   If it’s the first time for the performer or first instance related to the task, it’s tempting to label the event as an isolated incident. But when it comes back around again, it becomes apparent there was a “failure to conduct an in-depth investigation” to correct and prevent. Not surprisingly, a push back of “Operator Error as Root Cause” has forced organizations to look deeper into the root causes involving Humans.

Who’s human nature?

Recall that one of the categories of the researched obstacles was “conditions of the immediate managers”. This makes managers uncomfortable. With so much on their plates, managing a people performance problem is not what they want to see. A silver bullet like a re-training event is a nice activity that gets a big red check mark on their to-do list. However, Robert Mager and Peter Pipe, in their book, Analyzing Performance Problems, provide insights to managing direct reports that may lead to unintended consequences. A brief list can be found here – scroll to Tool: Performance Causes.  (It’s not always the performer’s fault.)

It takes all three to correct a performance problem

soln-people-performance-problemThe third category of researched obstacles clustered around “conditions of the organization”.  I’ve already discussed task interference above. To suggest that organizations are setting up their employees to fail is pushing it just a bit too far.   So I won’t go there, but it is painful for some leaders to come to terms with the implication. In order to prevent issues from reoccurring, an examination of the incidents and quite possibly a restructuring of systems have to occur, because automatic re-training is not the only solution to a “people performance problem”. –VB

Robinson DG, Robinson JC. Performance Consulting: Moving beyond training. San Francisco: Berrett-Koehler; 1995.

Mager R, Pipe P. Analyzing performance problems. Belmont: Lake Publishing; 1984.

Performance objectives are not the same thing as learning objectives

Some folks might say that I’m mincing words, but I beg to differ. The expectations for training delivery are that participants learn the content, aka learning objectives and then use or apply it back on the job thus improving departmental / organizational performance. So, do you provide the training and then keep your fingers crossed that they can deliver on their performance objectives or do you assure that employees can perform after the event is long over?

“Employee Qualification” is a successful program for the Life Sciences companies and they have been deploying variations of it for several years now. The essence of it is an observed assessment of performance by a qualified OJT Trainer. Simple in theory, yes nonetheless, implementation is a bit more structured. See Moving Beyond R & U SOP Training.

Employee Qualification is the ultimate Level 3 Training Evaluation

Referring to the well-known Kirkpatrick Model of Evaluation, Level 3 is behavior change The focus of Employee Qualification is about the employee’s ability to apply knowledge and skill learned during OJT back on the job / in the workplace setting.   Actual performance is the ultimate assessment of learning transfer. If an employee is performing the job task correctly during a formal performance demonstration, this meets the expectation for successful training.

Yet, according to the 2009 ASTD research study “The Value of Evaluation”, only 54.6% of respondents indicated that their organization conducts Level 3 evaluations.   The top technique used is follow-up surveys with participants (31%), while observation of the job was fourth (23.9%).   If on the job assessment is the “ultimate” measure of transfer, then why isn’t it being used more frequently?

Post training assessments are time and labor intensive.   About a quarter (25.2%) of their learning programs are evaluated for behavior according to the respondents.   But for organizations who have to meet compliance requirements (46.9% of survey respondents), documenting training effectiveness has fast become one of the top expectations of external regulators.     No longer satisfied with just providing LMS history records, many auditors are now asking to see the training effectiveness strategies for required compliance training.

Validating Your Training Effectiveness

Being “SOP” qualified is the demonstrated ability of an employee to accurately perform a task or Standard Operating Procedure independent of his OJT coach with consistency to meet acceptable quality standards.   Also see “From training logs to OJT Checklists and beyond”. An active Employee Qualification program also verifies that the training content in this case the SOP, accurately describes how to execute the steps for the task at hand. If either is not done well, the qualification is stopped and a cause analysis is conducted to examine contributing factors. Success starts with an effective training system.

Oh, but now we have Curricula!

Having training curricula and matrices is a huge step in identifying what is required for employees to learn. LMSes are also helpful for recording history, tracking overdue requirements and generating reconciliation reports. More sophisticated databases can provide functionality for quizzes. Quiz/ test/ knowledge check can measure knowledge retention and possibly comprehension if it includes challenge questions about real workplace situations. Making them a popular Level 2 evaluation tool. However, be mindful though of the danger of “teaching to the test” or using the search function within the e-doc system to find the answer in the SOP. Most of the knowledge retained is immediately flushed within hours of “passing the test” or satisfying the LMS generated quiz. So, having a quiz is not a guarantee that the knowledge transforms itself into a skill set back on the job.

The true measure of effectiveness

The use of “100% completed” reports is a metric for completeness only; a commonly used data point from the LMS. It does not address transfer of learning into performance back on the job. Neither does a 5-question multiple-choice quiz designed to measure the achievement of learning objectives.   The true measure of effective OJT training is an observed demonstration of the performance objective(s). Isn’t that what effective training is supposed to mean – a change in behavior? – VB

What happens when the performance demonstration becomes more of a "this is how I do it discussion" instead of an actual demonstration?

*The Value of Evaluation: Making Training Evaluations More Effective. An ASTD Research Study, 2009, ASTD.