Some folks might say that I’m mincing words, but I beg to differ. The expectations for training delivery are that participants learn the content, aka learning objectives and then use or apply it back on the job thus improving departmental / organizational performance. So, do you provide the training and then keep your fingers crossed that they can deliver on their performance objectives or do you assure that employees can perform after the event is long over?
“Employee Qualification” is a successful program for the Life Sciences companies and they have been deploying variations of it for several years now. The essence of it is an observed assessment of performance by a qualified OJT Trainer. Simple in theory, yes nonetheless, implementation is a bit more structured. See Moving Beyond R & U SOP Training.
Employee Qualification is the ultimate Level 3 Training Evaluation
Referring to the well-known Kirkpatrick Model of Evaluation, Level 3 is behavior change. The focus of Employee Qualification is about the employee’s ability to apply knowledge and skill learned during OJT back on the job / in the workplace setting. Actual performance is the ultimate assessment of learning transfer. If an employee is performing the job task correctly during a formal performance demonstration, this meets the expectation for successful training.
Yet, according to the 2009 ASTD research study “The Value of Evaluation”, only 54.6% of respondents indicated that their organization conducts Level 3 evaluations. The top technique used is follow-up surveys with participants (31%), while observation of the job was fourth (23.9%). If on the job assessment is the “ultimate” measure of transfer, then why isn’t it being used more frequently?
Post training assessments are time and labor intensive. About a quarter (25.2%) of their learning programs are evaluated for behavior according to the respondents. But for organizations who have to meet compliance requirements (46.9% of survey respondents), documenting training effectiveness has fast become one of the top expectations of external regulators. No longer satisfied with just providing LMS history records, many auditors are now asking to see the training effectiveness strategies for required compliance training.
Validating Your Training Effectiveness
Being “SOP” qualified is the demonstrated ability of an employee to accurately perform a task or Standard Operating Procedure independent of his OJT coach with consistency to meet acceptable quality standards. Also see “From training logs to OJT Checklists and beyond”. An active Employee Qualification program also verifies that the training content in this case the SOP, accurately describes how to execute the steps for the task at hand. If either is not done well, the qualification is stopped and a cause analysis is conducted to examine contributing factors. Success starts with an effective training system.
Oh, but now we have Curricula!
Having training curricula and matrices is a huge step in identifying what is required for employees to learn. LMSes are also helpful for recording history, tracking overdue requirements and generating reconciliation reports. More sophisticated databases can provide functionality for quizzes. Quiz/ test/ knowledge check can measure knowledge retention and possibly comprehension if it includes challenge questions about real workplace situations. Making them a popular Level 2 evaluation tool. However, be mindful though of the danger of “teaching to the test” or using the search function within the e-doc system to find the answer in the SOP. Most of the knowledge retained is immediately flushed within hours of “passing the test” or satisfying the LMS generated quiz. So, having a quiz is not a guarantee that the knowledge transforms itself into a skill set back on the job.
The true measure of effectiveness
The use of “100% completed” reports is a metric for completeness only; a commonly used data point from the LMS. It does not address transfer of learning into performance back on the job. Neither does a 5-question multiple-choice quiz designed to measure the achievement of learning objectives. The true measure of effective OJT training is an observed demonstration of the performance objective(s). Isn’t that what effective training is supposed to mean – a change in behavior? – VB
What happens when the performance demonstration becomes more of a "this is how I do it discussion" instead of an actual demonstration?
*The Value of Evaluation: Making Training Evaluations More Effective. An ASTD Research Study, 2009, ASTD.