When Your GXP Learning Provider Becomes Your Strategic Partner

“So, they want me to ask if you would do the GMP refresher for us.”

“No.  I did the GMP Basics last year.  It’s only been 4 months and I am not going to repeat the content or water it down or “just tell them the GMPs”. 

“Well, I can’t do it by myself, not after you’ve introduced us to activities.  I don’t know how to create meaningful exercises.”

“Let me think about this some more.  What’s currently happening on site, other than last year’s 483? Do you have any updates we can use?”

“Oh, there was an audit done and you know they found stuff.  Folks are kind of hot about it, actually.”

“Okay, now we have something to work with.  I’ll do it on two conditions.  1.) That we use the audit observations and 2.) I need 90 minutes.”

“Nope, not going to happen.  No way for the 90 minutes. Oh, and they want to keep the audit observations ‘confidential’.  We’re not allowed to use them.”

“Then you’ll have to find another consultant to do it. After the training we did last year, I cannot go back to lecture only.  I built trust with your employees.  And you and I learned that we need to add time so that they can complete the quiz at the end and not feel rushed”. 

“I really want you to do it.  I will go back to the management team and tell them you said yes. But I need a proposal justifying the 90 minutes and describing what you will do with audit observations.”

The Quality of Previous Learning Provider Engagements

This organization is a leading manufacturer and distributor of regulated products.  They received Form FD-483 observations the year before.  Among the remedial actions, was a “Back to GMP Basics” training program for the entire site including indirect support and non-GMP staff.   

Not only were sessions content tailored for varying roles, but employees were also introduced to interactive exercises and collaborative activities that encouraged them to share responses with their seatmates.  For many of them, this was a totally different GMP learning experience.  Employees were no longer grumbling about the training and management was able to produce the required paperwork to the agency. 

As part of the remediation plan, the client also pledged to increase the frequency of GMP refreshers to twice a year.  Given the favorable outcome from the previous GMP Training program, it was suitable that the management team extended an offer to conduct a refresher.  What was a surprise, however, was that the management team wanted their old – “gloom and doom, thou shall follow these regulations” style of telling them the GMPs from the podium.  Why? Was this a form of punishment?  It sure felt like it.

Management Team Knew Their Priority

At the time of the above request, 4 months of the year had already transpired.  During which, a follow-up audit of their GMP systems was conducted by a separate consulting group.  The audit found numerous examples of recent noncompliance.  The site leadership team was also reminded that the 1st GMP refresher was coming due.  The head of Quality made the decision to use the GMP refresher as a corrective action for the audit.

Kudos to the Head of Quality for doing three things.  First, he recognized that the c in CGMPs was the audit findings and resulting noncompliance examples.  Two, he leveraged the resource of QA Training and demonstrated just how strategic the role should be to the organization by elevating the importance of this refresher.  He used a systems approach when he tagged the delivery of this refresher to the corrective actions for the independent audit.  This was quite novel at the time.  And third, he approved the request to provide his Training Manager some help. He connected all the dots between the independent solos!

Their Decision to Outsource

Their “GMP Trainer” was also the manager of the company-wide training database.  He was a department of one whose primary duty was data entry into a complicated and non-user-friendly training database.  Time to develop and deliver the GMP refresher was something this manager did not have.  The sooner it could be delivered, the sooner the training manager could get back to the mounting stacks of attendance records awaiting data entry.

 “Just tell them the GMPs” one more time is what the management team was originally asking for.  The learners were expected to sit and listen.  After all, they used to be scolded for poor compliance results. One employee referred to it as “shame and blame” sessions with their heads down to avoid eye contact with the presenter; especially if it was a leader from Quality Operations.

The management team wanted to bring back their learning provider and they wanted to dictate how the refresher would be conducted.  To revert to lectures only would break the trust the learners had built with the GMP Learning Provider and jeopardize any successful transfer of behavior change to the workplace.

Corrective Actions are supposed to remediate, right?

The independent audit revealed several examples of non-compliant actions taken.  When categorized, these actions clustered around three main areas.  If the “corrective actions training” aka the GMP refresher was going to remediate these findings, just telling them what the regs say without any discussion or collaborative processing of what these examples were showing, would have no effect on eliminating or reducing further GMP “mistakes”.  What did the management team expect to happen because of the training?  I imagine no more examples of GMP violations?  

Aligning Business Needs with Quality Objectives Yield a Strategic Focus

Shifting the behavior towards appropriate compliance begins with shifting mindsets around compliance.  Working with the three categorized areas, the management team agreed to the following three business outcomes:

  1. Continued commitment to comply with our GMP Work Habits
  2. Follow Good Documentation Practices for our controlled documents
  3. Perform our responsibilities for operating equipment in our work areas

While these outcomes were great goals, they were also very broad and hard for learners to put into daily practice; let alone change their behavior.  Effectiveness checks for training was another commitment to the agency as part of the GMP Training program improvements.

Furthermore, measuring the effectiveness of the GMP refresher training would require more than a knowledge check if behavior change back at the workplace was their end game.  What would the CAPA effectiveness checks look like for this corrective action as well? Recall, the two quality system activities were now interlinked.

In further discussions with the Head of Quality, the business outcomes were transcribed into quality themes and key messages.  These then drove the decisions for specific content pieces and activities that targeted what compliant behavior should look like regarding the 3 quality themes. 

To further quell any negative ramifications using audit findings, the chosen examples were approved by the Head of Quality and a courtesy copy of the refresher materials was provided to the management team in advance. 

Designing, developing, and delivering the refresher course topics with even more interactive opportunities all the while ensuring that the previous GMP Basics content was refreshed, not just repeated, became the basis for the activities.  It was analogous to “a Part 2” in which employees were allowed to “interact” with both the regulations and real-time scenarios (selective audit observations). 

When a Learning Partnership Works

One of the advantages of working with a previous learning provider is the ability to create an ongoing learning path built upon previously delivered content; not just provide a “canned” presentation on the requested topic.  There is a willingness to work with internal resources and integrate existing artifacts such as audit findings into the course design so that the organization achieves its business outcomes beyond just closing the CAPAs and getting a checkmark for delivering the first of two promised refreshers. 

Value and Impact: Training Effectiveness Results

  • Assembling into pairs and groups was achieved in less time than when first introduced in the GMP Basics series.  The Learning Provider now had credibility and the employee’s earned trust. The relevance of the audit finding examples kept employees engaged with many of them asking private questions at the end of the sessions.  This was unheard of previously, thus initiating the “transfer back to my job”.  They were thinking about how these examples and the GMP content applied to them.
  • A 5-question quiz was administered as part of their effectiveness training check.  This is the classic tool for knowledge comprehension and their newly revised SOP now included Knowledge Checks (KCs) for GMP Training.  400+ employees completed the quiz.  The mean score was 91% with several achieving 100% and a few failing scores.

Why Knowledge Checks Don’t Tell the Whole Story …

Using an item analysis can reveal a lot of information about the construction of the GMP KC questions.

As always, it is possible that some memorization and sharing the correct lettered responses happen despite that there were three versions of the KC.  It is also possible that some folks do not take the time to read questions thoroughly.  And some folks don’t recognize when “all of the above” choice applies.  They are in a rush to exit the class. 

Pay attention to incorrect responses that are clearly wrong as in they don’t make sense.  It may indicate a “cheating trend” if only one version of a quiz is provided.  In this case, three versions of the quiz were used.

When responses are troubling; a follow-up discussion with an SME or a “legacy” employee is needed to determine if an old, outdated, and non-GMP practice was acceptable at one time.  Knowing this gives some insight into why it was chosen but disheartening that the behavior is still prevalent.  If the behavior is still happening, it warrants further discussions with site leadership and the Head of Quality.  It may be isolated to a few individuals and can be managed via the HR performance management system.

Sometimes verbal responses to a question in class, don’t match the chosen response in the quiz.  The quiz results could be a matter of confusion with the wording of the question especially for Assessment A in which the question had a NOT worded in it, rendering the answer to be False.  It’s akin to a double negative and it trips up a lot of people who are not test-savvy.  So be careful with questions that can be confusing or tricky.  For this client, it was worded to capture a real scenario that happens in the industry. 

This client was urged to address failing scores; results below 80%.  Their Training SOP had recently been revised to include formal assessments for GMP Training and Critical Task SOPs as part of their effectiveness checks for training.

  • The priority for the refresher was agreed to by the site management team and communicated to everyone during the sessions.  The FDA remediation plan was still a focused site objective, and this refresher was part of those activities; not something in addition to like an after-thought or add-on.
    • Three outcomes were identified as the business needs and became the driver behind content and decisions for activities. The “Big Why” for this refresher was clear and compelling.  Tangible work-behaviors were user-generated for peers to model as examples of complying with the GMP Work Habits.
    • There was continuity with previous GMP content.  The existing GMP Learning Provider connected the dots with prior content and reinforced the relevancy and importance of the GMP Work Habits.  Rapport was re-established and on-going trust for an engaging and interactive session of 90 minutes was achieved.  Minimal repeat of previous content; refresh enough to complete an activity.
  • Learning Provider negotiated with Site Management Team on behalf of the Learners Needs: to include real workplace audit observations and an additional 30 minutes of classroom time that included the GMP quiz and debrief of the correct answers. Employees left each session with the correct responses to 5 questions; their final take away from the sessions.
  • The design followed a 3-step learning model: Learn, Experience, Apply. 
    • Learn: Short lectures or vignettes was used to learn/refresh on content. 
    • Experience: Variety of ways in which the Learners engaged with the audit observations; not just a slide for each observation but small group discussion and tangible activity to complete and report back on.
    • Apply: peer feedback, debriefing.

Are you thinking about outsourcing your next compliance?  Why not form a training partnership with an industry GXP Learning Provider?  -VB

Like this blog? Tell Vivian.

HPISC eGuidebooks are available.

When to Outsource Your GMP Refresher

Tips for Writing KCs

Tips for Writing Knowledge Checks

              

About Vivian Bringslimark, President of HPIS Consulting, Inc.

Looking for more resources for GXP Training? Visit the GXP Training Menu Page.

Need to build a refresher or overhaul your GMP Basics Program?

(c) HPIS Consulting, Inc.

What’s the difference between Trainers and Performance Consultants?

Aren’t they one and the same?

 Since the original release of this blog in 2014, it continues to be the # 1 blog viewed on the Theory vs. Practice blog spot.  Thank you to all the viewers and future viewers.  This tells me that the question is still relevant today in 2021. 

Just another fancy title?

Some leaders think there is no difference; that we’ve just added one more title into the crowded lexicon of L&D jargon.  And others believe that performance consultants (PCs) want to expand their scope, budget and timelines.  And some simply hear excuses about why the requested training “course” is not immediately being embraced. 

Dana Gaines Robinson in her seminal book, Performance Consulting, provides 6 items to use when comparing a Trainer/Training event and a PC/performance-based solution.  Allow me to expand upon the 6 elements to illustrate the difference between the two and the depth of impact one has over the other.  

FOCUS

Training addresses the learning needs of employees.  Various definitions include closing the knowledge and skill gap of what they know now and what they know afterward.  It’s built on the assumption that the cause of the gap is a lack of knowledge and skill.  Performance Consulting addresses the business goals and performance needs of the affected employees.  Instructor-led training is just one of the possible solutions that can be used; not the only one. See HPI 6 “boxes” of performance solutions. 

OUTPUTS

A training solution delivers a structured learning event.  Whether it is a classroom or virtual or self-led, the event itself is the end goal. The assumption is that learning occurred and knowledge gained so, therefore, a change in behavior or in the learners’ performance should occur as well.  

Performance Consulting or Human Performance Improvement (HPI) projects are implemented to improve performance.  The end goal is not about the solution such as the specific HPI Project, but rather a positive change in performance that leads to the achievement of the business goal.  The endpoint is “further down the road”.  So it takes longer to produce the results. This frustrates site leadership.  They would rather check off the box that a learning event was delivered because it’s more tangible and occurs faster than quarterly metrics. 

ACCOUNTABILITY

With training, the Trainer is held accountable for the event.  In a lot of organizations, there is an implied but not spoken accountability for the results back on the job despite that Trainers lack the authority to direct their learners’ actions back in the workspace.  Without the proper systems and support mechanisms in place, many Trainers get “blamed” for training transfer failure.  Here’s the big difference for me. 

Performance Consultants (PCs) partner with their internal customers, system owners, and business leaders in support of the business goals.  The accountability for improved performance becomes shared across the relationships.

Differences between Training and Performance Consultants
ASSESSMENTS

Trainers typically conduct a needs analysis to design the best learning “program” or course possible.  Again, the assumption is that a learning course will close the training gap. When the directive comes from a senior leader in the organization, it is hard to initiate a dialogue about human performance improvement.  That is probably the least successful time to educate the leader.

PCs conduct performance gaps to assess causes that can go beyond knowledge and skills.  It’s called a performance cause analysis and often reveals other contributing factors that a training course cannot and will not fix. To a compliance trainer or quality systems professional, this sounds a lot like root cause analysis. 

Why we love root cause analysis

We get to document our compliance with CAPA requirements.  It allows us to use tools and feel confident that our “data doesn’t lie”.  This bodes well for our credibility with management.  And it provides the strategic connection between our HPI solution (as a corrective action) and the business goal.  This collected data can become the baseline for measuring the effectiveness of the chosen solution later on.   CAPA= Corrective Actions Preventive Actions.

The outcome of a performance analysis produces a 3 tiered picture of what’s encouraging or blocking performance for the worker, work tasks, and/or the workplace. And what must be done about it at these same three levels.  The solutions then become tailored to the situation, coordinated across the organization, and executed consistently over time.

MEASURES

Trainers very often use course evaluation sheets as a form of measurement.  In the Compliance Training arena, knowledge checks and quizzes have also become the norm.  Caution.  A learner can achieve 100% of the learning objectives and still fail to perform the skills necessary to achieve the business outcomes.  This is also known as a failure to transfer training or the learning objectives. PCs measure the effect on performance improvement and achievement of business objectives.

ORGANIZATIONAL GOALS

This is another key differentiator.  Training is viewed as a cost typically.  Compliance Trainers are all too familiar with the phrase, “GMP Training is a necessary evil”.  And more recently, compliance training has become synonymous with check the box training and “just get ‘er done”. PCs become business partners in solving performance gaps and accomplishing organizational goals.

But isn’t this still training?

Managers and leaders really all the benefits that come from performance consulting, but they don’t have the patience for it especially when many of the solutions end up looking like a “training event”.

If it looks like, smells like, and tastes like training …

Then it must be training, right? Not exactly. But nod your head anyway; at least they are still engaged with you!  If your client/sponsor/requestor is more comfortable with calling it training, let them do so.  Don’t push the HPI label at this point.  First, work on raising their awareness with your early projects and successes.  From your success, you can bridge to an explanation about HPI and gain more support for HPI projects. 

What’s your company’s definition of training, anyway?

Most folks will envision instructor-led classrooms, virtual instructor-led, and formal eLearning courses. Their frame of reference is the gap must be a lack of knowledge and training is used to close that gap.  Is closing a skill-based gap also considered training?  Most companies would define that as OJT.  What about “awareness training” and communication “training” sessions; are these considered training?  It is a form of closing a knowledge gap, the depth of the gap and the degree of required proficiency is the differentiator.  Again, what’s your company’s definition of training?  You may have several examples of differentiating levels of depth.

What’s your organization’s definition of training?

Closing Performance Gaps with the Right Solutions

The essence of HPI methodology is all about the right solution based on the data (evidence) and making a worthy impact on the bottom line when the performance gap closes.  Is this training, you tell me?

 I believe that this is what training is supposed to provide when you perform the proper cause analysis and identify what the business wants to achieve by resolving the performance gap. How would you explain it to your requestor?

Wait a minute. What is worthy performance?

Thomas Gilbert described it as engineering worthy performance in his groundbreaking book, Human Competence: Engineering worthy performance.   It’s when the cost of doing the task is less than the value of the results generated.  When they are the same or greater, we have a performance gap.  The eBook, “Triggering the Shift to Performance Improvement” is a short primer that explains human performance to management.

After the business analysis is conducted, the performance analysis (PA) follows next.  PA recognizes that performance occurs within organizational systems.  It is not a training needs analysis.  The emphasis during a PA is on first recognizing the drivers and barriers that get in the way of worthy performance.  The method gathers multiple perspectives on the problem, not just content for a training course. 

Human Performance Improvement Solutions is like opening up Pandora’s Box

Very often the recommended HPI solution(s) involves integration of linkages outside of the initiating department but within those same “organizational and quality systems” in order to ensure sustainable performance improvement. Otherwise, you have a fragment of the solution with high expectations for solving “the problem” which often falls short of performance improvement.

This requires cooperation of others

How solid are these relationships?  Would a request to fix someone else’s system go over well?   Or would you be reproached of starting a turf battle?  HPI projects have the potential of opening up unsettling issues similar to Pandora’s Box.  Image, perception, pending promotions, can all be impacted by what the Performance Analysis reveals, including the fear of losing one’s job.  And yet, this very opportunity to engineer worthy performance is what makes these projects so valuable for impactful results. 

HPI Project vs. a Training Event

Conclusion

A training solution closes a knowledge and skill gap, wonderful.  Rarely is lack of knowledge the only factor contributing to poor performance.  A performance solution may include a training piece, but it also closes a gap in Job Performance which in turn can close a gap in a Process Performance and resolve a gap in Business Results. That’s what an HPI project/solution does differently than a training solution. Being able to show this kind of impact on the business as a result of the work a Performance Consultant does go a long way to earning business leaders’ trust.  –VB

References:

Robinson DG, Robinson JC. Performance Consulting: Moving beyond training. San Francisco: Berrett-Koehler; 1995.

Gilbert T Human Competence: Engineering Worthy Performance. San Francisco: ISPI, Pfeiffer; Tribute Edition, 2007.

HPISC Library has articles, impact stories, and white papers.

Performance Analysis: the lean approach to solving performance problems

HPISC eBooklet: Trigger the Shift in Performance Improvement

So your retraining corrective action didn’t produce the results you wanted. And now it’s really becoming an urgent issue. I can help with your “Urgent Request”.

Who is Vivian Bringslimark?

Comments Welcomed, Feedback appreciated.

Like this blog? Subscribe to Theory vs Blog | navigate to the left sidebar and scroll up to Subscribe Button.

(c) HPIS Consulting, Inc.

Did we succeed as intended? Was the training effective?

When you think about evaluating training, what comes to mind? It’s usually a “smile sheet”/ feedback survey about the course, the instructor and what you found useful. As a presenter/instructor, I find the results from these surveys very helpful, so thank you for completing them. I can make changes to the course objectives, modify content or tweak activities based on the comments. I can even pay attention to my platform skills where noted. But does this information help us evaluate if the course was successful?

This blog has been merged with Why Knowledge Checks are Measuring the Wrong Thing.

(c) HPIS Consulting, Inc.