Why Do CAPAs Fail Their Effectiveness Checks?

When we start talking about deviations and CAPAs, we can’t help having a sidebar discussion about root causes and more specifically the rant about finding the true root cause.  I intentionally skipped that content in the previous blog.  It was my intention to kick off the new Deviation and CAPAs blog series by first looking at deviations by themselves.  And the learning opportunities deviations can provide us about the state of control for our quality systems.  From those deviations and ensuing CAPA investigations, I ask you this: are we improving anything for the long term (aka prevention).  Are we making any progress towards sustaining those improvements?

Corrective Actions Preventive Actions (CAPA) Steps

Let’s step back a moment and quickly review typical steps for CAPAs:

CAPA Components

The purpose of an Effectiveness Check (EC) is for verifying or validating that actions taken were effective and do not adversely affect product, device or process.  It goes beyond the statement in the investigation form to include a follow-up activity that closes the loop on the specific CAPA.  If an effectiveness check fails meaning the CA/PA was not effective or another deviation /nonconforming incident has occurred, we go back to the beginning and either start again or in most cases, we re-open the investigation.  The pressing question is why did the EC fail?  Almost instinctively, we believe that we did not find the true root cause.  Perhaps.  Was there a rush to close the investigation?  Probably.  Did the investigation team grab the first probable cause as the root cause because the “problem” felt familiar?  Maybe. Or is it a case of a fix that backfired into unintended consequences? Possibly. I will also suggest that the CA/PA may not have been aligned properly.

Ask these 3 questions about CA/PAs

  • Is the CA/PA Appropriate? The focus of this question is about the affected people.  What is the size of this audience? Is it mainly one person or groups of people?

Can the CA/PA be executed efficiently?  Is it for one site or multiple sites?

  • Is the CA/PA Economical? What budget is available?

Is it a “cheap” fix or a 3 – 6 month project? Or an expensive solution of more than 6 months and will need capital expenditure funding?

  • Is the CA/PA Feasible? The real question is about the timeline.

            Need it fast – within 3 months or

            Have time – don’t need until more than 3 months from now.

And then there is the unspoken 4th question – is the CA/PA “political”?  I experienced first hand what happens to CAPAs that are politically oriented.  Most of them failed their ECs.  Request “Can You Stay a Little While Longer”. The best CAPAs are the ones that map back to the root cause.

Introducing the HPISC CAPA Performance Chain

On the left hand side, you will recognize the 3 traditional tasks to complete.  After the EC is written, trace upwards to ensure that the EC maps back to the CA/PA and that the CA/PA maps back to the root cause; hence, the bottom up arrow.  On the right hand side are performance improvement activities that I use as a Performance Consultant (PC) to bring another dimension to the CAPA investigation, namely, Human Performance Improvement (HPI). 

Before I can write the root cause statement, I examine the “problem” also known as a Performance Discrepancy or an incident and I conduct a Cause Analysis that forces me to take a three tiered approach (the worker, the work tasks, the workplace) for the possible causes and not get bogged down in observable symptoms only.  The Performance Solution is more appropriately matched to the identified gap. In theory, this is what the corrective action(s) is supposed to do as well. During the performance solution planning, determination of success and what forms of evidence will be used happens with key stakeholders.  So that collecting the data happens as planned, not as an after thought, and the effectiveness is evaluated as discussed.    

What can we really control?

In RCA/CAPA meetings, I often hear about what management should do to fix the working conditions or how most of the operator errors are really managements’ fault for not taking the culture factor seriously enough.  While there may be some evidence to back that up, can we really control, reduce or eliminate the human factor?  Perhaps a future blog on understanding human errors will be released.

Management Can:

  • Design work situations that are compatible with human needs, capabilities and limitations
  • Carefully match employees with job requirements
  • Reward positive behaviors
  • Create conditions that optimize performance
  • Create opportunities to learn and grow professionally.

Clues for Failed Effectiveness Checks

One of the first activities to perform for a failed EC is to evaluate the effectiveness check statement.  I have read some pretty bizarre statements that challenge whether the EC was realistic to achieve at all. The conditions under which we expect people to perform must be the same as the conditions we evaluate them during an EC review.  So why would we set ourselves up to fail by writing ECs that don’t match normal workplace routines? What, because it looked good in the investigation report and it got the CAPA approved quicker?

Next, trace back each of the CAPA tasks to identify where to begin the re-investigation.  I also suggest that a different root cause analysis tool be used. And this is exactly what we did while I was coaching a cohort of Deviations Investigators.  Future blogs will discuss RCA tools in more detail. -VB

How committed to SOJT is your organization?

In this 2019 refreshed SOJT blog series, I’ve explored why curricula alone is not enough to call your OJT structured. And I shared what else needs to be incorporated into your OJT program to move from traditional OJT to SOJT.  These are all necessary components to provide direction and tools for the Qualified Trainer to deliver required training.  But what about management support for the QT’s and for delivering SOJT?  If only we were required to have a procedure for that! 

ICH Q10 – Pharmaceutical Quality System

While not mandatory, management needs to seriously take notice of ICHQ10 guidance document released in April 2009 (1).  In particular to the following:

  • MANAGEMENT RESPONSIBILITY 2.3 Quality Planning 

“(d) Management should provide the appropriate resources and training to achieve the quality objectives”.

  • “CONTINUAL IMPROVEMENT OF THE PHARMACEUTICAL QUALITY SYSTEM

4.3 Outcomes of Management Review and Monitoring

The outcome of management review of the pharmaceutical quality system and monitoring of internal and external factors can include:

(b) Allocation or reallocation of resources and/ or personnel training.”

Under a Quality System: Managers Expectations for Training

Referencing the Sept. 2006 issue of Guidance Document for Quality Systems: IV. Management Responsibilities | B. Resources (2), we see alignment with the training CGMPs for continued training so “as to remain proficient in their operational functions and in their understanding of CGMP regulations.”  “Typical quality systems training should address the policies, processes, procedures, and written instructions related to operational activities, the product/service, the quality system, and the desired work culture (e.g., team building, communication, change, behavior).”

And my personal favorite, “When operating in a robust quality system environment, it is important that managers verify that skills gained from training are implemented in day-to-day performance.”  The responsibility for training under a quality system is not assigned to just one person or one function.  It is a shared responsibility across the entire organization.

Goals of the Train-the-Trainer Program (TTT) vs. OJT Program vs. Employee Qualification Program

In order to have qualified employees, they need to receive structured on the job training delivered by a qualified trainer who is content qualified and training process qualified via a TTT course.   Each of the three “programs” has defined outcomes that are dependent upon each other.  Unfortunately, the term program has been a bit overused throughout the years and can have a variety of meanings for folks.  For purposes of this blog, TTT program means OJT Qualified Trainers workshop, OJT Program means OJT methodology / procedure and Qualified Employee program means having a robust quality training system beyond the newest Learning Management System (LMS).

As you can see from the diagram, Qualified Trainers are at the core for all three “programs”.   I’ve previously blogged and written several resources on what’s involved.  The OJT QT Workshop is designed to prepare the QT’s for the realities of life as a Qualified Trainer. It includes the pre-work of familiarizing themselves with the training procedures and then in class: exploring basic learning theory and committing to what the QT signature means. The OJT Program is about delivering structuring on the job training consistently following the approved methodology as discussed in the previous blog.  An Employee Qualification Program is the ultimate validation of training effectiveness of the OJT methodology and the clarity of the underlying procedures.

Readiness Factors for SOJT and an Employee Qualification Program

Let’s start with a written purpose statement for having qualified employees beyond it’s required.  What is the company’s philosophy on achieving qualified status?  Is there agreement among the leadership for the level of rigor required to demonstrate performance and achieve a pass rating?  Where are the OJT program goals written?  Does a schedule exist for OJT and qualification events other than a LMS printout with required due dates.  That is not a schedule for OJT.  Do you have clearly defined objectives for the QT workshop captured in a document or perhaps a procedure?  Is there a single owner for all three programs or is responsibility and accountability assigned accordingly? Check out the HPISC Readiness Factors for SOJT/RTS/Employee Qualification Checklist.

Ronald Jacobs and Michael Jones, in their 1995 ground breaking book, Structuring on-the-Job Training, inform us that SOJT as a system functions within a larger context, namely the organization.  SOJT is not a standalone program.  Conflicts, competing priorities and mixed messages can influence the success of your SOJT program.  What else is going on in the organization that will compete for the same set of QTs? Remember they are also your most experienced and technical subject matter experts.  How is the overall Employee Qualification program aligned with the other quality systems? 

Recognition for QTs and Qualified Status

Most QTs are not fully dedicated to delivering training for departments.   There are pros and cons for this decision.  For now, I will leave them out.  Suffice it to say, they are tasked with both their “day” job and the responsibility for delivering training when needed.  They are at times, doing two jobs.  Whether or not they are compensated additionally for delivering OJT, acknowledging their contribution to the department and the organization is part of management support.   It takes more than “you are doing a good job, keep it up”. 

Often supervisors and managers don’t know what else they are supposed to do to show their support, other than allow them to attend the QT workshop.  The interested ones will “pop” in during lunch and chat with their direct reports.  Others will show up at the end for the poster activity (equivalent to a written test) and some will come to learn about the parking lot issues that need follow up.  The energy in the room when this happens is amazing. 

To help ease the knowledge gap between a manager and their now Qualified Trainer, I started delivering the Leadership Briefing module prior to the QT workshop delivery.  The purpose is to provide an overview of the content highlights, alignment with initiatives / CAPAs/ agency commitments and more importantly to secure agreement for the following:

  • criteria for nominating a QT
  • roles and responsibilities of QT
  • scope of work QT’s can be assigned
  • expectations for QT’s post launch
  • what happens day one after workshop is done
  • what is the status of the SOJT checklists
  • scheduling and budget concerns.

If the organization says they support the qualification program, then what happens when employees achieve qualification status?  Nothing?  A nonevent? Or is it announced in newsletters, plasma screens and other company announcements?  Is it a big deal to be able to perform independently and free up a much-needed QT for another learner?  I keep hearing over and over again about how there aren’t enough QT’s to deliver OJT the right way.  One would think qualification status on SOPs, especially big complex processes deserves SOME kind of recognition.   Just how committed are the managers and supervisors?  QTs and employees draw their own conclusions about the level of real management support for the programs.  

If truth be told, after launching many supervisors privately don’t support the program.  They lose their top performers for the workshop participation and during the hours it takes to train someone. Forget about giving QTs adequate time to complete the paperwork properly! And then leadership wonders why good documentation practice (GDP) issues continue to be a problem? The non-distracted performance observations that QT’s are expected to conduct for the qualification demonstration drive supervisors and line managers crazy the most – what, they can’t do anything else but observe?  Hence, many QT’s are asked to multi task just to get the work done: not enough resources they are told.  For supervisors, productivity and the workload will always trump OJT and qualification events, until their bonuses include completion of training and qualification events.

What Real Support Is Supposed To Look Like

My key take away message is that attending the TTT program is not the end of the OJT program or the Employee Qualification Program but rather the launching point.  Management support needs to go beyond just nominating QTs and allowing them to participate in the workshop.  The real support is in the alignment of goals, clarifying expectations continuously, allocating resources for training and budgeting time to deliver OJT using an approved OJT methodology that includes qualification events.  This commitment of time and sponsorship for qualified employees is a culture shift for many line managers and site leaders.  But actions do speak louder than words.  -VB

Alas, we have come to the end of this SOJT blog series. For a recap, visit the individual blog links below:

(1) Guidance for Industry Q10 Pharmaceutical Quality System | US Department of HHS | FDA
| CDER | CBER April2009 ICH

(2) Guidance for Industry Quality Systems Approach to Pharmaceutical CGMP RegulationsU.S.Department of HHS | FDA | CDER | CBER | CVM | ORA| September 2006 Pharmaceutical CGMPs

Jacobs RL, Jones MJ. Structured on-the-job training: Unleashing employee expertise in the workplace.  San Francisco: Berrett-Koehler: 1995.

What’s Your Training Effectiveness Strategy? It needs to be more than a survey or knowledge checks

When every training event is delivered using the same method, it’s easy to standardize the evaluation approach and the tool. Just answer these three questions:

  • What did they learn?
  • Did it transfer back to job?
  • Was the training effective?

In this day and age of personalized learning and engaging experiences, one-size training for all may be efficient for an organizational roll out but not the most effective for organizational impact or even change in behavior. The standard knowledge check can indicate how much they remembered. It might be able to predict what will be used back on the job. But be able to evaluate how effective the training was? That’s asking a lot from a 10 question multiple choice/ true false “quiz”. Given the level of complexity of the task or the significance of improvement for the organization such as addressing a consent decree or closing a warning letter, it would seem that allocating budget for proper training evaluation techniques would not be challenged.

Do you have a procedure for that?

Perhaps the sticking point is explaining to regulators how decisions are made using what criteria. Naturally documentation is expected and this also requires defining the process in a written procedure. It can be done. It means being in tune with training curricula, awareness of the types of training content being delivered and recognizing the implication of the evaluation results. And of course, following the execution plan as described in the SOP.   Three central components frame a Training Effectiveness Strategy: Focus, Timing and Tools.

TRAINING EFFECTIVENESS STRATEGY: Focus on Purpose

Our tendency is to look at the scope (the what) first. I ask that you pause long enough to consider your audience, identify your stakeholders; determine who wants to know what. This analysis shapes the span and level of your evaluation policy. For example, C-Suite stakeholders ask very different questions about training effectiveness than participants.

The all purpose standard evaluation tool weakens the results and disappoints most stakeholders. While it can provide interesting statistics, the real question is what will “they” do with the results? What are stakeholders prepared to do except cut training budget or stop sending employees to training? Identify what will be useful to whom by creating a stakeholder matrix.

Will your scope also include the training program (aka Training Quality System) especially if it is not included in the Internal Audit Quality System? Is the quality system designed efficiently to process feedback and make the necessary changes that result from the evaluation results? Assessing how efficiently the function performs is another opportunity to improve the workflow by reducing redundancies thus increasing form completion speed and humanizing the overall user experience. What is not in scope? Is it clearly articulated?

TRAINING EFFECTIVENESS STRATEGY: Timing is of course, everything

Your strategy needs to include when to administer your evaluation studies. With course feedback surveys, we are used to immediately after otherwise, the return rate drops significantly. For knowledge checks we also “test” at the end of the session. Logistically it’s easier to administer because participants are still in the event and we also increase the likelihood of higher “retention” scores.

But when does it make more sense to conduct the evaluation? Again, it depends on what the purpose is.

  • Will you be comparing before and after results? Then baseline data needs to be collected before the event begins. I.e. current set of Key Performing Indicators, Performance Metrics
  • How much time do the learners need to become proficient enough so that the evaluation is accurate? I.e. immediately after, 3 months or realistically 6 months after?
  • When are metrics calculated and reported? Quarterly?
  • When will they be expected to perform back on the job?

Measuring Training Transfer: 3, 6 and maybe 9 months later

We can observe whether a behavior occurs and record the number of people who are demonstrating the new set of expected behaviors on the job. We can evaluate the quality of a work product (such as a completed form or executed batch record) by recording the number of people whose work product satisfies the appropriate standard or target criteria. We can record the frequency with which target audience promotes the preferred behaviors in dialogue with peers and supervisors and in their observed actions.

It is possible to do this; however, the time, people and budget to design the tools and capture the incidents are at the core of management support for a more vigorous training effectiveness strategy. How important is it to the organization to determine if your training efforts are effectively transferring back to the job? How critical is it to mitigate the barriers that get in the way when the evaluation results show that performance improved only marginally? It is cheaper to criticize the training event(s) rather than address the real root cause(s). See Training Does Not Stand Alone (Transfer Failure Section).

TRAINING EFFECTIVENESS STRATEGY: Right tool for the right evaluation type

How will success be defined for each “training” event or category of training content? Are you using tools/techniques that meet your stakeholders’ expectations for training effectiveness? If performance improvement is the business goal, how are you going to measure it? What are the performance goals that “training” is supposed to support? Seek confirmation on what will be accepted as proof of learning, evidence of transfer to the workplace, and identification of leading indicators of organizational improvement. These become the criteria by which the evaluation has value for your stakeholders. Ideally, the choice of tool should be decided after the performance analysis is discussed and before content development begins.

Performance Analysis first; then possibly a training needs analysis

Starting with a performance analysis recognizes that performance occurs within organizational systems. The analysis provides a 3-tiered picture of what’s encouraging/blocking performance for the worker, work tasks, and/or the workplace and what must be in place for these same three levels in order to achieve sustained improvement. The “solutions” are tailored to the situation based on the collected data and not on an assumption that training is needed. Otherwise, you have a fragment of the solution with high expectations for solving “the problem” and relying on the evaluation tool to provide effective “training” results. Only when the cause analysis reveals a true lack of knowledge, will training be effective.

Why aren’t more Performance Analyses being conducted?
For starters, most managers want the quick fix of training because it’s a highly visible activity that everyone is familiar and comfortable with. The second possibility lies in the inherent nature of performance improvement work. Very often the recommended solution resides outside of the initiating department and requires the cooperation of others.   Would a request to fix someone else’s system go over well where you work? A third and most probable reason is that it takes time, resources, and a performance consulting skill set to identify the behaviors, decisions and “outputs” that are expected as a result of the solution. How important will it be for you to determine training effectiveness for strategic corrective actions?

You need an execution plan

Given the variety of training events and level of strategic importance occurring within your organization, one standard evaluation tool may no longer be suitable. Does every training event need to be evaluated at the same level of rigor? Generally speaking, the more strategic the focus is, the more tedious and timely the data collection will be. Again, review your purpose and scope for the evaluation. Refer to your stakeholder matrix and determine what evaluation tool(s) is better suited to meet their expectations.

For example, completing an after-training survey for every event is laudable; however, executive leadership values this data the least. According to Jack and Patricia Phillips (2010), they want to see business impact the most. Tools like balanced scorecards can be customized to capture and report on key performing indicators and meaningful metrics. Develop your plan wisely, generate a representative sample size initially and seek stakeholder agreement to conduct the evaluation study.

Life after the evaluation: What are you doing with the data collected?

Did performance improve? How will the evaluation results change future behavior and/or influence design decisions? Or perhaps the results will be used for budget justification, support for additional programs or even a corporate case study? Evaluation comes at the end but in reality, it is continuous throughout. Training effectiveness means evaluating the effectiveness of your training: your process, your content and your training quality system. It’s a continuous and cyclical process that doesn’t end when the training is over. – VB

 

Jack J. Phillips and Patricia P. Phillips, “How Executives View Learning Metrics”, CLO, December 2010.

Recommend Reading:

Jean-Simon Leclerc and Odette Mercier, “How to Make Training Evaluation a Useful Tool for Improving L &D”, Training Industry Quarterly, May-June, 2017.

 

Facilitating the Shift from Passive Listening to Active Learning

On the one end of “The Learner Participation Continuum” is lecture which is a one way communication and requires very little participation.  At the other end, we have experiential learning and now immersive learning environments with the introduction of 3D graphics, virtual simulations and augmented reality.

In the middle of the range are effective “lectures” and alternate methods such as:

  • Demonstrations
  • Case Study
  • Guided Teaching
  • Group Inquiry
  • Read and Discuss
  • Information Search.

Shift one step to right to begin the move to active learningNow before you insist that the SME as Facilitator move to the far right and conduct only immersive sessions, a word of caution is in order. It’s really about starting with the learners’ expectations and the current organizational culture and then moving one step to the right. If they are used to lectures from SMEs, then work on delivering effective lectures before experimenting with alternate training methods. The overnight shift may be too big of a change for the attendees to adjust to despite their desire for no more boring lectures. Small incremental steps is the key.

How is this done? Upfront in the design of the course materials. The course designers have spent time and budget to prepare a leaders guide that captures their vision for delivering the course.  SMEs as Facilitators (Classroom SMEs) need to study the leader’s guide and pay attention to the icons and notes provided there. These cues indicate the differentiation from lecture, to an activity whether that be self, small group, or large group. While it may be tempting to skip exercises to make up for lost time, it is better for learner participation to skip lecture and modify an activity if possible.

During the knowledge transfer session/ discussion with the course designer and/or instructor, Classroom SMEs make notes of how the instructor transitions from one slide to the next and how s/he provided instruction for the activity. This is a good time for Classroom SMEs to ask how to modify content or an activity if certain conditions should occur. Especially important for SMEs to ask is what content is critical and what content can be skipped if time runs short. It is always a good idea for the Classroom SME to mark-up his/her copy of the materials. And then again after the first delivery to really make it their own leader’s guide. -VB

Speaking of personalizing their leaders’ guide, SMEs may want to experiment with different ways to “open a session” to get experience with a variety of techniques and observe which ones yield better results.

I’m in love with my own content!

Many QA /HR Training Managers have the responsibility for providing a train-the-trainer course for their designated trainers.  While some companies send their folks to public workshop offerings, many chose to keep the program in-house.   And then an interesting phenomenon occurs.  The course content grows with an exciting and overwhelming list of learning objectives.

The supervisors of the SMEs struggle with the loss of productivity for the 2 – 3 day duration and quickly develop a “one and done” mindset.   Given the opening to “train” newly identified SMEs as Trainers, the instructional designer gets one opportunity to teach them how to be trainers.  So s/he tends to add “a lot of really cool stuff” to the course in the genuine spirit of sharing, all justifiable in the eyes of the designer.  However, there is no hope in breaking this adversarial cycle if the Training Manager doesn’t know how to cut content.

I used to deliver a two-day (16 hour) workshop for OJT Trainers. I included all my favorite topics.  Yes, the workshop was long.  Yes, I loved teaching these concepts.  I honestly believed that knowing these “extra” learning theory concepts would make my OJT Trainers better trainers.  Yes, I was in love with own my content.  And then one day, that all changed.

 

Do they really need to know Maslow’s Hierarchy of Needs?

During a rapid design session I was leading, I got questioned on the need to know Maslow’s Hierarchy of Needs.  As I began to deliver my auto-explanation, I stopped mid-sentence.  I had an epiphany.  My challenger was right.  Before I continued with my response, I feverishly racked my brain thinking about the training Standard Operating Procedures (SOPs) we revised, the forms we created, and reminded myself of the overall goal of the OJT Program.  I was searching for that one moment during an OJT session when Maslow was really needed.  When would an OJT Qualified Trainer use this information back on the job, if ever I asked myself?

It belongs in the Intermediate Qualified Trainers Workshop, I said out loud.  In that moment, that one question exercise was like a laser beam cutting out all nice-to-know content.  I eventually removed up to 50% of the content from the workshop.

 

Oh, but what content do we keep?

Begin with the overall goal of the training program: a defendable and reproducible methodology for OJT.  The process is captured in the redesigned SOPs and does not need to be repeated in the workshop.  See Have you flipped your OJT TTT Classroom yet?

Seek agreement with key stakeholders on what the OJT QTs are expected to do after the workshop is completed.  If these responsibilities are not strategic or high priority, then the course will not add any business value.  Participation remains simply a means to check the compliance box.  Capture these expectations as performance objectives.

How to align purpose of a course to business goals

Once there is agreement with the stated performance objectives, align the content to match these. Yes, there is still ample room in the course for learning theory, but it is tailored for the need to know only topics.

In essence, the learning objectives become evident.  When challenged to add certain topics, the instructional designer now refers to the performance objectives and ranks the consequences of not including the content in the workshop against the objectives and business goal for the overall program.

 

What is the value of the written assessment?

With the growing demand for training effectiveness, the addition of a written test was supposed to illustrate the commitment for compliance expectations around effectiveness and evaluation.  To meet this client need, I put on my former teacher hat and created a 10 question open book written assessment.  This proved to need additional time to execute and hence, more content was cut to accommodate the classroom duration.

My second epiphany occurred during the same rapid design project, albeit a few weeks later.   What is the purpose of the classroom written assessment when back on the job the OJT QTs are expected to deliver (perform) OJT; not just know it from memory? The true measure of effectiveness for the workshop is whether they can deliver OJT according to the methodology, not whether they retained 100% of the course content!   So I removed the knowledge test and created a qualification activity for the OJT QTs to demonstrate their retained knowledge in a simulated demonstration using their newly redesigned OJT checklist.  Now the OJT QT Workshop is value added and management keeps asking for another round of the workshop to be scheduled.  -VB

Are you ready to update your OJT TTT Course?

 

 

 

I’ve fired my [TTT] Vendor!  

Sustaining Qualified Trainer’s Momentum Post Launch

Are you doing the Training 2 Step?

Has this situation ever happened to you? You are in a root cause meeting and the *CAPA investigator is conducting an interview with the primary individual involved with the discrepancy. When questioned why did this happen, he shrugs first and then quietly mumbles I don’t know. When pushed further, he very slowly says I just kind of went brain dead for a moment.  And then silence.

While that may be the honest truth, the investigator must resist the temptation to label it as Operator Error and explore possible causes. One of my favorite root cause analysis tools for “I Don’t Know Why” response is to use the Fish Bone Diagram also known as the 4 M’s diagram. This tool provides a structured focus to explore many possibilities and not just stop at the first plausible cause; such as Operator Error. Aptly nicknamed, the 4 M’s are Man, Machine, Methods and Materials.   When the results of this exercise points to a training or operator related issue, don’t stop at “operator error –> retrain”.

Consider for a moment, what this retraining session would look like. Will re-reading the procedure be enough to “jog his memory”? Will repeating the procedure be a good use of precious time when s/he already knows what to do? More than likely it won’t prevent “going brain dead” from happening again. Instead, do the HPISC Training 2 Step:

Step 1 – confirm the results of the gap analysis

What task, what step(s) or actions are in question?

Step 2 – address why the original training did not transfer back to the job.

Using the 4 M’s diagram as the framework, explore Man, Machine, Methods and Materials questions with regards to the training this operator receives. See diagram below. The full set of questions can be found in the eBook Training Cause Analysis.

4msfortrca

Is this really worth it?

I think it is. Conducting these 2 steps will accomplish two objectives.   It will provide further evidence that some kind of training is needed. And it will highlight what areas are in need of revising either for the performer, the training program or course materials. Yet, there are some who will resist this added work because it’s easier to find blame than to uncover the cause. Fixing the true root cause could trigger a re-validation of the process or an FDA filing if it’s a major process change.   Why create more work? Isn’t it easier to just retrain ‘em? No, not really. Finding the true root cause is the only effective way of eliminating many of the costly, recurring problems that can plague manufacturers.

But what if

Some folks will push back with the excuse – “this never caused a problem until now”, so it must be the operator’s fault! This may be the first time it was discovered but that does not mean the procedure is 100% accurate. Often, experienced operators know how to work around an incorrect step and don’t always report a misstep in the procedure while a less savvy operator follows the procedure and causes the non-conformance to occur. See Sidebar SOP Logic Rules. Is the procedure difficult, lengthy or requires weeks to become proficient let alone qualified? Was the qualification routine or performed as a simulation? Was the procedure written with support from a lead operator or qualified trainer? Did the draft version undergo some kind of field test or dry run prior to release? And the classic situation, are proposed changes hung up in change control awaiting effective release?screen-shot-2016-10-19-at-4-33-39-pm

Understanding Why Human Errors Occur

Industry practice is evolving to explore why people make the decisions they do by looking at the Organization’s systems. It’s usually a poor decision made somewhere in the error chain. We must believe that the person who made the poor decision did not intend for the error to occur. As part of CAPA investigations, we need to explore their physical environment as well; the conditions under which they make those decisions. The Training Program Improvement Checklist can be requested using this link to capture your findings.

If you are going to spend time and money on training, at least identify what the gap is; fix that and then “train” or provide awareness on what was corrected to prevent the issue from re-occurring again.   That is after all, the intention of *Corrective Action Preventive Action investigations. -VB

You may to explore these other library gems:

BandAidsPicture1
Why the Band Aids Keep Falling Off

OpErrorPicture1

What will it take to gain access to HPI/HPT Projects?

It’s more than a name change.
Adding Performance Consulting to your department name or position title sounds like a good idea at first. You know, help get the word out and ease into Performance Consulting projects, right? Well not exactly. Adding it on is exactly what happens; possible projects get added on to your workload and the “regular” training requests keep coming. It becomes a non-event. Dana Gaines Robinson and James C. Robinson, authors of Performance Consulting, strongly recommend that you create a strategic plan for your transition. And that’s exactly what I did in 1997.

Technical Training is now known as Performance Enhancement Dept.
But not without first discussing my plan with my boss and then pitching it to his staff at his weekly meeting. My plan included the need for the change and a comparison of the traditional training model and the performance model. In this comparison, I listed the percentages of training to consulting ratios and where the shift would occur. Training was never going away, but that we would do less and pick up more performance consulting work instead. I used the now familiar line – training is not always the answer. (Back then it was a very edgy statement.)

And I included in my pitch, the recommended pieces from the Robinsons’ seminal book: mission, vision, guiding principles, services, responsibilities and even who are customers were by percentages. Key to this plan and acceptance, was that we never said no to a training request, but re-framed it into why and how will we measure success. If we couldn’t design a measurement strategy from the beginning, we were obligated to turn the project down. And the General Manager agreed with that guiding principle.

NOW OPEN FOR BUSINESS!
While we waited for feedback and project requests, I invited myself to a quality meeting about a GMP concern from a Line Trainer. When no one volunteered to complete a suggested task, I raised my hand and took the assignment. Cheers, we had our first project and we were now open for business. The task was then assigned to a direct report who thought I was crazy or evil, but I described how this assignment could catapult us into the limelight and showcase exactly the kind of performance work we were capable of doing. Intrigued but still doubtful, he took on the research task and I took on the rest of the project since I had the vision and could connect the dots. We got 3 more requests after we went public with our first project.

One of my best requests that first year was a request for Peer Mentoring. Oh did I want this project. I met with the requester and listened to his case. I researched the topic (remember this was 1997) and got some ideas about a possible solution. When I pressed on about measuring the success, he was vague and said, you know, as part of organizational awareness. I was in love with the topic and what it could mean for the operators and for the new PE department, but I could not find enough support to measure the success nor justify the time and resources to make it happen. We had to scrap the project request. This was the Evaluator Role coming out loud and clear. And this news got around fast. The PE department was not a dumping ground for someone else’s yearly objectives.

Okay, that’s great, but who does GMP Training?
During our success, we still managed the Compliance Training requirements as part of our agreement. Folks got so used to us and how we managed both the compliance side and performance enhancement requests, that we no longer had to explain what PE was and who we were. So upon biennial inspection from FDA, the inspector asked, “Well then, who does GMP Training”? So, I was asked to put Training back into our department name and become known as TPE: Training and Performance Enhancement which felt like we were back to square one. But the requests kept coming and the projects got much better.

My favorite project was the “Checking Policy”. It had everything going for it. Unfortunately for the company, a very expensive error was made by an operator and site leadership wanted him terminated. The GM who was our unofficial sponsor knew there was a better way to manage this and he needed to find the true root cause of the performance discrepancy, so he reached out to me. The rest of the story is long, so I’ll spare you the details, but three additional projects resulted from this request and all three included operators as my SME Team. This was unheard of at the time and really highlighted what an asset they were to the company despite the costly mistake. Turns out it wasn’t his fault, what a surprise!

Alas, the time came for me to leave that company and take on external consulting full time. When given the opportunity to reinvent myself once again years later, I reflected on the times when I was most engaged and excited about going to work. It was those Performance Enhancement projects that gave me such powerful examples of successfully aligning improvement projects with the business needs. But rather than do it again for one single company, I created HPIS Consulting instead so I could share the approach with more than one company.

So as this “Gaining Management Support” series concludes, I summarize all the related blogs with this final question and provide an overview as the answer.

What will it take?
Developing trust with business partners for starters. Ongoing skill development as an Analyst, a Change Manager, an Evaluator and of course, a Performance Solutions Specialist to build credibility. A good transition plan with vision for 1-3 years and tentative plans for year 4 and 5. And the courage to take projects no one else wants if you want to become a Performance Consultant bad enough. We did it and I’ve never been happier! -V

Gaining Management Series includes the following blog posts:

If the only tool in your toolbox is a hammer …
Are you worthy of your line partner’s trust?
Wanted: Seeking a business partner who has performance needs
First, make “friends” with line management

If the only tool in your toolbox is a hammer …

A hammer is the right tool to drive a nail into wood or dry wall, etc. supporting the adage “right tool for the right job”. Until the closet you installed comes off the wall and you realize that perhaps you needed screws instead or an additional widget to support the anticipated load. It isn’t until “in-use” performance feedback is collected that the realization of a different tool and additional support mediums are needed. Providing training (as in formal instruction) as the solution to a performance issue is analogous to using a hammer for every job.

Site leaders want business partners who can help them succeed with organizational goals, yearly objectives and solve those pesky performance issues. The more valuable the “trainer-now-known-as-performance-consultant is in that desire, the more access to strategic initiatives. So, the more you want to be recognized as a business partner to the site leaders, you need to continue to build your “solutions toolbox” that includes more than delivering a training event or LMS completion report.

As we begin to wrap up this series on gaining management support, we’ve been exploring how to forge relationships with line managers and earn their trust by being trustworthy. In the blog (Are you worthy of your line partner’s trust?), I asked if you were also trust worthy as a Performance Consultant (PC).

Do you have the necessary competencies to tackle the additional performance solutions? A logical next step is to review the plethora of literature that has been published on the multiple roles for a Performance Consultant. These include Analyst, Change Manager, Solutions Specialist, and Evaluator. There are more, but let’s start with an overview of these four.

A trainer with strong instructional design skill could argue that s/he has loads of experience with 3 of the 4 roles sans solution specialist. To that end, ADDIE has been the methodology and the foundation for successful training events for years. A sound training design analyzes need first and incorporates change management elements and includes evaluation activities for level 1(reaction) and level 2 (learning) of the Kirkpatrick Evaluation Model. So how hard could it be to master the role of Performance Consultant? Doesn’t every solution have a training component to it anyway?

Maybe and maybe not. As the traditional role of technical trainer evolves into Performance Consultant, the skills needed are evolving as well to keep up with management expectations for alignment with business needs.

The PC wears the hat of Analyst when working the business analysis and performance analysis portion of the HPI methodology honing in with the skill of asking the right questions and being able to analyze all of the contributing factors for performance causes. This is more than a needs analysis for designing a course.

The Solution Specialist role relies heavily upon systems thinking skills and is already way outside the power point training solution. As a problem solver working the probable causes from the Performance Cause Analysis, s/he opens the toolbox and can look past the “training design tray” into other alternative performance solutions. There is much more than a hammer in their toolbox. NOTE:  For more details on those types of solutions, navigate to paragraph “HPI Approach” within the blog link).  Implementation experience grows with each executed solution and a great PC also develops good project management skills.

During implementation, the PC may also have to wear a dual hat of Change Manager.  Process changes, culture change and more require strong facilitation skills and process consultation HPI roles for Perf Consulttechniques to manage the different phases of change depending on the nature of the solution and the size of the change impact.

And the Evaluator role surfaces at or near end of project implementation as the solution launches and goes live. Feedback collection, standards setting and re-assessing the performance gap to determine success or additional gap analysis.

The role of Performance Consultant requires more variety of skills and depth of project experiences. While training solutions are part of the PC toolkit, a training manager’s toolbox typically does not offer other performance solutions. It’s usually a hammer when a swiss army knife is what’s needed.  –VB

References: William J. Rothwell Editor, ASTD Models for Human Performance Improvement: Roles, Competencies, and Outputs. 2nd Ed, 1999.

Are you worthy of your line partner’s trust?

In this current series of gaining management support we’ve been exploring how credibility, trust and access impact or influence relationships with our business partners. In Stephen Covey’s, The 8th Habit: From effectiveness to greatness, he informs us that you cannot have trust without being trustworthy.  As Performance Consultants (PCs) continue to demonstrate their character and competence, their line leaders begin to trust them more and more.

From those initial getting-to-you-know-you chats (see previous blog)  to requests for help discussions, the give trust and return trust has been reciprocated and continues to strengthen the relationship. With each request / opportunity, PCs are demonstrating their character traits and further developing their Human Performance Improvement (HPI) technical competence and experience.

Following the HPI/HPT model gives the PC the ability to articulate the big picture of how this request, this performance gap, this project, relates to organizational goals thus illustrating a strategic mindset. And by following the related methodology, PCs demonstrate strong project management skills while implementing changes systematically; not just a quick course to fix a perceived knowledge gap or motivation problem.

So PCs become worthy of receiving their partners’ trust.  Line partners in exchange merit their trust by giving it. Are you trustworthy as a Performance Consultant? Do you have the necessary competencies to tackle the additional performance solutions? Stay tuned for more blogs on what those competencies are and why they are so helpful for PCs. In the meantime, check out the sidebar “Ten Steps for Building Trust” from Alan Weiss in Organizational Consulting.  -VB

How to Build More Trust

References:
Covey,SR. The 8th Habit: From effectiveness to greatness, USA, Free Press, 2004.

Weiss, A. Organizational Consulting: How to be an effective internal change agent, USA, Wiley, 2003.