Please Tell Me There Are More Quality Tools Than Fish Bone and the 5 Whys!

Don’t get me wrong.  I am a huge fan of both these methods especially for training related errors.  However, there are a lot of other quality tools in the problem solving toolbox that aren’t as popularly used.  In this blog, I will continue part two of the previous blog, RCA Meetings vs. Problem Solving Discussions in which I presented an alternative problem-solving model that helps investigators take a deeper dive into understanding the problem first (#1), then asking a series of questions to generate possible causes (#2) that move from the event trigger to patterns to systems glitches.

So many tools, so little time!

Before Google became a verb, the best way to learn about a problem-solving tool was to ask someone, ask to borrow his or her “The Memory Jogger” pocket guide, or wait patiently to take your course.  Getting information at your fingertips instantly is no longer a barrier.  But the consequence of too many available tools can lead to confusion and indecision resulting in sticking to what is familiar instead. One way to mitigate the bewilderment is to choose these tools by which stage the investigation is in and whether the team needs to analyze the process or the collected data.

Where are you in the investigation?

Are you exploring and just learning more about the triggering event?  Are you generating possible causes and need an organized decision making technique?  Or are you verifying and eliminating causes?  Refreshing on the purpose of the tool helps the Lead Investigator make the right tool choice.  Consider the Job Aid: Which Tool to Use When?

WHAT IS THE ASSIGNABLE CAUSE?

Just because the team has generated more than one possible cause does not mean that one of them has to be the root cause by default.  Has the team asked the “second right answer”?  Charles “Chic” Thompson, in his book, What a Great Idea, explains that we’ve been trained to look for the right answer, to seek the one and only answer, ever since we started school.  Instead, he suggests that we ease “into a new frame of mind” by asking a host of questions in search of potential solutions.  Included in this perspective, is asking, “dumb” but penetrating questions that lead to finding “the true source of bottlenecks buried deep in habitual routines of the firm”.  

When the team has really explored all possible causes, then it’s time to switch tools and use data analysis tools such as histograms, run/control charts, and/or scatter diagrams.

“A fishbone represents opinions.  Opinions must be verified with facts before action to change is taken.”  

Waste Chasers: A Pocket Companion to Quality and Productivity, p.43.

Pareto charts can be very helpful when you need to display the relative importance of all of the conditions in order to identify the basic cause of the problem.  It helps the team to focus on those causes that will have the greatest impact if solved.  The team has to agree on the most probable cause aka the root cause.  One of the best decision making tools I’ve experienced is Nominal Group Technique.  NGT provides a way to give everyone in the group/team an equal voice in the selection.  Team members rank the items without being pressured by others.  Therefore, it builds commitment to the team’s choice through equal participation in the process.

What is the solution?

One of the most versatile tools in the arsenal is brainstorming.  It’s quick; it’s easy to administer and almost everyone has experienced a session or two.  We like it because it is an efficient way to generate a lot of ideas that are free of criticism and judgment when facilitated properly. 

At this point in the investigation, some members on the team finally get to share their ideas on how to solve the problem.  For teams pressed for time, having already made ideas sounds like a gift ready to be opened.  And yet, there can be a tendency to allow the eager problem solvers to dominate the discussion and solution path forward. 

A nice warm up exercise prior to the solution brainstorm, is to assign an individual idea map first.  By mapping individuals’ own thoughts first, they are not lost to the group’s brainstorming results when the time comes to participate in the session. The original ideas, thoughts, and even impulses can be easily shared since they have already been captured in their own idea map.

For teams stuck in the “same old way” mindset or wrestling with a frustrating repeat problem, try switching up the brainstorm trigger word.  For example, the trigger word “when solved” can help us work backwards.  Or use the opposite trigger word and brainstorm on ideas to make it true.  Try a random trigger word or a metaphorical trigger phrase such as (the problem …) is like … the day after a vacation when … .  I’ve seen amazing idea maps resulting from stepping up the brainstorming trigger words.  If the team is still stuck, compare all of the maps to look for commonalities or common themes.  And sometimes you just have to change your shoes or take a walk outside to shift your thoughts.

What is the best way to implement the solution?

Now, the team is responsible for bringing their ideas into reality aka the corrective actions / CAPA Plan.  One of my favorite tools is the Force Field Analysis when the team is working on corrective actions.  The tool helps team members to identify forces and factors that will support or work against the solution of an issue so that the positives can be reinforced and /or the negatives eliminated or reduced. It forces the team to think through all the aspects of making the desired change a permanent one. 

“After falling in love with your idea, you must also be prepared to fall out of love.”

Charles Thompson, What A Great Idea, p.161.

Now Just Do It

Implementing the corrective action(s) may be the longest duration depending upon the scope of the project and therefore needs to be managed using project management techniques.  And that could be the basis of another blog series. 

Close the Investigation

Before the investigation can be closed, the effectiveness check needs to be performed and evaluated as described in the CAPA plan. Over the course of the incident and its CAPA, the story evolves.  This is normal and expected especially as we learn more.  To an outsider reviewing the investigation report, the story and all the supporting evidence has to be understandable, even to someone internal to the organization but not involved in the investigation, the CAPA or the project. 

The Lead Investigator must review the entire file one more time to finalize the problem statement and confirm the cause statement given what is known at this final point of the investigation.  The following is a list of questions to ask:

  • Is the report easy to follow? Is there a logical flow?
  • Is the report free of unnecessary documentation? Is it uncluttered?
  • Is there sufficient information to back up the investigation, results and the outcome?
  • Does the report support decisions about product disposition?

Conclusion

Part One of this blog suggested the use of a more creative problem-solving approach to help us see our problems using systems thinking perspective.  Systems Thinkers understand complex relationships and their interdependencies. They step up and take responsibility to fix the problem.

In part two, I suggested different problem solving tools to use depending on the investigation stage and the team’s task.  As a Performance Consultant, the problem solving approach and the plethora of “quality” tools has me excited about identifying the true root cause and implementing systematic solutions as the corrective actions because we have to get better at solving our problems.  System Thinking,

  • puts a problem into a context of the larger whole with the objective of finding the most effective place to make an appropriate change and
  • it can help us identify and respond to a series of changes before those changes lead to more unwanted deviations and CAPAs. 

Isn’t that the purpose of the PA in CAPA, the preventive actions? -VB

Root Cause Analysis and Qualifying Investigators Coaching Briefs
  • Charles “Chic” Thompson, What A Great Idea: Key Steps Creative People Take, HarperPerennial, 1992.
  • The Memory Jogger: A Pocket Guide of Tools For Continuous Improvement, GOAL/QPC, 2nd 1988.
  • Waste Chasers: A Pocket Companion to Quality and Productivity, Conway Quality, Inc.

Conducting Root Cause Investigation Meetings vs. Facilitating Problem Solving Discussions

I began this Deviation and CAPA series with the intent to share what I call the human performance improvement (HPI) touch points; where HPI and the CAPA quality system have cross -over connections.  See The Big Why of Deviations and Why Do CAPAs Fail Their ECs.  In this two-part blog, I will reveal another noteworthy HPI crossover connection.

If you perform a Google search on CAPA process and the ensuing steps, the results are varied but in general, industry follows a routine process that looks like the following:

  • Compliant “CAPA” Program includes procedures for:
    • Discovery and Notification
    • Containment
    • Impact Assessment
    • Product Investigation
    • RC Investigation
    • Corrective Actions (CA)
    • Prevention Actions (PA)
    • Effectiveness Checks (EC)
    • Close

If you are satisfied with the results you are getting, great, more success to you!  One of the misconceptions about today’s CAPA process is that it suggests the steps start and end cleanly and the process moves in a linear fashion. I have one of those theory vs. practice moments where what is supposed to happen “if you correctly follow this process” does not happen in reality.  That somehow it’s the fault of the lead investigator if it’s not progressing as described. 

Deviations and CAPA investigations are not theoretical. As the investigation unfolds and the true story is discovered, we learn quickly that the process is really iterative.  Often we find ourselves back at the beginning, rewriting the event details and fine tuning the problem statement or re-examining the root cause because a repeat deviation occurred.  Frustrated that the clock is ticking and the pressure to get ‘em closed is always in the forefront of the investigation, there is no time allowed to think outside this process; no room for creatively solving the problems.  The mere mention of creativity makes some compliance professionals break out in hives. 

I get it.  The intention of the process is to provide a structured approach to conduct the investigation and manage the related CAPA when assigned.  Otherwise, it’s too overwhelming and unwieldy.  We could go down a rabbit hole and never close an investigation or come up with totally out of the question corrections.  However, it seems to me that we’ve ingrained these steps into routine habit, that our root cause investigation meetings have also become check the boxes to get the CAPA closed on time!  Hence, our industry is still getting “inadequate investigation” observations.  Could there be a causal relationship?

So, if you seek something else, I am proposing a problem-solving model below that mirrors very closely with CAPA.  The following problem-solving steps give us the freedom to consider new ideas and new possibilities for solving the same repeated problems without violating our Deviation and CAPA procedures.

DEFINING THE PROBLEM

It sounds simple enough to do.  Collect the event details.  Answer the universal questions.  And lo behold, the problem should be evident.  SMEs unwittingly suffer from the curse of too much knowledge and as they learn more about the event, they tend to form early hypothesis about the problem especially if they’ve seen a similar event.  They tend to collect evidence about problems they are familiar with and understand.  Unfortunately for most, they seldom go beyond their current knowledge base. 

In the Qualifying SMEs as Investigators Program, nominees are asked to discuss and prepare their response to the following question: Is the problem different from the cause?  Initially, everyone nods his or her head yes with a baffled look as to why further discussion is needed.  In their break out groups, I ask them to prepare their group rationale. And then it happens, a few will inadvertently slip in the cause with the problem explanation and to their amazement they are stunned at hearing these words.  Alas, that’s the point.  Before you can assign the root cause, you need to know what the problem really is.

WHAT DO I ASK TO FIND THE CAUSES?

“To find the answer, ask a lot of questions first”.  The discovery of a non-conformance, deviation or discrepancy is usually triggered by an event and thus the investigation begins here.  In the event stage, not only are investigators collecting details they also interview the individual performers who are involved.  However, there are three levels of interaction that can influence performance:

  • The individual performer level
  • The process level
  • The systems level.

To identify the possible causes not just the obvious one, the investigation team needs to move beyond just the event triggers.

Moving from Events to Patterns and Systems Influences

If we keep the scope limited to the event and performers involved “as an isolated incident”, we often find that the easy way out (aka the fastest way to close the investigation) will often lead back in with more repeat deviations.  So, we need to expand the investigation to include other individuals and /or other similar incidents to look for a pattern or a reoccurring theme.

(c) HPIS Consulting, Inc.

As investigators, we need to keep asking questions that enable information sharing to drive a deeper understanding of what the individuals were/are experiencing when performing their part of the tasks within the process. 

Mapping the task/process helps SMEs to re-trace the steps or sub-tasks rather then rely on their memory (their expertise reservoir).  Many SMEs think they know the procedure and will often state a step from an older version of the SOP or will be surprised to see new changes in the most current version of the procedure.  In this group review, SMEs can look for weaknesses and offer their insights about the point of discovery. 

Within this process analysis/review, ask are there other procedures that have been implicated in other deviations?  What about the hand offs; the linkages between the procedures and other quality systems?  Are these clearly defined and understood or are these the basis of “miscommunication” causes between other groups?  The big question to ask is what would happen if we left it alone? What other systems would be impacted?  When investigation teams are given the time to explore (investigate) enough, they often find these system glitches just waiting to contribute to another deviation.

Problem Solving ala Systems Thinking

Peter Senge in his 1990 ground breaking book, The Fifth Discipline: The Art and Practice of the Learning Organization, describes a system as being perceived whole whose elements “hang together” because they continually affect each other over time and operate toward a common purpose.  There’s a pattern of interrelationships among the key components of the system, including the ways decisions are made.  Often invisible until someone points them out.

Systems thinking allows problem solvers to see the event in the context of the whole system not just individual performers or procedure.  They see patterns of recurring problems that deepen their understanding of the situation and leads them to reframe the problem, thus providing a new framework from which to generate an additional set of possible causes.

The next blog will continue describing the problem-solving model and explore other quality tools investigation teams can use. – VB

Subscribe (at the left side bar menu) now so you don’t miss part two of this blog.

(c) HPIS Consulting, Inc.

Why Do CAPAs Fail Their Effectiveness Checks?

When we start talking about deviations and CAPAs, we can’t help having a sidebar discussion about root causes and more specifically the rant about finding the true root cause.  I intentionally skipped that content in the previous blog.  It was my intention to kick off the new Deviation and CAPAs blog series by first looking at deviations by themselves.  And the learning opportunities deviations can provide us about the state of control for our quality systems.  From those deviations and ensuing CAPA investigations, I ask you this: are we improving anything for the long term (aka prevention).  Are we making any progress towards sustaining those improvements?

Corrective Actions Preventive Actions (CAPA) Steps

Let’s step back a moment and quickly review typical steps for CAPAs:

CAPA Components

The purpose of an Effectiveness Check (EC) is for verifying or validating that actions taken were effective and do not adversely affect product, device or process.  It goes beyond the statement in the investigation form to include a follow-up activity that closes the loop on the specific CAPA.  If an effectiveness check fails meaning the CA/PA was not effective or another deviation /nonconforming incident has occurred, we go back to the beginning and either start again or in most cases, we re-open the investigation.  The pressing question is why did the EC fail?  Almost instinctively, we believe that we did not find the true root cause.  Perhaps.  Was there a rush to close the investigation?  Probably.  Did the investigation team grab the first probable cause as the root cause because the “problem” felt familiar?  Maybe. Or is it a case of a fix that backfired into unintended consequences? Possibly. I will also suggest that the CA/PA may not have been aligned properly.

Ask these 3 questions about CA/PAs

  • Is the CA/PA Appropriate? The focus of this question is about the affected people.  What is the size of this audience? Is it mainly one person or groups of people?

Can the CA/PA be executed efficiently?  Is it for one site or multiple sites?

  • Is the CA/PA Economical? What budget is available?

Is it a “cheap” fix or a 3 – 6 month project? Or an expensive solution of more than 6 months and will need capital expenditure funding?

  • Is the CA/PA Feasible? The real question is about the timeline.

            Need it fast – within 3 months or

            Have time – don’t need until more than 3 months from now.

And then there is the unspoken 4th question – is the CA/PA “political”?  I experienced first hand what happens to CAPAs that are politically oriented.  Most of them failed their ECs.  Request “Can You Stay a Little While Longer”. The best CAPAs are the ones that map back to the root cause.

Introducing the HPISC CAPA Performance Chain

On the left hand side, you will recognize the 3 traditional tasks to complete.  After the EC is written, trace upwards to ensure that the EC maps back to the CA/PA and that the CA/PA maps back to the root cause; hence, the bottom up arrow.  On the right hand side are performance improvement activities that I use as a Performance Consultant (PC) to bring another dimension to the CAPA investigation, namely, Human Performance Improvement (HPI). 

Before I can write the root cause statement, I examine the “problem” also known as a Performance Discrepancy or an incident and I conduct a Cause Analysis that forces me to take a three tiered approach (the worker, the work tasks, the workplace) for the possible causes and not get bogged down in observable symptoms only.  The Performance Solution is more appropriately matched to the identified gap. In theory, this is what the corrective action(s) is supposed to do as well. During the performance solution planning, determination of success and what forms of evidence will be used happens with key stakeholders.  So that collecting the data happens as planned, not as an after thought, and the effectiveness is evaluated as discussed.    

What can we really control?

In RCA/CAPA meetings, I often hear about what management should do to fix the working conditions or how most of the operator errors are really managements’ fault for not taking the culture factor seriously enough.  While there may be some evidence to back that up, can we really control, reduce or eliminate the human factor?  Perhaps a future blog on understanding human errors will be released.

Management Can:

  • Design work situations that are compatible with human needs, capabilities and limitations
  • Carefully match employees with job requirements
  • Reward positive behaviors
  • Create conditions that optimize performance
  • Create opportunities to learn and grow professionally.

Clues for Failed Effectiveness Checks

One of the first activities to perform for a failed EC is to evaluate the effectiveness check statement.  I have read some pretty bizarre statements that challenge whether the EC was realistic to achieve at all. The conditions under which we expect people to perform must be the same as the conditions we evaluate them during an EC review.  So why would we set ourselves up to fail by writing ECs that don’t match normal workplace routines? What, because it looked good in the investigation report and it got the CAPA approved quicker?

Next, trace back each of the CAPA tasks to identify where to begin the re-investigation.  I also suggest that a different root cause analysis tool be used. And this is exactly what we did while I was coaching a cohort of Deviations Investigators.  Future blogs will discuss RCA tools in more detail. -VB

The Big Why for Deviations

As part of my #intentionsfor2019, I conducted a review of the past 10 years of HPIS Consulting.  Yes, HPISC turned 10 in August of 2018, and I was knee deep in PAI activities.  So there was no time for celebrations or any kind of reflections until January 2019, when I could realistically evaluate HPISC: vision, mission, and the big strategic stuff.  My best reflection exercise had me remembering the moment I created HPIS Consulting in my mind.

Human Performance Improvement (HPI) and Quality Systems

One of the phases for HPI work is a cause analysis for performance discrepancies.  The more I learned how the HPI methodology manages this phase the more I remarked on how similar it is to the Deviation /CAPA Quality System requirements.  And I found the first touch point between the two methodologies.  My formal education background and my current quality systems work finally united.  And HPIS Consulting (HPISC) became an INC.  

In my role of Performance Consultant (PC), I leverage the best techniques and tools from both methodologies.  Not just for deviations but for implementing the corrective actions sometimes known as HPI solutions.  In this new HPISC blog series about deviations, CAPAs, and HPI, I will be sharing more thoughts about HPISC touch points within the Quality Systems. For now, lets get back to Big Why for deviations.

Why are so many deviations still occurring? Have our revisions to SOPs and processes brought us farther from a “State of Control”? I don’t believe that is the intention. As a Performance Consultant, I consider deviations and the ensuing investigations rich learning opportunities to find out what’s really going on with our Quality Systems.

The 4 cross functional quality systems

At the core of the “HPISC Quality Systems Integration Triangle” is the Change Control system.  It is the heartbeat of the Quality Management System providing direction, guidance and establishing the boundaries for our processes.  The Internal Auditing System is the health check similar to our annual physicals; the read outs indicate the health of the systems.  Deviations/CAPAs are analogous to a pulse check where we check in at the current moment and determine whether we are within acceptable ranges or reaching action levels requiring corrections to bring us back into “a state of control”.  And then there is the Training Quality System, which in my opinion is the most cross-functional system of all.  It interfaces with all employees; not just the Quality Management System.  And so, it functions like food nourishing our systems and fueling sustainability for corrections and new programs.

Whether you are following 21CFR211.192 (Production Record Review) or ICHQ7 Section 2 or  820.100 (Corrective and Preventive Action), thou shall investigate any unexplained discrepancy and a written record of the investigation shall be made that includes the conclusion and the follow up. Really good investigations tell the story of what happen and include a solid root cause analysis revealing the true root cause(s) for which the corrective actions map back to nicely.  Thus, making the effectiveness checks credible. In theory, all these components flow together smoothly.  However, with the continual rise of deviations and CAPAs, the application of the Deviation /CAPA Management system is a bit more challenging for all of us.  

Remember the PA in C-A-P-A?

Are we so focused on the corrective part and the looming due dates we’ve committed to, that we are losing sight of the preventive actions? Are we rushing through the process to meet imposed time intervals and due dates that we kind of “cross our fingers and hope” that the corrective actions fix the problem without really tracing the impact of the proposed corrective solutions on the other integrated systems? Allison Rossett, author of First Things Fast: a handbook for performance analysis, explains that performance occurs within organizational systems and the ability to achieve, improve and maintain excellent performance, depends on integrated components of other systems that involve people. 

Are we likewise convincing ourselves that those fixes should also prevent re-occurrence? Well, that is until a repeat deviation occurs and we’re sitting in another root cause analysis meeting searching for the real root cause.  Thomas Gilbert, in his groundbreaking book, Human Competence: engineering worthy performance tells us, that it’s about creating valuable results without using excessive cost.  In other words, “worthy performance” happens when the value of business outcomes exceeds the cost of doing the tasks.  The ROI of a 3-tiered approach to solving the problem the first time, happens when employees achieve their assigned outcomes that produce results greater than the cost of “the fix”. 

Performance occurs within three tiers

So, donning my Performance Consulting “glasses”, I cross back over to the HPI methodology and open up the HPI solutions toolbox.  One of those tools is called a Performance Analysis (PA). This tool points us in the direction of what’s not working for the employee, the job tasks a/or the workplace. The outcome of a performance analysis produces a 3 tiered picture of what’s encouraging or blocking performance for the worker, work tasks, and/or the work environment and what must be done about it at these same three levels.  

Root cause analysis (RCA) helps us understand why the issues are occurring and provides the specific gaps that need fixing.  Hence, if PA recognizes that performance occurs within a system, then performance solutions need to be developed within those same “systems” in order to ensure sustainable performance improvement.  Otherwise, you have a fragment of the solution with high expectations for solving “the problem”.  You might achieve short-term value initially, but suffer a long-term loss when performance does not change or worsens. Confused between PA, Cause Analysis and RCA? Read the blog – analysis du jour.

Thank goodness Training is not the only tool in the HPI toolbox!   With corrective actions /HPI solutions designed with input from the 3 tiered PA approach, the focus shifts away from the need to automatically re-train the individual(s), to implementing a solution targeted for workers, the work processes and the workplace environment that will ultimately allow a successful user adoption for the changes/improvements.   What a richer learning opportunity than just re-reading the SOP! -VB

  • Allison Rossett, First Things Fast: a handbook for Performance Analysis; 2nd edition 
  • Thomas F. Gilbert, Human Competence: Engineering Worthy Performance
You might want to also read:

Did we succeed as intended? Was the training effective?

When you think about evaluating training, what comes to mind? It’s usually a “smile sheet”/ feedback survey about the course, the instructor and what you found useful. As a presenter/instructor, I find the results from these surveys very helpful, so thank you for completing them. I can make changes to the course objectives, modify content or tweak activities based on the comments. I can even pay attention to my platform skills where noted. But does this information help us evaluate if the course was successful?

Formative vs. Summative Distinction

Formative assessments provide data about the course design. Think form-ative; form-at of the course. The big question to address is whether the course as designed met the objectives. For example, the type of feedback I receive from surveys gives me comments and suggestions about the course.

Summative assessments are less about the course design and more about the results and impact. Think summative; think summary. It’s more focused on the learner; not the instructional design. But when the performance expectations are not met or the “test” scores are marginal, then the focus shifts back to the course, instructor/trainer and instructional designer with the intent to find out what happened? What went wrong? When root cause analysis fails to find the cause, it’s time to look a little deeper at the objectives.

Objectives drive the design and the assessment

Instructional Design 101 begins with well-developed objective statements for the course, event, or program. These statements aka objectives determine the content and they also drive the assessment. For example, a written test or knowledge check is typically used for classroom sessions that ask questions about the content. In order for learners to be successful, the course must include the content whether delivered in class or as pre-work. But what are the assessments really measuring? How much of the content they remember and maybe how much of the content they can apply when they return to work?

Training effectiveness on the other hand is really an evaluation of whether we achieved the desired outcome. So I ask you, what is the desired outcome for your training: to gain knowledge (new content) or to use the content correctly back in the workplace? The objectives need to reflect the desired outcome in order to determine the effectiveness of training.

What is your desired outcome from training?

Levels of objectives, who knew?

Many training professionals have become familiar with Kirkpatrick’s 4 Levels of Evaluation over the course of their careers, but less are acquainted with Bloom’s Taxonomy of Objectives. Yes, objectives have levels of increasing complexity resulting in higher levels of performance. Revised in 2001, the levels were renamed for better description of what’s required of the learner to be successful in meeting the objective. Take note, remembering and understanding are the lowest levels of cognitive load while applying and analyzing are mid range. Evaluating and creating are at the highest levels.

If your end in mind is knowledge gained ONLY, continue to use the lower level objectives. If however, your desired outcome is to improve performance or apply a compliant workaround in the heat of a GMP moment, your objectives need to shift to a higher level of reasoning in order to be effective with the training design and meet performance expectations. They need to become more performance based. Fortunately, much has been written about writing effective objective statements and resources are available to help today’s trainers.

Accuracy of the assessment tools

The tools associated with the 4 levels of evaluation can be effective when used for the right type of assessment. For example, Level 1 (Reaction) surveys are very helpful for Formative Assessments. Level 2 (Learning) are effective in measuring retention and minimum comprehension and go hand in hand with learning based objectives. But when the desired outcomes are actually performance based, Level 2 knowledge checks need to shift up to become more application oriented such as “what if situations” and scenarios requiring analysis, evaluating, and even problem solving. Or shift altogether to Level 3 (Behavior) and develop a new level of assessments such as demonstrations and samples of finished work products.

Trainers are left out of the loop

But, today’s trainers don’t always have the instructional design skill set developed. They do the best they can with the resources given including reading books and scouring the Internet. For the most part, their training courses are decent and the assessments reflect passing scores. But when it comes to Level 4 (Results) impact questions from leadership, it becomes evident that trainers are left out of the business analysis loop and therefore are missing the performance expectations. This is where the gap exists. Trainers build courses based on knowledge / content instead and develop learning objectives that determine what learners should learn. They create assessments to determine whether attendees have learned the content; but this does not automatically confirm learners can apply the content back on the job in various situations under authentic conditions.

Performance objectives drive a higher level of course design

When you begin with the end in mind namely, the desired performance outcomes, the objective statements truly describe what the learners are expected to accomplish. While the content may be the same or very similar, how we determine whether employees are able to execute post training requires more thought about the accuracy of the assessment. It must be developed from the performance objectives in order for it to be a valid “instrument”. The learner must perform (do something observable) so that it is evident s/he can carry out the task according to the real work place conditions.

To ensure learner success with the assessment, the training activities must also be aligned with the level of the objectives. This requires the design of the training event to shift from passive lecture to active engagement intended to prepare learners to transfer back in their workspace what they experienced in the event.   This includes making mistakes and how to recognize a deviation is occurring. Michael Allen refers to this as “building an authentic performance environment”. Thus, trainers and subject matter experts will need to upgrade their instructional design skills if you really want to succeed with training as intended. Are you willing to step up and do what it takes to ensure training is truly effective? – VB

 

Allen,M. Design Better Design Backward, Training Industry Quarterly, Content Development, Special Issue, 2017, p.17.

Are you doing the Training 2 Step?

Has this situation ever happened to you? You are in a root cause meeting and the *CAPA investigator is conducting an interview with the primary individual involved with the discrepancy. When questioned why did this happen, he shrugs first and then quietly mumbles I don’t know. When pushed further, he very slowly says I just kind of went brain dead for a moment.  And then silence.

While that may be the honest truth, the investigator must resist the temptation to label it as Operator Error and explore possible causes. One of my favorite root cause analysis tools for “I Don’t Know Why” response is to use the Fish Bone Diagram also known as the 4 M’s diagram. This tool provides a structured focus to explore many possibilities and not just stop at the first plausible cause; such as Operator Error. Aptly nicknamed, the 4 M’s are Man, Machine, Methods and Materials.   When the results of this exercise points to a training or operator related issue, don’t stop at “operator error –> retrain”.

Consider for a moment, what this retraining session would look like. Will re-reading the procedure be enough to “jog his memory”? Will repeating the procedure be a good use of precious time when s/he already knows what to do? More than likely it won’t prevent “going brain dead” from happening again. Instead, do the HPISC Training 2 Step:

Step 1 – confirm the results of the gap analysis

What task, what step(s) or actions are in question?

Step 2 – address why the original training did not transfer back to the job.

Using the 4 M’s diagram as the framework, explore Man, Machine, Methods and Materials questions with regards to the training this operator receives. See diagram below. The full set of questions can be found in the eBook Training Cause Analysis.

4msfortrca

Is this really worth it?

I think it is. Conducting these 2 steps will accomplish two objectives.   It will provide further evidence that some kind of training is needed. And it will highlight what areas are in need of revising either for the performer, the training program or course materials. Yet, there are some who will resist this added work because it’s easier to find blame than to uncover the cause. Fixing the true root cause could trigger a re-validation of the process or an FDA filing if it’s a major process change.   Why create more work? Isn’t it easier to just retrain ‘em? No, not really. Finding the true root cause is the only effective way of eliminating many of the costly, recurring problems that can plague manufacturers.

But what if

Some folks will push back with the excuse – “this never caused a problem until now”, so it must be the operator’s fault! This may be the first time it was discovered but that does not mean the procedure is 100% accurate. Often, experienced operators know how to work around an incorrect step and don’t always report a misstep in the procedure while a less savvy operator follows the procedure and causes the non-conformance to occur. See Sidebar SOP Logic Rules. Is the procedure difficult, lengthy or requires weeks to become proficient let alone qualified? Was the qualification routine or performed as a simulation? Was the procedure written with support from a lead operator or qualified trainer? Did the draft version undergo some kind of field test or dry run prior to release? And the classic situation, are proposed changes hung up in change control awaiting effective release?screen-shot-2016-10-19-at-4-33-39-pm

Understanding Why Human Errors Occur

Industry practice is evolving to explore why people make the decisions they do by looking at the Organization’s systems. It’s usually a poor decision made somewhere in the error chain. We must believe that the person who made the poor decision did not intend for the error to occur. As part of CAPA investigations, we need to explore their physical environment as well; the conditions under which they make those decisions. The Training Program Improvement Checklist can be requested using this link to capture your findings.

If you are going to spend time and money on training, at least identify what the gap is; fix that and then “train” or provide awareness on what was corrected to prevent the issue from re-occurring again.   That is after all, the intention of *Corrective Action Preventive Action investigations. -VB

You may to explore these other library gems:

BandAidsPicture1
Why the Band Aids Keep Falling Off

OpErrorPicture1

The Silver Bullet for Performance Problems Doesn’t Exist

Oh but if it did, life for a supervisor would be easier, right? Let’s face it, “people” problems are a big deal for management. Working with humans does present its challenges, such as miscommunications between staff, data entry errors, or rushing verification checks. Sometimes, the task at hand is so repetitive that the result is assumed to be okay and gets “a pass”.  Add constant interruptions to the list and it becomes even harder not to get distracted and lose focus or attention to the detail.

Actual behavior vs. performing as expected

In their book, Performance Consulting: Moving Beyond Training, Dana Gaines Robinson and James C. Robinson describe performance as what the performer should be able to do. A performance problem occurs when the actual behavior does not meet expectation (as in should have been able to do).   Why don’t employees perform as expected? Root cause analysis helps problem solvers and investigators uncover a myriad of possible reasons.   For Life Sciences companies, correcting mistakes and preventing them from occurring again is at the heart of CAPA systems (Corrective Actions Preventive Actions).

A closer look at performance gaps

Dana and James Robinson conducted research regarding performer actions and sorted their results into three categories of obstacles:

  • Conditions of performers
  • Conditions of the immediate managers
  • Conditions of the organization

A checklist for common Performance Causes  – scroll down for the Tool.

But, weren’t they trained and qualified?

Hopefully, employees are trained using an approved OJT (On the Job Training) Methodology in which they are shown how to execute the task and then given opportunities to practice multiple times to become proficient. During these sessions, they are coached by Qualified Trainers and given feedback on what’s right (as expected) and given specific instructions to correct what’s not right with suggestions for tweaking their performance so that their final performance demonstration is on par with their peer group. At the conclusion of the qualification event, employees must accept that they now own their deviations (mistakes) from this point forward. So what gets in the way of performing “as they should” or in compliance speak – according to the procedure?

Is it a lack of knowledge, skill or is it something else?

The Robinson’s explain that performance is more than the training event. It’s combination of the overall learning experience and the workplace environment that yields performance results. Breaking that down into a formula per se, they suggest the following: learning experience x workplace environment = performance results.

The root cause investigation will include a review of training and the qualification event as well as a discussion with the performer.

  • Is it a lack of frequency; not a task often performed?
  • Is it a lack of feedback or delayed feedback in which the deviation occurred without their awareness?
  • Is it task interference?

The work environment includes organizational systems and business unit processes that together enable the performer to produce the outcomes as “expected”.   These workplace factors don’t always work in perfect harmony resulting in obstacles that get in the way of “expected” performance:

  • Lack of authority – unclear roles, confusing responsibilities?
  • Lack of time – schedule conflicts; multi-tasking faux pas?
  • Lack of tools – reduced budgets?
  • Lack of poorly stored equipment/tools – lost time searching?

Isn’t it just human nature?

Once the root cause investigation takes on a human element attention, it’s easy to focus on the performer and stop there.   If it’s the first time for the performer or first instance related to the task, it’s tempting to label the event as an isolated incident. But when it comes back around again, it becomes apparent there was a “failure to conduct an in-depth investigation” to correct and prevent. Not surprisingly, a push back of “Operator Error as Root Cause” has forced organizations to look deeper into the root causes involving Humans.

Who’s human nature?

Recall that one of the categories of the researched obstacles was “conditions of the immediate managers”. This makes managers uncomfortable. With so much on their plates, managing a people performance problem is not what they want to see. A silver bullet like a re-training event is a nice activity that gets a big red check mark on their to-do list. However, Robert Mager and Peter Pipe, in their book, Analyzing Performance Problems, provide insights to managing direct reports that may lead to unintended consequences. A brief list can be found here – scroll to Tool: Performance Causes.  (It’s not always the performer’s fault.)

It takes all three to correct a performance problem

soln-people-performance-problemThe third category of researched obstacles clustered around “conditions of the organization”.  I’ve already discussed task interference above. To suggest that organizations are setting up their employees to fail is pushing it just a bit too far.   So I won’t go there, but it is painful for some leaders to come to terms with the implication. In order to prevent issues from reoccurring, an examination of the incidents and quite possibly a restructuring of systems have to occur, because automatic re-training is not the only solution to a “people performance problem”. –VB

Robinson DG, Robinson JC. Performance Consulting: Moving beyond training. San Francisco: Berrett-Koehler; 1995.

Mager R, Pipe P. Analyzing performance problems. Belmont: Lake Publishing; 1984.

Tired of repeat errors – ask a Performance Consultant to help you design a better corrective action

In this last “Making HPI Work for Compliance Trainers” series, I blog about one of the biggest complaints I hear over and over again from Compliance Trainers – management doesn’t really support training.  It’s hard to ask for “more of the same” even though you know your programs are now different.  In previous blogs, I shared why management hasn’t totally bought into the HPI methodology yet. See the blog Isn’t this still training?

 

Given the constant pressure to shrink budgets and improve the bottom line, managers don’t usually allow themselves the luxury of being proactive especially when it comes to training.  So they tend to fall back on quick fix solutions that give them a check mark and “clears their desk” momentarily.  For the few times this strategy works, there are twice as many times when those fixes back fire and the unintended consequences are worse.

 

In the article, “Why the Band Aids Keep Falling Off”, I provide an alternate strategy that emphasizes moving away from events-only focus to exploring the three levels of interaction that influence performance: individual performer, task/process, organizational quality systems.  These same three levels are where performance consultants carry out their best work when supported by their internal customers.  The good news is that the first step is the same; it begins with a cause analysis.  See the blog Analysis du jour  for more thoughts on why these are essentially the same approach.

 

The difference is that the corrective action is not a reactive quick fix but a systems approach to correcting the issue and preventing it from showing up again.  System based solutions are the foundation of many HPI/HPT projects that require cross functional support and collaborative participation across the site / organization.  And this is where a PC needs support from senior leaders.

 

We wrap up this series here and introduce the next series – Gaining Management Support – where I blog about credibility, trust, and access and how these 3 concepts impact relationship management.

What’s the difference between Trainers and Performance Consultants: Aren’t they one and the same?

After 10 years of HPI consulting, I’m still being asked this question a lot.  In the blog, “Isn’t this still training”, I shared why it still looks like training.  Alas, this blog brings us to the beginning of another series within the Human Performance Improvement (HPI) arena.  I’m calling it “HPI: Making it Work for Compliance Trainers”. So, in this blog, I will expand upon 6 elements of comparison to illustrate the difference between the two and the depth of impact one has over the other. 

FOCUS

Training addresses the learning needs of employees.  Various definitions include closing the knowledge and skill gap of what they know now and what they know afterwards.  It’s built on the assumption that the cause of the gap is a lack of knowledge and skill.  Performance Consulting addresses business goals and performance needs of the affected employees.  Training is just one of the possible solutions that can be used; not the only one.

OUTPUTS

A training solution delivers a structured learning event.  Whether it is a classroom or virtual or self -led, the event itself is the end goal.  Performance Consulting or HPI projects are implemented to improve performance.  The end goal is not about the solution such as the specific HPI Project, but rather a positive change in performance that leads to the achievement of the business goal.  The endpoint is “further down the road”.  So it takes longer to produce the results.

ACCOUNTABILITY

With training, the Trainer is held accountable for the event.  In a lot of organizations, there is an implied but not spoken accountability for the results back on the job.  But without the proper systems and support mechanisms in place, many Trainers get “blamed” for training transfer failure.  Here’s the big difference for me.  Performance Consultants (PCs) partner with their internal customers, system owners and business leaders in support of the business goals.  The accountability for improved performance becomes shared across the relationships.

ASSESSMENTS

Trainers typically conduct a needs analysis to design the best learning “program” or course possible.  PCs conduct performance analyses gaps assessments to identify causes that can go beyond knowledge and skills.  See the blog, “Analyses du jour”.

MEASURES
Trainers
very often use course evaluation sheets as a form of measurement.  In the Compliance Training arena, knowledge checks and quizzes have also become the norm.  PCs measure the effect on performance improvement and achievement of business objectives.

ORGANIZATIONAL GOALS

This is another key differentiator.  Training is viewed as a cost typically.  Compliance Trainers are all too familiar with the phrase, “GMP Training is a necessary evil”.  PCs become business partners in solving performance gaps and accomplishing organizational goals.

For a visual graphic and expanded description of these 6 elements, you can request HPISC white paper, Why They Still Want Training?

I also recommend that you request the HPISC white paper, Performance Analysis: lean approach for performance problems.