Michaels & Associates

According to Paul Kearns (Evaluating the ROI from Learning), the first step involves deciding where the Learning & Development (L&D) teams, business units and enterprise fall on his six stage Learning Maturity Model (or LMM).

There can be many L&D teams in a single enterprise and each can exist in a varying stage of maturity. What is the reference to "maturity"? Think about these scenarios:

• You receive an email stating a department needs a five-hour course on X Operation in 30 days. Without much planning, you quickly throw together a presentation or something similar. In this situation, your reactive mode places you in Stage 1.


• You have worked toward making the line managers aware that L&D follows normal business processes equivalent to theirs, and that a good quality product might result from adequate budgeting and time. In this situation, you are likely at Stage 3.


• You attend meetings with business unit executives and become aware of initiatives that will require segments of employees to learn new skills. You present a strategy with a budget, an achievable deadline, and a commitment to prove the strategy will execute as planned. The strategy is then approved. In this situation, you are in Stage 5, or a true learning partner.


• The executive team knows about your learning strategies and helps to steer information to you so you can include corporate strategies into your work. In this situation, you are at Stage 6. You could also be a learning consultant who is teaching the entire organization how to be a learning corporation. Studies show that learning corporations are more profitable than those that don't invest in and support learning across the organization.

Where on the Learning Maturity Model is your L&D team?
Where is your business unit?
Where is your corporation?

These questions are important to consider and here's why. If you are at Stage 1, it can be almost impossible to measure the effectiveness of your training because your circumstances are not conducive to constructing instructionally sound, engaging training. Learners are often predictably disappointed and the results of Stage 1 assessments or "smile sheets" will consistently reflect this. At this point, time is better invested in moving your L&D team further along the maturity scale rather than focus on measuring effectiveness.

As you evaluate your situation, consider how your L&D team works internally, with the other business units and within the enterprise. Is your team more likely to perform in a strategic manner with all key stakeholders actively involved? If so, you already far along on the maturity scale and you're more likely to obtain stronger measurement results.

If L&D is lower on the scale and training development occurs more frequently in a reactive approach, there may be a little more work to do to strengthen the reliability of your results. The good news is with consistency of effort, you can make tremendous strides in a short period of time.

For example, let's say your L&D team is at Stage 1. What things can you do incrementally to move forward in maturity? Here are some strategies to try:

Strategy #1

Begin with a through training needs analysis in all business units the L&D unit supports. A good training needs analysis involves the process of defining on-the-job performance requirements, the gaps between the requirements, and what employees are presently doing.

Strategy #2

Approach every training request with questions about what the requester is trying to achieve. Turn your questions into an analysis of whatever depth you can manage to help guide the development of the deliverable.

Strategy #3

Make allies of the line managers and ask them about their objectives, goals and strategies. They can give you current performance metrics for their employees. Since they are your customers, it's important to pay attention to what they want to achieve, and plan for how you might help them.

Strategy #4

Talk to your Human Resources department to obtain job descriptions and the competencies required to perform those jobs. Compare these with what the line managers are telling you.

Strategy #5

Analyze your information sources, define the opportunities you find, and eliminate the non-training issues. You may want to report non-training issues (process or systemic) as they come up and make recommendations regarding them, but your focus is to address those training solvable issues.

Strategy #6

Develop an L&D strategy for managing training development. Begin discussing with your ally-managers how to plan for training, and what you need to be successful so you can make them successful.

Strategy #7

Develop a replicable process for creating instructionally sound training. Paul Kearns recommends the Deming Cycle: Plan, Do, Check, Act. Another well-known and effective process is ADDIE (or Analyze, Design, Develop, Implement, Evaluate). Either process model is fine, as long as you find it replicable, consistent and defendable within your organization. Keep in mind that line managers should respect that you have a process that works.

Strategy #8

Practice saying "no" in a nice way. Example: "It's possible to get you a presentation in a few hours, but for training that boosts our metrics and ensures your employees learn, I'm going to need to go through our process to define your objectives and make sure the training meets them. Would you like to meet on Monday to get started? We could begin by determining what you'd like your employees to be able to do at the end of this training."

Other steps people use to increase the accuracy and dependability of training measurement results include:

• Determining their definition of Return On Investment (or ROI). Is it the same as the business unit manager's or the Chief Financial Officer's?


• Learning more about how to address the variability in results that learning professionals can encounter. For an excellent discussion on these variables, see Ron Drew Stone's CLOMedia article, "ROI is Like a Box of Chocolates".


• Considering how to isolate the results of training from outside variables, such as the economy, a new marketing campaign, a product release, personnel adjustments and so forth. Any one or all of these factors can distort and invalidate ROI calculations.


• Ensuring the training program assessment addresses the needs of business unit managers, who are in pursuit of overall corporate goals.

Although these steps are valid, the challenges and variables that surround ROI can be great. It can often be an easier (and better) task to prove that the training your L&D team develops has value instead.

Next time: We'll discuss the concept of Proving Learning Value and how you can ensure your team is a true business partner within your organization and make current training investments more effective and efficient.

In the meantime, please join the conversation in the comments section of this article or feel free to send us an email. We'd love to hear your thoughts!

~ ~ ~ ~ ~ ~ ~ ~ ~ ~
Continue reading:
Measuring Training Effectiveness: Back to Basics (Part 1) →
Measuring Training Effectiveness: Formative versus Summative Assessments (Part 2) →
Measuring Training Effectiveness: Proving Learning Value (Part 4) →