This case study is from a presentation at ATD 2018 titled “How to Improve and Measure Sales Training ROI” with Michael Carpenter, Sales Readiness Information and Technology at Tableau.

Tableau, a Seattle-based developer of business intelligence software, uses sales learning and coaching tools to boost sales performance and evaluate the impact of training on behavior and ROI.

Convincing Tableau’s sales readiness team to adopt a sales learning and coaching platform was a “really easy sell” for Michael Carpenter.  

Carpenter, the company’s Manager of Sales Readiness Information and Technology, saw that he could use a tool like this to modernize learning at the company, and recognized the opportunity to better track the outcomes of his learning programs with it.  His ultimate goal was to improve learning at the company by measuring not only reactions to training programs and assessment results, but to measure behavior change and actually quantify the training’s impact on revenue.

But he knew he needed to win over the sales team first for it to take off.  Positive feedback after the first exercises using Allego signaled to Carpenter that he was onto something.

“Previously, it had always been like pulling teeth to get people to actually participate in the live objection handling exercises with their peers,” Carpenter said in a presentation at the ATD 2018 International Conference & Expo. “But when we used the videos to preload people’s brains with things they could say — ideas they could build on — we had the best session ever.”

Wanted: Hard Data that Training Actually Works

However, Carpenter knew he’d need more than anecdotal evidence to persuade Tableau to put Allego to work for its roughly 1,000 sales reps worldwide.

“I couldn’t just stroll into my CFO’s office and say, ‘We had a great objection handling class. Give me money.’ At Tableau, we’re a visual analytics company. We care about data above all else.  But I saw an opportunity to collect rich data from the coaching exercises… a way to creep our way up the Kirkpatrick pyramid.” The Kirkpatrick pyramid is a popular four-level training evaluation model that helps trainers measure the effectiveness of their programs in an objective way.

Carpenter wanted to move beyond Levels 1 and 2 (measuring learners’ reactions to training as well as simple assessment results) to gain deeper insight to the effectiveness of his programs by actually measuring behavior change and quantifying the training’s impact on revenues (Levels 3 and 4 of the pyramid).

“Then we can start to make the case for how effective these tools are.”

Historically, reaching Levels 3 and 4 has been costly and time-consuming. The biggest challenges are identifying which outcomes are most closely linked to the training, and developing effective ways to measure them over the long term.

Flash Drills as a Data Mine

For Carpenter, one source of results-oriented data is Allego’s Flash DrillsⓇ, which he uses to push out 30-second microlearning exercises to Tableau reps on an automated daily schedule. In addition to helping reps reinforce their newly acquired knowledge and skills, the drills help Carpenter mine valuable data.

“We track each user’s mastery score over time. This has a couple of benefits. Number one is gamification. I can build a dashboard from Allego data that breaks down mastery by office location and manager. All of a sudden, the manager of a poor-performing team might say, ‘Hey guys and gals, let’s ramp up the Flash Drill, we look kind of bad.’  It’s … a soft stick that people have actually appreciated, and it also helps us determine which questions are harder than others,” Carpenter said.

“This is just [Kirkpatrick pyramid] Level 2. We’re reading people. We’re understanding where they’re having issues with questions. But this has been a big win for us.”

Objection Handling in Sales

To evaluate behavior (Level 3) and enhance sales performance, Carpenter and his colleagues focused on objection-handling responses. A member of the sales intelligence team records a short (six or seven-minute) video detailing various responses reps could give to certain objections in a competitive sales situation. The reps can then practice and record these responses, sending them back to the team for evaluation, scoring and feedback.

Once a sufficient number of responses roll in, Tableau ranks each rep’s capabilities to see which individuals, regions and office locations are struggling.

“For the first time … I can actually see behaviorally how people are articulating the messages we are trying to teach. We can start to see where we need to focus more training, or whether our training is even hitting the mark.”

An Experiment to Measure the ROI of Sales Training

To reach Level 4 of the pyramid, a quantitative assessment of Allego’s impact on sales training ROI, Carpenter and his team conducted an experiment with a small sample of sales reps.

Although the experiment’s results are proprietary, Carpenter shared that, compared to reps who did not receive the training, the 20 or so trainees produced a statistically significant gain in sales revenue. The improvement rate wasn’t large, but “If you play a mental math game, [expanding these results] to over 1,000 sellers, that’s a lot more money over time. You can really blow these numbers out and start to make compelling cases.”

“I’m not ready to go to my CFO yet and say, ‘Look at this. We made all this money’… but we’re going to watch those numbers carefully in comparison to what we did before.”

Carpenter is finally on the road to designing his training programs for ROI using insights gleaned from hard data. “We have plans to roll out these types of focused learning programs to more teams throughout the year. We’re going to start to build a nice set of data. Sure, we might find that it doesn’t move the needle, but based on past results I’ve got a sneaking suspicion that it will.”

For a deeper dive into measuring ROI on sales learning and coaching platforms, check out this article in ATD on how Tableau uses data to track the effectiveness of sales training and our ebook 4 Ways to Measure the ROI of Sales Training Technology

 

Ⓡ Flash Drills is a registered trademark of Allego Software.