Can a business get away with only using feedback surveys rather than investing the time and resources into a fully featured mystery shopping program? The temptation is understandable, since a manager might think he can just put a few questions into an online form and pass a hyperlink out to the customers to get those answers. That would be easier, right?
But there’s a fundamental flaw in customer feedback surveys: these surveys are almost exclusively taken by the kinds of people who are either happy or angry enough to take the time. Even when you attempt to incentivize further participation through giveaways or coupons, you’ll find the needle barely moves because most people are simply too busy to bother unless their experience was truly remarkable.
Have you ever heard the term “survivorship bias?” Put simply by Wikipedia, “Survivorship bias or survival bias is the logical error of concentrating on entities that passed a selection process while overlooking those that did not. This can lead to incorrect conclusions because of incomplete data.”
Learning from History
An important historical example of this problem comes to us from World War II, when the U.S. Navy was trying to find ways to improve the survivability of their bomber aircraft in the Pacific theater. When analyzing the returning aircraft, the U.S. Navy got the impression that the Imperial Japanese Navy was targeting the bombers’ wingtips and center fuselages since that where the vast majority of damage occurred.
Their response was to reinforce the armor on those hotspots in the hopes of offering better protection. But these modifications did not improve survivability, and may have actually made it worse. For one, the added armor reduced speed and maneuverability, but just as crucially, this decision failed to account for all of the bombers that didn’t return.
Consider this: the surviving bombers had the majority of their damage concentrated on the wingtips and center fuselages. But what about the ones that didn’t survive? The U.S. Navy failed to account for this because they were only considering the “entities that passed a selection process”; that is, the returning bombers and not the ones that were lost to enemy fire.
Long story short, the correct areas to reinforce were not the wingtips and center fuselages but actually the engines and cockpits—because aircraft that sustained significant damage to those areas never made it home at all. That might seem obvious now in retrospect, but very intelligent people with lots of combat experience overlooked this.
How Mystery Shopping Completes Your Data
Don’t get us wrong: feedback surveys do have their uses. Because they’re biased towards spotlighting particularly glaring examples, good and bad, they do still represent training opportunities—or even second chances for managers to reach out to aggrieved customers to fix problems. But most of your customers do not have experiences on the extreme ends of the spectrum. In order to gauge average experiences—that is, the ones that nearly every customer has every day—you need consistent evaluation from people whose sole mission is to do just that.
Mystery shoppers are recruited from normal consumers to ensure your business receives real-world feedback. Regular consumers have all had both good and bad experiences with restaurants, retail stores, car dealerships, doctor’s offices, and more, and they know what they like and don’t like. So, when they come to your business and go through your customer experience processes, the feedback they have for you will absolutely make sense in full context without the limitations of feedback surveys that mostly account for extreme examples.
Ready to learn more? Drop us a line and let’s develop a program that makes sense for your business and for your customers.
In the meantime, Merry Christmas!
