‘Would you recommend this show to a friend?’

It seems like we’re bombarded with customer surveys these days. Where are these surveys coming from — and do they work?
Guests
Fred Reichheld, Bain fellow and founder of Bain’s Loyalty practice, which helps companies achieve results through customer and employee loyalty. Creator of the Net Promoter system of management.
Brad Anderson, president of products and engineering at Qualtrics, one of the largest survey data companies.
Also Featured
Clayton Mantell, General sales manager at Ramsey Auto Group in New Jersey.
Transcript
Part I
DEBORAH BECKER: This is On Point. I’m Deborah Becker, in for Meghna Chakrabarti.
Today we want to talk about those seemingly ubiquitous customer surveys. Doesn’t it feel as if we’re flooded with them? The grocery store, the doctor, the pharmacy, the rideshare, the online retailer — why are so many companies asking, “How did we do? How likely are you to recommend us to a friend?” So we surveyed some On Point listeners:
(MONTAGE)
DREW BOURRUT: Asking me to fill out a survey gets you one star.
JANE MOONEY: I’ve grown so weary of all of these surveys and customer satisfaction pushed in front of me, whether it be by text or email or on my receipt.
UNNAMED CALLER: It’s always linking to something else and more questions.
BRAD: Rate your service on a 1 through 10. Fast checkout, was it friendly associates, was it we got the right stuff in stock. We’re always told to tell the customers to rate us a 10.
JONATHAN EISENHOWER: Most people don’t fill them out. I think we on average, had maybe a 1% return rate on that.
HOWARD TURNER: I do occasionally fill out the customer surveys. Normally it’s when I have something to complain about.
DOUGLAS BARNHARDT: All it takes is one bad survey to ding your score for that week.
JEYSA WRIGHT: It basically ranks your servers from best to worst.
CLAYTON MANTELL: If you’re getting a 10 or a nine, great. Essentially anything at like seven or below is a zero.
JOSEPHINE GIAIMO: I don’t believe my information is going to actually improve the product or service.
BRAD: They probably gave us a low rating because we didn’t have the eggs that they wanted. That’s something we can’t control. If there’s no eggs in the warehouse for us to get to the store, there’s nothing we can do.
BECKER: Those folks called us from Virginia, Wisconsin, California, Indiana, Washington, Georgia, Colorado, and New Jersey. So, where are these surveys coming from? Do they work? And what is happening with all that survey data?
Joining us is Fred Reichheld. He’s a fellow at Bain and founder of Bain’s Loyalty Practice. Fred, welcome to On Point.
FRED REICHHELD: Nice to be here, Deborah.
BECKER: So I understand that we should be thanking you, blaming you maybe, for the question, “How likely are you to recommend us to a friend?” You came up with that survey question for businesses, is that right?
REICHHELD: I did. I invented it. You called them ubiquitous. I would call them annoying.
BECKER: (LAUGHS)
REICHHELD: But in all honesty, they started out 20 years ago or more very effectively. Because I saw a company, Enterprise Rent-A-Car, who had grown to be the largest in the world by treating customers better than their competitors. They’ve been a great success story.
And one of the things they used that I admired was a very short survey, two questions. And they didn’t send it to billions of customers. They got just enough customers at each branch to have a reasonable response that’s trustworthy. And they didn’t design it as a big market research project for corporate to learn more about their customers. They designed it for each individual branch to learn how to get better. It was genius.
And I just took that and used it as a model to build something I called Net Promoter Score, which is a simple way almost any business can measure, “Of all the lives I touch, how many are enriched, how many are diminished?” And that’s a pretty worthwhile goal.
BECKER: And basically, Net Promoter Score means, will you promote to someone else? Will you recommend?
REICHHELD: One question. “How likely would you recommend this to a friend, zero through 10?” Some people use a five star system, same idea. And, you know, when you get a 10, at least in the old days when people could be honest without fearing that their service provider would be fired or dinged in their bonus, a 10 meant that you’ve made my life better. I would recommend you to a friend, which is the highest standard you can have.
BECKER: Mm-hmm. But how do you know that, I’m wondering where you get the data, or where maybe Enterprise got the data? How do you know that when someone is saying they would recommend that that is accurately reflective of the quality and service of the product that they’re being asked about? They could be talking about something completely different. How do you zero down on exactly what the customer is talking about when they might be a promoter of that business?
REICHHELD: Well, Enterprise actually didn’t use likelihood to recommend as their question. But I wanted to test — you know, this was the young Fred, economist, math and statistics major — I tested which question best predicted future customer behaviors that would have indicated that their life was enriched. They expanded their purchases. They made referrals to friends.
And we tested, “How satisfied are you? Would you recommend to a friend? Does this company deserve your loyalty? Do you intend to return?” All the classics. But I knew people’s time is precious. I just wanted one question as the scoring question, and it turned out likelihood to recommend was the best predictor of those real behaviors. And that’s why we picked that one question.
BECKER: And you think it is reflective then of really customer satisfaction, which means what for a business?
REICHHELD: Well, the highest standard is that your life has been so enriched by an experience that you want to share that with a loved one. Thus the recommendation. And a recommendation is an act of love. I want my friend, my colleague, to have this wonderful experience. I trust this brand, this company, is going to do it for them. That’s pretty awesome. It also turns out to be that’s what drives company success. But since companies don’t measure it very well, they don’t see it. They think accounting measures success which actually takes them down the wrong path more often than not.
BECKER: But also there could be misinterpretations of some of the answers, right?
REICHHELD: Of course. Think about Amazon with their rating system. They want you to give a star rating, but is it about the delivery experience or the shopping experience or the product itself or the service that did or didn’t get applied to that product? You know, it gets fuzzy.
And I’ll take my car dealer who is just always begging for 10s because somehow I know that they must fire the bottom 10% of salesmen based on their how many 10s they get. “Only a 10 is a passing grade.” How many times have I heard that? The reason I’m not happy with my car is the battery in the key fob went bad after three weeks of this brand new car and left me stranded in a parking lot. It had nothing to do with the salesman, but he’s still afraid that I’ll give a low score that’ll affect his future.
BECKER: And why do you think it is that we are so inundated, annoyed, as you might say, with these surveys at this particular time? Why are so many companies deciding that now is the time to do them? What does it do for the businesses involved?
REICHHELD: Well, Aristotle understood this hundreds and hundreds of years ago, that there is a tragedy of the commons. When there’s a free good that everyone gets to use or abuse, eventually it gets abused. And whether it’s the common, the village green where they, you know, graze the cows or the forest that gets cut down if everyone has rights or fishing. It’s a common phenomenon. And that’s what’s happened with these surveys. Customer goodwill has been abused by people misusing surveys. And we’re now at the stage where it needs a little bit of reinvention and rethinking.
BECKER: Mm-hmm. But so, but why? What was the point of using these surveys to begin with? Aristotle didn’t have the internet.
REICHHELD: Well, I hope they read my book! (LAUGHS)
BECKER: (LAUGHS)
REICHHELD: I made a pretty strong case that companies who do the best job of earning the loyalty of their customers are the ones that grow profitably, make their employees proud and their investors rich. There’s a really strong economic argument that underlies this idea. And I think leaders intuitively understand that.
You know, you don’t want to lose customers at a rapid rate and have to refill that leaky bucket. That’s terrible. And the notion that, gosh, a referred customer is a higher quality customer, they stay longer, I didn’t have the acquisition cost, that, you know, more referrals makes everyone rich. So there’s common sense underlying it as well as my more Bain rigor analysis.
But I think the reason it’s falling apart is because it’s being misused. People are linking scores to employee bonuses and therefore the employee doesn’t really care whether you are happy or not. They just care about your score.
BECKER: Right, right. And so then how do they need to be restructured? What would be your main recommendation in terms of restructuring them? Do less of them? Or not link them to employee compensation? Or what do we do?
REICHHELD: I would do way less. Well, you, in one of the interviews that you had played a little bit earlier on the show said only 1% of people respond to these surveys. That’s about right. So 99% of survey requests are being ignored. Now that has a cost. Aristotle knew this cost. The customer is saying, “This is an inappropriate waste of my time. It’s a communication that’s wasting my energy.” It’s bad. It weakens the brand.
But 1% are responding now. Now, are these 1 percent typical of your customer base so you can actually learn from them reliably? I don’t think so. The other, you know, you asked what do I do? Yes, far fewer surveys. They’re obviously failing. And what do you replace it with? I need to know whether I’m enriching a customer’s life. Am I earning their loyalty? Will they come back for more and bring their friends? That’s the core engine that drives prosperous business growth.
I think in today’s digital world, it’s simple. And this is a case I make in Winning on Purpose, my last book: Just track repeat purchase and referrals. Back for more and bring your friends. Those are signals that are in your own accounting system — well, at least back for more is in your accounting system. But referrals can be tracked today. And use that as the signal of whether you made the customer happy or not.
Then if you need surveys or outreach with conversations and so forth, great. But much of the learning can be done just with analyzing data that already exists in your system.
BECKER: But I still don’t understand — and we’ve only got about a minute before we take a break here — but why with a 1% response rate, so many businesses have decided that this is the vehicle that they want to measure their customers by.
REICHHELD: They’re desperate to hold someone accountable for delighting customers. The CEOs know if they don’t make customers happy, their company won’t grow. So they’ve delegated down into the silos of the organization this survey methodology that they think will hold those people accountable to turning customers into happy customers, loyal customers.
Part II
BECKER: Also with us right now is Brad Anderson. He’s president of products and engineering at Qualtrics, which is one of the largest survey data companies.
Brad, welcome to On Point.
BRAD ANDERSON: Hey, thank you. I appreciate you being here.
BECKER: We want to note that Boston University, which owns the license for WBUR, where On Point is produced, is a customer of Qualtrics. BUR is editorially independent.
And Brad, I just wonder, we want to get a look at how big this survey industry is. And I’ve read that Qualtrics has analyzed billions, with a B, billions of people. Billions of survey responses, and your business continues to increase even more than you’ve projected. So, I’m wondering, can you explain, you know, how many surveys Qualtrics is currently analyzing and businesses you’re working for? How big is this industry?
ANDERSON: Yeah. You know, so in 2024, we had 1.2 billion surveys come back to the platform that we analyzed. That actually grew 15% year over year at the beginning of the year. To your point, I had projected that surveys would grow between four and 5%. So it grew much, much faster than I thought that it would. And so, you know, survey fatigue is a real thing. And so getting it right, in some of the points that Fred made, is just critical.
BECKER: Mm-hmm. And so, generally then, to see this kind of growth, why? Why? It’s the same question that I asked Fred, why are we seeing so many businesses doing surveys?
I can understand that you want to know whether your customers are satisfied, but is it because it’s less expensive and easier to do this, and we have tools to do this online? It takes, you know, minutes to do an email or send a text. Or is there something else happening that you really need to try to hold on to customers and understand where they’re at and perhaps improve their experience? Or is it all of those things? Why are we seeing so much of this now?
ANDERSON: I think organizations, as they see so much of their business going digital, are struggling to understand how do they build that connection with people? You know, when everything goes digital, if you’ve just got some kind of a flat experience, it isn’t personalized, it’s very hard to build that relationship. And the brands that build relationships with their users are the ones who get the points that Fred was making, repeat, referrals, all those kinds of things.
And so really what I, as I’m talking to the organizations who I would classify as the leaders in really listening and understanding and most importantly acting on customer feedback, it really is about focus on that personalization and building a relationship with people.
BECKER: And so when you design a survey for a particular business, I’m assuming that you go in and work with folks to design the questions. I mean, what are the rules for, general rules for, what you would say is an effective way to reach out to customers to make that connection and get the company what it wants in terms of customer data, but also make sure that the customer responds to the questions?
ANDERSON: Yeah. A couple of things come to mind. First of all, you want to be brief. You don’t want to ask too many questions. We have, you know, more than a hundred I.O. psychologists on staff here that we can have work with organizations as they’re developing the questions in a way that we know that they’ll be answered.
And so there’s a couple of things that I would say. First of all, be brief, be crystal clear and don’t flood your people, you know, your customers with emails. It’s interesting, you know, as you were talking in the previous section, I was just pulling up some data here while I was listening in. Our leading customers, the customers that I would say are really doing the best and getting this feedback are actually getting 13, 14, 15% response rates on their surveys. A lot more than 1%.
But I think one of the most important things here is there are so many other ways to listen in to what customers and employees are saying that are not surveys. And that is —
BECKER: Yeah.
ANDERSON: Go ahead.
BECKER: No, no, go ahead.
ANDERSON: Okay. And so what I’m seeing is, you know, let me, let me give you an example. Take, you know, a hotel or an airline. In a given year, they may receive five or six million surveys back. But as we pull down from them all the reviews that are being posted on all the social sites, that’s another 10 million. As we pull in their chats and the calls that they’re having with customers, we analyze all that. That’s another 20 million interactions.
And so what I see is surveys are definitely growing. But what is growing much, much faster is organizations listening to what customers are already saying about their experiences without having to ask that. And that actually is growing four times what surveys. In 2024, the number of chats, reviews, calls that came through our system was 2.2 billion, and that grew 60% year over year.
BECKER: Mm-hmm. So it’s basically monitoring customers online activity to get a fuller picture of exactly what the customer experience might have been?
ANDERSON: That’s right.
BECKER: Okay. But I wonder, you know, when you said, you know, minimize time, designing questions, 100 psychologists on staff, that number I found really astounding. But I guess, is there sort of a general framework? So when I get an email, say, from someone that I’ve purchased something from, I tend to get a few repeat emails. They give me a deadline by which to respond.
I mean, what’s the general framework there in terms of how many emails might be okay and not annoying? Do you give a deadline? Do you compensate people for responding to a survey with a gift card or 10% off their next purchase. I mean, what are some of the, the general things that you’ve found that perhaps give you that higher response rate than what Fred had mentioned initially?
ANDERSON: Yeah, the biggest thing that we see is the proximity of the time when you asked for the feedback relative to the experience is actually the biggest driver of one, response rate and two, quality of the response.
And so, for example, if someone has an experience and they get a link, whether it be an email or text to take a survey and it’s two days, three days, five days later, you know, they’ve already moved on. But if you can actually collect that feedback in the moment — we’ll talk about that in a minute about how AI is changing all of this. But if you can collect that feedback in the moment or immediately thereafter, you do get much better response rates and higher quality data.
BECKER: So do it fast. Don’t do it repeatedly.
ANDERSON: That’s right. And so we have rules within our platform that a brand can go in and say, “Hey, if a customer has received a survey from us in the last 30, 60, 90 days, don’t send them another one.” And so we have all of these things that we have built in to help organizations govern that.
But I’ll tell you where this is heading to is, you know, what we have been innovating on is how does generative AI just completely transform a survey? If you think about historically, a survey is a flat, non-personalized, just a set of questions. It’s interesting, the average survey has 14 questions in it. And in that average survey, five of them are open-ended. Just “tell us what you think,” not multiple choice.
So what we have been doing now is applying generative AI to the survey. And so, for example, taking, you know, what Fred and team built, if someone comes back and says, “How likely would you be to recommend us?” And they say a seven, the first question every organization has is, “Well, what could we have done better to get a nine or a 10?” And so our AI could detect that and then automatically say, “Well, what could we have done better?” And then as that feedback comes in with generative AI, we’re actually able to have an interview with the customer, which gets much better data.
BECKER: Okay, we’re going to talk about AI in just a moment. Let’s just talk about what typically is being done right now with these surveys and why they are so ubiquitous. And so, let’s talk about in terms of what companies do with the data. Let’s get to that because you may be able to collect all kinds of data and generative AI may make it so much easier. But the company’s got to do something once it has that information about its customers. And so typically then, where do you come in in advising businesses on what they do with customer data?
ANDERSON: Yeah, there’s two things that I think really make the difference here. The first one is culture. And, you know, you talked earlier about reward systems and how people are rewarded, the reward system does have a profound impact on how any organization works. And so getting the reward system right to where you just don’t have, for example, salespeople at a car dealership, trying to game the system but really having a culture that says, “We are curious, we are inquisitive, we want to understand what we can do better.”
And that, for the companies that are really doing this well, starts at the top. When I say starts at the top, you have CEOs who are not saying, “We have to have, you know, an NPS of X,” they’re saying, “We want to deliver an incredible, unique, differentiated experience. How do we learn what that is?” That’s the first thing. And then the second thing is you have to take action. You know, there was the one of the comments from one of the callers that said, “I don’t believe anything’s going to happen from this.”
BECKER: Right. I think that’s probably why a lot of people don’t respond. They don’t feel that it’s worth their time.
ANDERSON: That’s right. And so if they don’t see action and that action can be personalized. Then, you know, the likelihood of them filling out another survey for you is going to go down.
BECKER: Can you give me an example of a company that demonstrated that it took action because of customer feedback? I mean, I’m just not thinking of anything off the top of my head. Is there one that you know of that said, “Our customers said that this wasn’t working for them” and so they took action?
ANDERSON: Yeah, I’ll give you a couple of examples. Let’s start with Hilton Hotels, which is a, you know, global brand, 8,500 locations. And what they did is instead of asking questions all the time, they started listening in to the chats that were coming in through the application. They were listening into what was being posted up on social websites.
And what they found is like for families, the number of towels in the bathroom has a profound impact on the satisfaction of the stay. And their comment back to me was, “We never would have known to ask the question, ‘Do you need more towels?’ in a survey.” And so this ambient listening coupled with surveys really gives an organization the ability to understand in a much more broader way, in a much more comprehensive way, the things that they can be doing to delight their customers.
BECKER: Yeah.
ANDERSON: That’s one example. I’ll give you another example, which was Delta Airlines has talked about what they are doing. You know, Delta Airlines is well recognized for having the world class experience. And, you know, if you take a look at what their CEO said at their event here at CES last week, it’s all about understanding and personalizing the experience.
And so, you know, now when you go onto a Delta flight, the in-seat console actually is personalized. You can log into that with your SkyMiles number. It understands who you are. It understands what your preferences are back in the system. You know, we’ve collected things like does the individual, if there is an issue with the flight, do they like a red eye? Do they want a direct flight? Do they want a hotel? So we have all those kinds of preferences that we have collected.
Now, when Delta Airlines takes action, like, you know, there’s been a flight canceled. They can take all the preference data that we have collected and then personalize the action that they take in the moment to fix that situation.
BECKER: Brad, as a journalist, when you say things like collected preferences that I might have as a consumer or listening in to chats and aggregating data for a business, it makes me nervous. What kind of consent do you need from a customer to be able to aggregate this data and collect preferences on someone?
ANDERSON: Yeah. So, you know, when you call a call center. One of the things that you’ll mostly hear almost every time you call a call center is that little recording that says, “This data may be used to improve the product, improve the quality of the service.” That is what our customers put in front of things like calls.
For things that are posted on social sites, that’s public data. So we can pull that back and don’t have to ask for approval. You know, the individual has given their approval when they post it. And then of course, as we think about our survey data, you know, what we do is we, we give the individual the ability to identify what things they want to share with the company that’s using us.
And so, you know, if you’re asking a set of questions about, you know, what are your preferences here? We’re not trying to discover this. What we are doing is taking what the customer tells us they want the company to know about them and then that’s what we store. So we only store in the profiles that we create things that the customer has said, “This is what I want you to know about me.”
BECKER: So legally you’re covered?
ANDERSON: Oh, absolutely.
BECKER: Mm-hmm.
ANDERSON: And it’s honestly less about the legal piece, it’s less about being legally covered. It’s about what is doing right and doing things in the right way to where customers actually have more confidence, more trust with their organizations that they’re getting services from. So that really is the focus we have.
BECKER: Mm-hmm. You know, Fred Reichheld is still with us and I want to bring him back into the conversation here. And, Fred, I’m assuming you’ve been listening to much of what Brad has said and he’s said a few things that were different from what you said, particularly in terms of the response rate to customer surveys.
He says it’s more like 15%, you said it was about 1%. And also just the size of evaluating 1.2 billion survey responses and the enormous growth of companies like Brad’s, like Qualtrics. I mean, what are your thoughts, Fred, when you hear some of the things that we’ve been talking about?
REICHHELD: Well, I have, let me say, I greatly admire companies like Qualtrics. My daughter, Jenny, works at Qualtrics, so I know it well.
BECKER: (LAUGHS) There’s a disclaimer.
REICHHELD: Don’t give them your only daughter if you don’t think highly of them!
BECKER: (LAUGHS)
ANDERSON: We love Jenny!
RECIHHELD: And yes, there are ranges of response rates. I think the best companies can find ways to increase response rates. But I’m still, you know, I might say, hey, if 85% of your survey requests are still being ignored, that can’t be considered a wonderful, winning process. There is something fundamentally wrong there.
So what do I see in the great — I’ll go back to Enterprise Rent-A-Car where, where I learned this in the first place. Two of the core concepts. Three concepts. One, local learning. This isn’t for Big Brother at headquarters. This is for the local team. Two, two questions. Not gonna waste anybody’s time. And three, which I didn’t mention, if you get anything other than a high score, if you get a detractor score, there is a requirement that someone from that branch closes the loop, calls the customer, apologizes, tries to fix the problem, and then figures out a way systematically to fix it for any other future customer.
So when I look at a company and say, do you have a good Net Promoter system, a good feedback system? I ask them, “what percentage of detractors have a closed loop within 24 or 48 hours?” And most companies just throw up their hands and say, “Oh, I could never do that. I don’t have enough people.” And that’s saying, “Oh, so your time is valuable, but the customer’s time — and especially a customer who’s told you they didn’t get what they paid for, we just ignore that.” What a screaming communication that I don’t really love my customers.
BECKER: And Brad, what do you tell the businesses you work with about that, about sort of following up with the customer? We only have a few seconds here before we have to take a break.
ANDERSON: Yeah. So I, you know, Fred and I are actually aligned on the things that he is saying, you know, you’ve got to do this right. So be brief, be clear and close the loop. And so the other thing that I would say here is, you know, NPS is a system. It’s a culture. And so the organizations that really are using NPS to the fullest extent have infused it through the culture. And I think Fred’s measurement of how many times are you closing the action is the absolute best measure.
Part III
BECKER: At some jobs, customer survey results can determine how much money people make. Jeysa Wright in Littleton, Colorado, spent four years as a server at a chain restaurant. She says her restaurant had tablets on each table where customers could pay their bill and fill out a survey.
JEYSA WRIGHT: All of this data from the surveys is put into a little poster in the back of the kitchen, available for all staff to see. It basically ranks your servers from best to worst. This could be kind of fun if you wanted to see how you were doing compared to your other coworkers. But it could sometimes be stressful if you had a bad serving day. Or sometimes management would use those rankings to either give or take away shifts based off performance.
BECKER: So basically, how customers answered a survey could affect how many shifts she worked.
And at car dealerships, where many salespeople work on commission, high customer service scores can mean more money. Clayton Mantell is a sales manager at Ramsey Auto Group in New Jersey. He says an even slightly lower score could mean missing out on tens of thousands of dollars in bonuses, depending on how many cars someone’s sold.
CLAYTON MANTELL: If you’re getting a 10 or a nine, great. But if you’re getting an eight or lower, you’re basically screwed. Essentially anything at like, seven or below is a zero. You’d think, “Well, just tell your customer,” but that’s also a survey metric that typically would lead to you not being paid, if you don’t have like 90% of your customers saying you didn’t influence the survey in a specific manner. I routinely see guys get beat outta money when customer comments come in with it. “Great.” “Best salesman ever.” Stuff like that.
BECKER: And Mantell is concerned. He says customers might also take off survey points for things that aren’t related to him, like maybe the dealership’s vending machine being out of a particular item or they’re just having a bad day.
MANTELL: People buy cars having walked out the door in the morning getting told, you know, their wife’s cheating on them and she’s out. And she’s taking the dog, you know. And they walk in there. And like, you gotta be the guy to talk to them. We want people to have 10 out of 10 experience, obviously, with our dealership. To be rated on a scale that’s like a warped funhouse mirror is just crazy.
BECKER: Mantell says there are industry stories of car salespeople quote-unquote “forgetting” to send surveys to customers who they think might leave bad reviews — or even getting their coworkers to send in reviews instead.
MANTELL: I believe Hyundai back in the day used to do a phone interview survey. And the dealer could just basically put in like, their buddy’s phone number or a pay phone or whatever and just tell them great news, stuff like that. I’m sure that happens, you know, it doesn’t happen a lot. But with so much money on the line and the goalposts being so high, like, it’s going to happen.
BECKER: That’s Clayton Mantell, general sales manager at Ramsey Auto Group in New Jersey.
And I wonder, Fred, could I get your reaction first to this idea of compensating employees based on survey results and the inherent conflict in that? What can you do to get around that to make sure, you know, that folks are delivering good service, but also make sure that your survey results aren’t skewed here?
REICHHELD: This is a vital, vital issue. And if you read any one of the books I’ve written that defines the Net Promoter system and guides people toward best practices, you will read these words: “Do not link survey scores to employee bonuses. Only bad things will happen.”
And it might take a year or two, but eventually, employees feel like the job is to get a high score no matter what it takes because they get desperate. It’s a horrible idea. And you know, I know companies do this with good intention. They want to show that they’re serious about this and they take scores just as serious as they do accounting results. But really, surveys are soft. 1% response rate, sampling error, all of the biases and cheating and gaming from the ideas you just reviewed. Don’t use surveys in the wrong way. That’s part of why we have 1% response rates.
BECKER: But if customers need to feel that what they’re telling a particular business is meaningful and they need to see action, you know, it would almost seem like maybe it is a tool to help. Maybe you don’t directly link bonuses or shifts worked at a restaurant to it, but it could help guide someone if a particular employee might need support in some area to deliver better customer service, couldn’t it?
REICHHELD: One of my sons worked at Apple in the retail, in the stores, for at least a decade. And Apple has used Net Promoter the right way. They do not link people’s compensation to it. They do not put the rank ordering of employees that week on their Net Promoter Score on the locker room door. That’s humiliating. And it makes the employee fixate on the score, when in fact the score for the employee should be a signal. They just want to learn when a customer is not happy, what do they do that they can change?
And, you know, they celebrate the 10s. They think of them as standing ovations. So they get all the energy of an employee enriching someone’s life. And the team hears these stories. And they keep the detractor, they close the loop, the same thing I said that any good company does. Well, Apple has one of the store managers reach out to those customers and try to fix it. And then they talk with employees one on one if there’s individual feedback that’s appropriate.
So, done right, it’s a great system. It’s just it’s done right very, very infrequently these days.
BECKER: Brad, what do you tell folks about tying survey results to compensation or something beneficial to employees?
ANDERSON: Yeah, I’ve always believed and I certainly learned this, you know, as I was on the front row of the transition from Balmer to Satya at Microsoft and just saw how that culture changed. The reward system and the things that you measure and reward are really important. And so to Fred’s point, if you’re going to use something like the NPS system, it has to be to drive a culture of continuous improvement. It’s not punitive, but it has to be something to where the culture of the entire organization is curious. And you’re digging in, “What could we have done better?”
Now this is a place, I think the example that was given by the car dealership is really an interesting one to spend a second on. You know, in the past, someone would get a survey and the survey would say, you know, “Would you like to recommend us? And let’s say they say a seven. Well then now, using generative AI, our solution can say, “Well what could we have done better?” And the answer that comes back to the customer could have been like, “Well the food was terrible.” Well, “give us more information about that.” And we would find out that it’s the vending machine.
BECKER: Mm-hmm.
ANDERSON: And so by using AI, you can now pick apart and understand what are the things that could be used to help coach the car salesman or the agent, and what are the things that are structural and are meta issues that need to be solved at, you know, a dealership or at a brand level. So that’s where the power of AI and really understanding the context and the specifics about what caused a great experience or what caused a poor experience, and then having that culture of curiosity and continuous improvement.
BECKER: So don’t necessarily do away with bonuses based on surveys, but make sure that the survey is accurately reflecting what the employee’s performance was here before you do so. Is that what you’re saying?
ANDERSON: Yeah. Well, that’s definitely part of it. I do believe that if you reward great customer service, it can drive the right behaviors. But, you know, to the points that were being made before, it can become embarrassing. It can become dismissive. And so you have to do it right.
BECKER: What about compensating consumers for doing surveys? What do we know about that? Does that help improve response rates or, you know, responses and get more stars if you give somebody a discount or a gift card or something along those lines? Or does it skew the results? What do you say about that?
ANDERSON: You know, I don’t see a lot of organizations who are giving rewards to customers to fill out surveys. Where I see rewards being given is when they’re doing research.
So research could be of prospective customers or, you know, “What’s the next service or product that we should build?” And I do see rewards there because then, you know, it’s generally speaking with those kinds of feedback and those kinds of surveys, they’re much more time consuming.
And so I don’t believe that there is a requirement to reward that’s going to increase their response rates. In fact, what I’ve actually seen is there are organizations that will look for those kinds of reward and then try to game the system. And so we’ve had to build AI that is able to understand is this a human, is this not a human, is this a bot to make sure that organizations are not just paying some kind of a crime organization in another country to fill out a survey.
BECKER: Hmm. So let’s go to generative AI because you’ve mentioned it a couple of times, and it is really fascinating because it obviously it has this ability to aggregate a lot of data, which you’re dealing with 1.2 billion survey respondents, I think you said at the start of our conversation. So tell me, what are some of the most interesting things that you’re working on right now, perhaps the most, we’ll pick one, that you think is really going to help improve this sort of customer service interaction with businesses?
ANDERSON: Yeah, far and away, the thing that I am seeing is the most impactful, you know, of the dozen-plus AI capabilities that we’ve built into the product over the last year is what we call conversational feedback. And this transforms a survey from a flat, impersonal experience to an experience that is personalized and adapts while the survey is being given.
And what that allows an organization to do is, first of all, get much better data, and then second, really have a personalized experience for the individual filling out the survey. So let me give you just two data points that — or two examples you’ll find interesting.
So what we have a bunch of measurements that we do as we get these open-ended comments coming back to us. And so, for example, with these conversational feedback surveys, what we’re able to do is take a look at the first answer, how many syllables in it. We have some lexicon measurements that we use to understand the quality of the response, and then we can compare that with the second response that comes in when we prompt for more information.
And so what we see, for example, is seven times the number of words or syllables that come in on the second response. From a lexicon perspective, the quality of the second response is much higher. And then third, the most important thing is organizations who are seeing this are actually seeing double the number of key things that they can go structurally fix in order to drive improvement in the customer experience. And so those are very, very specific measurements.
And it’s interesting, we, with a lot of our customers who have used these capabilities, we’ve asked them to allow us to put a question at the end of the survey so we can understand how the consumer filling out the survey felt. And resoundingly, when it gets personalized and it has this adaptive conversation, the feedback coming from the individuals filling out the survey, the satisfaction of the survey is high. So I think that that’s a good sign.
Second example I’ll give you, Deborah, that I thought was fascinating is we had a healthcare organization a couple of months ago who was using these adaptive conversational feedback capabilities. And there was a survey that was being given to, you know, an older individual who had had a poor experience. Well, our AI detected the poor experience, showed empathy in the questions that were being asked, in the follow-up questions.
So much of the impact of this was that the individual actually commented, “I appreciate you showing me empathy. I love working with an organization like this.” And so these conversational AI capabilities can really improve their relationship, which I think is what every organization is trying to do.
BECKER: But basically it’s — it is really the AI-generated conversation. The computer is having this conversation, a human being is not.
ANDERSON: That is right. And so, you know, let’s talk about, you know, getting high quality data. With surveys, you know, there’s a level of data that you can get. And if you compare that to, for example, with a five minute personal interview that would be done by a human, of course, you’re going to get a lot more data in the interview because the human will adjust to ask more questions to get down to, okay, what makes this actionable?
Well, we automate all that through conversational AI, through generative AI. And so, yes, it is the generative AI having the conversation. But we can, for example, train the AI to have the tone of the company, to understand what rewards can be given. And you think about where this will head. You can see where, you know, as an organization is conducting a survey, and there’s a poor experience, the system will automatically be able to close the loop and say, “Oh, we apologize about this. Here are some points. Here is a gift certificate to come back on us.” And you can have all that be controlled based upon the feedback that is being given.
And to the point that Fred made earlier, the best measurement of are you doing the right things? Are you closing the loop? And so you can see where this conversational AI is going to take us to to where we can close the loop in the moment when that feedback is being given.
BECKER: Mm. But if personal customer experience is so important and a customer feeling like they are important, it would seem that we’re not at a point yet where — perhaps with your response from, you know, the patient who went to the hospital and felt that the AI conversation was empathetic, perhaps I’m not right here, but it would seem that we’re not at a point yet where we could use AI to be authentic enough to create a personal experience for a customer. But you think we’re almost there? Or we are there? What would you say, Brad?
ANDERSON: Yeah, we’re definitely on the path there. You know, there’s a lot of conversation in the industry right now about agentic AI. And that really is the next frontier. And what agentic AI is autonomous agents that have been trained to understand the customer experience, that have been trained to understand the organization, their brand, their tone. And so, as I think out the next two to three years, as these agentic solutions come out, you’ll see a lot more of exactly what you were saying of where you’ll have agents who will be able to take action on behalf of customers.
And so as I fast forward two or three years from now, you’ll start seeing things rather than an individual coming to a website or in an app and then, you know, navigating that to accomplish something, they’ll be able to come up to that app or website and just describe what it is that they want or what they’re trying to accomplish. And then you’ll have a set of autonomous, agentic AI that’s operating in the background, these agents, to take that action.
And our belief is you want to infuse all those agents with an understanding of the human experience, which is where our investments have been for the last five years. You know, yes, we continue to improve surveys, but all the investments, the majority of the investments has been in how do we build the AI to understand the human experience to then allow organizations in these systems like an NPS system to be able to understand what the lived experience of the customer and the employee was, and then take action.
BECKER: Okay, 30 seconds left. I know, Fred, there’s a lot there, but we’re going to give you the last word here. We have come a long way from developing the question, “Would you recommend us to a friend?” it seems. (LAUGHS)
REICHHELD: You know, Brad started out saying that Qualtrics analyzed 1.2 billion responses. You know, at a 1% response rate — and I don’t know what market share Qualtrics has in the market, it’s very substantial — but it means there’s hundreds of billions, maybe a trillion responses or requests for surveys there’s a problem out there. The tragedy of the commons is upon us.
I think we need to simplify. Yes, AI can help. But my simplification is even simpler. The real goal is to treat a customer so well that they come back for more, and especially they refer their friends. Don’t ask in a survey!
BECKER: Okay. Thanks, Fred.
link