Allegiance Blog

In Tamilnadu, South India, the chef is complimented following a well-cooked meal with the phrase “Kai Manam,” meaning the knowledge, care and soul the cook’s hands imparted to the meal.

This is also true for Voice of the Customer champions who try to convey a similar sense to their customers through surveys, analysis and the action thereafter.  By acquiring knowledge to understand the needs of the customer and communicating care, VOC experts strive to deliver improved products and services and create happier customers.

Hence, we undertake this exercise to learn and explore the skills harnessed by top chefs in the kitchen to drive best practice in survey creation.

Have a vision

The master chef plans before execution.

Before the survey creation process, a few things should be accomplished:

• Identify the purpose and business objectives of the project .
• Determine what you are planning to measure. Your questions will differ accordingly.
• Include all internal and external stakeholders and determine that you are not asking customers for duplicate information.
• Plan your touch point rules across the organization to increase response rates and decrease survey fatigue.

Keep it simple

The master chef knows that simplicity is the secret to making ingredients sing.

In the Power of Survey Design Iarossi states, “The survey should use language that is simple in both words and phrases.”

• Use words and expressions that are simple, direct and familiar.
• Avoid buzz words, abbreviations and acronyms. Provide help text if buzz words cannot be avoided.
• Use simple sentences to avoid ambiguity or confusion.

Taste test

The master chef creates the well-balanced dish by tasting at every step of the creation process.

“Taste test” the survey for readability, usability and accurate data collection.

Source: Jakob Nielsen’s AlertBox

According to leading usability specialist Jakob Nielsen, just five users would reveal about 85% of all problems with your website. Only two test users would likely find the majority of usability problems.

Don’t be afraid of doing a little; any testing is better than none!

Serve it hot!

Freshness is key! The master chef always serves the meal up hot.

• Send transactional survey when the incident or event is still fresh in your respondents mind.
• React to survey results when your findings are hot.

The chart on the left indicates that faster response time has a significant impact on the probability that the customer will return and buy again.

Tools for success

The right tool for the job is a key to success for the culinary master.  A selection of knives is a chef’s best friend.

• Wield the advanced, user-friendly filters in Allegiance Engage7 to slice and dice your data.
• As all data is not created equal, it is vital to filter your data based on attributes and customer segments.
• Analyze the data as a whole and in subsets to concentrate on the metrics that matter the most to help prioritize activities that address the hottest issues of your high value customers.

Then, refine your survey based on what you’ve learned from prior deployments. The result will be a survey that your customers will relish.

Allez cuisine!

My Facebook status today reads, “I’m hoping this morning’s travels will inspire another interesting blog post.” Right after I posted that, a thought popped into my head:  it’s not my travels today as much as my destination that has inspired me. I’m on my way to our headquarters in Utah to meet with my team for our Quarterly Business Review. This got me thinking about how our clients might review their businesses…especially as it pertains to their Voice of the Customer (VOC) initiatives.

Gathering feedback from your customers helps you understand what you’re doing well and not so well for them. However, if that’s all you do, it’s like living in a bubble. It’s important to put all of your scores and feedback into the broader context – how well do you do stack up against your competitors? This is applies not only to financial/operational metrics, but also with your VOC data.

I’m talking about benchmarking, which means making comparisons to help you understand the perception of your business relative to the competition in the minds of your customers.

1. Competitive Benchmarking helps you determine your performance relative to a primary competitor or a set of key competitors. Competitive Benchmarking data can be obtained in several different ways.

          Third-Party Surveys:  Engage with a third party to conduct a blind competitive survey. This is the cleanest survey approach, but it’s also the most expensive.

          Your Surveys:  Add some questions to the end of your relationship survey that ask your customers to rate one or two of your competitors with which they’ve done business. This approach is a little less clean and perhaps even a bit biased because you’re asking these questions only of your customers. As long as you view the responses in that light, you can still get a decent benchmark.

          External Metrics:  Get access to syndicated results for ACSI, JDPA indexes, NPS, Forrester CxPi Customer Experience Index, etc. that are relevant for your industry, product, etc. Ask a comparable question or set of questions in your own survey(s) to benchmark.

2. World Class Benchmarking is a slightly different approach where you’re not necessarily interested in benchmarking question to question or score to score. In World Class Benchmarking, you ask your customers to tell you about a “world class experience” they had with another company – any company, regardless of industry. What you’re looking for is a way to identify who your customers look up to when it comes to service, products, literature, training, etc.  You then study that company inside and out – you might even partner with them or find a mentor in that organization, depending on who it is – to identify best practices that you can put to work in your own company.

My final thought about how clients review their businesses brings me to Internal Benchmarking, which entails taking the feedback you’ve gathered and comparing scores, ratings, or indexes internally – within your own business, i.e., benchmark business units, locations, sites, etc. against each other. Identify your stars and your dogs, compare practices, and have your stars mentor your dogs.

Ok, it’s time to go update my Facebook status to “another one’s in the can.”

Have you ever noticed that more people start your online surveys than finish them? It’s like they start clicking through the answers and then suddenly they have a heart attack and it’s curtains!

What is happening to those hapless survey takers?

Life insurance actuaries, the Census Bureau, and the Social Security Administration use a tool to understand mortality: life tables. Life tables measure the odds of dying at a particular age. Age 0 to 1 has a higher risk of death than age 1 to 5. Past age 5, our odds of death steadily climb.

One might assume that a similar pattern would be true for online surveys, and that the odds of abandoning a survey increase with each increasing page. So, I decided to check it out. Do survey abandonment curves match similar patterns to human mortality tables? I used one of Allegiance’s client surveys to examine the trends.

In the chart below I used one axis to plot the number of survey takers who had completed each page. On the secondary axis, the red line, I plotted the odds of abandonment. The odds of abandonment is the number of survey takers lost to page x+1 divided by the survey takers remaining at page x.

Turns out, survey abandonment curves are not quite as smooth as human mortality curves. Survey abandonment curves are spiky. It’s like the Grim Reaper inhabits certain pages but not others.

Clearly, one can see where the problem pages are, where abandonment spikes. It turns out that this survey has a bunch of open-ended questions on pages 2, 3 and 10. Answering open-ended questions is a lot of work. Rather than work, people abandon the survey altogether.

There is no way this client will get rid of their open-ended questions. There is nothing I can do to make the questions less prone to abandonment. However, what we can do is re-order the pages and put the low-abandonment pages at the front of the survey and the high-abandonment pages at the back. The thought is that with more survey takers sticking around for longer, we may get to gather more data.

We can measure the increased data the same way the census bureau measures “life-years.” I’ll call this measurement “page-completes.” If we accumulate more page-completes by the end of the survey then we’ve made an improvement. Below is a simulation of re-ordering the pages to put the high-abandonment pages at the end.

Sure, the same number of respondents made it to the end of the survey, 1910. However by re-ordering the survey, the page-completes grow from 28,828 before to 32,319 after. Re-ordering may help me capture more survey data overall.

Kyle LaMalfa, Best Practices Manager and Loyalty Expert, Allegiance

Many people think that surveys are just a bunch of questions, but they’re not. Every survey is a culmination of a challenging seven-step process. And each step of that process poses unique challenges to researchers. After all, sound business decisions much come from high quality information.

And here is a seven-step process that you can use to ensure you create a high-quality survey that will produce more accurate and informative insights:

1. Establish your goals and objectives – A good survey research project must start with clearly defined goals and objectives. Create project goals and answer the questions: what is unknown?; what has already been done; what decisions will be made based on the result?; what do you think the results will look like?’ and what if you’re wrong? What if the results are inconclusive?
2. Determine your audience – Contacting the right mix of respondents is critical to an accurate outcome – just as important as asking the right questions.
3. Design your questions – Use question design best practices to minimize bias and optimize the quality of response.
4. Do a pilot test of your survey – Always conduct a pilot test with a small group prior to launching the survey. Look for misunderstandings, extremely unexpected results, technology problems, and also evaluate the time it takes a typical person to complete the survey. Update and improve your survey. This process will help you catch problems before they become costly mistakes.
5. Launch the survey – Many modern survey projects are launched via email and completed online. Above all, make sure your email invitations are accurate, genuine and compelling. Even the best survey will fall flat if people don’t bother to take it.
6. Analyze the results – Prepare to analyze the data when you are developing the questions. Use techniques such as cross-tabbing and weighting to understand and contextualize data.
7. Communicate the results to others – Action may never be taken unless others know about your survey discoveries. Communicating results clearly and efficiently is the key to a compelling argument for taking action.

Kyle LaMalfa, Best Practices Manager and Loyalty Expert, Allegiance

Without getting into deep research, it seems to me that the average marketing manager should be able to put together a sensible survey simply by using some common sense. Somehow, this is not happening as often as I would expect. My speculation is that people are so hungry for feedback on so many items that they can’t resist asking their customers for feedback on all of them. The result of this is that survey abandonment goes up proportionate to the length of the survey and the demographic of the recipient.  And, you wind up with skewed results, since certain classes of respondents, as a group, are more inclined to abandon than others. Here’s a recent experience of mine to make my point.

I recently stayed at a hotel in southern Utah. Two days after my stay, I received a survey request from them. I like this particular hotel chain, so I had no problem opening the survey and giving them my feedback. They had a nice little progress bar on the screen so I knew exactly how far into the survey I had gone. After three or four pages of multiple ranking pages, however, I was still only 40% complete. The next page had 15 ranking questions on everything from their toiletries, to the beds, to the TVs, etc. I bailed out of the survey.

There are two issues in how the hotel should have designed their survey:

1.  The hotel knew who I was and from my profile, should already have known whether or not I was a frequent business traveler or a pleasure/family traveler. Knowing that, they should realize that getting frequent survey responses from me would be very valuable to their business, but also knowing that I am a business person with very little time, they should ask, at most, no more than 5 questions. They could have easily asked me 5 questions out of a set of 20 and by doing this randomly across all their business travelers, still have received the feedback they desired. This would especially be true since their abandonment rate would probably drop by a factor of two or three.

2.  Even if the hotel did not know I was a business traveler, they still should have done the same process outlined above because hotel stays are generally a repeated service. This means unlike, for instance, a car purchase, you are likely to repeat business with them more often than once every few years. Common sense says that recipients of surveys who have made higher dollar, more infrequent purchases will be more likely to tolerate a longer survey. If you are a provider of a more frequent service, you want to design a survey that is quick and easy for the recipient to take so that you will get feedback EVERY time you deliver that service.

So, use common sense when surveying. Understand your recipient. Spread the feedback items across the audience, especially when the sample size and frequencies are high. Know your key goals and cut questions that are not absolutely necessary to meet them.

Terence Fugazzi, VP Demand Marketing, Allegiance