Traditional business intelligence systems that analyze structured data are very good for statistically reporting the current state of customers and markets. Sales are up or sales are down. Customers are more satisfied or customers are less satisfied. This region seems to be performing better than that region. Although these are important facts to understand, the key insights that are missing are why those things are happening now. Answering the “why” behind the data is typically not possible, even with investments in interpolation, modeling, and statistical analysis on traditional structured data.
However, when you combine structured data with unstructured data, such as freeform replies to open-ended survey questions or comments on the Internet, you add another layer of depth that can give you a complete picture. For example, you can see what customers are saying about a poorly performing product, why customers in a specific region for a specific type of product and for a specific time period are unhappy, and what were the key issues that drove low satisfaction.
Text analytics is the key to understanding these questions. Well-designed surveys will typically ask for customers to rate products or services, then ask “Why did you give us that rating?” or “Why were you dissatisfied with our service?” The answers to those questions provide powerful insights. However, until recently this has been difficult to analyze. Businesses have traditionally relied on verbatim coding systems where outsourced vendors or analysts manually review a random sample of a few hundred responses, and then create codes to categorize them into common issues.
Although manually reviewing a sample of responses provides some level of accuracy, there are some inherent flaws in that process. First and foremost is that you are not looking at all of the data. If you have thousands or hundreds of thousands of responses, you are only able to cost effectively analyze a small fraction of the available information.
The second flaw is human bias. Whenever humans are making decisions about the data, there is always a tendency for people to respond and categorize based on the way they are feeling that day. Eye strain and fatigue also play a role in delivering inconsistent results. One day an analyst may categorize a particular issue as a customer service problem, the next day or week they may think it is more of a product problem.
In addition, customers may have complex issues that are not easily categorized with traditional coding schemes. In this case, you may need multiple interdependent codes, but that can make it even more difficult for human analysts to be consistent. All of these challenges to analyzing freeform, open-ended comments in surveys are prevalent today. Text analytics delivers the capability to automatically process and analyze large volumes of freeform text with consistency and accuracy.
Coming Soon: The new way to do text analytics in a VOC program
Eric Weight is Director Text Analytics Products at Allegiance, Inc.