Lonely Data

I once read a beautiful story by Sherman Alexie in his book "Ten Little Indians". The story was about a woman who checks out an old book from the library and finds that it has never been checked out before. She wonders how the book must feel; neglected on the shelf, passed by by so many potential readers. I, too, felt for the book and its author.

What about data that is collected and never used? Every survey that I have either taken or designed has had the "Is there anything else..." open-ended text question at the end. I know some of those open-ended responses are rarely, if ever, analyzed.

Sure, one can fill Excel, SAS, Access, and SPSS with rows of quantitative data then slice and dice it. But open-ended, qualitative, responses are different. If one collects 100 responses, its a manageable task to sort through them. Even 1000 is manageable with a couple of people helping, although that presents its own challenges. But what about 10,000 responses? Do those responses just wait for some ambitious analyst to download and analyze them?

That is exactly what I found. Thousands of open-ended text responses collected every few weeks from customers. Thousands of potential solutions to the customer pain points just waiting, while the customers asked, "Why don't they listen to us?"

The survey showed, via questions using a likert scale, that a product "scored" a 2 out of 5. "That is bad, we need to fix this," said the product managers. They put together a team to assess the situation, came up with recommendations, implemented them, and the product still scored a "2 out of 5". "What did we do wrong," they asked.

They didn't listen to the users. Quantitative data will tell us there is a problem. Yes, 2 on a scale of 5 is a red flag. However, qualitative data, e.g., open-ended text data will tell us what the problem is and how to solve it.

Lets propose a hypothetical situation based on a situation I encountered; we have a store that specializes in woman's clothing, specifically evening dresses. They sell the type of dress a woman may wear on a night out. The store manager realizes that a lot of her customers bring along their mates (spouses, significant others) for feedback on the potential purchases. Eventually, these men are milling around the front of the store glassy-eyed, or getting in the way of the female customers. She has even seen some arguments break out with the customer leaving the store, empty-handed. This is not good.

She designs a survey, and gets the sales people administer them. The survey contain only three questions, much like an NPS survey. The survey is as follows:

• If male, ask "How likely would you be to recommend our store to your friends as a place where they should take their girlfriend or wife?"
• If female, ask "How likely would you be to recommend our store to your friends?"
• Ask both female and male, "Why did you give that answer?"

The results are not surprising. The men would not recommend the store, but the women would. "I know," the manager says, "I'll make it more comfortable for the men. I'll put in a TV and set it to the sports station. That should make them more comfortable." And it does. A few weeks later she repeats the survey, but now the ratings have flipped; the women would not recommend the store while the men will. And, the manager has noticed more arguments between the women and the men. What went wrong?

She didn't read the comments. One would assume that with an NPS-style survey, the manager would read these comments. However, she didn't. Some of the comments she missed:

• "There is not enough space around the changing room. I don't have anywhere to wait while my wife changes then comes out to show me the dress."
• "Its like a fashion show, we are all moving around trying to get out of the way so we can see our girlfriends."
• "I can't talk to my wife while she is putting on the dress. I saw a dress I'd like her to think about on another person, but I couldn't tell her to try it."
• "Maybe like a fashion show would be cool. The ladies could come out and model the dresses and we could see. If I see one I like I could tell my wife to look at it."
• "My girlfriend is kind of shy. She won't come out and show me if a lot of people are around. It would be nice if I could see her without everyone seeing her. Maybe with a video camera like on that one TV show."
  

Although this a hypothetical situation, it is clear what recommendations I would make. Clearing the area around the changing room and adding chairs would be an inexpensive way to increase satisfaction. The ideas of a "fashion show" and a "private screening" are interesting - I would assess the data for more like responses and perhaps make a recommendation based on those ideas. Communication seems to be a theme. Perhaps there is a way to get messages to the women from their husbands/boyfriends.

Open-ended text can be a difficult data type to analyze but it can be a useful source of information. There are many tools available to help with this analysis. I used SPSS Text Analytics for Surveys. In future posts I will share some insights I had from using the tool.