A poorly defined global segmentation can create significant implementation and strategy problems. When we review segmentation solution options with our clients, we reassure them that we’re presenting only statistically sound alternatives. This allows them to focus on evaluation criteria such as face validity (do the groups feel real?), comprehensiveness (are any groups missing?), and actionability (do they drive different business actions?).
That said, it’s important for every client to take a critical look at their segmentation findings when they come in since not all solutions actually are statistically valid. But a segmentation typically relies on sophisticated analytics applied to survey responses or first party behavioral data, so it can be difficult for end users to assess the validity of the segments they receive from their research partner or internal data scientist.
What are some signs your segmentation data might not be statistically sound? Here are three red flags to look out for when you’re evaluating new segments.
Do you have one segment that is high on everything and perhaps another that is low on everything? Or does your high engagement group balloon in high rater countries like Mexico and India, but shrink in middle-rater countries like Japan? Either of those signs may indicate that the algorithm grouped people based on how they use rating scales rather than what they are trying to communicate with their ratings.
This is not to say that the presence of high and low engagement groups is a bad thing. Most categories have groups like these, but it’s important to be sure that the groups emerged due to high quality measurement methods and not as the result of scale usage differences. A low engagement person should score high on measures of price sensitivity and indifference.
Even if it is not a global segmentation study, we strongly recommend segmenting based on metrics that reduce or eliminate scale usage differences, which are a catastrophe across countries but also problematic within countries. Examples of good segmentation inputs are bipolar scales (where people choose between opposing statements rather than indicating how much they dis/agree with one of those statements) and partial or complete rankings (depending on the list length). We do not recommend using MaxDiff scores as inputs since those scores are modeled by borrowing information from the very respondents you hypothesize may belong to a different group!
Lengthy typing tools (the set of questions you need to ask in order to predict what segment someone belongs to) drive up future research costs and deter the organization from deploying the segmentation in all relevant research settings. If you can’t squeeze your typing tool into your brand tracker, then how are you supposed to judge the effectiveness of your marketing activities which are presumably geared toward target segments more than others?
Clients that have received really lengthy typing tools tend to request a shorter, and necessarily less accurate, typing tool for some use cases.
Furthermore, if it takes so many questions to identify segment membership accurately, it suggests that the definitions of the segments are more complex than your understandings of them. We attempt to follow Occam’s razor to seek an elegant solution that defines segments based on just the key differentiating traits, allowing us to generate typing tools that are efficient, highly accurate, and consistent with how you think about the segments.
This might be a problem with the framework itself or simply with the quality of the report writing and presentation. Signs that you have a problem with the framework include:
Too often, a market segmentation is a case of “buyer beware.” Once you receive the results, it can feel like you have to make do with what you’re given, but that’s not the case. If you don’t like your segmentation, send it back! And if you don’t trust your partner to solve the problems on their own, contact us. It would not be the first time we’ve been hired to re-analyze the data from someone else’s segmentation study.