Applying Assignment Weights to Brand Ratings

In my last blog, “Who Should Rate Which Brands?” I addressed brand assignment methodologies when obtaining consumer brand ratings. Having consumers rate every brand in a category just isn’t feasible or even desirable. Most clients would not want to spend the entire budget or questionnaire real estate on brand ratings. Moreover, respondents would tire out and then tune out, negatively impacting data quality. Accordingly, respondents typically rate only some brands, with study objectives and budget driving the choice of assignment methodology.

Some brand assignment approaches, while helping us answer our business problems, may introduce bias into the survey that we can address with assignment weights. Assignment weights are weights applied only to the brand ratings and, if a person rates more than one brand which requires ratings weights, that person will have different weights for each brand. Why might brand assignment weights be needed?

Imagine a common case where we prioritize assignment to consumers’ primary brand and, at the same time, assign everyone familiar with the client brand to rate it. If we report these brand ratings directly, with no adjustment for the assignment rules, the client may look terrible since competitors were more often rated by their loyalists… it’s not a fair fight.

To level the playing field, you would want to weight the competitor ratings to reflect a market-representative mix of usage relationships. If this is not possible, such as instances where competitors were ONLY rated by their users, it will be further necessary to filter the client brand ratings (in this case, to its own users) when comparing to competition.

To avoid such potentially extreme weighting or limiting filter requirements, you may restrict your survey to random or most-needed assignment rules.

Most-needed assignment is a system of always assigning people to rate their lowest incidence qualifying brands; it is employed to boost sample sizes on niche competitors. It is extremely risky, if over-used, since it limits the ratings of market leaders to consumers unfamiliar with the niche options. These consumers, having a limited view of the competitive set, are likely not going to view the majors the same way that a more informed or discerning customer does.

It is recommended that no more than half of the rating slots be assigned on a most-needed basis. And the composition of all brand ratings bases should be compared to the composition of those eligible to rate the brand; if those assigned to rate deviate notably from those eligible on category diversity or usage status, then an assignment weight should be applied.

Even if we just assign brands on a purely random basis, there may still be distortions in the data as those eligible to rate more brands are under-represented within the rating base of every brand. The problem is less severe and more evenly distributed across brands, than in a most-needed assignment situation, so you may choose to skip this extra brand-specific weighting step, but its applicability should at least be considered. As the length of the brand list increases relative to the number of brands to be rated per respondent, the magnitude of the assignment bias increases.

One way to avoid the need for most-needed assignment rules, and minimize the likelihood of needing assignment weights when employing random rules, limit your brand list to only those players big enough to obtain naturally robust base sizes.

When getting brand ratings and associations, let your business goals and budget drive your decisions about who should rate which brand and remember to consider assignment weighting to address possible bias introduced. Do this in the name of high quality data, which is paramount to good decision making.