Checking the sample behind a retail satisfaction survey

To judge a retail satisfaction result, establish how many eligible customers answered the specific question, how they entered the survey and whose experience the result is meant to represent. A large overall survey can still leave a small or selective group for one retailer.

Define the customer group

Who qualified to rate the retailer? Did respondents need to have purchased recently, visited a store, used delivery or merely recognised the brand? A score from recent purchasers answers a different question from one collected from everyone who has dealt with the business. Check whether online and in-store customers, regions and customer types are combined.

Published awards make this distinction easy to miss. Roy Morgan says its Australian Single Source survey covers more than 60,000 consumers annually and requires a minimum sample size for an organisation to qualify for its customer satisfaction awards.

The annual survey total is not a retailer-level or monthly rating base. The awards page does not specify a universal minimum number. A close comparison still needs the relevant retailer bases, periods and categories.

Roy Morgan Customer Satisfaction Awards: Sample & Methodology Highlights

Check selection and missing answers

Look for a methods note explaining recruitment and response. An open feedback form may attract customers with unusually strong experiences. A survey sent only after an online order misses people who abandoned checkout or bought in a shop.

If the publisher reports weighting, check which population characteristics it aligns. Weighting cannot add experiences from customers who could not enter the survey.

Find the denominator for the precise result. Some people skip a service question because they did not use that service. Excluding non-delivery customers makes sense for a claim about delivery users; it does not support a claim about all customers.

The ABS distinguishes sampling error from non-sampling errors such as coverage, non-response and inaccurate answers. A larger sample generally reduces sampling error, but it does not automatically remove the other problems. Interpret a published confidence interval in light of how the sample was selected.

Pros and Cons of Using Weighting in Retail Satisfaction Surveys

Compare retailers on the same basis

Before comparing satisfaction percentages, check question wording, response scale, treatment of neutral answers, field dates, eligibility and sample bases. A share marked “satisfied” and an average score cannot be subtracted meaningfully. Identically labelled scores may still reflect different customer mixes, such as delivery users at one business and in-store buyers at another.

If those details are unavailable, describe the result as the publisher’s measure for its surveyed customers. Do not present a narrow lead as a settled quality difference. A useful next check is whether the pattern holds within the same channel, region and recent-purchase period, and whether complaints or repeat purchasing tell a compatible story.

Retail Satisfaction Survey: Key Factors for Valid Comparison