Stop Horrible Focus Group Recruitment - New Tactics

Summary

The Research blogsphere is alive with talk about a Tropicana repackaging disaster, where a consumer backlash forced a reversion from a new design created by focus groups. The reality is that the discussion on the new packaging probably provided invaluable exposure on an otherwise dull product change, but does raise the ongoing question about guaranteeing quality participants for focus groups. AFS uses a distinct “quantitative” recruitment approach to traverse the main issue with groups and specialist recruiters – the experienced, Professional participant.

New York Magazine recently exposed how easily professional recruitment can be fixed. Author Will Leitch attended groups that he certainly did not qualify for, these included;
- people who grew up in Long Island;
- victims of severe asthma;
- travelers who had back-packed through Mongolia;
- people of Italian descent;
- Johnnie Walker drinkers; and
- sufferers of profuse, gland-related perspiration.

He was able to earn as much as AUD$450 a session, up to four times a week, to gain entry he simply lied, wasn't screened or was coached into.

Leitch's story caused outrage, several large research organisations, including the Market Research Association condemned the author, but did not censure the agencies involved.

Big players such as Procter & Gamble and Unilever have for some time moved away from focus groups toward more rigid methods including ethnography and CATI quantitative studies to make their business decisions.

Finding participants can be a difficult task, and recruitment agencies can be less than stringent in their selection procedures – particularly when the pressure is on and the respondents are more difficult to find than anticipated. There is evidence of a growing mass of semi-professional focus group participants in Australia, particularly among retirees and fostered out of experiences with online panels rewards. AFS receives a constant stream of people looking for this type of 'work.'

Focus group veterans are bad for insight because they usually agree with the moderators' questions and embellish or invent experiences. Leitch even describes the 'cardinal rule' of focus group faking: don't offer your opinion, just confirm whatever they want you to confirm.

Traditional recruiters use 'snowballing tactics' asking qualified respondents 'if they know anyone else...' this encourages mates and like minded people to become participants, this issue is clearly that groups contain similar voices. Some recruiters will also look for locals near the venue - this can invalidate research unless you specify a geographic requirement.

How recruitment works – How to improve it

Reputable recruiters have ways of assuring professionals are contained, strategies include database tracking and quarantining participants from making regular appearances. Remember Leitch's companies had these policies in place, but most are not even ISO credentialed. Unfortunately the only way you'll really know is when results conflict with your subsequent research experiences or you just happen to get lucky and catch respondents in a lie. Obvious triggers are when group members agree 'too much' or elaborate on positions already made.

You can generally tell the quality of the respondent by the price tag, but another way is to look at the raw source and how the sample is constructed. Pure online panels use incentives to draw in their participants in while traditional recruiters use a snowballing process, if they don't have an immediate house list that suits, relying on relationships to find the right people (both are excellent at providing low cost and/or low incidence participants) so long as you re-screen efficiently. Unfortunately both suffer from the bias of having a prior (and often ongoing) relationship with the recruiter which leads for more rewards, this pressures and motivates the respondent to participate.

Quantitative recruiters like AFS use a broader approach – taking a purchased consumer or business list depending on the criteria (not active panel members) or use random sample – this can be a more expensive proposition per group (although incentives are generally lower) but often it requires a less number of groups when compared to panels or traditional snowballing techniques.

The key difference is Researchers get to 'ask research virgins' who perfectly match the desired profile without question and any relationship bias. Researchers often comment on the difference; that participants are less forthcoming in expressing their views but 'do not easily toe the line,' are less precious about time limits and are generally more challenging in their participation (than they are used to with other recruitment methods.) Ultimately they find they can really trust the results gleaned from these participants, resulting in a requirement for less groups. Overall this leads to a shorter reporting cycle and greater confidence in results.

Conclusions

All researchers know focus groups are a challenge, professionals, bias toward outspoken participants, inconclusive sample sizes, just some of the reasons why quantitative and ethnographic methods [the study of real consumers with real peers in real locations, engaged in the actual purchase or consumption of products and services] - are growing in influence in make important business decisions.

For Fiona Jack, chair of the UK's Association for Qualitative Research professional respondents are an increasing problem, yet focus groups will remain a central tool. So if you do rely on them frequently, be wary to guarantee their quality every time, as invariably for some engagements, they are a vital first step in an iterative approach to finding research insights. Choose wisely.