Understanding Incidence as it Effects Data Collection Costs

Summary

You would be surprised how many experienced Researchers do not always fully consider the implications of incidenceits implications; at what can be a cost to both themselves and their client’s. Don’t you be one of them.

Challenge

In a very basic sense the Incidence is simply the extent or frequency that an item exists in a population (often shown as a percentage.)  For example, the incidence of Fords is 20% of all motorcars – they occur 1 in 5 times.

The challenge is to identify the incidence of your target as closely as possible before commissioning full research; this ensures the research costs can be kept within an acceptable range.  

So far so good – complexity comes when you need to research a subset of a population. Let’s investigate the implications and how to manage them.

At the start of each project an assessment needs to be made of the effective Incidence of the groups under study.  In the most common commercial research, a customer base will be used and the Incidence easier to define.  If we are researching all customers the Incidence is 100%.

But consider the impact on the Incidence if we need to research only females between 24 -54 with a household income of $150,000 plus. The incidence or occurrence of these in a population will be far less than 100%.

The cost implications are easy to imagine, if we take the Ford case and only 1 in 5 are suitable for research, it will take approximately five times as long to recruit for the research.

 Finally, consider the implications of studying multiple groups with different incidences: 

–         Females 25-35 that think they look good in jeans, an incidence of 5%

–         Males 25-35 that think they look good in jeans 15%

–         Males 50-60 that think they look good in jeans 1%

With a higher incidence, the Males 25-35 will fill up quicker, three times quicker than Females 25-35 and fifteen times faster than Males 50-60.

At the start of the data collection process, the initial incidence will be a combination of the groups at 21% (15%+5%+1%), but after the Males 25-35 fill, the incidence will drop to 6% and then to 1% searching for the Males 50-60.

Now consider you don’t know the incidence of any of the three groups when you design your research. 

You may hope you have an incidence of 21% overall, but you could end up with an incidence closer to 5% or 1% overall.  Ouch!

Solutions

Thankfully there are a number of ways to reduce the unknowns almost entirely, but it is important to realise the implications of even small deviations in Incidence.  As a rule, try to get all your groups to have an Incidence of 80%+.

There are sources of Incidence information available, the ABS, past studies and similar studies.

Key 1:  Understand the sample source

Will it be from a customer database, commercially available list or panel, a study of the general population or passing traffic? 

Can the contacts be pre-defined so that they match the desired target more closely? i.e. if you can separate the male contacts you increase your incidence from 50% to 100%.

Lesson:Pre-define your sample list as much as possible; spending time here will give you options to manage the risk of unknown incidences.  Explore different sample sources for this, to try to get as predefined as your design will allow.

Key 2:  Do you really need them? Can you live with less?

If you can’t get the incidence up beyond 30% for a target group, consider whether it is practical for the project?  It is far better to deliver comfortably on what you know than over promise on the unknown.  Best practice is to allow for individual target incidences when designing your quotas and understand the potential costs of any boost requirements.

From a commercial viewpoint, if they are hard to reach for research they are equally difficult to communicate to in a marketing sense.  Your client will have an opinion on this and other ways to reach them, but expressing the additional costs of such segments may re-orientate their views on their value.

Lesson:Think hard about the value of low incidence respondents – perhaps consider the option of a smaller sample size studied in more in depth

 Key 3:  Run a Pilot, Workshop the Results

If you’ve done your homework you should be pretty close.  A pilot is very important, particularly for telephone interviewing, a 10% pilot will give you an indication of where the incidence of each group is falling.     

A pilot will provide you some definition on which groups are likely to cause an issue.  It is only in the pilot that items such as contact list accuracy or traffic flow can be tested.  Be sure to discuss the results and their different implications with both your supplier and client before proceeding.

So often the implications of incidence post a Pilot go unheeded under the pressure of ‘moving things forward,’ be sure to consider all the options before locking in the remainder of the project.  It is better to make changes at this point than to have to make adjustments to costs and timeframes down the line.

A quality data collection partner should be able to help in sourcing respondent ideas, pre-planning, issue flagging and damage control options.  

Lesson:Run a pilot and really analyse the results, ask questions.  You will not only understand if you’re design is working, you’ll also be able to assess if your budget and assumptions were correct, giving you and your client even greater faith in the results.

Back to Best Practices Library