ISSUING OUTPUT SPECIFICATIONS:
All too common Data Analysis Issues
and How to Solve Them. 

Summary
The end of a data collection project produces a mass of data to be formulated, interpreted and communicated, the assistance of a skilled Data Processing Analyst will speed this process up, ensuring you get the most out of what you collect.  Like Researchers, there are varying skill levels in DP professionals, they are “propeller heads” – no doubt about it, and communicating what you need, can sometimes be akin to speaking a foreign language.  If you find a good DP professional, with great communication, responsiveness and the ability to glean insights from similar projects to bring to your own, then they can add immeasurably to your reporting and importantly protect you from making amateur mistakes or basic oversights.  Here I describe the think I've learnt from others mistakes, and what I’m thinking when you are issuing instructions to about your data. 

Challenges and Solutions       

Recurring Issue No.1: Not ensuring the base is the base you need – Instructions for each question can contain filters that ensure the segment of the population you’re studying maps to the responses you receive.  The segment is often referred to as the base number i.e. Of 200 people surveyed, we want to know what 100 males thought in question 5, the base is therefore 100 males, not the 200 people.  You can see that in this example, to get it wrong would junk the results.  Hint: if the data you received doesn’t look right, this is the number one reason.  

Recurring Issue No.2: Beginning without the end in mind - this might be a fundamental but it’s worth repeating as it often brings the inexperienced unstuck.  If the data collection doesn’t contain the data you think you now need or is in a format that is confusing to interpret, there is little to be done without major costs to resolve this.  The solution is to design the output specifications as you design the questionnaire – the best in the business do this, it speeds up the process and ensures you’re thinking about this while you have it top of mind, not as an afterthought.  Talk this out with your DP contact, work out what is possible – creating fully designed specifications creates efficiencies at each end of the process. 

Good DP teams like AFS will run a logic check program to map out the results and provide valuable feedback on the following:

• Which questions to apply Table column Banners
• Which questions will benefit from significance testing results
• Logic of questionnaire Skips
• Extending the code frames for each question (as above)
• Insights into filters
• Etc etc…

Recurring Issue No.3: Not checking the “in field” data properly – similar to above, yet not all Researchers thoroughly consider what the initial “hole counts” (a colloquial term for the basic counts of the data for each question in the first days of collection) are telling them.  Most importantly check to see fields are being populated with responses in the way you expected, if not you have a programming issue (unlikely – ed!) or the question responses/the respondents are not as you expect. As a time saver ask for an interim file (say 25%-50% through the program) if there is any chance you think you may find data that you don’t expect and will need to alter the output specifications for.  Waiting till the end, once you’ve received your final data, then finding out there is a new avenue which you’d like to explore, can add anything from a day or more to the process.  When you’re in the flow of writing the report, this not the time you want to stop and wait.  Take the pressure off yourself and your DP relationship by theorizing about this with actual data while the project is still in field. 

Recurring Issue No.4: Not standardising data – once in field there maybe recurring answers you did not consider which need code frames developed, the most frequent is don’t know or refused to answer how you treat these answers can impact the quality of the data and the logical flow of the questionnaire.  Your organisation may have an internal policy of how to treat (label) these answers, so try to be consistent and work from a template that includes how to treat these answers and include them in each question.  Culturally your organisation may use a specific format – binomial or multi-max for multi-choice questions, you will need to instruct the DP team on this.  Remember your data maybe in someone else’s hands, they need to interpret it the same way you do.    

Recurring Issue No.5: Always ask for an SPSS/ASCII file – Frequently you will complete a project and you’ll need to return to it for further interrogation, this maybe in a couple of months or a couple of years, for that client or another.  Having received Excel or Word table outputs may have been efficient at the time but having the raw source data is better.  Also if you have it, this doesn’t tie you down to returning to the same data collection house or DP operative you used.

Enjoy your Week, Vishal.

See Researching Older Australian's Article

See Customer List Traps Article

Return to Article Library

See a further Article on Researching Doctors and Specialists