Top 25 Biases of Research Readers 

This article examines validity and bias issues from the perspective of both the researcher and report user. It proposes issues and questions to assess the validity of research undertaken while highlighting the research users personal interpretation bias which are often overlooked.

Weaknesses occur in Research. Evaluating the validity and reliability of the information you find is a crucial step in Evidence-Based Management. Some weaknesses are the result of bias, and some are the result of confounding variables.

Research bias is any inferences from the truth because of the way(s) in which the study is conducted. Two key points that must be introduced:

1. Bias does not necessarily carry an allegation of prejudice, such as the investigators’ desire for particular results, but it should be the role of users of research reports to determine whether the weakness is serious enough to warrant reinterpretation of the study's finding.
2. There is more potential for bias in the recipient of the research report than in the research itself. In providing a valid research report the advocate should identify the possible bias for the user to consider.

This articles examines and summaries these points of validity and bias.

Describing what is Valid Research

There are two types of validity: internal validity and external validity. Internal validity must be established before external validity. Internal validity tries to determine the connection between the independent variable and dependent variable - Internal validity therefore asks, "Did the action cause the effect?" To begin to determine the validity use the questions below;

• Is there a clear research question which creates such an hypothesis?
• Does the research method match the question?
• Is the sample drawn from the population to which the researchers seek to generalise the results?
• Is randomisation employed when appropriate at sampling and response level?
• How is the analysis performed, and what statistical methods are used?
• Are the conclusions supported by the study's findings about both variables?

External validity is the ability to generalise the study results to a larger population. The most common loss of external validity comes from initially drawing a small study sample population from a single geographic location.

Questions to consider concerning external validity:

• How representative of the larger population are the study groups - %?
• How large are the samples?
• How diverse are the samples, do they cover a broad range of key segments?
• Do the demographics of the study samples match the demographics of the larger population?

Corollary - In many research designs, there may be a "trade-off" between internal validity and external validity, whereby measures are taken to increase the degree of internal validity may also limit the generalisability of the findings.

Informed Bias

What is not produced from validity tests is how the user of research applies their own bias Blind spots (the tendency not to compensate for one's own cognitive biases) when interpreting a report. It is important for researchers to identify bias cues so as to properly present the nature of the reality that the report's data contains. Below are the applicable biases; by no means exhaustive – I'm sure you can identify these in your;

1. Confirmation bias — the tendency to search for or interpret information in a way that confirms one's preconceptions.
2. Base rate fallacy — ignoring available statistical data in favor of particulars.
3. Congruence bias — the tendency to test hypotheses exclusively through direct testing, in contrast to tests of possible alternative hypotheses.
4. Déformation professionnelle — the tendency to look at things according to the conventions of one's own profession, forgetting any broader point of view, also Group think.
5. Distinction bias — the tendency to view two options as more dissimilar when evaluating them simultaneously than when evaluating them separately.
6. Framing — Using an approach or description of the situation or issue that is too narrow, also a framing effect — drawing different conclusions based on how data is presented or collected.
7. Illusion of control — the tendency for human beings to believe they can control or at least influence outcomes that they clearly cannot.
8. Information bias — the tendency to seek information even when it cannot affect action.
9. Irrational escalation — the tendency to make irrational decisions based upon rational decisions in the past or to justify actions already taken.
10. Mere exposure effect — the tendency for people to express undue liking for things merely because they are familiar with them.
11. Need for closure — the need to reach a verdict in important matters; to have an answer and to escape the feeling of doubt and uncertainty. The personal context (time or social pressure) might increase this bias.
12. Neglect of probability — the tendency to completely disregard probability when making a decision under uncertainty.
13. Not Invented Here — the tendency to ignore that a product or solution already exists, because its source is seen as an "enemy" or as "inferior".
14. Outcome bias — the tendency to judge a decision by its eventual outcome instead of based on the quality of the decision at the time it was made.
15. Reactance — the urge to do the opposite of what someone wants you to do out of a need to resist a perceived attempt to constrain your freedom of choice.
16. Von Restorff effect — the tendency for an item that "stands out like a sore thumb" to be more likely to be remembered than other items.
17. Authority bias — the tendency to value an ambiguous stimulus (e.g., an art performance) according to the opinion of someone who is seen as an authority on the topic – also Base rate fallacy.
18. Hindsight bias — sometimes called the "I-knew-it-all-along" effect, the inclination to see past events as being predictable.
19. Positive outcome bias — a tendency in prediction to overestimate the probability of good things happening to them (refers also to the aptly named Wishful thinking effect and the Ostrich effects.)
20. Primacy and recency effect — the tendency to weigh initial events more than subsequent events.
21. Mental effort effect — the tendency to prefer the easy to understand rather than the correct.
22. Disregard of regression toward the mean — the tendency to expect extreme performance to continue.
23. Texas sharpshooter fallacy — the fallacy of selecting or adjusting a hypothesis after the data is collected, making it impossible to test the hypothesis fairly. Refers to the concept of firing shots at a barn door, drawing a circle around the best group, and declaring that to be the target.
24. Fundamental attribution error — the tendency for people to over-emphasize personality-based explanations for behaviors observed in others while under-emphasizing the role of situational influences on the same behavior.
25. System justification — the tendency to defend and bolster the status quo. Existing social, economic, and political arrangements tend to be preferred, and alternatives disparaged sometimes even at the expense of individual and collective self-interest

Back to Best Practices Library