Print.
A A

Evaluating research

In an information-rich society we are bombarded with press releases about the latest research, consumer market surveys, crime statistics, health studies, public opinion polls and new advances in technology.

How can we tell the good research from the poorer research? How can we, for instance, tell the difference between credible research and cleverly disguised marketing?

As an organisation that commissions, collates and analyses research that can be used to advise government, the Families Commission has a vested interest to ensure that research is reported accurately. The following checklist will help to determine the quality or appropriateness of the research.

 

A simple checklist for evaluating research

Reprinted with the kind permission of Research First

Dimension Ask yourself Details to check
Face value
 What do the results say?
  • Do the results make sense?
  • How easy are the results to understand?
Context What do other studies have to say about this issue?
  • How does this study fit with others?
  • What else does this study have to say? (have results simply been cherry-picked?)
  • Who else can you talk to about this?
Motivation Who did the research and why?
  • Who funded the research?
  • Who conducted the research?
  • Is there a vested interest?
  • Are the researchers reporting their findings or attempting to influence an audience.
Quality How do the researchers know the things they claim?
  • How many people did they ask?
  • How did they select these people?
  • How did they ask them?
  • What did they ask them?
  • When did they ask them?
  • What analytical techniques were used?
  • How did they interpret the results?
  • Have the data been weighted?
  • Are the researchers clear about the limitations of the research?
Philosophical Has the research addressed common sources of bias?

How have the researchers avoided common biases such as:

  • selection bias
  • confirmation bias
  • publication bias
  • the Hawthorne Effect
  • the implicit referent
  • the ecological fallacy.

Remember:

  • Always be wary if you cannot get access to the original research questionnaire, data coding schedule, and the raw data
  • Be wary of research that concludes with the call for more research.

 

Research Companies in New Zealand

The Market Research Society of New Zealand is the professional body for the market research industry. A list of member organisations is available on their website.

AttachmentSize
A simple checklist for evaluating research (PDF)254.35 KB
Back to top