Research Methods: Meta-Analysis (Class)

What is it?
A meta-analysis is a survey in which the results of the studies included in the review are statistically similar and are combined and analysed as if they were one study.” (
It’s basically where a number of studies are combined together if they returned similar results, and are treated as one batch of information.

What kinds of questions/problems might it be useful for?
In general, situations where statistics are lacking and so a broader study with lots of gathered data would be beneficial to the subject as a whole, such as investigating

How could it be used in IT research?
Meta-analysis can be used to verify or justify new hypothesis or theories regarding IT development decisions, such as finding the audience for new technology or user interest in certain developments. Meta-analysis is useful for gaining a statistical view of the research topic, and so would likely be useful for establishing where IT development is wanted or needed through studies on user response and interest to new technologies, GUIs or just small features or a program. In general meta-analysis seems like it would be more useful for developing technologies than just feedback for the improvement of pre-established technologies due to the multi-study nature of meta-analysis, but it would depend on the circumstance of the studies and the subject.
Decisions about the utility of an intervention or the validity of a hypothesis cannot be based on the results of a single study, because results typically vary from one study to the next. Rather, a mechanism is needed to synthesize data across studies.” (

What are the main strengths of the approach?
Large amounts of data from meta-analysis can be valuable, especially when investigating new ground of a subject. It generally increases the power of one study over that of an individual study.

What are the main weaknesses of the approach?
The studies need to be very well/carefully done in order for the data to be considered valid, and so the reasoning for doing a meta-analysis needs to also be very valid. “Indeed, it is our impression that reviewers often find it hard to resist the temptation of combining studies when such meta-analysis is questionable or clearly inappropriate.” (Egger et al. (2002)), “In reality, if carefully performed, it yields useful information, but a meta-analysis of badly designed studies produces erroneous statistics and may be misleading.” (Julien I.E. Hoffman (2015),