Quantitative Appraisal of Research Design and Analysis of Data
Appraisal of Research Design in a Quantitative Study
1. Was the design experimental, quasi-experimental, or nonexperimental?
What specific design was used? Was this a cause-probing study? Given
the type of question (therapy, prognosis, etc), was the most rigorous
2. What type of comparison was called for in the research design? Was
the comparison strategy effective in illuminating key relationships?
3. If the study involved intervention, were the intervention and control
conditions adequately described? Was blinding used, and if so, who
was blinded? If not, is there a good rationale for failure to use blinding?
4. If the study was nonexperimental, why did the researcher opt not to
intervene? If the study was cause-probing, which criteria for inferring
causality were potentially compromised? Was a retrospective or
prospective design used, and was such a design appropriate?
5. Was the study longitudinal or cross-sectional? Were the number and
timing of data collection points appropriate?
6. What did the researcher do to control confounding participant
characteristics, and were the procedures effective? What are the threats
to the study's internal validity? Did the design enable the researcher to
draw causal inferences about the relationship between the independent
variable and the outcome?
7. What are the major limitations of the design used? Were these
limitations acknowledged by the researcher and considered in
interpreting results? What can be said about the study's external
Questions for Appraising Analysis of Data
1. What statistics were used to describe the characteristics of the sample?
What statistics were used to analyze the data that were collected? Were
the statistics appropriate for the level of measurement?
2. Were measures of central tendency provided? If so, which ones were
used? Are they the most appropriate? How sensitive to outliers is the
3. Were measures of variation/dispersion provided? Were standard
deviations reported for each mean that was reported?