PSYC 4483 Research Methods

 

Use this checklist to critically evaluate your research articles.

 

From Meltzoff, J. (1998). Critical thinking about research: psychology and related fields. Washington, DC: American Psychological Association.

 

Introduction

 

Is the research question clearly stated?

 

Does the introduction, statement of the problem, and overview of the literature adequately set the background for the reader? Is this material consistent with the research question?

 

Given the research question and the background material, are the hypotheses appropriate and clearly stated?

 

Are the key terms well defined?

 

Variables and Measures

 

Is the IV appropriate given the question of the study? Are the levels of the independent variable appropriate, valid, and reliable?

 

Is the DV appropriate for the study?

 

Are the criterion and criterion measure of the dependent variable appropriate, valid, and reliable? Are the scoring, rating, or judging procedures valid and reliable? Is the apparatus (if any) accurate and reliable?

 

Are the controls appropriate? Can the results be affected by variables that have not been controlled? Are control or contrast groups (if present) properly selected?

 

Is the research design suitable to test the hypotheses and answer the research question?

 

Participants

 

Are the methods and procedures clearly described in sufficient detail to be understood and replicated?  Are the participants properly oriented and motivated? What is their understanding of the task? Are the instructions sufficiently clear and precise? Is communication among the participants a factor? Are there signs of experimenter bias in the design, data collection, assessment, analysis, or reporting?

 

Are the participants properly selected? Is the sample representative and unbiased? Do the procedures adhere to the guidelines for the protection of experiment participants? Is the size (N) of the sample appropriate? Are appropriate procedures used to assign participants to groups, treatments, or conditions? Are suitable techniques used to establish group equivalence, such as matching, equating, or randomizing? Does participant attrition occur, and if so, does it bias the sample? 

 

Data and Statistics

 

Are the statistical tests appropriate, and are the assumptions for their use met? Are the degrees of freedom correct and the error measures valid? Are there errors in calculation or presentation of statistical results?

 

Are the tables and figures clearly labeled and accurately presented? 

 

Are results correctly interpreted, properly reported, given meaning, and placed in context?

 

Is the discussion reasonable in view of the data?

 

Conclusion

 

Are the conclusions valid and justified by the data?

 

Are the generalizations valid?

 

Do the References match the citations in the text?

 

Are ethical standards adhered to in all phases of the research?

 

What would you do to improve or redesign the study?