Reading Reports of Empirical Studies

The Method section is a detailed description of exactly how the study was conducted. The detail is important so the reader can determine the appropriateness of the method for answering the scientific questions. It is also important for another researcher to be able to replicate the study. This section reads a lot like a recipe. The following questions will help you evaluate the method:

• Who are the participants in the study?
• Are the participants appropriate for the study?
• What is the research design?
• Is the design appropriate for the research question(s)?
• What are the measures?
• Are the measures appropriate for addressing the research question(s)?
• What ethical considerations are important to address? Are they all addressed in the article?

These questions are a mix of fact-finding questions and critical thinking questions. The Method section contains a lot of scientific jargon and it is essential to understand exactly what was done in order to evaluate whether it was done "right". Read the Method section in the article then work through the summary and questions below.

Are the participants appropriate for the study?

What is the research design?

Is the design appropriate for the research question(s)?

What are the measures?

Are the measures appropriate for addressing the research question(s)?

What ethical considerations are important to address? Are they all addressed in the article?

Participants

A large number of grade 11 students from Minneapolis, Minnesota, U.S.A., Taipei, Taiwan, and Sendai, Japan participated. The participants were selected to be representative of the city in which they lived. The cities were chosen to be as comparable as possible in terms of socioeconomic and cultural status within their respective countries.

Materials

Interview items were developed simultaneously in each language. The items were designed to obtain information about how the participants spent their day, including time spent in various academic and nonacademic activities.

A mathematics test, covering information taught in all three countries, was constructed to measure achievement.

Procedure

Participants were interviewed and tested after school by native speakers. Data on time use when the participants were younger was also available.