Gabriel A. Reich's article, Testing Historical Knowledge: Standards, Multiple-Choice Questions and Student Reasoning, addresses the following research questions:
1. How can the knowledge and reasoning used by relatively
successful and unsuccessful participants to answer a set of
multiple-choice history questions be characterized?
2. Did the items evoke the thinking called for in the
standards?
1. How can the knowledge and reasoning used by relatively
successful and unsuccessful participants to answer a set of
multiple-choice history questions be characterized?
2. Did the items evoke the thinking called for in the
standards?
- qualitative study
- knowledge and reasoning = used in multiple choice test taking per the Regent's exams
- successful and unsuccessful participants = sampling from “Metropolitan School” which is a small, urban high school of fewer than 400 students - 15 students from a single 10th grade history class participated
The article explains the history, methodology, findings, discussion and conclusions for the above research questions. A small sample of 15 students in Mr. Heche's 10th grade history class at Metropolitan School (NY state) were given 15 multiple-choice questions, observed taking their "test" on historical topics and interviewed. The researchers used codes in their observations to assess students cognitive processes and used additional codes to express their inferences. The article provides many findings that point toward the tendency for teachers to focus on literacy skills and content knowledge in their teachings, vs. discipline-based skills, but that Mr. Heche did not do this. He did, however, abide by a curriculum that followed the test standards closely.
Questions:
- What was the motivator for using such a small sample size? (15 students vs. that many students from several classes) Was it due to the selection criteria and if so why wasn't that criteria modified so that the results using a larger sample could be more generalizable?
- If the research question is to explore how independent variables like knowledge and reasoning are to be classified - why was history, versus math, chosen as the test subject for the tests?
- Did the objectives stated in the New York State World History and Geography standards use the same variables (cognitive processes) to express knowledge in reasoning or were those imposed by researchers?
- Would a rival hypotheses be to use an equal sample size with the same characteristics (class/test subject, SES, age, etc...) but with a teacher found not to be teaching to the test standards? If the results were similarly accurate could this add to the test's validity?
- Were there any events (history) that affected the outcome of the research?
- In the conclusion a generalization is made that teachers tend to focus on literacy skills and content knowledge in their teachings. Is this drawn from the literature research and not from the actual qualitative study?
