Evaluating Information from
Research Articles and Presentations

Material on classroom practices and interventions can be useful to educators who examine it critically, looking for quality indicators and evidence of a scientific approach. The No Child Left Behind Act identifies scientific research as “vigorous, systematic, and objective”. Empirical studies are those based on and verifiable by experience, observation, and experiment. Resulting data are reliable and valid no matter who performs the experiment or how often. Before trying something recommended in an article or presentation, teachers can and must ask if those who recommend it can back up their claims with evidence, and also check bibliographies for independent, reliable sources.

It's useful to remember that the scientific method, an organized, step-by-step way of approaching a question or problem, has been the research standard for the fields of medicine, psychology, and health professions such as nursing and counseling. These are the scientific method's steps.

1. Define the question or problem for study.
2. Collect available information about the question from libraries, the internet, interviews, and observation.
3. State the study's hypothesis, a brief if/then or cause/effect statement to test.
4. Design and implement an experiment that gathers data under controlled conditions to test the hypothesis.
5. Analyze and interpret the resulting data.
6. Form a conclusion that can be independently verified. It may support the hypothesis or not.

Peer-reviewed educational journals, those where the material submitted is read and approved by other researchers in the same field or by independent experts, publish articles that represent one or more of these steps. For instance, a school might report great results observed with a set of instructional materials, or a company could announce information about a new piece of assistive technology. Published educational research represents four stages of investigation:

• Stage 1- ideas, hypotheses, observations, pilot work
• Stage 2- controlled lab experiments, classroom-based demonstrations and experiments
• Stage 3- randomized classroom trials
• Stage 4- informed classroom practices.

There are several types of studies that are accepted as evidence-based. Single-subject research or case studies can consist of observations or experiments that report results for one individual in a typical school environment. Correlational studies examine for cause and effect in randomly selected test and control populations using currently accepted best practices in measurement. Qualitative studies observe what happens in a specific context. These types of research have in common that they

• test a theory
• document change or lack of it
• manipulate one variable to record change in a second
• use valid measurements, appropriate for the data
• interpret the data correctly
• produce results that can be reported, discussed and replicated
• result in scientific evidence that can lead to adoption of a teaching practice.

So a “Hey, we tried this” article can be valuable as a starting point or pilot work, but hasn't generated enough evidence to justify adopting the practice without further study. It's the first stage of research and in need of rigorous, high-quality testing.

References

Carnahan, D. & Fitzpatrick, M. (2003). Don't get buried under a mountain of research. [Electronic version] Journal of Staff Development, 24(2). www.nsdc.org/library/jsd/carnahan242.html

Graham, S. (Ed.) (2002). Criteria for evidence-based practice in special education. [Special Issue] Exceptional Children, 71(2).

Stanovich, P.J. & Stanovich, K.E. (2003). Using research and reason: How teachers can use scientifically based research to make curricular and instructional decisions. [Electronic version] National Institute for Literacy, Jessup, MD.
www.nifl.gov/partnershipforreading/

U.S. Department of Education. (2003). Identifying and implementing educational practices supported by rigorous evidence: A user friendly guide. [Electronic version] (Institute of Education Sciences). Washington, DC: Author.
www.excelgov.org/evidence/ , publications page

U.S. Department of Education, Office of the Deputy Secretary. (2004). No Child Left Behind: A toolkit for teachers. [Electronic version] (Education Publications Center) Jessup, MD: Author.
www.ed.gov/teachers/nclbguide/nclb-teachers-toolkit.pdf

Research on Instruction | Collect Classroom Data