The Education Technology Industry Network (ETIN), a division of the Software & Information Industry Association (SIIA), has released an updated guide to research methods, authored by a team at Empirical Education Inc. The Guidelines for Conducting and Reporting EdTech Impact Research in U...
is designed to advise producers of educational technology who do not have research expertise, yet need to provide evidence that their product is effective and meets the current ESSA standards of evidence.
The authors walk the reader through the stages of Getting Started, Designing, Implementing, and Reporting while recommending 16 best-practice standards of research. Each standard is discussed in light of its background and rationale, with examples provided to illustrate points.
"Getting Started" talks about the early stages of generating evidence for a product and prior research to consider when building a rationale for its expected impact. In the "Designing the Research" stage, the guidelines focus on what does and does not count as evidence for ESSA. These guidelines include:
-Decide who is being tested: students, teachers, schools, or a combination
-Consider the four levels of evidence defined in ESSA
-Use random assignment if you have control over who gets the program now and who gets it later
-Use comparison group studies to show evidence of impact
-Use correlational designs to find a program's promise and how usage relates to outcomes of interest
The "Implementing" section discusses how to best make a study happen effectively, and "Reporting the Results" discusses best reporting practices and the risk in advocating a product with negative or un-robust outcomes.