Common Guidelines for Conducting Education Research
education from the project? What knowledge do the PIs believe they will gain? What
evidence supports that belief? The guidelines
lay out a logical sequence or “pipeline” of
evidence gathering, as shown in Figure 1. The
pipeline view illustrates a high-level introduction to the guidelines rather than a detailed
depiction of the entire Common Guidelines
document. The guidelines identify six types
of knowledge and organize them into three
groups representing projects that:
1. Contribute to our core knowledge of
how people learn.
2. Develop solutions to achieve a learning
3. Contribute to the body of evidence
that demonstrates the success of the
interventions and strategies developed
in achieving the intended goals.
The questions to the right of each type
of research reflect the question it asks. The
complete guidelines outline the purpose,
significance, theoretical, and empirical
basis for each type of research. They also
describe project outcomes and relevant
research and external feedback plans for
each, with concrete examples of awards to
illustrate each type of research.
Software developers will note the similar-
ity between the last three types of research,
shown in blue in Figure 1, and alpha, beta,
and production testing. At each stage, the
audience broadens and the contexts in which
the PIs gather data more closely represents
an operational deployment of the innovation.
PIs new to education research often ask
what metrics and evidence can be used when
a novel intervention or strategy is added to
a course, and small class sizes and limited
offerings of the course make it infeasible or
impossible to conduct a study with a control
group, or where no benchmarks exist. Such
instances call for a bit of ingenuity. For example, the PIs can conduct formative assessment
workshops that invite faculty, students, and
stakeholders to the PIs’ institution(s) to review
and test the idea. PIs can ask participants to
complete surveys and pre- and post-tests and
participate in focus group discussions that
provide valuable feedback to the PIs. PIs can
include in their grant proposals small stipends
to compensate participants for their time and
travel expenses. Alternatively, a PI with the
habit of saving scores for individual problems
on assignments, exams, and quizzes, can
use this data from pre-intervention runs of a
course to provide benchmark data for evaluating their hypothesis.
While Common Guidelines identifies a
pipeline, it also emphasizes the actual complexity of building knowledge. The guidelines
are descriptive rather than prescriptive. In software engineering, the waterfall model is a way
to think about traceability. However, the model
only works as a system development pipeline
in a perfect and unrealistic world in which
developers know all requirements, stakeholders, software, and hardware before the project
begins, and exclude updates as platforms
evolve and innovations as developers gain
knowledge about the system. Similarly, the
pipeline in Common Guidelines provides a
framework for understanding, analyzing, and
structuring education research projects and
their relationship to one another. The research
journey looks more like the highly-connected
network of Figure 2 in which each type of
research can inform, grow out of, and lead into
the others. Researchers are free to use any of
the nodes in the network as the starting point
for their work, providing they establish the
appropriate purpose, significance, theoretical, and empirical basis as well as project
outcomes and relevant research and external
feedback plans for that type of research.
Reducing the fifty-three pages of the
Common Guidelines into 1000 words is
an impossible task. My hope is that this
column provides an orientation to the
guidelines and motivates you to explore
the report and rely on it to inform your
education research projects.
Tip: Solicitations often include a list of
references relevant to the solicitation. They
are there for a reason—a wise PI will become
familiar with those references before writing
a proposal in response to the solicitation.
1. Erlinger, M. and Tymann, P. NSF Program Officers’
Views – Inside the NSF. ACM Inroads, 7, 3 (2016), 17-18.
2. Erlinger, M. and Tymann, P. NSF Program Officers’
Views – Inside the NSF. ACM Inroads, 7, 4 (2016), 40-41.
3. Institute of Education Sciences and National Science
Foundation. Common guidelines for education
research and development; https://www.nsf.gov/
pubs/2013/nsf13126/nsf13126.pdf. (2013). Accessed
2016 November 12.
4. National Science Foundation—ATE; Advanced
Technological Education (ATE); https://www.nsf.gov/
pubs/2014/nsf14577/ nsf14577.htm. Accessed 2016
5. National Science Foundation—ECR; http://www.nsf.
gov/pubs/2015/nsf15509/ nsf15509.htm/. Accessed
2016 November 12.
6. National Science Foundation—EHR; http://www.nsf.
gov/ehr/about.jsp. Accessed 2016 November 12.
7. National Science Foundation—S-STEM; NSF
Scholarships in Science, Technology, Engineering, and
Mathematics (S-STEM); http://www.nsf.gov/pubs/2016/
nsf16540/ nsf16540.htm/. Accessed 2016 November 12.
8. National Science Foundation—STEM; Improving
Undergraduate STEM Education: Education and
Human Resources (IUSE: EHR); http://www.nsf.gov/
pubs/2015/nsf15585/ nsf15585.htm/. Accessed 2016
9. National Science Foundation—STEM+C; STEM +
Computing Partnerships (STEM+C); https://www.
2016 November 12.
Stephanie E. August
NSF – DUE
4201 Wilson Blvd
Arlington, Virginia 22203 USA
Figure 1: A pipeline view of education research.
Figure 2: The web-like reality of education
research. Each type of research can contribute
to an evidence base that informs and justifies
other types of research [ 3].