Design-based
research can produce
knowledge that
normally could not
be generated by
theoretical analysis
or traditional empirical
approaches.

that it be fully specified [ 4]. On the other hand, the development of theoretical constructs and standards without their grounding in a concrete design often leads to a range of problems, as shown in Henning’s discussion about the reasons for the decline of CORBA (Common Object Request Broker Architecture) [ 7]. Henning concluded that standards consortia must ensure they standardize only existing best practices and that no standard should be approved without a reference implementation and without having been used to implement a few projects of realistic complexity.

 

September + October 2011

interactions

variables, and we can control all extraneous variables that might affect the outcome. These conditions significantly limit the scope of experimental research, and in many real-world situations we cannot fulfill them, as a researcher usually cannot maintain control over all factors that may influence the result of an experiment. Attempting to simplify a real-world situation so it can be subjected to experimental research often leads to studying unrealistically simple situations. Attempting to establish an experimental control in a real-world setting, on the other hand, may lead to negative phenomena, such as the Hawthorne effect, in which those who perceive themselves as members of the experimental, or otherwise favored, group tend to outperform their controls, often regardless of the intervention.

Although significantly different,
controlled experiments and design-
based research are compatible
forms of research that can be and
often are used together. Controlled
experiments, for example, can
guide design decisions and test
particular elements of a design
on a smaller scale and in more
controlled conditions. A decision
about which input control to use in
a user interface, for instance, may
be based on the results of a con-
trolled experiment comparing the
efficiency of users’ data input with
several alternatives. Controlled
experiments may provide reli-
able information that something
“worked,” but they often do not pro-
vide sufficient information about
exactly what it was that worked, or
why or how it worked. Design-based
research can help us to character-
ize and identify relevant variables,
create an explanatory framework
for the results of the experiments,
and provide us with more insights
about why and how some elements
of a design work.

Why Design Can Reveal Things
That Other Methods Cannot

Design-based research complements existing research methods in its ability to employ in a greater amount the tacit, implicit, intuitive knowledge and skills of both designers and users. Schön calls such knowledge knowing-in-action, revealed only in the way in which we carry out tasks and approach problems: “The knowing is in the action. It is revealed by the skillful execution of the performance— we are characteristically unable to make it verbally explicit” [ 8].

In other words, though we cannot explain such knowledge and skills, we can demonstrate them by being engaged in a particular activity. This observation is supported by studies of embodied cognition, which emphasizes the formative role that the environment plays in the development of cognitive processes [ 9].

A design activity can set in motion our intuitive and tacit knowledge accumulated through years of research and experience. Much of such valuable knowledge is not captured in existing theories and guidelines. Often, we are not