cisions, describing why we did something, not only what we have done. This
is not easy, as it often requires making
explicit the elements practitioners use
intuitively. But such documentation is
extremely valuable, especially for others, to understand why some things
worked well or not.
Generalization. Knowledge obtained from practice can be viewed as a
generalization of experiences. In a normal design effort the primary goal is
to create a successful product, and lessons learned are restricted to the particular design and the people involved
in it. To be useful to others, some effort has to be invested in generalizing
these lessons.
Generalization enables correlating
different experiences, which other wise
may look too specific. In the process
of generalization, practitioners need
to expand their focus beyond the current design situation, viewing the design problem, solutions, and processes
as instances of more general classes.
However it is also important not to
“over-generalize.” Practice usually
cannot provide us with insights to develop “grand” and universal theories.
Rather, emphasis should be between
narrow truths specific to some situation, and broader knowledge covering
several similar situations.
Evaluation. Evaluation is a necessary part of any learning process. This
is especially true for domains, such
as software engineering or human-computer interaction, where we have
to deal with complex human and social issues for which we do not have
strong theories, models, and laws. By
evaluation I do not necessarily mean
formal evaluation activities conducted
in “laboratories” (even though these
may be sometimes used). I see evaluation as a systematic effort to get feedback on our findings and quality of our
work that is not based on our intuition.
Even simple techniques, such as peer
reviews may be incredibly efficient in
identifying shortfalls in our problem
analysis, the solution, and the design
procedure. In order to enable such a
process, practitioners need to be prepared to make their reasoning explicit,
public, and open to critical reflection
and discussion. The key is to make
your intuitive decisions more explicit
and “vulnerable” to the critique of others and empirical findings.
Iterations. Lastly, to maximize
learning, all of the previous elements
should be applied in a number of iteration. While a single event can have an
impact, it usually takes many events
to extract general features and generate rules from experience. In everyday
work, you should try to combine elements of preparation, actual practical
work, and evaluation.
Learning should not stop at the
end of a project. New insights and
broader generalizations will often oc-
cur through retrospective analyses
of lessons learned and data collected
through a whole project. Brooks, for
example, spent several years analyzing
and reflecting on the lessons learned
in design of OS/360, producing the
influential The Mythical Man-Month
[ 5]. These macro-iterations of retro-
spective analyses can also happen on
a broader scale, covering several proj-
ects from different contributors. For
example, in special issues of journals,
editors often spend some time sum-
marizing and generalizing findings
from individual articles.
CONCLUSION
Doing research in practice is valuable both for the practice and for the
research community. But for you as a
practitioner, doing research in practice
may be a competitive advantage. You
may bring to practice original ideas and
results of cutting edge research, and
you may bring to the research community lessons researchers from academia
do not have the opportunity to obtain.
To read the full version of this article,
please visit http://xrds.acm.org/article.
cfm?aid=2611810.
References
[ 1] Obrenović, Z. Design-based Research: What we learn
when we engage in design of interactive systems.
Interactions 18, 5 (2011), 56-59.
[ 2] Obrenović, Z. Rethinking HCI education: teaching
interactive computing concepts based on the
experiential learning paradigm. Interactions 19, 3
(2012), 66-70.
[ 3] Obrenović, Z. Research and Practice: The curious case
of ‘small’ researchers-practitioners. Communications
of the ACM
56, 9 (2013), 38-40.
[ 4] Simon, H. A. The Sciences of the Artificial. MI T Press,
Cambridge, 1996.
[ 5] Brooks, F.P. The Mythical Man-Month. Addison-Wesley
Professional, 1995.
[ 6] Brooks, F.P. The Design of Design: Essays from a
Computer Scientist. Addison- Wesley Professional,
2010.
Biography
Željko Obrenović is a technical consultant with the
Software Improvement Group in Amsterdam, The
Netherlands. Before joining SIG he worked as a researcher
and best practices consultant at Backbase, an assistant
professor at the Technical University in Eindhoven, and as
a researcher at C WI in Amsterdam. In his work, he aims at
bridging software design research and practice, trying to
get best of both worlds.
Practice is a rich
and still hugely
unexplored
area, and as a
practitioner you
may be in a unique
position to witness
or make important
discoveries in
many areas of
computing.
Only 12% of U.S. executives
identify as being in a minority,
although minorities make up
35% of all U. S. employees.