Why is it important to move from
craft to engineering?
Doing so will help us cope with the
ever-increasing challenges of a more
automated, more interconnected
world—where small improvements in
software performance can make the
difference between profit and loss;
where a reputation for robustness,
scalability, and security can add millions to the share price; and where software is more and more the public face
of the business.
The codified knowledge and professionalism of an engineering discipline
are necessary for:
˲Sustaining and growing delivery
capability through changes in technologies, teams, and suppliers.
˲Predictably scaling operations
from early prototypes to global rollouts.
˲ Taking control of investments and
knowing when to pivot to solutions
more likely to deliver favorable returns.
˲ Systematically growing the levels
of reuse and interoperability of solution components and systems.
˲Producing long-lived solutions
with affordable costs of ownership.
Perhaps not everybody needs to move
from craft to engineering. As Mary Shaw
says, “The greatest need for engineering
discipline exists for software systems
that are fully automated and are operating unattended and where the consequences of failure are catastrophic.
Examples are telecom equipment, nuclear safety devices, medical implants,
self-driving cars, and stock-trading
programs. The need for engineering in
software development depends upon
how serious the consequences are when
things go wrong and whether human
beings can take action in time to minimize the consequences.” There is also
a strong need for engineering systems
used for e-commerce, finance, electronic medical records, and even human resources. The consequences of failures
in such systems may not include the immediate loss of life, but they can still be
“catastrophic” to either the businesses
or the individuals affected.
Thus, for many organizations and
software systems craft is not enough.
The good news is there is a way for-
ward that maintains the values of agile
while making software development
more of an engineering discipline than
a craft. It involves:
their teams change and their systems
mature.
Another frequent example of unsus-tainability is in the way many companies are facing an uncontrolled explosion in the number of applications they
have to support and the overall cost of
ownership of IT as a whole.
So industrial-scale agile requires
much more than just being able to
scale agile. It also means taking a disciplined approach to ensuring IT investments result in sustainable benefits for
both the producing organization and
its customers.
This involves adopting a different
approach to many aspects of agility.
We need to look beyond small-scale agile, beyond independent competitive
islands of agile excellence, beyond
individual craftsmanship and heroic
teams, and beyond the short-term instant gratification that seems to be the
focus of many well-intentioned but self-centered agile teams. It is this adoption
of a more holistic approach that we call
moving from craft to engineering. (See
Jacobson3 for more background.)
From Craft to Engineering
The move toward agility has led to many
benefits for the software industry. It has
broken the tyranny of the prescriptive
waterfall approach to software engineering, an approach that was causing
more and more large project failures,
and it has allowed software developers
to keep up with the ever-increasing demand for more innovative IT solutions.
It has enabled many companies to
do great things but in many cases has
led to a culture of entitlement, heroic
programming, and short-term thinking that threatens the sustainability of
the parent companies and the IT solutions on which they depend. Little or
no thought is put into maintainabil-ity, the heroes become potential single
points of failure, and the cost of keeping the lights on just keeps growing
and growing.
What is needed is a way to maintain
the values of agility while making software development more an engineering discipline than a craft—a new form
of agile software engineering fit for the
Internet age.
What are craft and engineering?
The term craft is usually applied to peo-
ple occupied in small-scale production
of bespoke goods and trades where
skills are passed in person from master
to apprentice. Engineering, on the other
hand, is defined by Wikipedia as “the
application of mathematics, empiri-
cal evidence and scientific, economic,
social, and practical knowledge in or-
der to invent, innovate, design, build,
maintain, research, and improve struc-
tures, machines, tools, systems, com-
ponents, materials, and processes.”
There have been many discussions
about whether or not the term engi-
neering should be applied to software
development and whether or not soft-
ware engineers are actually engineers.
With the rise of cloud computing, big
data, and the Internet of Things, how-
ever, it is clear there are many types of
software and many aspects of software
development that would benefit from
an engineering approach.
In her 1990 seminal paper, “Pros-
pects for an Engineering Discipline of
Software,”
7 Mary Shaw suggested a defi-
nition of software engineering would
include these clauses: “Creating cost-
effective solutions … to practical prob-
lems … by applying scientific knowledge
… building things … in the service of
mankind.” She also said about software
work that “most tasks are routine and
not innovative,” but it “is treated more
often as original than routine,” imply-
ing that there is a lot of potential for im-
proving quality and shortening time to
market “if we captured and organized
what we already know” by codifying our
knowledge, possibly even automating it.
Her observations are still highly rele-
vant; at the Go To Amsterdam 2015 con-
ference on software development, she
talked about the progress made toward
establishing a software engineering dis-
cipline. According to Shaw, the charac-
teristics of engineering are as follows:
˲ Limited time, knowledge, and re-
sources force decisions on trade-offs.
˲ The best-codified knowledge, preferentially science, shapes design decisions.
˲ Reference materials make knowledge and experience available.
˲ Analysis of design predicts properties of implementation.
Although software development
shares many of the characteristics of
an engineering discipline, we are not
there yet. The rise of agile is not a problem unless this is where we stop.