In mitigation, we reduce on-target
testing with more static verification.
Secondly, if we know that code is completely unambiguous, then we can justify testing on host development machines and reduce the need to repeat
the test runs on target. Hardware simulation can give each developer a desktop virtual target or a fast cloud for the
deployment pipeline. While virtual-ization of popular microprocessors is
common, high-fidelity simulation of
a target’s operating environment remains a significant challenge.
On one embedded project, all development of code, static analysis, and
testing is done on developers’ host machines, which are plentiful, fast, and
offer a friendly environment. A final
re-run of the test cases is performed
on the target hardware with the expectation of pass-first-time, and allowing
the collection of structural coverage
data at the object-code level.
High-integrity practices can comple-
ment Agile. We previously mentioned
the use of static verification tools.
While we have a preference for devel-
oper-led, sound analysis, we recog-
nize that some projects might find
more benefit in unsound, but easier to
ways able to accept delivery of the
product and use it immediately. This
is not realistic for high-integrity proj-
ects. Some customers have their own
acceptance process, and regulators
may have to assess the system before
deployment. These processes can be
orders-of-magnitude slower than a
typical Agile tempo.
iFACTS uses a deeper pipeline and
multiple iteration rates, with at least
four builds in the pipeline:
˲ Build N: in operation with the customer.
˲ Build N+1: undergoing customer
acceptance. This process is subject to
regulatory requirements, and so can
˲ Build N+2: in development and
˲Build N+3: undergoing requirements and formal specification.
All four pipeline stages run concur-
rently with multiple internal iteration
rates and delivery standards. The de-
velopment team can deliver to our test
team several times a day. A rapid build
can be delivered to the customer (in,
say, 24 hours), but comes with limita-
tions on its assurance package and
allowed use: it is not intended for op-
erational use, but for feedback from
the customer on a new feature. A full
build (perhaps once every six months)
has a complete assurance package, in-
cluding a safety case, and is designed
for eventual operation. The trick is
to make the iteration rates harmonic,
both with each other and with the cus-
tomer and regulator’s ability to accept
and deploy releases.
Embedded Systems Issues. Agile
presumes plentiful availability of fast
testing resources to drive the development pipeline. For embedded systems, if the hardware exists, there may
be just one or two target rigs that are
slow, hostile to automation, and difficult to access. We have seen projects
revert to 24-hour-a-day shift-working
to allow access to the target hardware.
of fast testing
resources to drive the
High-integrity Agile evidence engine.