other uses for modeling
Modeling is an iterative process. It
should not be thought of as just some
type of performance test. Here is a list
of items that could be added to further
the evaluation process.
˲ ˲ Use the model to evaluate various
infrastructure choices. These could
include messaging middleware, operating system and database-tuning parameters, network topology, and storage system options.
˲ ˲ Use the model to create a performance profile for a set of hardware,
and use that profile to extrapolate
performance on other hardware platforms. Any extrapolation will be more
accurate if the model is profiled on
more than one hardware platform.
˲ ˲ Use the performance profiles to
determine if multiple instances of the
publisher (horizontal scaling) are likely
to be required as the system grows. If
so, this capability should be built into
the design and modeled appropriately.
Converting components designed to
be singletons could be very expensive.
˲ ˲ Use the model to explore the set of
possible failure scenarios. Availability is one of the primary attributes of
a quality system. Waiting to address it
after a system is built can cost an order
of magnitude more.
The examples used in this article
can be seen in abstractions of many
systems. Similar modeling approaches should be undertaken for any material component. When interrelated
models have been built and tested
they can then be combined for more
comprehensive system modeling. The
approach of building one model at
a time allows the system behavioral
knowledge to be gained in steps rather
than attempting to understand—not
to mention build—one all-encompass-ing model.
One key element present in almost
all systems is some type of data store.
Evaluating a database design can be
complex. There are a number of steps
that are similar to the system model-
ing already discussed, however. Once a
draft of the database model (columns,
tables, and so on) is available, it can
be populated with enough generated
data to enable some performance test-
ing. The effort required to write a data
generator for this purpose will give an
idea of how easy it will be to work with
the database during the development
process. If this generator seems too
difficult to tackle, it may be a sign the
database model is already too complex.
instrumentation and maintenance
Regardless of the type of driver/com-ponent combination, instrumentation is vital to both modeling and the
long-lasting health of a system. It is not
just a luxury. Flying blind about performance is not advised. Visual flight
rules (that is, without instrumentation) can be used only when the skies
are clear. How often is that true for
modern systems? The functional and
technical complexity typically clouds
the ability to see clearly what is happening. System performance can be
like floating down the river in a raft.
If you do not observe the speed of the
water periodically, then you might not
notice an upcoming waterfall until the
raft is hopelessly plunging over the
edge. As mentioned previously, when
the volume of instrumentation data is
too high, consider using “tracers” and/
or statistical sampling.
There are numerous advantages to
keeping the drivers and model components up to date as a system evolves:
˲ ˲ They can be used for general regression testing for performance,
availability, or scalability, when changes are proposed.
˲ ˲ They can be used for capacity planning by extrapolating performance
from a smaller set of resources. The
only practical way to do this is by fully
understanding the resource usage
characteristics.
˲ ˲ They can model infrastructure or
other large-scale changes that may
need to be made to an existing system.
˲ ˲ At times there are factors outside
the control of the maintenance/devel-opment team (for example, infrastructure changes). The drivers could be
used to test an isolated portion of the
system. If any degradation was caused
by the outside factors, then the results
could provide “defensive” data to have
the changes altered or rolled back.
˲ ˲ When some type of performance,
availability, scalability, or other infrastructure problem arises, it would be
much quicker to pull out the model
and drivers than to take on the possibly
overwhelming task of updating them
while under pressure to troubleshoot a
production problem.
Modeling is an extremely powerful
method to understand and improve
the overall quality of a system. For
systems expected to last for years this
improvement translates into real monetary savings. Development organizations can then spend their budgetary
money on providing functionality. If
the models and associated drivers are
sustained, then this functional focus
can be widely celebrated.
Related articles
on queue.acm.org
hidden in Plain Sight
Bryan Cantrill
http://queue.acm.org/detail.cfm?id=1117401
Visualizing System Latency
Brendan Gregg
http://queue.acm.org/detail.cfm?id=1809426
Performance Anti-Patterns
Bart Smaalders
http://queue.acm.org/detail.cfm?id=1117403
References
1. booch, g. Object-oriented Analysis and Design with
Applications (2nd edition). benjamin cummings,
redwood city, ca, 1993.
2. Petzold, c. Programming windows. Microsoft Press,
1988.
Kevin Montagne has over 25 years of experience in
the it field working with large-scale systems where
performance and availability were critical. he spent 20 of
those years in the financial industry, including more than a
decade as an architect of front-office trading systems.