other uses for modeling

Modeling is an iterative process. It should not be thought of as just some type of performance test. Here is a list of items that could be added to further the evaluation process.

˲ ˲ Use the model to evaluate various infrastructure choices. These could include messaging middleware, operating system and database-tuning parameters, network topology, and storage system options.

˲ ˲ Use the model to create a performance profile for a set of hardware, and use that profile to extrapolate performance on other hardware platforms. Any extrapolation will be more accurate if the model is profiled on more than one hardware platform.

˲ ˲ Use the performance profiles to determine if multiple instances of the publisher (horizontal scaling) are likely to be required as the system grows. If so, this capability should be built into the design and modeled appropriately. Converting components designed to be singletons could be very expensive.

˲ ˲ Use the model to explore the set of possible failure scenarios. Availability is one of the primary attributes of a quality system. Waiting to address it after a system is built can cost an order of magnitude more.

The examples used in this article can be seen in abstractions of many systems. Similar modeling approaches should be undertaken for any material component. When interrelated models have been built and tested they can then be combined for more comprehensive system modeling. The approach of building one model at a time allows the system behavioral knowledge to be gained in steps rather than attempting to understand—not to mention build—one all-encompass-ing model.

One key element present in almost
all systems is some type of data store.
Evaluating a database design can be
complex. There are a number of steps
that are similar to the system model-
ing already discussed, however. Once a
draft of the database model (columns,
tables, and so on) is available, it can
be populated with enough generated
data to enable some performance test-
ing. The effort required to write a data
generator for this purpose will give an
idea of how easy it will be to work with
the database during the development
process. If this generator seems too
difficult to tackle, it may be a sign the
database model is already too complex.

instrumentation and maintenance

Regardless of the type of driver/com-ponent combination, instrumentation is vital to both modeling and the long-lasting health of a system. It is not just a luxury. Flying blind about performance is not advised. Visual flight rules (that is, without instrumentation) can be used only when the skies are clear. How often is that true for modern systems? The functional and technical complexity typically clouds the ability to see clearly what is happening. System performance can be like floating down the river in a raft. If you do not observe the speed of the water periodically, then you might not notice an upcoming waterfall until the raft is hopelessly plunging over the edge. As mentioned previously, when the volume of instrumentation data is too high, consider using “tracers” and/ or statistical sampling.

There are numerous advantages to keeping the drivers and model components up to date as a system evolves:

˲ ˲ They can be used for general regression testing for performance, availability, or scalability, when changes are proposed.

˲ ˲ They can be used for capacity planning by extrapolating performance from a smaller set of resources. The only practical way to do this is by fully understanding the resource usage characteristics.

˲ ˲ They can model infrastructure or other large-scale changes that may need to be made to an existing system.

˲ ˲ At times there are factors outside the control of the maintenance/devel-opment team (for example, infrastructure changes). The drivers could be used to test an isolated portion of the system. If any degradation was caused by the outside factors, then the results could provide “defensive” data to have the changes altered or rolled back.

˲ ˲ When some type of performance, availability, scalability, or other infrastructure problem arises, it would be much quicker to pull out the model and drivers than to take on the possibly overwhelming task of updating them while under pressure to troubleshoot a production problem.

Modeling is an extremely powerful method to understand and improve the overall quality of a system. For systems expected to last for years this improvement translates into real monetary savings. Development organizations can then spend their budgetary money on providing functionality. If the models and associated drivers are sustained, then this functional focus can be widely celebrated.

 

Related articles
on queue.acm.org
hidden in Plain Sight

Bryan Cantrill

http://queue.acm.org/detail.cfm?id=1117401

Visualizing System Latency

Brendan Gregg

http://queue.acm.org/detail.cfm?id=1809426

Performance Anti-Patterns

Bart Smaalders

http://queue.acm.org/detail.cfm?id=1117403

References

1. booch, g. Object-oriented Analysis and Design with Applications (2nd edition). benjamin cummings, redwood city, ca, 1993.

2. Petzold, c. Programming windows. Microsoft Press, 1988.

Kevin Montagne has over 25 years of experience in the it field working with large-scale systems where performance and availability were critical. he spent 20 of those years in the financial industry, including more than a decade as an architect of front-office trading systems.