being used instead of artificial keys, sometimes referred to as surrogate keys. The system generates them ( typically as integers) to represent the “real” keys. This can improve performance and maintenance when dealing with complex key structures and/or when the actual key values can change.

We determined that material improvements were possible if we restructured the database design and changed the associated SQL statements. The programs were written in such a way that would have made the changes very expensive, however. Our conclusion was that the system would need a major overhaul if it were to be successful. Since the project had already spent well over $10 million, this recommendation was a hard sell.

After an additional $5 million, the project was canceled, and my team’s focus was redirected to other efforts. The modeling process had taken only about six weeks. The point to be made here is that it would be possible to use modeling to vet the major architectural decisions before committing large expenditures. It is vastly less expensive to discover that a design will not perform or scale before a system is built rather than after it has been placed in production.

modeling new Systems

It should be standard practice to research the architectural options for new systems—or when making substantial overhauls to existing ones. The experiments should be with lightweight models rather than a full system, but it is vital that these models accurately capture the evolving behavior of the system. Otherwise the value of the modeling process is diminished and may lead to erroneous conclusions.

I typically start by trying to understand the functional problem space in an abstract fashion. Is the primary functionality a user-requested action followed by a system reply (such as, request/reply)? Is it a request followed by a stream of notifications (for example, ticking quotes) or bits (for example, music or video)? Is it to process some input data and send the result to another process or system (such as, flow-through)? Is it to crunch through a massive dataset in search of information (decision support system)? Is it a

it should be
standard practice
to research the
architectural
options for new
systems—or when
making substantial
overhauls to
existing ones.

combination of these, or something altogether different?

Some may ask: how do I know which portions of the system to model and how much time and effort should be spent in the process? It is a simple case of risk management. The modeling should focus on the areas that would be the most expensive to get wrong. The process should continue until the high-risk decisions can be justified. Make an effort to retest the decisions as often as practical.

One of the most challenging aspects in modeling is in finding the right balance between capturing enough of the system behavior and keeping the model from becoming too complex (and expensive) to implement. This is easier with an existing system. As you progress through the modeling iterations, if the observations begin to mimic aspects of the system, then you are probably pretty close. You can begin to alter the modeling drivers and components to explore more of the behavior. For a new system I typically look to model components that can be used as shells for the real component. The goal is to provide the responsible developer with a starting point that allows the focus to be on the functionality rather than having to explore the critical nuances of the underlying technology and infrastructure.

There are numerous technical modalities to consider when designing or evaluating architecture: performance, availability, scalability, security, testability, maintainability, ease of development, and operability. The priority ordering of these modalities may differ across systems, but each must be considered. How these modalities are addressed and their corresponding technical considerations may vary by system component. For example, with request/reply and streaming updates, latency is a critical performance factor, whereas throughput may be a better performance factor for flow-through message processing or bulk-request functionality. A perhaps subtle but nonetheless important message is to avoid mixing different modality implementations within the same component. Failure to adhere to this lesson puts the architecture on a sure path to complexity.

It is far too common to hear the ex-