Fighting Physics: A Tough Battle

applications, causing some cost in storage and throughput (to maintain the cache) to be incurred. In practice, this is almost always a good bet when replicas can be made, because growth in storage capacities and network throughputs appears to be increasing on a steady exponential. Prefilling the cache with data likely to be used means that some capacity will be wasted (what is fetched but not needed) but that the effects of some delays will be mitigated when predictions of what is needed are good.

Think relentlessly about the architecture of the distributed application. One key observation is that a distributed system can be distributed based on function. To return to the design of a system with a live data store (such as a stock market), we might place the program trading of stocks near the relevant exchanges, while placing the user interaction functionality, account management, compliance logging, etc. remotely in less exchange-local real estate. Part of such a functional decomposition exercise is identifying where latency makes a difference and where the delay must be addressed directly rather than via caching techniques.

Where possible, adapt to varying latencies. The example of protocols maximizing throughput by adapting to bandwidth-delay capacities shows how a wide range of latencies can be accommodated. For distributed applications, this might be accomplished by dynamically relocating elements of a system (e.g., via process migration or remote evaluation).

None of these suggestions will allow you to overcome physics, although prefetching in the best of circumstances might provide this illusion. With careful design, however, responsive distributed applications can be architected and implemented to operate over long distances.

SUMMARY

Propagation delay is an important physical limit. This measure is often given short shrift in system design as application architectures evolve, but it may have more performance impact on real distributed applications than

bandwidth, the most commonly used figure of merit for networks. Modern distributed applications require adherence to some rules of thumb to maintain their responsiveness over a wide range of propagation delays. Q

ACKNOWLEDGMENTS

Comments from the ACM Queue editorial board, particularly Eric Allman, and from Craig Partridge greatly improved this article.

REFERENCES

1. Mohr, P. J., Taylor, B. N. 2005. CODATA recommended values of the fundamental physical constants. Reviews of Modern Physics 77(1): 1-107.

2. 40-gig router test results. 2004. Light Reading; http://www.lightreading.com/document.asp?doc_ id=63606&page_number= 4&image_number= 9.

3. Partridge, C. 1994. Gigabit Networking. Addison-Wesley Professional.

4. Shaffer, J. H., Smith, J. M. 1996. A new look at bandwidth latency tradeoffs. University of Pennsylvania, CIS TR MS-CIS-96-10; http://repository.upenn.edu/cgi/ viewcontent.cgi?article=1192&context=cis_reports.

 

LOVE IT, HATE IT? LET US KNOW feedback@queue.acm.org

JONATHAN M. SMITH is the Olga and Alberico Pompa Professor of Engineering and Applied Science and a professor of computer and information science at the University of Pennsylvania. He served as a program manager at DARPA from 2004 to 2006 and was awarded the OSD (Office of the Secretary of Defense) Medal for Exceptional Public Service in 2006. He is an IEEE Fellow. His current research interests range from programmable network infrastructures and cognitive radios to disinformation theory and architectures for computer-augmented immune response. © 2009 ACM 1542-7730/ 09/0200 $5.00