John Doyle (doyle@caltech.edu)
John G Braun Professor
Control and Dynamical Systems, BioEngineering, and Electrical Engineering
Caltech, Pasadena, CA

Our research is increasingly focusing on the issue of architecture, the most universal, high-level, and persistent elements of organization, usually defined in terms of protocols— the rules that facilitate interaction between the diverse components of a network [28]. The nature of complexity, robustness, and evolvability has been a persistent source of errors and confusion both throughout science and between science and society, and highly evolved or designed architectures are perhaps at the heart of this confusion, which we aim to resolve.
This research builds on insights about the fundamental nature of complex biological and technological networks that can now be drawn from the convergence of three research themes.  1) With detailed descriptions of many components of biological networks, and the growing attention to systems biology, the organizational principles of these networks are becoming increasingly apparent 2) Advanced technology provides engineering examples of networks with complexity approaching that of biology. While the components differ from biology, we have found striking convergence at the network level of architecture and the role of layering, protocols, and feedback control in structuring complex multiscale modularity ( [1], [2], [4], [6], [13], [15], [16], [22]). Our research is leading to new theories of the Internet and related networking technologies, and to new protocols that are being tested and deployed ([3], [8], [10],  www.hot.caltech.edu,  netlab.caltech.edu).  3)  Most importantly, there is a new mathematical framework for the study of complex networks that suggests that this apparent network-level evolutionary convergence both within biology and between biology and technology is not accidental, but follows necessarily from the requirements that both biology and technology be efficient, adaptive, evolvable, and robust to perturbations in their environment and component parts (www.cds.caltech.edu/sostools, [2], [11], [17], [21],[22],[24],[25],[26],[27]). This theory builds on and integrates decades of research in pure and applied mathematics with engineering, and specifically with robust control theory.
                One of the simplest but most important observations is that biological systems are robust and evolvable in the face of even large changes in environment and system components, yet can simultaneously be extremely fragile to small perturbations. Such universally robust yet fragile (RYF) complexity is found wherever we look. The amazing evolution of microbes into humans (robustness of lineages on long timescales) is punctuated by mass extinctions (extreme fragility). Diabetes, obesity, cancer, and autoimmune diseases are side-effects of biological control and compensatory mechanisms so robust as to normally go unnoticed.
                RYF complexity is not confined to biology. The complexity of technology is exploding around us, but in ways that remain largely hidden.  Modern institutions and technologies facilitate robustness and accelerate evolution, but enable catastrophes on a scale unimaginable without them (from network and market crashes to war, epidemics, and climate change). Understanding RYF means understanding architecture — the most universal, high-level, persistent elements of organization — and protocols. Protocols define how diverse modules interact, and architecture defines how sets of protocols are organized.
            The Internet is an obvious and familiar example of how a protocol-based architecture facilitates evolution and robustness.  The hourglass protocol “stack” has a thin, hidden “waist” of universally shared feedback control (TCP/IP) between the visible upper (application software) and lower (hardware) layers. This allows “plug-and-play” between modules that obey shared protocols; any set of applications that “talks” TCP can run transparently and robustly on any set of hardware that “talks” IP, accelerating the evolution of TCP/IP-based networks. Diverse applications use additional protocols to exchange objects such as web files, images, and email.  In the hardware layer, signals are also exchanged via shared protocols and interfaces. Indeed, recent theory proves that with suitable versions of TCP/IP and network provisioning, an otherwise arbitrary (e.g. large and heterogeneous) network has guaranteed global dynamics that robustly optimizes aggregate application utility.  (Similar results hold for flocking and oscillator synchronization.) Other theory shows that overall system robustness obeys conservation laws, suggesting RYF behavior can be managed but not completely eliminated.
Within a layer these protocols can be visualized as “bowties” with large fan-ins and -outs of energy, materials, and information via a thin `knot” of universal protocols specifying carriers, building blocks, or interfaces.  Other engineered examples of bowties include networks which connect energy sources to consumers via carriers and standard socket-plug interfaces, sellers to buyers via money, fibers into diverse clothing via sewing, and raw materials to assemblies via standardized building blocks for advanced manufacturing.  In these and the biologic examples below, the currencies, carriers, intermediates, precursors, plugs, packets, conserved residues, and interfaces in the bowtie “knot” are highly constrained by protocols.  Yet this shared universality allows robust adaptation and evolution to extreme diversity at the inputs (energy, sellers, fibers, materials, nutrients, receptors) and outputs (consumers, buyers, clothing, assemblies, products, effectors), as long as they have appropriate (and typically hidden) layers of feedback control.
The robustness and evolvability of the cell exemplify reliance on layers and protocols analogous to those in TCP/IP and other engineered architectures, but with wondrously different hardware components and more extensive and sophisticated use of feedback control. Perhaps the ultimate bowtie involves the universal, central protocols that transcribe and translate DNA to proteins via conserved codes, polymerases, and RNA intermediates. As in IP, feedback control of polymerases yields a functional network with regulated levels of macromolecules.  A simple cell in a constant environment needs little more, but real cells add robustness to changing demands with a TCP-like feedback layer of allostery and post-translation modifications acting on faster time scales. These TCP/IP-like layers (the waist of a cell’s hourglass) of control protocols constitute much of biochemistry texts. 
In the microbial biosphere, genes that “talk” transcription and translation protocols can move by horizontal gene transfer (HGT), thus accelerating evolution in a kind of a “bacterial internet.”  Furthermore, proteins newly acquired by HGT or through duplication and divergence or domain rearrangements can work effectively because they can exploit additional shared protocols. For example, a bacterial cell’s application layer consists of biochemical fluxes with various additional bowtie protocols such as core metabolism (nutrients to biosynthesis and energy supply via conserved precursors and carrier-based group transfers) and signal transduction (receptors to downstream effectors via conserved residue pairs).  Thus selection acting at the protocol level could evolve and preserve this variety of shared and ultimately conserved architecture, essentially “evolving evolvability”.
                All life and advanced technologies rely on protocol-based architectures. The evolvability of microbes and IP-based networks illustrate how dramatic, novel, dynamic changes on all scales of time and space can also be coherent, responsive, functional and adaptive, despite implementations that are largely decentralized and asynchronous. New genes and pathways, laptops and applications, even whole networks, can plug-and-play, as long as they obey protocols.  Biologists can even swap gene sequences over the Internet in a kind of synthetic HGT. A related aspect of sophisticated architectures is that actuator signals (e.g. from intermediate fluxes and concentrations in core metabolism to cardiopulmonary rates and renal concentrating effects) have extremely high variability in order to keep other critical signals (e.g. from key metabolic products to blood oxygenation and pH, to core body temperature,) tightly regulated despite fluctuating supplies and demands.  Such natural physiological variability can be a source of confusion if homeostasis is misinterpreted as implying that all signals, including actuators, are held nearly constant.
                Typical behavior is fine-tuned with this elaborate control and thus appears boringly robust despite large internal and external perturbations and fluctuations.  As a result, complexity and fragility are largely hidden, often revealed only by rare catastrophic failures. Since components come and go, control systems that reallocate network resources easily confer robustness to outright failures, whereas violations of protocols by small malicious or even random rewiring can be catastrophic.  So programmed cell (or component) “death” is a common strategy to prevent local failures from cascading system-wide.  The greatest fragility stemming from a reliance on protocols is that standardized interfaces and building blocks can be easily hijacked. So that which enables HGT, the web and email also aids viruses and other parasites. Large structured rearrangements can be tolerated, while small random or targeted changes that subtly violate protocols can be disastrous.  Thus another source of high variability is in the external behavior of RYF systems, where typical behavior is boringly robust but rare events can be catastrophically large.
                Chaos, fractals, random graphs, criticality, and related ideas from statistical physics inspire a popular and completely different view of complexity, where behaviors that are typically unpredictable and fragile “emerge” from simple and usually random interconnections among homogeneous components. The attraction is understandable because a variety of features, continually recycled in series of “new sciences,” greatly simplify modeling and analysis.  For example, random internal rewiring has little effect.  Tuning, structure, and component heterogeneity is minimal, as is system functionality, robustness, and environmental uncertainty.  Perhaps most appealing is that “modularity” reduces to superficial self-similar patterns in ensemble averages, avoiding the complexity of protocols and architecture.  Unfortunately, highly evolved and architecture-based RYF complexity is utterly bewildering when viewed from this perspective, which has led to widespread and remarkably persistent errors and confusion about complex systems of all kinds.  There are striking parallels with irreducible complexity and intelligent design, which has even more successfully popularized persistent errors and confusion, though more narrowly targeting biology and evolution.
Particularly persistent errors surround the origin and nature of power laws.  The statistical invariance properties of Gaussians and exponentials for low variability phenomena are well-known enough that their presence is little cause for surprise.  But for high variability phenomena, power laws have even richer statistical invariants, and could rightly be called “more normal than Normal.”  Thus their ubiquity in the abundant high variability of architecture-based RYF systems should be no surprise, and certainly no evidence on their own for any specific mechanism beyond the presence of such high variability.
One thing all can agree on is that the search for a “deep simplicity and unity” underlying natural and technological complexity remains a common goal.  Fortunately, our growing need for robust, evolvable technological networks means the tools for engineering architectures and protocols are becoming more unified, scalable and accessible, and happen to include the more popular complexity views as extreme special cases within a much richer structure, organized, and dynamical perspective.  These developments are bringing much-needed rigor and relevance to the study of complexity generally, including biology, but will not eliminate the need for attention to domain-specific structure and detail. Quite the contrary: both architectures and theories to study them are most successful when they facilitate rather than ignore the inclusion of domain-specific details and expertise.

References

1. Csete M , Doyle J, Reverse Engineering of Biological Complexity, Science 295 (5560): 1664-1669 Mar 1 2002
2. Carlson JM, Doyle J, Complexity and robustness, P Natl Acad Sci USA 99: 2538-2545 Suppl. 1 FEB 19 2002
3. Low SH, Paganini F, Doyle JC,  Internet congestion control, IEEE Contr Syst Mag 22 (1): 28-43 FEB 2002
4. Csete M.E. and J.C. Doyle, 2004, Bow ties, metabolism, and disease, Trends in Biotechnology, Vol 22, Issue 9, pg. 446-450
5. J. Stelling, U. Sauer, Z. Szallasi, F. J. Doyle III, and J. Doyle, 2004, Robustness of cellular functions, Cell, October, 2004.
6. Doyle, J; Csete, M, 2004. Imitation of life: How biology is inspiring computing, NATURE. OCT 21             2004
7. H. El-Samad, H. Kurata , J.C. Doyle , C.A. Gross, and M. Khammash, 2005, Surviving Heat Shock: Control Strategies for Robustness and Performance, PNAS. PNAS (8): 2736-2741 FEB 22, 2005
8. Jin C, Wei D, Low SH, Bunn J, Choe HD, Doyle JC,et al,  FAST TCP: From theory to experiments  IEEE NETWORK 19 (1): 4-11 JAN-FEB 2005
9. Paganini F, Wang ZK, Doyle JC, et al. Congestion control for high performance, stability, and fairness in general networks , IEEE-ACM TRANSACTIONS ON NETWORKING 13 (1): 43-56 FEB 2005
10. Wang JT, Li L, Low SH, Doyle JC. (2005) Cross-layer optimization in TCP/IP networks, IEEE-ACM TRANSACTIONS ON NETWORKING 13 (3): 582-595 JUN 2005
11. Manning M, Carlson JM, Doyle J (2005) Highly optimized tolerance and power laws in dense and sparse resource regimes PHYSICAL REVIEW E 72 (1): Art. No. 016108 Part 2 JUL 2005
12. R. Tanaka, T-M Yi, and J. Doyle (2005) Some protein interaction data do not exhibit power law statistics, FEBS letters, 579 (23): 5140-5144 SEP 26 2005
13. Doyle et al, (2005), The “Robust Yet Fragile” Nature of the Internet, P Natl Acad Sci USA. vol. 102 no. 41, October 11, 2005
14. Zhou T, Carlson JM, Doyle J (2005) Evolutionary dynamics and highly optimized tolerance, JOURNAL OF THEORETICAL BIOLOGY 236 (4): 438-447 OCT 21 2005
15. L Li, D Alderson, JC Doyle, W Willinger (2005) Towards a Theory of Scale-Free Graphs: Definition, Properties, and Implications, Internet Math, 2005
16. D Alderson, L Li, W Willinger, JC Doyle (2005) Understanding Internet Topology: Principles, Models, and Validation, IEEE/ACM TRANSACTIONS ON NETWORKING, Dec. 2005.
17. R. Tanaka, M. Csete and J. Doyle, Highly optimised global organisation of metabolic networks, IEE Proceedings Systems Biology, 152 (4): 179-184 DEC 2005
18. MA Moritz, ME Morais, LA Summerell, JM Carlson, J Doyle (2005) Wildfires, complexity, and highly optimized tolerance, P Natl Acad Sci USA, December 13, 2005; 102 (50) ,
19. T Brookings, JM Carlson, and J Doyle (2005) Three mechanisms for power laws on the Cayley tree, Phys. Rev. E 72, 056120
20. J. Doyle and M Csete (2005), Motifs, stability, and control, PLOS Biology, November 2005,Volume 3, Issue 11
21. H El-Samad , A Papachristodoulou, S Prajna, J Doyle, and M Khammash (2006),  Advanced Methods and Algorithms for Biological Networks Analysis, PROCEEDINGS OF THE IEEE, 94 (4): 832-853 APR 2006
22. L. Chen, S. Low, M. Chiang, and J. Doyle. Cross-layer congestion control, routing and scheduling design in ad hoc wireless networks. In Proc. of IEEE Infocom, 2006.
23. Kurata, H El-Samad, R Iwasaki, H Ohtake, JC Doyle, et al. (2006) Module-based analysis of robustness tradeoffs in the heat shock response system. PLoS Comput Biol 2(7):  July 2006  (PMID: 16863396)
24. M Chiang, SH Low, AR Calderbank, JC. Doyle (2006) Layering As Optimization Decomposition, PROCEEDINGS OF THE IEEE, Volume: 95  Issue: 1  Jan 2007
25. Martins NC, Dahleh MA, Doyle JC (2007) Fundamental Limitations of Disturbance Attenuation in the Presence of Side Information,  IEEE Trans Auto Control, Feb 2007
26.  H Sandberg, JC Delvenne, JC Doyle (2006)  The Statistical Mechanics of Fluctuation-Dissipation and Measurement Back Action- American Control Conference 2007. Preprint available at http://arxiv.org/abs/math.DS/0611628.
27. J.-C. Delvenne, H. Sandberg, and J. C. Doyle, Thermodynamics of linear systems, European Control Conference 2007.
28. Doyle J, and Csete M, Rules of engagement. NATURE 446 (7138): 860-860 APR 19 2007 (PMID: 17443168)

Presentations

Invited speaker: “Design Principles in Biological Systems," The Banbury Center, Cold Spring
Harbor Laboratory, 7-10 May 2006

Invited speaker: International Worksho on Recent Advances in Biological Control, Frontier Research System 20th Anniversary, July 29-30, 2006, Bio-Mimetic Control Research Center, RIKEN, Nagoya, Japan

Organizer: Connections Workshop, Caltech, Aug 15-17, 2006

Invited panelist: Grace Hopper Celebration, Irvine, CA, Oct 4-5, 2006

Invited speaker: International Conference on Systems Biology, Oct 9-13, Yokohama, Japan.

Invited speaker: ICCAI 2006 Fall 2006 meeting of the Society for Complexity in Acute Illness, Washington, DC Oct 19-21, 2007

Keynote speaker: Engineering Principles in Biological Systems, Cold Spring Harbor Labs Conference,
December 3-6, 2006

Main plenary speaker: Workshop on Complex Networks and their Applications, January 22-24, 2007, Georgia Tech, Atlanta, GA

Invited speaker: "Navigating Complexity: Doing more with less"  IBM Almaden Institute Conference, April 11-12

Invited speaker: Design Principles in Biological Systems: Banbury workshop, Cold Springs, May 6-8, 2007 ()

Invited Speaker: Random and Dynamic Graphs and Networks, IPAM, UCLA, May 7 - 11, 2007