But is it Art(ificial Intelligence)?
Download as docx, pdf0 likes194 views
The document discusses the evolution of networking towards automation and formal design methods, drawing parallels with toolsets used in software and hardware engineering. It introduces the concept of network design automation and highlights ongoing developments in using formal methods for network verification and reachability analysis. The author emphasizes the urgent need for these advancements as networks grow more complex and the demand for efficient operations increases.
But is it Art(ificial Intelligence)?
- 1. But is it Art(ificial Intelligence)? And is it Good? And Who Says So? Its D辿j vu all over again. - Yogi Berra No army can stop an idea whose time has come. - Victor Hugo Last week, I blogged for my employer on machine learning in networking which was based on a podcast that our CTO had given on the same topic. We have an enterprise version of an open- source automation platform called StackStorm that wires apps, services and workflows together, and whose incipient vision was to build a self-driving data center. Not as fancy as this propeller head, but very useful and popular, and less vulnerable to this sort of nonsense. But thats not what I wanted to talk about today. Ive been thinking about trends in networking that may or may not (I think not) qualify as machine learning based on the definitions, which Ill paraphrase (read: oversimplify) as: Algorithms, trained by data, perform better over time (they get better based on what they learn) They add parameters for decision making based on continual new data They apply to situations where, practically speaking, its overly difficult to define rules manually This is a fancy set of requirements that emphasize the learning part of the definition. It generated discussion and is a hot topic to be sure. And I am ~100% sure we will get there (and that, like all AI, it will be something to treat with caution!). However, I there are clearly intermediate steps, and a broader cross-domain view of networkings next frontier, that may be even more interesting, if only because we are right at its doorstep. In this article, Im going to describe a movement thats underway (hint: its sort of a second wave of SDN) in both the academic and commercial side of networking, and give what I expect will be the first of many other examples of how we are taming this discipline over the next few years.
- 2. Lets (For a Change) Get Formal My thoughts as to whats coming next come from formal methods that George Varghese and Nick McKeown sum up as network design automation. Briefly, Varghese draws upon programming history and electronic design automation, finding (along with McKeown) methods and tool kits in both domains that now have relevance in networking: Programmers have access to a panoply of tools including compilers, debuggers, static and dynamic checkers. Hardware designers have synthesis, place-and-route, and timing verification tools. (Source here) But data network operations are still largely peppered with venerable thirty year old ping and traceroute. By comparison to programming and hardware, this is paper clips and rubber bands, and we at least need Legos. Side Note: Do follow the ping link if you want a little bit of folklore from its (sadly deceased) author, early Internet super-hacker Mike Muuss. Mike was what we can safely call a real programmer. But the building blocks are in place, its an ideal time for networking to start taking on the toolsets that have helped formalize and ensure engineering success in software and hardware development and design. Programmers, after all, have long had access for decades to compilers, debuggers, intermediate code generators, static and dynamic checkers, and linkers. The first full screen debuggers for programming languages were available in the 1980s. Similar tools for Computer Aided Design and Manufacturing (CAD/CAM) came shortly thereafter. Very-large-scale integration (VLSI) has incorporated hardware description languages that inform optimal gate- level designs, place and route analysis tools, and timing verification tools. Indeed, by the turn of the century, design and simulations systems for FPGAs and ASICs took on many of the characteristics of more flexible software development tools: hardware designers could code their systems in software, verify and check them in software, debug them in software, etc. Why Has Networking Lagged? In 2011, Scott Shenker gave a talk entitled The Future of Networking and the Past of Protocols in which he recounted an under-abstracted approach of one new protocol per problem. McKeown details (SIGCOMM, 2014) how this makes debugging difficult: there are complex interactions between multiple protocols on a switch or router, and between state on different
- 3. switches and routers. Operators cant control and observe the ever-changing state. Network engineers wrestle daily with misconfigurations that could not possibly have seemed unreasonable at the time they were committed. This also has something to do with the physical nature of networks. Networks just kind of grew up to be spacious and complex, with different design approaches depending on their range, morphing into many types of spaghetti cabling (with varied propagation delay and unpredictable physical length), varietal decision boxes (switches, routers, hubs, optical equipment), and interface cards supporting many different speeds and opportunities to be misaligned. A perfect storm of complexity, one new feature bolted onto another. As Varghese observes, this conglomeration does not resemble code, and is harder to get control of than either a software system or a circuit board. But still, in the spirit of SDN (kind of a next step from the original central control vision), the network must be tamed by being treated as a program that takes packets from specified inputs and sends them (by rewriting headers for the decision boxes). So improvement time is nigh, and in fact thats what were seeing. This is especially important because the networks themselves are growing more complicated. Load balancers spread traffic using opaque hash functions, firewalls prohibit messages based on cryptic rules and syntax that varies between vendors. Essential Symmetry: Software, Hardware, Networks To get to the answer--and find the right level of abstraction for the problem--lets look at the questions being asked of networks: Is a host (or a network) reachable? Why are these interfaces losing packets? Why is my Internet connection intermittent? Are the virtual network slices (VLANs, subnetworks) truly segregated and not vulnerable to broadcast storms? And, conversely, do subnetworks have ways to communicate when they need to? Moreover, networks support users, applications, devices, access to backend systems, cloud services, etc. This is their reason for being. So related questions include: Are my users free from attacks? Even ones they bring on themselves (by being phished, for instance) What about other endpoints, such as servers or applications? Can network data help identify ideal workload placement? Can reconfiguration optimize application performance? To make a cloud service more accessible and higher performing, should I use a regional hub or a larger interface?
- 4. But the Costs! Quick segue to the money. Thats what drives the design and the sales side of the house. Network failures are, by any account, are a constant source of runaway operational expenditure, particularly in the age of tablets and mobile devices, many of them being brought into enterprise environments and accessing cloud services in the age of BYOD. Framework Now lets take a look at what McKeown and Varghese see as the potential parallels between EDA and similar toolsets for networks. Figure 1: Digital Hardware Design as Inspiration (Source: McKeown SIGCOMM 2012) There are many topics that we can deal with over time to address this movement for formalized design automation. Here we will address just one: verification. Example: Verification This verification example is for reachability and will address functional verification (EDA) versus verification/reachability (NDA) side of the equation.
- 5. The example uses an abstraction for end-to-end reachability called Header Space Analysis (HSA). This allows you to treat the network as a program: you can model the following: Packet headers as points in header space Routers as functions (taking actions) on headers: possibly rewriting but determining reachability and potential headers that can reach next hops Networks as composite functions: composing the router functions to form a network program that takes points at entrances and transforms them via intermediate spaces to points at the network exit. Again oversimplifying, operations are as follows: 1. The system models sets of packets, based on relevant TCP/IP header bits, as subsets of {0,1, wildcard (*)} in three-dimensional header spaces (cubes) 2. We can perform set algebra on these header spaces as packets traverse the network 3. We can abstract away middle boxes (from your favorite vendors) as transfer functions on sets of headers 4. Thus, packets that reach across paths are expressed as compositions of these transfer functions 5. When we find all the packets that reach specified nodes you can check the results against your original reachability specification. We can use HSA (Figure 2) to compute all packet headers if any that can reach between any two hosts, say A and B.
- 6. Figure 2: Header Space Analysis (Source: George Varghese: When Clarke Meets Cerf) The large cube at Host A shows the entire header space: all packets that can be sent. As packets go through transfer functions at intermediate boxes, the header spaces become smaller. All hypercubes in the figure represent sets of possible packets. Ultimately, the complexity depends on the headers, the paths, and the number of rules in each box and in the entire network. The problem is to establish all permutations of reachability between Hosts A and B. To do this, we inject the space of all packets headers that A can send into Box 1. As packets traverse each box, the transfer functions subdivide the header spaces. As the simulation begins to cover the network, exact sets of packets that can reach B from A are expressed as the composition of the transfer functions of the boxes in all possible paths. This algorithm can be extended and modified for different of these middle boxes: routers, switches, load balancers, firewalls, etc.
- 7. Back to the Original Questions The answer to the title question is, of course, it depends. There was certainly machine learning in the early vision for StackStorm, the tool I mentioned above that serves as the brains behind workflows. But the need for formalization, akin to software programming workbenches like Visual Studio and EDA tools that fulfill the functional blocks of Figure 1, is very keen. Major work is being done to foster this concept for networks, and the momentum is clearly building because its time has come. Other Links for Fun and Profit A Distinguished Series Speaker Talk by George Varghese, discusses the growing work in treating computer networks as programs. I originally came across Varghese through his somewhat popular (well, I have it and see it around) work, Network Algorithmics: a multidisciplinary approach to networking that draws upon principles for best practices in hardware, software and systems. Youtube video of Varghese for Distinguished Lecture Series at UC Irvine: When Hoare Met Cerf Similar talk by Professor Varghese at University of Washington Header Space Analysis: the principal inventor is Peyman Kazemian.