Class 27 CS 439 23 April 2013 On the board ------------ 1. Last time: networking 2. Today: more networking --------------------------------------------------------------------------- 1. Last time --Networking --emphasis was on how information is encoded ... --... and on how it travels --Today: --other functions of a network --put the pieces together: see what happens when a browser makes a request 2. Networking A. Intro B. Physical layer C. Link layer D. Network layer E. What do we mean by layering? F. Zoom out G. Transport layer H. Application layer A. Intro --computer networks are interesting --end-points highly programmable, middle kind of boring (only kind of). --can program all of the nodes! --extremely easy to innovate and develop new uses of the network --contrast: telephone network: end-points ridiculously simple, middle has complexity. --worse, can't program most phones, need FCC approval for new devices, no visibility, etc. --If you're interested in this stuff, take classes in networking! Or program away! Or read the RFCs (short for "Request For Comments" but despite the name, they are standards). Few things are as open and well-documented as the various protocols that form, and run over, the Internet --Network classically explained as being divided into sharply distinguished layers. In reality, things are messier. But still incredibly useful to think about layering. B. Physical layer [you studied this last time] Note that physical connectivity is rare..... --instead, communications usually "hop" through multiple devices --[DRAW PICTURE: source --> bunch of switches --> destination ] --Allows links and devices to be shared for multiple purposes --Must determine which bits are part of which messages intended for which destinations Two kinds of ways to create this indirect connectivity: --Circuit-switched: provide virtual links. Dump bits in at source, they come out at the destination --example: the old telephone network. dialing the number set up a virtual circuit (and before that, human operators set up an actual circuit) --Packet-switched: --Pack a bunch of bytes together intended for same destination --Slap a _header_ on packet describing where it should go --Most networks today are packet switched C. Link layer Ethernet: classic technology History: developed at Xerox PARC by Metcalfe and others, intended to help with the office of the future, amazing technology. used constantly. however, not used much in its original configuration (of shared medium) because many links now point-to-point. --but if you plug your computers into a hub, your hardware is still going to use Ethernet's key features. originally designed for shared medium (coaxial cable) D. Network layer Internet Protocol (IP): classic technology (took over the world, pretty much literally) --IP used to connect multiple networks --Runs over a variety of physical networks --Most computers today speak IP Fundamentals --Every host has a unique 4-byte IP address (Or at least thinks it has, when there are address shortages) --for example: diligence.cs.utexas.edu is 128.83.130.113 www.cs.utexas.edu is 128.83.120.139 --Based on a destination's IP address, packets are routed --Address space structured to make routing practical at global scale --For example, UT Austin gets: 128.62.*.* 128.83.*.* 146.6.*.* etc. [see http://whois.arin.net/rest/org/UTAA/nets for a full list] (the top-level assignment is by IANA, who delegates to ARIN (for north america), who assigns to either UT or UT's providers.) --NOTE: there is a sharp separation between an entity's IP address and its attachment point in the network --*routing* solves the problem of knowing where all of the hosts are attached, and how to reach them --Dijkstra's algorithm, Link state, path vector, etc., etc. --Result: number of routing entries across the Internet vastly smaller than the number of addresses --this was hugely important for scaling. still is, though becoming less so (as memory gets cheaper) Upshot --Packets need IP addresses in addition to MAC addresses --Refer to picture E. Key idea: layering --packets inside packets (though different layers packetize differently from each other, so the picture below is a simplification) --[DRAW PICTURE 2x, ONE ON EACH SIDE] [app_payload] [TCP header | app_payload] [IP header | TCP header | app_payload] [Eth header | IP header | TCP header | app_payload] --[MAP THIS ONTO THE DIAGRAM OF THE BIG PICTURE, SHOWING THAT IP PIECE TRAVELS MOSTLY UNADULTERATED] --An IP router _forwards_ a packet from one Ethernet to another, creating a new Ethernet packet containing the same IP packet --In principle, layers should not depend on each other. In practice, there are annoying dependencies (TCP's checksum depends on fields in IP header) --Different layers have different functions --link layer: framing and media access --network layer: --forwarding --routing (NOTE: routing != forwarding) --------------------------------------------------------------------------- Encourage you to poke around: --"arp -a" (Unix) --"ifconfig -a" (Unix) --"netstat -arn" (Unix) --"ipconfig /all" (windows) --"route print" (Windows?) --------------------------------------------------------------------------- F. Zoom out: where are we? Above (mainly Sebastian's class) should have convinced you that if (a) a computer knew the IP address of a local router; and (b) that computer knew the IP address of the destination; and (c) we have a network that knows how to forward packets then --that computer could arrange for packets to travel to its destination Okay, but how do we get (a)--(c)? (a) two possibilities: --manual configuration --BTW, even edge routers get this thing configured manually. A third-tier ISP is told: "here's the IP address of the other end of this link." --If you have a cable modem, it does this --DHCP (b) Naming system: Domain Name System (DNS) (c) [DRAW PICTURE OF ROUTING: BGP, OSPF, etc.; ANOTHER FUNCTION OF THE NETWORK LAYER] WHAT'S NEXT? --we do not yet have a way to indicate what application or process on the destination computer gets the packet --we also don't cleanly handle things like failure, congestion in the network, etc. G. Transport layer Motivation: failure, demultiplexing, flow control, etc. DRAW PICTURE: layer role TCP UDP ICMP("ping") {flow control, port space} IP {forwarding} Ethernet {framing} radio copper_wires fiber {signal propagation} Several types of error can affect packet delivery --Bit errors (e.g., electrical interference, cosmic rays) --Packet loss (packets dropped when queues fill on overload) --Link and node failure In addition, properly delivered frames can be delayed, reordered, even duplicated How much should OS (or the networking modules) expose to application? --Some failures cannot be masked (e.g., server dead) --Others can be (e.g., retransmit lost packet) --But masking errors may be wrong for some applications (e.g., old audio packet no longer interesting if too late to play) UDP and TCP most popular protocols on IP --Both use 16-bit _port_ number as well as 32-bit IP address --Applications _bind_ to a port and receive traffic to that port (discuss later what the interface is) UDP: User Datagram Protocol --Exposes packet-switched nature of Internet --Sent packets may be dropped, reordered, even duplicated (but generally not corrupted). Application's problem to deal with these errors TCP: Transmission Control Protocol --Provides illusion of a reliable "pipe" between two processes on two different machines --Masks lost and reordered packets so apps don't have to worry --Handles congestion and flow control Uses of TCP --Most applications use TCP --Easier interface to program to (reliability) --Automatically avoids congestion (don't need to worry about taking down network) Many issues involved in implementing TCP: --Wants multiple packets outstanding --But want to react to congestion in the network (want to save network from congestion collapse) --TCP has to "learn" parameters per-connection --Connection set-up and tear-down is complicated --sender never knows if it's last packet was lost --so has to keep state around after connection close --Tons of hacks for good performance Issues directly for OS too: --Have to track unacknowledged data --Keep a copy around until recipient acknowledges it --Keep timer around to retransmit if no ack --Receiver must keep out of order segments and reassemble --When to wake process receiving data? --E.g., sender calls write (fd, message, 8000); --First TCP segment arrives, but is only 512 bytes --Could wake recipient, but useless w/o full message --TCP sets PUSH bit at end of 8000 bytes, to force write data --When to send short segment, vs. wait for more data --Usually send only one unacked short segment --But bad for some apps, so provide NODELAY option --Must ack received segments very quickly --Otherwise, effectively increases RTT, increasing bandwidth-delay product but without increase in bandwidth --> useful throughput declines Servers typically listen on well-known ports SSH: 22 Email: 25 Finger: 79 Web / HTTP: 80 --Example: Interacting with www.cs.utexas.edu --Browser resolves IP address of www.cs.utexas.edu --Browser connects to TCP port 80 on that IP address --Over TCP connection, browser requests and gets home page H. Application layer Example: HTTP Normally, HTTP servers, otherwise known as Web servers, run on port 80 when your Web browser connects to a URL, it knows to always make requests on port 80, meaning it stamps "80" in its packets you can direct your Web browser to make requests on any port, though, like this: http://:port_num In that case, the browser itself will address its packets to the IP address that corresponds to the name of the machine and destination port port_num instead of destination port 80. Messages look like this: Browser --> Server: "GET /pics/dog.jpg HTTP/1.0\r\n" Server --> Browser: "HTTP/1.0 404 Not found\r\n" or "HTTP/1.0 200 OK\r\n header1: value1\r\n header2: value2\r\n \r\n [the bytes in dog.jpg]" [Keep in mind that the above is happening inside TCP, and that TCP is presenting a reliable byte stream to the layers above it.] QUESTION: where does NFS sit in this picture? [answer: runs over UDP or TCP on some port, either well-known, or determined with a port mapping service running on the server] K. What is the interface to the networking stack? --Application programmer classically sees *sockets*. Inspired by pipes int pipe(int fds[2]) --Allow Inter-process communication on one machine --Writes to fds[1] will be read on fds[0] --Can give each file descriptor to a different process (with fork) The idea is: let's do the same thing across machines: **SOCKETS** Write data on one machine, read it on another *sockets* can represent many different network protocols, but: --classically an interface to TCP/IP and UDP --sometimes an interface to IP or Ethernet (raw sockets) --sockets API /* senders and receivers */ int sockfd = socket(AF_INET, SOCK_STREAM|SOCK_DGRAM|, 0); [note: with AF_INET in the first position, the setting of SOCK_STREAM vs SOCK_DGRAM controls whether the app's data is going to go over TCP or UDP]. [with UDP sockets, send atomic messages that may be reordered or lost] [with TCP sockets, bytes written on one end are read on the other, provided no failures. but no guarantees that reads will return the full amount requested ... or that the data will be packetized according to the number of times the sender called send(). With TCP, you *must* sit there in a loop and keep reading. You know you're done because either (a) the application-level protocol is expected to understand where message boundaries begin and end or (b) the first machine closed its connection to the server] int rc = close(); select(); struct sockaddr_in { short sin_family; short sin_port; uint32_t sin_addr; char sin_zero[8]; }; /* senders */ int rc = connect(sockfd, &addr, addrlen); int rc = send(sockfd, buf, len, 0); int rc = sendto(sockf, buf, len, 0, &addr, addrlen, 0); /* receivers */ int rc = bind(sockfd, &addr, addrlen); int rc = listen(sockfd, backlog_len); int rc = accept(sockfd, &addr, &adddrlen); int rc = recv(sockfd, buf, len, 0); int rc = recvfrom(sockfd, buf, len, 0, &addr, &addrlen); NOTES: * connections are named by 5 components: protocol (TCP), local IP address, local port, remote IP address, remote port * UDP does not require connected sockets * OS tracks all of this state in a PCB (protocol control block). --What does kernel see, and what interfaces does it invoke? TX direction: --usually gets payloads from higher levels and implements TCP/IP, UDP, IP, and part of Ethernet --usually hands most of an Ethernet frame to the network device --but not always: could imagine a Web server implemented entirely in the kernel, or even a Web server implemented on a network card RX direction: --when a packet arrives, use 5-tuple (above) to find PCB and figure out what to do with packet Note that to avoid lots of copies, OS may not actually store packets contiguously. May store linked list of buffers. Each buffer is either a packet header or a payload Network interface cards (NICs) --Used to be dumb --Now sometimes do lots of stuff Kernels also do *routing* --A machine has multiple NICs connected to different networks, kernel gets a packet (either from one of the NICs or from an application), now which NIC does it go out? --kernel generally looks at the destination address of the packet and does a lookup in a table that it maintains: [IP address, prefix-length] --> next-hop next-hop is the physical interface to send the packet out This is the same routing function that Internet routers do there are data structures to make it efficient in time and space (radix trees are a decent first cut)