Why layering: the continuum idea

Part 05 ended with a simple picture: a web app is a Client talking to a Server. The natural next question is where that server should actually sit, and the answer turns out to be "not in one place, but anywhere along a range".

The path is not an empty wire

Picture a student in Dhahran opening a web app whose only server is in a data center on another continent. Every click travels the full distance out as a request and the full distance back as a response. Now add three stops along the way: one in the campus building, one in the city, one in the region. Each stop is closer to the client than the far server is, so any work that can run at a nearer stop comes back sooner.

Seen this way, the space between client and server is a range of places where compute and storage can live. Latency grows as you move away from the client, and so does capacity, because far sites are larger. Placing layers along that range lets each request be answered by the nearest layer that is powerful enough to handle it.

This is not an invented teaching device. NIST describes fog computing as a layered model for accessing a shared continuum of computing resources, with nodes organized by their latency and distance to the end devices, and with different layers providing different services while working together as a continuum (NIST SP 500-325).

What makes a range a continuum

The word Continuum is doing real work here, so pin it down with the clearest example there is: visible light. Walk along the spectrum from 400 nm to 700 nm. Move one nanometer and the color does not visibly change. There is no single nanometer where blue stops and green starts. Yet 400 nm (violet) and 700 nm (red) are obviously different colors. That is the whole definition in one picture: neighboring points barely differ, while the two ends differ a lot. The Britannica Dictionary and Wiktionary put it the same way: a continuous series where no part is noticeably different from its neighbors, although the ends are very different from each other.

The familiar names on the wider electromagnetic spectrum (gamma, X-ray, UV, infrared, microwaves, radio) are labels people put on that smooth range because labels are useful. They are not walls that nature built.

Now map it back to computing. In the Edge-to-cloud (E2C) continuum, the two endpoints are "computing on the device itself" and "computing in a hyperscale cloud region". Everything between them is a candidate location for the layers above. The names you will meet in Part 08, Mist, Edge, Fog and Cloud, work exactly like UV and infrared: convenient labels on one range, where latency to the client and available capacity both grow as you move away from the device.

Layering along a distance is not new. The network you use every day already splits one long path into segments, each built with the technology that suits its span.

Make it concrete. A student sits in a KFUPM lab and sends a print job to a partner office in Riyadh. The first hop is the lab's Ethernet switch. The middle of the journey crosses routers in a wide area network. The last hop is the office's own switch, which delivers the job to the printer. Three segments, three tiers along the distance.

Why one technology cannot run end to end

A switch works at the Data link layer and forwards frames by MAC address. It fills its forwarding table by flooding and learning, which is cheap and automatic inside one building but does not scale to the whole world: flooding every unknown frame to every port of every network on Earth would be absurd. A router works at the Network layer and forwards by IP address, using tables computed by routing algorithms over hierarchical address blocks, which is exactly what scales across many networks (Kurose and Ross, Chapter 6). Each technology is excellent inside its own range and poor outside it, so the path is built from pieces that each fit a stretch of it.

This is the networking Continuum: distance grows along the path, and connectivity is assembled from tiers that each serve one stretch. Keep the shape in mind, because the edge-to-cloud continuum does the same with compute instead of connectivity, and the memory hierarchy, next, does it with time instead of distance.

The second place layering already exists is inside every computer. Here the axis is not kilometers but time: how long it takes the CPU to reach a piece of data.

Imagine a loop that sums a large array. The running total is touched on every instruction, so it lives in a register, right inside the processor. The chunk of the array being read right now is touched thousands of times a second, so it lives in cache. The rest of the dataset sits in RAM, and the file it came from sits on disk. Each step outward is slower to reach, but holds more.

This arrangement is the Memory hierarchy, a Continuum of access times. The rule is simple: the more urgently and frequently data is needed, the closer to the CPU it lives. Hennessy and Patterson describe the same picture: as you move away from the processor, each level is slower and larger, with times ranging from picoseconds up to milliseconds for magnetic disks. Notice the match with the network path: near tiers are small and fast, far tiers are large and slow.

Why a small fast layer is enough

If fast memory holds only a tiny fraction of the data, why does it help at all? Because of locality. Programs tend to reuse data they touched recently (temporal locality) and data stored near it (spatial locality). So a small, fast layer that keeps those items can answer most requests. Denning notes that this one principle shapes processor caches, disk caches, storage hierarchies and edge caches for the Web. That last item is the bridge to this course: an edge site works for the same reason a CPU cache does.

A ranking tells you the order of the levels; measurements tell you how much each step costs. To measure a level you need two quantities: bandwidth (how much data per second it can move) and latency (the fixed access delay paid on every fetch). Storage is simply the size of each level.

Start with a video editing workstation. The frame on screen right now sits in cache or registers, because it is touched constantly. Today's raw clips are pulled from the SSD into RAM so they can be processed quickly. Last year's projects stay on slow archive storage and are fetched only when you open them. That is the Memory hierarchy used as a placement policy, and the question is how big the gaps between levels really are.

The measured hierarchy

The Cornell Virtual Workshop publishes figures measured on Intel Skylake server processors. Reading outward, bandwidth is roughly cut in half at each level while latency roughly quadruples. The real steps vary, so read this as a rule of thumb.

The bandwidth figures come from a Sandia National Laboratories evaluation of Skylake (Hammond, Vaughan and Hughes, 2018), and the latencies were measured on a closely related Xeon model. Latency is also quoted in clock cycles, such as "4 cyc." and "14 cyc.". 2 ns in 4 cycles implies about 0.5 ns per cycle, a clock of roughly 2 GHz (our own arithmetic, not a published figure).

Turning two resources into one time

Bandwidth and latency only become useful for decisions once you combine them into the time a fetch actually takes. Cornell gives a simple model for any level:

Here T is time, S is the size of the data, B is bandwidth and L is latency. For a tiny fetch, S/B is close to zero, so latency decides everything. For a huge transfer, the S/B term dominates and bandwidth decides.

The analogy the course is built on

Now every piece is in place. CPU cache is to RAM as Edge is to Cloud. Both put a small, fast, nearby tier in front of a large, slower, distant one. Both answer most requests from the near tier because of locality, and both fall back to the far tier on a miss. Interactive requests are small, so by T = S/B + L they are latency-bound and benefit most from a nearer tier. Bulk analytics is not interactive, so a few extra milliseconds of latency cost it little, and it needs far more compute than an edge site has. That is why Offloading it to the cloud, where capacity is, makes sense, as long as the data can be moved there or already lives there.