COE 558Lecture 01Full guide
Introduction
The whole lecture on one page, taught concept by concept. Work through the parts in order, mark each concept once you understand it, and open the slide chips when you want the original slides.
- Parts
- 10
- Concepts
- 45
- Slides
- 77
- Reading
- 270 min
Part 01: Course map and syllabus
What the course covers, how it is graded, and how to plan the research project from week one.
3 concepts, slides 1-7
Why this part matters
This course is graded mostly on work you do outside the exam hall. The research project alone is worth 35%. The opening logistics are really a plan: which topics form the foundation, when the Week 8 midterm arrives, and how paper critiques turn into the research question behind your final study. Understand the plan well and you start the project in week one, not week twelve. The topic list is also a map of the skills you need to design a real edge and cloud service, from packaging it in containers to deciding where each step runs.
By the end you can
- Explain why Lecture 01 climbs from system models and networking to latency before it can answer where a task should run.
- Separate the Internet from the Web, and place each lecture topic in the chain that leads to the latency budget.
- Group the nine course topics into a foundations block and a build-and-operate block, and define image, container, registry, elasticity and the NIST models.
- Compute how the 100% splits between coursework and exams, and within the 35% research project.
- Plan the project as a pipeline: three-pass paper reading, critiques that end in a research question, and office hours to agree scope early.
Start from the question this lecture ends on. A client sends images to a pipeline that resizes each image, detects objects and draws bounding boxes. Where should each step run: on the client device, on a nearby edge box, on a fog node or in a distant cloud data center? And which latency number backs that choice? The last part of this lecture builds exactly that answer, and every idea before it supplies one piece.
The outline looks like a list, but read it as a chain in which each item answers a question the previous one raises. Take the running example of the lecture, a lamp you switch on from a web browser. To build it you first need a model of the system: which parts exist and who talks to whom. That model immediately asks how bytes move between machines, which is the job of the TCP/IP stack. Moving bytes is not enough; the browser and the lamp need a shared language for asking something to happen, and that language is HTTP. Your code then needs a programming interface for sending and receiving over the network, the socket. With those in place you can write an actual application, with a frontend in the browser and a backend on a server.
Only now can you ask the question the course is named after: where should that server run? The range of places between a device and the cloud where computation can live is the edge-to-cloud (E2C) continuum. The tool for choosing a place on it is the latency budget, the ceiling τ that a request's end-to-end response time must stay under. Placement is a latency decision, and latency comes from the network, which is why the lecture cannot skip the networking material to get to the interesting part faster.
| Outline item | Site part | Slides |
|---|---|---|
| Course syllabus | Part 1 | 1 to 7 |
| Motivation | Part 2 | 8 to 21 |
| The Internet (TCP/IP stack) | Part 3 | 22 to 29 |
| The Web (HTTP), network applications (sockets) and web development | Parts 4 and 5 | 30 to 45 |
| The E2C continuum | Parts 6 to 9 | 46 to 71 |
| Latency budget and example E2C applications | Part 10 | 72 to 77 |
Recall
Why does the lecture reach the TCP/IP stack before the E2C continuum?
A syllabus is a contract about scope: it tells you what can appear in exams and what counts as fair ground for your research project. COE 558 has two versions of that contract, and knowing both saves you from picking a project topic that is too narrow.
The slides describe what is taught this term. The official KFUPM catalog entry is the formal scope: it lists COE 558 as 3-0-3 (three lecture hours, no lab, three credits) with Graduate Standing as the prerequisite, and its description is broader than the nine topics taught in class.
The practical rule: study the taught topics for exams, but when you pick a research topic, anything in either list is in scope. Storage models and big-data frameworks get only one short line in class, yet they are rich research areas.
The foundations: three questions about the same application
The nine topics split into two blocks. The first three are the foundations, and they read as three questions about the lamp application from the lecture arc.
- How do things communicate? Topic 1, this lecture: the TCP/IP stack, client and server roles, sockets, frontend and backend, and how latency shapes a web service on the edge-to-cloud continuum.
- Where can it run, and how is it packaged? Topic 2: system thinking applied to the edge, fog and cloud layers, then virtualization: virtual machines, hypervisors and Docker containers.
- Where should it run? Topic 3: what makes something a cloud, what edge computing adds, and when to combine both.
Packaging is the first place the course gets concrete. Docker's own documentation gives three short definitions worth memorizing now. An image is a read-only template with instructions for creating a container. A container is a runnable instance of an image. A registry stores images, and Docker Hub is a public registry. A virtual machine, by contrast, runs a whole guest operating system on virtualized hardware managed by a hypervisor, which is why containers are said to deploy faster. Treat that claim as a simplification. Docker says a container is relatively well isolated by default, but containers share the host kernel while each VM runs its own, so faster startup comes with weaker isolation than a VM, a trade-off you will study in topic 2.
Once something is packaged, the next question is what it means to run it in a cloud. The standard definition comes from NIST Special Publication 800-145. It names five essential characteristics: on-demand self-service, broad network access, resource pooling, rapid elasticity and measured service. The course emphasizes self-service, scalability and elasticity. NIST describes rapid elasticity as capabilities that can be elastically provisioned and released, in some cases automatically, to scale rapidly outward and inward commensurate with demand.
Quick check
Which NIST characteristic means capacity scales outward and inward to match demand?
Recall
Image, container, registry: one line each.
Build and operate: the remaining six topics
With the foundations in place, the other six topics move from understanding to building and operating real services. In order: what you rent from a provider (service and deployment models), a real open-source cloud you can run yourself (OpenStack), how services talk to each other (HTTP, REST and JSON through a Web API), how to run them at scale (containers, orchestration, monitoring), how to handle data at Internet scale, and how to keep all of it secure.
The thread running through them is the service. Service-oriented architecture builds an application from separate services that talk over the network. Microservices push that idea further: many small services, each deployed on its own. REST (REpresentational State Transfer, capitalized to spell out the acronym) is the usual way those services expose a web API: each thing is a resource with a URL, and clients act on it with standard HTTP methods such as GET and POST.
What you rent determines how much of that service stack is yours to manage. NIST defines three service models. With Infrastructure as a Service you rent virtual machines, storage and networks and manage everything above them. With Platform as a Service you deploy your code onto a managed runtime. With Software as a Service you simply use a finished application. NIST also defines four deployment models, which describe who the cloud is for: private, community, public and hybrid.
OpenStack makes those models tangible: it is an open-source platform for building your own cloud, assembled from separate services that talk to each other through APIs. In other words, it is itself a service-oriented system. The course names four of its services, and the official documentation describes each in one line.
| Service | Role | Source |
|---|---|---|
| Nova | Provisions compute instances (virtual servers) | OpenStack Nova docs |
| Keystone | API client authentication, service discovery and multi-tenant authorization | OpenStack Keystone docs |
| Horizon | The dashboard: a web-based user interface to the other services | OpenStack Horizon docs |
| Neutron | Network connectivity as a service between interface devices | OpenStack Neutron docs |
Quick check
Which pairing of OpenStack service and its role is correct?
Recall
Name the four NIST deployment models.
Where the scope meets your project
Two of the build-and-operate items connect straight back to the placement problem at the end of this lecture. Client-service interaction for cloud and edge deployments is about requests crossing the continuum, and end-to-end processing pipelines are chains of steps that can each run in a different place (a service pipeline). Deciding which step runs on the edge and which in the cloud, under a latency limit, is a natural research direction, for example latency-aware task placement. Start a list of candidate topics now; the next concept explains why that list needs to exist in week one.
Start with the numbers, because they decide how you should spend your semester. The course has five assessments, and the research project inside them is split into three components.
Worked example
Where the 100% comes from
Coursework versus exams
Coursework is 10% + 10% + 35% = 55% (assignments, quizzes, project). Exams are 20% + 25% = 45% (midterm, final). More than half the grade is decided outside exam halls.Inside the project
Reviews, seminar and final study give 8% + 7% + 20% = 35%. That makes the project the single largest assessment, bigger than the final exam at 25%.One component against one exam
The final research study alone (20%) weighs exactly as much as the midterm (20%).Result
The project, not the exams, is the thing to plan the semester around. The tables add up: 10 + 10 + 35 + 20 + 25 = 100 and 8 + 7 + 20 = 35.
Read the rubric as a pipeline
The three project components are graded separately, but they work best as stages of one pipeline, much like the image pipeline this lecture ends on: each stage consumes the output of the one before it.
- Reviews and critiques (8%). Critical reading, identifying a research gap (a question or setting existing papers leave open), analysing the methodology, weighing strengths and weaknesses, and finally proposing an initial research question. The question is the output of this stage.
- Seminar (7%). Technical understanding, presentation quality, critical discussion and Q&A. You are graded on whether you can defend a paper under questions, not only summarize it, so present a paper close to your research question.
- Final research study (20%). Problem formulation, literature review, proposed methodology, implementation or simulation, results and the final report. That is months of work, not the last two weeks.
Quick check
Which research project component weighs the same as the midterm exam?
Quick check
According to the grading rubric, what should a paper critique end by proposing?
Feeding the pipeline: reading papers in three passes
The first stage only works if you can read many papers without drowning in them. S. Keshav's widely used method reads a paper in up to three passes, each deeper than the last, so you spend real time only on the papers that earn it. The first pass takes about 5 to 10 minutes: title, abstract, introduction, section headings and conclusions. It should let you answer the five Cs: category, context, correctness, contributions and clarity. The second pass takes up to an hour: read carefully, skip proofs, study the figures and mark references you have not read.
The third pass means virtually re-implementing the paper, making the same assumptions and recreating the work in your head. Keshav estimates about four or five hours for beginners and about an hour for an experienced reader. By the end you can name implicit assumptions and weak points, which is exactly what the critique rubric asks for. For a literature survey he suggests finding three to five recent papers, reading their related-work sections, then following the citations and venues they share.
Protecting the pipeline: office hours
A pipeline is only as good as its earliest decisions, and in a research project the expensive ones are scope and method. A concept you did not follow can be re-read here. A research question that is too broad, or a methodology the instructor considers infeasible, costs weeks if you discover it at the seminar. That is what office hours are for.
Section, schedule and contact details
- Section
- 558-2
- Class time
- Monday and Wednesday, 6:45 PM to 8:00 PM
- Venue
- Building 24, Room 244
- Instructor
- Dr. Muhammad Afaq, muhammad.afaq@kfupm.edu.sa
- Office
- 22-216
- Office hours
- Monday and Wednesday, 3:00 PM to 4:00 PM, or by appointment via email or Teams
The venue is written as Building #24 244, which reads as Building 24, Room 244. If you are unsure on the first day, check the room number on the registrar schedule.
Quick check
Which assessment carries the largest share of the course grade?
Recall
List the three research project components with their weights, and say what each one produces.
Recall
What are Keshav's three passes, and roughly how long does each take?
Recap
If you remember nothing else
- Lecture 01 is a chain: system model, TCP/IP, HTTP, sockets, web apps, then latency and where each task should run on the edge-to-cloud continuum.
- Topics 1 to 3 are the foundations (communication, virtualization, cloud and edge). Topics 4 to 9 cover building, running and securing cloud services. The catalog scope is broader and all of it is fair ground for research.
- Docker: an image is a read-only template, a container is a running instance, a registry stores images. OpenStack: Nova compute, Keystone identity, Horizon dashboard, Neutron networking.
- NIST has five cloud characteristics, three service models and four deployment models (private, community, public, hybrid). The course lists public, private, hybrid and multi-cloud.
- Grades: assignments 10, quizzes 10, project 35, midterm 20 (Week 8), final 25. Coursework is 55 and exams are 45.
- Project: reviews 8, seminar 7, final study 20, read as a pipeline. The review stage ends in an initial research question.
- Read papers in three passes, review each lecture one to two weeks later, and use office hours to agree project scope.
Sources
- List of COE Graduate Courses (COE 558: Cloud and Edge Computing)DocsKFUPM Computer Engineering DepartmentCredit hours 3-0-3, Graduate Standing prerequisite, official catalog description(opens in a new tab)
- The NIST Definition of Cloud Computing (SP 800-145)DocsNIST, Mell and Grance, 2011Five essential characteristics, three service models, four deployment models, rapid elasticity(opens in a new tab)
- What is Docker?DocsDocker DocsDefinitions of image, container and registry(opens in a new tab)
- OpenStack Compute (Nova)DocsOpenStack DocumentationProvisions compute instances(opens in a new tab)
- Keystone, the OpenStack Identity ServiceDocsOpenStack DocumentationAuthentication, service discovery, multi-tenant authorization(opens in a new tab)
- Horizon: OpenStack DashboardDocsOpenStack DocumentationWeb-based user interface(opens in a new tab)
- Neutron: OpenStack NetworkingDocsOpenStack DocumentationNetwork connectivity as a service(opens in a new tab)
- How to Read a PaperPaperS. Keshav, ACM SIGCOMM Computer Communication Review 37(3), 2007Three passes with time estimates, the five Cs, literature survey steps(opens in a new tab)
- Repeated retrieval during learning is the key to long-term retentionPaperKarpicke and Roediger, Journal of Memory and Language 57, 2007Repeated recall raised one-week retention by more than 100%(opens in a new tab)
- Spacing effects in learning: A temporal ridgeline of optimal retentionPaperCepeda, Vul, Rohrer, Wixted and Pashler, Psychological Science 19, 2008Optimal study gap is a fraction of the delay until the test(opens in a new tab)
- Learn How to Study Using... Retrieval PracticeArticleThe Learning ScientistsPractical guide to self-testing for students(opens in a new tab)
Part 02: Designing systems with mental models
Ad-hoc versus structured design, and the four questions that turn a lamp in a browser into a system: system, components, relationships, layers.
6 concepts, slides 8-21
Why this part matters
Every exam question about where a service should run, and every research project architecture, starts the same way: someone draws a system as boxes, arrows and layers. This part gives you a four-question checklist for drawing that picture, the vocabulary to name each arrow precisely (protocol versus interface, safe versus idempotent), and the idea the whole course is built on: the server layer can live in the cloud, in the fog or at the edge.
One running story carries the part. You want to switch a lamp on from Chrome and see how much energy it has used. Each concept adds one layer of precision to that story, until the lamp becomes a small but complete system whose server could run anywhere on the continuum.
By the end you can
- Explain why design reasons through a mental model, and why ad-hoc design fails to scale or integrate.
- Decompose a system with the four design questions into client, server and device, and justify offloading.
- Define a relationship as protocol plus interface and keep lower-level relationships out of scope on purpose.
- Specify interactions as request and reply pairs and compute energy in kWh without confusing power and energy.
- Map interactions onto HTTP and justify GET as safe and PUT as idempotent, including retry behavior.
- Distinguish the server component from the server layer and connect the layer to cloud, fog and edge.
Before you wire anything for a lamp you want to control from a browser, you already carry a picture in your head: the browser talks to something, and that something talks to the lamp. That picture is a mental model, and the first claim of this part is that good design starts there, not with code and not with hardware.
The claim is really about order. You do not jump from the problem straight to a solution; you first build a simplified representation of how the system works, and then derive the solution from it. The tool for building that representation is system thinking: describe the problem as parts, the connections between them, and the levels they sit on. A system seen this way is made of components, each one box with a job, and layers, each one a grouping of things that sit at the same level of abstraction.
The problem: a lamp, a browser, and a mysterious arrow
Now make it concrete. Draw Chrome on one side, a light bulb on the other, and one double-headed arrow between them. The browser is the client you already know. The lamp is the device. The arrow is where everything interesting hides. A browser tab cannot switch mains power; it can only send and receive messages over a network. So something must sit between the tab and the bulb, receive those messages, and turn them into electricity flowing or not flowing. The rest of this part is an answer to one question: what is inside that arrow?
The tempting answer: ad hoc
The first answer most people give is a part: "use a sensor", "buy a smart plug", "write a script". That is ad-hoc design, a component chosen before anyone has asked what the system actually is. The phrase comes from Latin ad (to) and hoc (this): for this specific purpose, a sense recorded in English from 1879. An ad-hoc design answers the immediate situation without previous planning.
For one lamp in one room, that can work. Now scale it: 50 lamps, a phone app next to the browser, a monthly energy report. With no named components and no agreed interfaces, each addition is another one-off wire or script, and nothing fits together. Ad-hoc designs may work for small isolated tasks, but they fail to scale or to integrate with larger systems.
The better answer: structured design
Structured design takes the same lamp but fixes the order of work: questions first, components second, code last. Design becomes a process, not a random walk. Notice what comes out of that process. It is not a sensor, a script or a board: it is a system, a named set of parts with defined connections. Named parts are what let a phone app or a second lamp plug in later without rework, and that is why structured design holds up under complexity, scalability and integration.
Control a lamp over the Web
Answer the four design questions in order
Client, server and device with defined relationships
| Aspect | Ad hoc | Structured |
|---|---|---|
| Planning | None | Up front |
| Works for | One small task | Growing systems |
| Adding a feature | Rework | Plug into a named interface |
| Integration | Hard | Defined by relationships |
Quick check
What is the main weakness of an ad-hoc design?
The structured process in this lecture is not mysterious. It is four questions, asked in a fixed order, and their highlighted words are the whole vocabulary of system thinking.
The order is forced by dependency. You cannot name relationships before you have components to relate, and you cannot group things into layers before you know what the things are. Treat the list as a reusable checklist: your research project architecture section must answer exactly these questions, and so must any exam answer that asks you to design or critique a system. This concept answers the first two for the lamp; the rest of the part answers the other two.
Recall
What four questions turn a design problem into a system?
Question one: what is a system?
For the lamp, the boundary is the room plus the network, the parts are the browser and the lamp, and the things that happen between them are "switch on" and "how much energy have you used?". Generalize that and you get the definition: a system is an environment in which interactions occur between components. Three ingredients, each one necessary. The environment draws the boundary. The components are the boxes inside it. Each interaction is an arrow.
Question two: what are the components?
The lamp is decomposed in two refinement steps, and each step adds one idea. First, separate software from thing: browser and bulb become Application and Device, where the application is everything that is software and the device is the physical lamp. Second, split the application in two. The client is the user-facing interface, the part you touch. The server processes client requests and communicates with the device. That second component is the answer to the mystery of the arrow: it is the thing that sits between the tab and the bulb.
User-facing interface
Processes requests, talks to the device
The lamp
Read the visual left to right: the click starts at the client, the server relays it, and only when the path reaches the device does the lamp glow. No box can be skipped.
Why split the application: offloading
Why not let the browser do everything? Because a browser tab is a poor place to keep things. It is not running while the tab is closed, so it cannot record energy readings around the clock, and a phone and a laptop would each hold their own copy of the lamp state. The server is always on and shared, so it can do both.
That is offloading: the client hands functions it cannot or should not perform to the server, because the server has the processing and storage. The server hosts those functions as services, such as "set the lamp state" and "read energy use", and the client simply calls them. Deciding what to offload, and to where, is exactly what the task mapping later in the lecture varies: a task can run on the device, at the edge, in the fog or in the cloud.
Recall
Define a system in one sentence, then name the three lamp components.
Question three asks what connects the components. Start concrete: between client and server the lamp needs at least two exchanges, "set the state" and "read the energy".
Each exchange is one interaction, and the relationship is the whole set of them. In set notation, an interaction d is an element of a relationship R. That framing is useful because it tells you how to specify a relationship: list its interactions, and then say what every one of them has in common.
What they have in common splits into two halves. The protocol is the grammar both sides agree on: message format, order and meaning. For the Web that is HTTP, with its message format, methods, headers and status codes. The interface is the list of operations the server offers: the endpoints, each a method plus a path, that a client may call. Neither half is enough on its own. A shared language with nothing to ask for is useless, and a menu of operations nobody can phrase is equally useless.
| Part | Question it answers | Lamp example |
|---|---|---|
| Protocol | How do we talk? | HTTP |
| Interface | What can I ask for? | /lamp/state and /lamp/energy |
The relationship we deliberately ignore
There is a second arrow in the lamp system, between server and device, and the lecture is blunt about it: it is not our concern. That is a design decision, not laziness. You may assume the server runs on a microcontroller board wired to the lamp. Inside that board, the program calls a hardware library; the library switches a pin; the pin drives a relay; the relay switches the lamp. That chain is a separate, lower-level relationship with its own protocol and its own interface.
The web client never sees any of it. It sees only the interface the server exposes. This is the payoff of defining a relationship by protocol and interface: each side can ignore the other side's internals, which is abstraction doing its job. Swap the relay for a smart bulb and the client does not change at all.
Quick check
The lamp server offers GET /lamp/energy. In system-thinking terms, what is that endpoint?
A relationship is a set of interactions, so the next step in specifying the lamp is to write that set down. It has two members, and each is a pair: a request from the client and a reply from the server. The client always speaks first.
| Interaction | Request carries | Reply |
|---|---|---|
| Set_on_off | The target state (on or off) | ok |
| read_energy_usage | Nothing extra | An energy value in kWh |
In the visual, each interaction is a request drawn toward the server followed by a reply drawn back to the client, and the meter inside the server fills once the kilowatt-hour reply lands. The device sits behind the server, never talking to the client directly, exactly as the previous concept required.
One design detail matters more than it looks. Set_on_off should carry the value you want (on, or off), not mean "toggle". If the network drops the reply and the client sends the request again, a toggle flips the lamp back, while "set on" twice still leaves it on. Hold on to that thought: the next concept gives the property a name and shows that HTTP builds it into its methods.
What the energy reply actually means
The read interaction returns energy, and that is where many students slip. A watt measures power, which is a rate: how fast energy is being used. A watt-hour measures energy: power kept up for a length of time. A kilowatt-hour is 1 kW (1,000 W) used for one hour, and the usual appliance formula is wattage times hours, divided by 1000.
Worked example
Energy of the lamp
Power
100 WTime
10 hMultiply
100 W × 10 h = 1,000 WhConvert
1,000 Wh ÷ 1000 = 1 kWhResult
read_energy_usage returns 1 kWh.
Quick check
A 60 W lamp stays on for 5 h. What should read_energy_usage return?
Recall
A 2 kW heater runs for 3 h. What energy, in what unit?
So far the interactions are abstract. On the Web they become concrete: set becomes HTTP PUT, and read becomes HTTP GET. The arrows carrying them are the protocol, HTTP, and the set and read operations exposed on the server edge are the interface, a Web API. The formula from earlier now has a real instance.
In the visual, the PUT and GET lines are the protocol, and the two sockets they seat into, set and read, are the interface. Only when both halves meet does the relationship work.
The actual messages
Here is what the two interactions could look like on the wire. Host and paths are illustrative.
Set: PUT request and response
- Request line
- PUT /lamp/state HTTP/1.1
- Host
- lamp.example.com
- Content-Type
- application/json
- Body
- {"on": true}
- Response status
- HTTP/1.1 204 No Content
Read: GET request and response
- Request line
- GET /lamp/energy HTTP/1.1
- Host
- lamp.example.com
- Response status
- HTTP/1.1 200 OK
- Body
- {"kWh": 1.0}
The 204 No Content reply is the HTTP form of the abstract ok: RFC 9110 defines it as the server having fulfilled the request with no additional content to send.
Why GET fits read: safe
RFC 9110 calls a method safe when its semantics are essentially read-only: the client does not request, and does not expect, any state change on the server. Reading energy changes nothing, so GET is the natural fit. RFC 9205 adds a practical reason: implementations can and do retry GET requests that fail, so GET handling should not change application state.
Why PUT fits set: idempotent
Recall the toggle problem from the interaction list. HTTP has a name for the property that solves it. A method is idempotent when the intended effect of several identical requests is the same as the effect of one. PUT, DELETE and the safe methods are idempotent. PUT asks for the target resource to be created or replaced with the state enclosed in the request, so sending {"on": true} twice still leaves the lamp on.
This matters on real links. RFC 9110 says an idempotent request can be repeated automatically if the connection fails before the client reads the response, while a client should not automatically retry a non-idempotent request unless it knows the request is actually idempotent, or that the original request was never applied. Edge devices often sit on flaky wireless links, so this choice decides whether a retry is harmless or a bug.
| Method | Interaction | Safe | Idempotent | Changes lamp |
|---|---|---|---|---|
| GET | Read | Yes | Yes | No |
| PUT | Set | No | Yes | Yes |
| POST | None (for contrast) | No | No | Depends |
Quick check
The client sends PUT /lamp/state with on true, and the connection drops before any reply. What should it do?
Recall
Why is PUT a better fit than a toggle for setting the lamp?
Recall
Relationship equals what? Give the lamp example of each part.
Question four asks for the layers. For this system the answer is a single layer, the one where processing and storage happen: the server layer. Answering it forces you to notice that the word "server" has been doing two jobs.
The server component is the program, for example the Python or C code that answers requests for the lamp state. The server layer is that program together with everything it needs to run: the processor executing it, the network carrying requests, the memory holding the current state, and the disk keeping energy history. The component is what you write; the layer is what you must provision.
What each part of the server layer does for the lamp
- Program
- Handles the HTTP requests
- Processor
- Runs the program
- Network
- Carries requests and replies
- Memory
- Holds the current lamp state
- Disk
- Stores the energy history
The visual starts with just the program lit, which is the component. The resources rise beneath it and the outline closes around all five: that outline is the layer.
The modern names for the server layer
Now the question the whole course answers: what do we call this layer today? Cloud, fog or edge, depending on where it is placed. Close the lamp story with that in mind. If the program runs on a board next to the lamp, the layer sits on the device side of the continuum (mist, or edge on a nearby gateway). If it runs in a distant data center, it is the cloud.
The standard definitions line up with the layer you just built. NIST defines cloud computing as on-demand network access to a shared pool of configurable computing resources, such as networks, servers, storage, applications and services, that can be rapidly provisioned and released: the same resources as the server layer, pooled at scale. NIST also describes fog computing as a layered model for access to a shared continuum of scalable computing resources, built from fog nodes that sit between smart end devices and centralized cloud services. Cloud, fog and edge together form the edge-to-cloud (E2C) continuum that the rest of the lecture studies.
Quick check
Besides the server program, what does the server layer contain?
Recall
How is the server component different from the server layer?
Recap
If you remember nothing else
- A mental model built with system thinking sits between the problem and the solution.
- Ad-hoc design is for one immediate situation; structured design handles complexity, scale and integration.
- The four questions, in order: system, components, relationships, layers.
- A system is an environment where components interact. The lamp system is client, server and device.
- The client offloads processing and storage to the server, which hosts services.
- A relationship is the set of interactions (d ∈ R), defined by a protocol plus an interface.
- GET is safe and idempotent; PUT changes state but is idempotent, so it can be retried after a failure.
- 100 W for 10 h is 1,000 Wh = 1 kWh (energy), not 1,000 W (power).
- Server component = program. Server layer = program, processor, network, memory, disk, today called cloud, fog or edge.
Sources
- RFC 9110: HTTP Semantics (STD 97, June 2022)RFCIETFSections 9.2.1 safe methods, 9.2.2 idempotent methods, 9.3.4 PUT, 15.3.5 204 No Content(opens in a new tab)
- RFC 9205: Building Protocols with HTTP (BCP 56, June 2022)RFCIETFGET handling should not change application state because GET requests are retried(opens in a new tab)
- PUT request methodDocsMDN Web Docs (Mozilla)PUT is idempotent but not safe(opens in a new tab)
- IdempotentDocsMDN Web Docs (Mozilla)(opens in a new tab)
- Mental models and human reasoning (Johnson-Laird, 2010)PaperProceedings of the National Academy of SciencesModels mirror the structure of what they represent and represent only what is true(opens in a new tab)
- ad hocArticleOnline Etymology DictionaryLatin for this specific purpose; the sense 'appointed for some particular purpose' is recorded from 1879(opens in a new tab)
- Glossary: Kilowatt, KilowatthourDocsU.S. Energy Information Administration(opens in a new tab)
- Energy Series: Estimating Appliance and Home Electronic Energy UseArticleVirginia Cooperative Extension, Virginia TechWattage times hours divided by 1000 gives kWh(opens in a new tab)
- SP 800-145: The NIST Definition of Cloud Computing (Sept 2011)DocsNIST(opens in a new tab)
- SP 500-325: Fog Computing Conceptual Model (March 2018)DocsNIST(opens in a new tab)
- Computer Networking: A Top-Down Approach, 9th ed. (Kurose and Ross)BookPearsonBackground reading on client-server and HTTP(opens in a new tab)
Part 03: The Internet and the TCP/IP stack
Why connectivity is split into five layers, what each layer sends and addresses, and where each layer lives inside a computer.
5 concepts, slides 22-29
Why this part matters
Every idea later in this lecture assumes you can say exactly what crosses the network: which header each hop reads, and which address identifies what. Latency budgets, sockets, containers and the mapping of tasks onto device, edge, fog and cloud all rest on this vocabulary.
This part turns the TCP/IP stack from a list to memorize into a working mental picture. We follow one real request from the lamp system in part 02 as it is wrapped, addressed, forwarded and delivered, and then we open up the computer to see where each step physically happens. The layer table and the address-per-scope table are classic exam questions, and the same vocabulary is what your research project uses to describe where sensing, processing and storage sit.
By the end you can
- Explain why connectivity is split into five layers, and name each layer with one protocol.
- Trace encapsulation from message to bits, stating each PDU and the payload it carries.
- Match the E2E, H2H and P2P scopes to port, IP and MAC addresses, and say which change per hop.
- Tell a socket, a connection and a flow apart, and write out the 5-tuple.
- Place each layer in application space, the OS kernel or the NIC, and name the seams between them.
In part 02 a browser controlled a lamp with HTTP requests: PUT /lamp/state to switch it and GET /lamp/energy to read its energy. Each request started on a laptop on KFUPM Wi-Fi and had to reach a server that might sit in a data center on another continent. Neither program knew the path in between, and neither needed to. That is the whole point of the TCP/IP stack.
Every host runs the same set of layers, so a client and a server can exchange bytes as if a private pipe joined them. The stack has a single purpose: connectivity. A program should only have to say who it wants to talk to and what it wants to send. Everything else in this part (layers, PDUs, addresses, sockets) is simply how that one job gets done.
Why split connectivity into layers
Hiding the path is a large job, far too large for one piece of software. Think about an air journey. You buy a ticket, check a bag, pass a gate, the plane takes off from a runway and air traffic control routes it. Each step relies on the step below it and offers a service to the step above. Kurose and Ross use exactly this example to motivate network layering.
The Internet divides connectivity the same way. The stack has five layers, and each layer provides one well-defined service using its own protocols. The payoff is explicit structure and modularity: a layer can change how it works internally without the rest of the system noticing. Wi-Fi can be swapped for 5G, and HTTP does not change a single byte.
Read the plus signs as composition, not arithmetic. No single layer delivers a web page. Connectivity emerges only when all five cooperate, the same way no single airport worker delivers you to another city.
| Layer | Abbreviation | One-line job | Example protocols |
|---|---|---|---|
| Application | AL | Supports network applications | HTTP, SMTP, IMAP |
| Transport | TL | Process-to-process data transfer | TCP, UDP |
| Network | NL | Routes datagrams from source host to destination host | IP, routing protocols |
| Data link | DLL | Moves data between neighboring network elements | Ethernet, 802.11, PPP |
| Physical | PL | Puts bits on the wire or into the air | Signal encodings of Ethernet and Wi-Fi |
Five layers versus seven
Part 02 used OSI as its example of a layered system, so it is worth lining the two models up. The OSI reference model (ITU-T X.200) defines seven layers. The Internet stack has no separate session or presentation layer. RFC 1122 says the Internet application layer essentially combines OSI presentation and application, and Kurose and Ross note that any such services must be implemented by the application itself.
| OSI layer | TCP/IP layer |
|---|---|
| 7 Application | Application |
| 6 Presentation | Folded into the application |
| 5 Session | Folded into the application |
| 4 Transport | Transport |
| 3 Network | Network |
| 2 Data link | Data link |
| 1 Physical | Physical |
Recall
Which two OSI layers have no separate layer in the TCP/IP stack, and where do their functions go?
The stack as the band between people and the physical world
Now zoom out to a whole system. Picture a distance sensor such as the HC-SR04 wired to a Raspberry Pi Pico. The Pico reads a distance, a Wi-Fi-capable board sends it across the network, and a dashboard in a control room shows it to a person. A command can travel the other way, down to an actuator.
The five layers form a band in the middle of that picture. Users sit above it and the physical world of sensing and actuation sits below it. The dashboard does not need to know how bits cross the air, and the device does not need to know how the dashboard draws its charts. This is modularity again, now at the scale of an entire system.
Read sensor values and send commands.
Five layers that give the two ends connectivity.
Measure the world and act on it.
This is the same shape as the lamp system from part 02, and it is also the shape of your research project. In parts 08 to 10 the same pipeline is stretched across the edge to cloud continuum, and the question becomes where along it each piece of processing should run.
Layers cooperate by handing data to each other, and the way they do it is the single most useful mechanism in this part. Take a simple read request, GET /lamp HTTP/1.1. The application layer hands it down as a message. Transport puts a TCP header in front and it becomes a segment. Network adds an IP header and it becomes a packet. Data link adds a MAC header in front and a frame check sequence (a CRC) trailer behind, making a frame. Physical sends that frame as bits.
Each layer's unit of data is its protocol data unit (PDU). The process of wrapping is encapsulation, and it follows one rule: a PDU is this layer's header plus a payload, and the payload is the entire PDU of the layer above. On the receiving host the steps run in reverse. Each layer reads and removes its own header, then passes the payload up. Step through it yourself before reading on.
- Layer
- Application
- PDU
- Message
- Address
- None: the app just writes data
- Adds
- The HTTP request itself
- Request
- GET /lamp HTTP/1.1
| Layer | PDU | Payload | Example protocols | Address |
|---|---|---|---|---|
| Application | Message | Application bytes | HTTP | None |
| Transport | Segment (TCP) or datagram (UDP) | Message | TCP, UDP | Port number |
| Network | Packet (also called datagram) | Segment or UDP datagram | IP | IP address |
| Data link | Frame | Packet | Ethernet, 802.11, PPP | MAC address |
| Physical | Bits | Frame, as signals | Physical specs of Ethernet and Wi-Fi | None |
Who reads which header
Encapsulation is also what lets the middle of the network stay simple. Kurose and Ross label the headers Ht (transport), Hn (network) and Hl (link). A switch processes only the link and physical layers. A router processes network, link and physical. Only the two end hosts open the transport header and the application message. The IP header even carries a Protocol field that names the next-level protocol in its data (RFC 791, section 3.1), which is how the receiver knows whether to hand the payload to TCP or UDP.
Minimum header sizes
- IPv4 header
- 20 bytes
- TCP header (no options)
- 20 bytes
- UDP header
- 8 bytes
These sizes come from RFC 791, RFC 9293 and RFC 768, and together they are the 40 bytes of layering overhead mentioned earlier. They also reveal a division of labor: IP has no acknowledgments, no retransmissions and no flow control (RFC 791, section 1.4). Reliability is added one layer up, by TCP, and only at the end hosts.
Quick check
In the TCP/IP stack, what does a network-layer packet carry as its payload?
Recall
What is the payload of a data-link frame, and what is the payload of that payload?
The header table ended with an address column, and that column is not decoration. Each header names something different because each layer connects something different. Your laptop on campus loads a page from a cloud server. The path goes over one Wi-Fi hop to the access point, through several routers, and finally over a data-center link to the server. Along that path there is one conversation between two programs, one route between two machines, and many separate links.
These are the three connection scopes. The transport layer provides an end-to-end (E2E) connection between two application processes. The network layer provides a host-to-host (H2H) path between two machines, however many routers lie between them. The data link layer handles point-to-point (P2P) transfer across a single hop between two directly attached interfaces.
| Scope | Between | Virtual or physical | Count along one path |
|---|---|---|---|
| E2E (transport) | Two application processes | Virtual | One |
| H2H (network) | Two hosts, across routers | Virtual | One |
| P2P (data link) | Two adjacent interfaces | Physical link | One per hop |
The rule to take away: the higher the layer, the wider its scope and the more virtual its connection. Only the P2P links are real cables or radio channels. The E2E connection is not a wire at all. It is state held in the two end hosts: RFC 9293 (section 3.3.1) describes the TCP control block that stores the local and remote IP addresses and ports. TCP creates that state with the three-way handshake (section 3.5), while IP itself is connectionless. UDP keeps no connection state at all, yet its datagrams are still delivered process to process.
Quick check
Which connection scope exists between two application processes rather than two machines?
Recall
How many E2E, H2H and P2P connections exist when your laptop reaches a server through four routers? (Treat the Wi-Fi access point as part of the first link.)
One address per scope
If each scope connects a different kind of thing, each needs its own way to name that thing. Look back at the encapsulation stepper. The server is 3.120.0.10, the service on it is port 443, and the next hop's interface is f4:8c:50:2b:e1:04. Three addresses, one per scope.
E2E names a process with a port number. H2H names a machine with an IP address. P2P names a network interface on the local link with a MAC address. (The course materials write "Mac"; the standard spelling is MAC, for Media Access Control.)
Address sizes
- Port number
- 16 bits, values 0 to 65535
- IPv4 address
- 32 bits (four octets)
- MAC address
- 48 bits (EUI-48, six octets)
Ports are 16-bit fields in both TCP (RFC 9293) and UDP (RFC 768). IPv4 addresses are a fixed four octets (RFC 791, section 2.3). MAC addresses are 48 bits, burned into the NIC but sometimes settable in software, written like 1A-2F-BB-76-09-AD.
| Range | Name | Example |
|---|---|---|
| 0 to 1023 | System ports | HTTPS on 443 |
| 1024 to 49151 | User ports | Registered application services |
| 49152 to 65535 | Dynamic ports | Client source port 51514 |
What changes at each hop
Putting scopes and addresses together gives the most useful insight of this concept. Along the whole path, ignoring NAT, the source and destination IP addresses and ports stay the same, because their scopes span the whole path. The MAC addresses are replaced at every hop, because their scope is one link: each router strips the old frame and builds a new one for the next link. To build it, the router uses ARP (RFC 826) to map the next hop's IP address to its 48-bit Ethernet address.
Quick check
A router forwards your packet toward a cloud server. Ignoring NAT, which address does it replace for the next hop?
Recall
Along a path from your laptop to a cloud server, which addresses stay the same and which change at every hop?
Addresses name endpoints, but a conversation needs more than one name. Open two browser tabs to the same server. Both tabs run on one laptop with one IP address, and both talk to one server port. Yet every reply lands in the right tab. Something must tell the two conversations apart.
Worked example
Two tabs, one server
Server socket
3.120.0.10:443Tab A socket
10.0.0.7:51514Tab B socket
10.0.0.7:51515Only the source port differs
Both flows are TCP from 10.0.0.7 to 3.120.0.10:443. Only the source ports 51514 and 51515 differ, so the OS delivers each reply to the right tab.
The general rule builds up in three steps. A socket is one endpoint, an IP address plus a port number. A connection is a pair of sockets, one at each end, which is exactly how RFC 9293 defines it. When networks need to tell flows apart, they add the protocol and use the 5-tuple, which RFC 6437 (section 3) lists as destination address, source address, protocol, destination port and source port. The protocol field is what separates a TCP flow from a UDP flow that happens to share the same addresses and ports.
Notice how this ties back to the scopes. The 5-tuple uses only the E2E and H2H addresses, the ones that survive the whole path. The MAC address plays no part in identifying a connection, because it covers only one hop and would change before the reply ever came back.
Quick check
Two browser tabs on one laptop connect to the same server on port 443. What lets the operating system tell the two connections apart?
Recall
What identifies a socket, and what identifies a flow?
So far the layers have been abstractions. The last step is to find them inside a real computer. A Node.js server asks the operating system for a socket and gets back a file descriptor. It writes bytes to that descriptor. Inside the kernel, TCP and IP headers are added. A driver hands the finished frame to the network card, which turns it into electrical or radio signals.
That story maps the layers onto three parts of one machine. The application layer lives in application space: the browser or server process, including its HTTP library. The transport layer and network layer live in the operating system kernel. The data link layer and physical layer live in the network interface and its driver.
Two seams separate these spaces. The socket API is the door between application and OS, which is why the socket of the previous concept is also a programming object. The Linux socket(7) manual calls sockets the uniform interface between the user process and the network protocol stacks in the kernel. Drivers are the door between OS and hardware. For the Web, the concrete stack reads: browser or server, HTTP, socket API, TCP, IP, Ethernet, then wired or wireless media.
| Space | Layers | Example |
|---|---|---|
| Application space | AL | Browser or Node.js server with its HTTP code |
| Operating system kernel | TL, NL | TCP and IP implementations, reached through sockets |
| Hardware (NIC and driver) | DLL, PL | Ethernet or Wi-Fi chip, firmware and driver |
The seams are also boundaries of privilege. An application cannot forge TCP state directly; it has to go through the socket API. This matters later in the course. Latency is added at every crossing, and virtual machines and containers in lecture 2 virtualize exactly these seams.
Quick check
On a laptop browsing the Web, where does the TCP implementation normally run?
Recall
Where do TCP and IP normally run in a computer, and what is the interface an application uses to reach them?
Recap
If you remember nothing else
- The TCP/IP stack exists to give a client and a server connectivity, and connectivity emerges from all five layers working together.
- OSI has seven layers, and TCP/IP folds session and presentation into the application.
- The PDUs are message, segment, packet (datagram), frame and bits, and each payload is the PDU of the layer above.
- Routers process only the network, link and physical layers. Only end hosts read transport headers and application messages.
- Transport connects processes end to end using 16-bit ports.
- Network connects hosts using 32-bit IPv4 addresses.
- Data link connects adjacent interfaces using 48-bit MAC addresses, which are rewritten at every hop.
- A socket is an IP address plus a port, a connection is a pair of sockets, and a flow is identified by the 5-tuple.
- Applications live in user space, TCP and IP in the kernel behind the socket API, and link and physical in the NIC and its driver.
Sources
- Computer Networking: A Top-Down Approach, 9th editionBookPearson (Kurose and Ross)Published June 2025(opens in a new tab)
- Kurose and Ross companion siteDocsUniversity of Massachusetts AmherstSlides, animations, Wireshark labs(opens in a new tab)
- Chapter 1 slides, 8th edition (why layering, protocol stack, encapsulation, ISO/OSI)DocsKurose and Ross, hosted by Simon Fraser UniversitySlides 1-66 to 1-68(opens in a new tab)
- Chapter 6 slides, 8th edition (where the link layer is implemented, MAC addresses)DocsKurose and Ross, hosted by Simon Fraser UniversitySlide 6-40 on 48-bit MAC(opens in a new tab)
- RFC 791: Internet ProtocolRFCIETF32-bit addresses (2.3), header (3.1), no retransmission (1.4)(opens in a new tab)
- RFC 9293: Transmission Control Protocol (TCP)RFCIETFObsoletes RFC 793; 16-bit ports, socket pair, TCB, three-way handshake(opens in a new tab)
- RFC 768: User Datagram ProtocolRFCIETF8-octet header(opens in a new tab)
- RFC 1122: Requirements for Internet Hosts, Communication LayersRFCIETFFour-layer suite (1.1.3), relation to OSI(opens in a new tab)
- RFC 6335: IANA Procedures for Service Name and Transport Protocol Port Number RegistryRFCIETFPort ranges (6)(opens in a new tab)
- RFC 6437: IPv6 Flow Label SpecificationRFCIETF5-tuple definition (3)(opens in a new tab)
- RFC 826: An Ethernet Address Resolution ProtocolRFCIETFIP to 48-bit Ethernet address(opens in a new tab)
- X.200: OSI Basic Reference Model, The basic modelDocsITU-TApproved 1994, seven layers(opens in a new tab)
- Guidelines for Use of EUI, OUI, and CIDDocsIEEE Registration AuthorityEUI-48 is six octets, used for MAC addresses(opens in a new tab)
- socket(7), Linux manual pageDocsman7.org (Linux man-pages project)Sockets as the interface between user processes and kernel protocol stacks(opens in a new tab)
Part 04: The Web and network applications
The Web as one network application, decentralized versus distributed designs, the two-dimensional design space, and the request-response loop.
5 concepts, slides 30-39
Why this part matters
Part 03 described the Internet as plumbing: a stack of layers that moves bytes between machines. This part climbs to the top of that stack and asks what people actually build on it.
Every edge or cloud system you will study or design for your research is a network application whose server side has been cut into components. The story here runs in one line: an application uses the Internet, its server side is really several components, each component can be placed on a different layer and resource, and every request a client sends makes those components spend compute, storage and communication. The central question, which component goes on which layer and why, is the core question of edge computing research and a very likely exam question.
By the end you can
- Explain why the Web is one network application on the Internet, and sort any app by the protocol it speaks.
- Use the lecture's definitions of decentralized and distributed correctly, and name your definition when papers disagree.
- Break a server side into components and place each one in the design space as a layer choice plus a resource instance.
- Justify a placement with latency, compute, storage, cost, privacy, sharing and availability.
- Separate the logical HTTP conversation from the physical path of bytes through sockets and the TCP/IP stack.
- State who starts the request-response loop and what each response costs the server.
Picture two things you do before lunch. You open a KFUPM page in a browser. Then your mail program hands a message to the KFUPM mail server over SMTP. Both need the Internet. Only the first one is the Web. Getting this distinction right is the first step to thinking clearly about applications.
Infrastructure and the services built on it
The Internet is the infrastructure: the TCP/IP stack, the routers and the providers that carry packets. The World Wide Web (WWW) is one application built on top of that infrastructure. What makes something "the Web" is that it fetches linked resources using HTTP. MDN puts it plainly: "The Internet is an infrastructure, whereas the Web is a service built on top of the infrastructure."
History makes the order of events obvious. The Internet was carrying remote logins, email and file transfers for years before anyone had heard of a web page.
What counts as a network application
Once the Web is seen as one application among many, the natural next question is what the whole family looks like. Open HungerStation on your phone and order dinner. The app on your phone (the Client) sends the order across the Internet to the company's backend (the Server), which passes it to the restaurant and a courier. Without the network the app is just a menu picture.
The lecture's definition of a Network application is broad on purpose: any application that uses the Internet as its transport medium. That phrase has a precise meaning in the stack you met in part 03. The application hands its messages to the Transport layer through the Socket API, and everything below that line is someone else's job. What the messages mean is decided by the application.
Familiar apps hide an interesting mix. Some run inside a browser, some are native programs, and one exchanges its files with other users instead of a company server. Try sorting them yourself before reading the table.
| App | Runs in | Talks to | Architecture |
|---|---|---|---|
| Chrome (opening a site) | A browser | Web servers over HTTP | Web, client-server |
| Gmail in a browser tab | A browser | Google's mail servers over HTTP | Web, client-server |
| Outlook desktop | A native program | A mail server (over SMTP or IMAP) | Client-server, mail protocols |
| Netflix or HungerStation app | A native phone app | The company's backend servers | Client-server, HTTP APIs |
| uTorrent | A native program | A tracker to find peers, then other BitTorrent peers directly | Peer-to-peer |
uTorrent is the odd one out. It speaks the BitTorrent protocol, where the specification says that when many people download the same file at once, "the downloaders upload to each other". Each copy of the program acts as a client and a server at the same time. This arrangement is called peer-to-peer, and it returns in the next concept when we ask what decentralized means.
The protocol decides the circle
A useful picture, adapted from Connolly and Hoar's Fundamentals of Web Development, draws the Internet as a large circle with several smaller circles inside it: the Web, email, online gaming and FTP.
The rule for deciding which circle an application belongs to is its Application layer Protocol. If it speaks HTTP to fetch web resources, it is in the Web circle. If it speaks SMTP to move mail between servers, FTP to move files, or a custom game protocol, it sits in a sibling circle. Siblings share the Internet but not the Web. The circle sizes are illustrative only; they make no claim about traffic shares.
Email is a good exam trap. When you read Gmail in a browser tab, your browser talks HTTP to Google, which is the Web. When Google then delivers your message to a KFUPM mail server, the two mail servers speak SMTP, which is not the Web. One everyday "email" experience touches two circles.
Recall
Name two network applications that are not part of the Web, and say what protocol makes them not Web.
Quick check
Which statement correctly describes how the Internet and the Web relate?
Take one calculator and build it three ways. First, a desktop calculator: one program on one machine. Second, a browser page that sends "2+3" over HTTP to a separate server program, which sends back 5. Third, the same service, but the server's work and data are spread over several machines, with addition on one node and the history database on another.
By the lecture's definitions, the first build is neither decentralized nor distributed. The second is a Decentralized application: the application is split into two standalone components (Client and Server), usually on different machines, that talk through a Protocol using sockets. It is also a Distributed application in the lecture's sense, because computation is shared between client and server and state lives on both. The third build pushes distribution further: the server's own computation and data are spread over several machines.
Keep these two questions apart, because they become the two axes of the design space in the next concept: decentralization is the vertical axis, distribution the horizontal one.
| Aspect | Decentralized (lecture) | Distributed (lecture) |
|---|---|---|
| What is split | The application itself, into separate client and server programs | The computation and state, between client and server or across several machines |
| What the definition stresses | Structure: separate programs that talk through a protocol and sockets | Resources: where computation and state physically live |
| Lecture example | BitTorrent, Bitcoin | Google Cloud |
| Design-space axis | Vertical: which layer a component sits on | Horizontal: which compute or storage resource it uses |
The lecture's examples fit. BitTorrent and Bitcoin have no single controlling center: the Bitcoin paper opens with the goal of sending payments "directly from one party to another without going through a financial institution". Google Cloud is one provider that spreads computation and state across its own machines in many regions and zones around the world.
Recall
Using the lecture's definitions, what makes an application decentralized, and what makes it distributed? Give the lecture's example of each.
Quick check
Using the lecture's own definitions, which example is labelled a distributed application?
So far "the server" has been a single word. Look inside the calculator's server side and it turns out to be several separate pieces, and that discovery is what makes edge computing possible: separate pieces can live in separate places.
The server side is a set of components
Follow one request. The Client sends "multiply 6 by 7". It does not know or care how the server side is organized. It simply knocks on one door. That door is the API gateway, a single entry point that receives every client request and routes it to the right service. AWS describes its own gateway product as a "front door" for applications to reach backend services. Behind it sit four operation services (Add, Sub, Mul, Div), each exposing its own Interface, like a small Web API, and each able to read or write a Database.
Sends one request and waits.
Reads the operation and picks a service.
Each does one kind of arithmetic.
Keeps results and history.
Worked example
One multiply request, step by step
Client sends
The client sends an HTTP request asking for 6 × 7 to the gateway's address.Gateway routes
The API Gateway sees the operation is multiply and forwards it to Mul.Service computes
Mul computes 42 and writes a history record to the Database.Answer returns
Mul replies to the gateway, which returns the response to the client.Result
One client request touched three server components (gateway, Mul, Database). The client saw only one door.
Everything except the client forms the Server layer, typically hosted in the Cloud. But nothing forces those components to live together. Because each one is a separate program with its own interface, Mul could run close to the user while the Database stays in a large data center. The question becomes: can we assign server components to different layers, and if so, how do we describe and justify the assignment?
Two coordinates for every component
The answer is a grid that works like a seating chart. "Mul sits on Layer 1 using Compute 1." "The Database sits on Layer 3 using Compute 2 plus Storage." Every component gets two coordinates.
The two-dimensional design space separates two independent decisions about each Component, and they are exactly the two questions from the previous concept. Moving a box up or down changes its Layer: that is the decentralization axis, how far the server side is split across layers. Moving a box left or right changes the resource it uses on that layer: Compute 1, or Compute 2, which can also offer storage (only the Database uses it). That is the distribution axis, how computation and state are spread across resources.
A column is a type, each layer has its own instance
One subtlety makes the grid precise. A column names a resource type, not a machine. Each layer has its own copies of those types, and the lecture marks them with suffixes: Compute 2_a + Storage_a on the Database, Compute 1_b on Add, and Compute 2_b on Sub. The letter tells you which layer's copy: _a belongs to Layer 3 and _b belongs to Layer 2. So Add and Sub share a layer, but run on two different instances.
This is what makes the application a Distributed application in practice: the same kind of resource exists as several instances in different places, and the application's computation and state are spread across them. "Compute 2" could be a virtual machine type in a Cloud region, a server in a Fog micro data center, or a box at the Edge.
Worked example
Name the instance for every box
Database (Layer 3, column 2)
Compute 2_a + Storage_a, given in the lecture.Add (Layer 2, column 1)
Compute 1_b, given in the lecture.Sub (Layer 2, column 2)
Compute 2_b, given in the lecture.API Gateway (Layer 3, column 1)
Following the pattern, Compute 1_a.Mul (Layer 1, column 1)
Following the pattern, Compute 1_c.Div (Layer 1, column 2)
Following the pattern, Compute 2_c.Result
Six boxes, six separate instances. Two boxes share a letter only when they share a layer, and share a number only when they use the same resource type.
Criteria for assigning components to layers
The grid tells you how to write a placement down. It does not tell you which placement is good. For that the lecture asks: what criteria decide which layer a component goes on? There is no single right answer, only trade-offs. Each criterion pushes a component in a direction.
| Criterion | Pushes component toward | Calculator example | Source |
|---|---|---|---|
| Latency | Lower layers, close to the client | A live result preview that must update as you type | Satyanarayanan 2017; NIST SP 500-325 |
| Compute demand | Higher layers, where capacity exists | A heavy matrix-inversion service added to the calculator | Satyanarayanan 2017 |
| Storage and state | A well-provisioned layer with durable storage | The Database holding every user's history | Lecture design space |
| Cost (bandwidth) | Lower layers, near where data is produced | Summarize raw sensor input locally, send only results up | Satyanarayanan 2017 |
| Privacy and residency | Lower layers, or a region inside the right borders | Strip personal data before anything leaves the site | Satyanarayanan 2017; Google Cloud locations |
| Sharing across users | A layer every client can reach | The API Gateway, the single entry point for everyone | AWS API Gateway guide |
| Availability | Add a nearby fallback below the cloud | Keep basic Add working during a short cloud outage | Satyanarayanan 2017 |
The evidence behind the table comes from the edge computing literature. Satyanarayanan writes that "reliance on a cloud datacenter is not advisable for applications that require end-to-end delays to be tightly controlled to less than a few tens of milliseconds." NIST adds that Fog computing "minimizes the request-response time" by working close to end devices. So a time-critical component moves toward the Edge, lowering Latency and Response time (RT).
The pull goes the other way too. Heavy computation and durable, shared data need resources that shrink as you go down the layers, which is why the Database sits on the top layer. Satyanarayanan also notes that analyzing raw data near its source cuts the bandwidth sent to the cloud, that a nearby cloudlet "can enforce the privacy policies of its owner prior to release of the data to the cloud", and that local resources give the "ability to mask transient cloud outages". Google Cloud, for its part, sells regions partly so data can stay resident inside a country's borders.
Quick check
In the lecture's two-dimensional design space, what does moving a component up the vertical axis change?
Quick check
A face-blurring component must process camera frames before any raw video leaves the building. Which criterion mainly pushes it to a lower layer?
Recall
List five criteria for assigning a server component to a layer, each with a one-line reason.
Placing components on layers only works because they can talk to each other across a network. So zoom in on one conversation. Open the calculator in a browser and press equals. The front-end JavaScript sends GET /add?a=2&b=3. A Node.js back-end reads that request from its socket, computes 5, and writes the response back. Along the way the bytes travel through routers you will never see.
Three levels in one conversation
At the top is the application conversation: an HTTP request and response. In the middle are the endpoints, the sockets. At the bottom is the network path through the Internet. The Frontend runs in the browser, and the Backend runs in Node.js.
Builds the request and renders the answer.
Reads the request, computes, replies.
Shows the result to the user.
RFC 9110, the HTTP standard, defines the roles by behavior. A Client "is a program that establishes a connection to a server for the purpose of sending one or more HTTP requests". A Server "is a program that accepts connections in order to service HTTP requests by sending HTTP responses". And HTTP itself "is a stateless request/response protocol for exchanging messages across a connection".
Sockets are endpoints, connections are pairs
A socket is named by an IP address plus a Port number. The connection is the pair of sockets, one at each end, as in this example using documentation addresses:
One HTTPS connection seen as a socket pair
- Client socket
- 192.0.2.10:52814
- Server socket
- 198.51.100.7:443
- Transport protocol
- TCP
- Connection
- Identified by both sockets together, set up by TCP after the client calls connect()
The HTTP arrow is logical, the bytes move vertically
Protocol pictures usually draw a dashed arrow between the two HTTP boxes, as if browser and server talked to each other directly. They do, but only logically. No HTTP byte ever travels sideways along that arrow.
The real journey is vertical. The browser writes an HTTP message. It goes down through TCP on the client and the Socket, into the TCP/IP network (the "pipe"), and back up through TCP on the server until Node.js reads it. This is Encapsulation from part 03: each layer wraps the data on the way down and unwraps it on the way up. Peer layers "talk" to each other only through that round trip.
The same picture also shows who owns what. The Application layer (AL) is the web application, written by developers: HTML, CSS and JavaScript in the Frontend, and Node.js in the Backend. The Transport layer (TL), Network layer (NL), Data link layer (DLL) and Physical layer (PL) are infrastructure, provided by the operating system, network cards, cables and routers. That split is the same line that defined a network application in the first concept: the app owns meaning, the infrastructure owns delivery.
Recall
Why is the HTTP arrow between browser and server drawn dashed, and what path do the bytes really take?
Step back from a single message to the rhythm of the whole exchange. Your browser asks the calculator for a result. The server wakes up, spends processor time computing it, reads and writes the history, and sends bytes back. Then it waits for the next request. It never phones you first.
That cycle is the Request-response loop. The Client initiates, and the Server only responds. RFC 9110 builds HTTP around exactly this pattern: a client sends a request message, and a server answers it with a response message.
Every response has a price
This is where the part comes full circle. Preparing each response costs the server three resources. Compute is the processor time, which you will later measure as Processing time (PT). Storage is reading and writing data. Communication is sending the response back over the network. For one user that is trivial. Multiply it by, say, 10,000 users pressing equals at once, and those three costs decide where each component in the design space should live.
It is also why placement matters to users. The time a user waits, the Response time (RT), is the trip to the server, the processing, and the trip back. NIST describes Fog computing as a way to minimize the request-response time. Moving the right component closer shortens the loop.
Quick check
Who initiates communication in the request-response loop between browser and server?
Recall
In the request-response loop, who initiates, and which three server resources does each response consume?
Recap
If you remember nothing else
- The Internet is the infrastructure. The Web is one network application on it, using HTTP.
- A network application is any app that uses the Internet as its transport medium: web, email, streaming, peer-to-peer.
- Lecture definition: decentralized means split into separate client and server programs that talk through a protocol and sockets (BitTorrent, Bitcoin).
- Lecture definition: distributed means computation is not on a single machine but spread between client and server, and state is distributed too (Google Cloud).
- The server side of an app is several components (an API gateway, services, a database), and each can be placed separately.
- Design space: the vertical axis picks the layer (decentralization). The horizontal axis picks the compute or storage resource type (distribution); the layer's copy of that type is the instance.
- Placement criteria: latency, compute demand, storage and state, cost, privacy, sharing and availability.
- A socket is an endpoint. HTTP talks logically between application layers, while the bytes move through the TCP/IP stack.
- The client initiates. Each response costs the server compute, storage and communication.
Sources
- RFC 9110: HTTP SemanticsRFCIETF / RFC EditorClient, server and request/response definitions, sections 3.3 and 3.4(opens in a new tab)
- A short history of the WebArticleCERN(opens in a new tab)
- How does the Internet work?DocsMDN Web Docs (Mozilla)(opens in a new tab)
- Fundamentals of Web Development, 3rd ed. (Connolly and Hoar, 2021)BookPearsonLikely origin of the nested-circles and request-response figures(opens in a new tab)
- Computer Networking: A Top-Down Approach, 9th ed. (Kurose and Ross)BookPearsonApplication layer, sockets and layered communication(opens in a new tab)
- A brief introduction to distributed systems (van Steen and Tanenbaum, Computing 98, 2016)PaperSpringer (open access)(opens in a new tab)
- Distributed Systems, 4th ed. (van Steen and Tanenbaum, 2023)Bookdistributed-systems.net(opens in a new tab)
- Bitcoin: A Peer-to-Peer Electronic Cash System (Nakamoto)Paperbitcoin.org(opens in a new tab)
- BEP 3: The BitTorrent Protocol SpecificationDocsBitTorrent.org(opens in a new tab)
- On Distributed Communications: I. Introduction to Distributed Communications Networks (Baran, 1964)PaperRAND CorporationCentralized, decentralized and distributed topologies. Verified via search listing(opens in a new tab)
- The Meaning of Decentralization (Buterin, 2017)ArticleMediumArchitectural, political and logical decentralization. Verified via search listing(opens in a new tab)
- Google Cloud locationsDocsGoogle Cloud(opens in a new tab)
- What is Amazon API Gateway?DocsAmazon Web Services(opens in a new tab)
- RFC 6455: The WebSocket ProtocolRFCIETF / RFC Editor(opens in a new tab)
- NIST SP 500-325: Fog Computing Conceptual Model (Iorga et al., 2018)PaperNIST(opens in a new tab)
- The Emergence of Edge Computing (Satyanarayanan, IEEE Computer, Jan 2017)PaperIEEE Computer Society (author copy, CMU)(opens in a new tab)
Part 05: Sockets and web app architecture
The socket API between your program and the transport layer, TCP versus UDP sockets, and the frontend and backend of a web application.
4 concepts, slides 40-45
Why this part matters
Every service you will place on the edge-to-cloud continuum, whether a camera pipeline, an IoT gateway or a cloud API, ends up as processes talking through sockets. The socket type decides what happens to a lost or late packet. The split between frontend and backend decides which parts can move to the edge and which must stay near the database.
Exams ask for the socket call sequence and for a justified choice between TCP and UDP. Your research project needs the same reasoning to explain where latency comes from. This part builds both, climbing from the point where a program first touches the network up to a complete web application.
By the end you can
- Explain a socket as a file descriptor, addressed by IP and port, that the socket API creates
- Order the socket calls a TCP server and client make, and say what accept returns
- Choose TCP or UDP for a workload by what a lost or late unit of data costs
- Split a web application into browser client, server and database, and trace a saved form
- Read a backend route as method plus path plus handler and trace one HTTP request
Think about what your browser does when you open the KFUPM portal. It cannot put signals on the Wi-Fi radio or build TCP segments by itself. Instead it makes a few requests to the operating system: open a connection to this address and port, send these bytes, give me whatever comes back. The operating system does everything below that. Understanding exactly what that request looks like is the foundation for everything else in this part.
An API is a menu, and the socket API is the networking menu
The set of operations one piece of software offers to another is an Application Programming Interface, or API. Earlier in this lecture you saw that a relationship between components is a protocol plus an Interface. Here the interface sits between the Application layer (your program) and the Transport layer (TCP and UDP inside the operating system). For networking, that interface is the Socket API.
Kurose and Ross describe the socket as the door between the application process and TCP. As a developer you control everything on the application side of that door, and you have very little control over the transport side. The rest of this concept opens the door and looks at what is actually there.
What a program actually holds: a file descriptor
Make it concrete. In C, calling socket() returns a small integer, for example 3. From then on the program writes write(3, ...) exactly as it would for an open file. POSIX states that when the descriptor refers to a socket, write() is equivalent to send() with no flags, and the Linux manual says the only difference between recv() and read() is the flags argument.
So to a program, a Socket is an operating system object that it reads and writes like a file. The kernel takes the bytes you write and turns them into TCP segments or UDP datagrams, which is why the application never needs to know how either protocol works. Be careful with the words: the socket API is the interface, and a socket is the endpoint that the API creates and names.
Naming the endpoint: the socket address
A file on disk has a path; a socket has an address: an IP address plus a Port number, for example 10.0.0.5:443. The IP address finds the host and the port finds the process on that host, the same two-level addressing you met with the TCP/IP stack. RFC 9293 identifies a TCP connection by a pair of sockets, one at each end, so a connection is really four numbers: client IP and port, server IP and port.
One more ingredient completes the name. The protocol is fixed when the socket is created, so TCP port 53 and UDP port 53 are different endpoints even on the same host. DNS, as RFC 1035 specifies, listens on both.
The lifecycle of a connection
A Server and a Client play different roles, so they use different calls. The server must be findable before anyone arrives: it creates a socket, binds it to a well-known address, marks it as listening, then accepts connections one by one. The client already knows where to go, so it only creates a socket and connects. Once connected, both sides send and receive, then close.
Socket lifecycle, in call order
- socket()
- Both sides. Creates an endpoint and returns a file descriptor such as 3. It has no address yet.
- bind()
- Server. Attaches a local IP address and port, for example 0.0.0.0:443. The Linux manual calls this assigning a name to a socket, so clients know where to knock.
- listen()
- Server. Marks the socket as passive, ready to accept connections. The backlog argument caps the queue of pending connections. Connection-oriented sockets only (SOCK_STREAM or SOCK_SEQPACKET).
- connect()
- Client. For TCP it starts the connection, which runs the three-way handshake. For UDP it only records a default destination.
- accept()
- Server. Takes the first pending connection and returns a new connected socket with a new descriptor, for example 4. The listening socket is unaffected and keeps waiting.
- send() / recv()
- Both sides. Move bytes. On a socket, write() is send() with no flags, and read() is recv() with no flags.
- close()
- Both sides. Releases the descriptor and ends that side of the connection.
Worked example
One HTTPS request through the socket calls
Server prepares
The server calls socket() and gets descriptor 3, calls bind() to 0.0.0.0:443, then listen().Client connects
The browser calls socket() and connect() to the server address. The TCP three-way handshake runs.Server accepts
accept() returns descriptor 4, a socket dedicated to this client. Descriptor 3 keeps listening.TLS handshake
Both sides exchange TLS handshake messages with send() and recv() on the connected sockets. From here on the bytes on the wire are encrypted.Request
The client calls send() with the encrypted request bytes. The server calls recv() on descriptor 4.Response
The server calls send() on descriptor 4; the client calls recv().Finish
Both sides call close() on their connected sockets.Result
The server held two sockets for this exchange: descriptor 3 listening and descriptor 4 connected.
Recall
What does a program actually hold when it has a socket, and which ordinary calls can it use on it?
Recall
List the server-side calls in order. What does accept return, and what happens to the listening socket?
Quick check
An HTTPS server process has 1000 clients connected on port 443. Counting the listening socket, how many TCP sockets does it hold?
Once you know a program holds a socket, the next decision is which kind. Take two jobs. First, downloading a 2 GB dataset for your project: one missing byte corrupts the file, so every byte must arrive, in order. Second, a live video call: a voice frame that arrives 1 s late is useless, so waiting for it to be resent makes the call worse, not better. These two jobs need different sockets, and the difference is fixed the moment socket() is called.
Two services behind the same API
A TCP socket is a stream socket. RFC 9293 describes TCP as a reliable, in-order, byte-stream service. It sets up a connection with a three-way handshake, numbers every byte, checks it with a checksum and retransmits what is lost. That reliability costs setup time and state on both hosts, which is exactly the connection that listen(), connect() and accept() manage.
A UDP socket is a datagram socket. RFC 768 says UDP offers a minimum of protocol mechanism: delivery and duplicate protection are not guaranteed, and the header is just four 16-bit fields (source Port number, destination port, length, checksum), 8 bytes in total. There is no listen or accept: each datagram stands alone. POSIX names the two kinds SOCK_STREAM and SOCK_DGRAM.
| Property | TCP socket | UDP socket |
|---|---|---|
| POSIX type | SOCK_STREAM | SOCK_DGRAM |
| Connection setup | Three-way handshake before any data | None, each datagram stands alone |
| Delivery | Lost segments detected and retransmitted | No delivery or duplicate guarantee |
| Ordering | Bytes reach the application in order | Datagrams may arrive out of order |
| Message boundaries | A continuous byte stream, no boundaries | Each datagram is one whole message |
| Header overhead | 20 bytes minimum, plus connection state | 8 bytes, four 16-bit fields |
| Typical uses | Web pages, file transfer, email | DNS queries, voice and video calls |
Choosing a socket for a workload
The right question is not which is faster. Ask what happens to your application when one unit of data is lost or late. If it must be recovered, use TCP. If late data is worthless, or the exchange is tiny and easy to retry, UDP avoids overhead that buys you nothing. For interactive media this is really about Latency: a retransmitted frame arrives after its moment has passed.
| Workload | Socket | Why |
|---|---|---|
| Web page over HTTP/1.1 or HTTP/2 | TCP | Every byte of HTML and script must arrive intact. Default ports 80 and 443. |
| File transfer (FTP) and email | TCP | One missing byte corrupts the file or message. |
| DNS query | UDP | A small single exchange that can simply be retried. Port 53, with TCP 53 for zone transfers and long answers. |
| Video call (WebRTC) | UDP | A late frame is useless, so retransmission only hurts. TCP through a relay is a fallback. |
| Web page over HTTP/3 | UDP (carrying QUIC) | QUIC adds reliable streams and encryption on top of UDP. |
The last row of that table shows that the choice is not always binary. RFC 9114 defines HTTP/3 as the same HTTP semantics over a new transport, QUIC, and RFC 9000 says QUIC packets are carried in UDP datagrams. QUIC rebuilds reliable streams, flow control and encryption in user space on top of UDP. So choosing UDP does not always mean giving up reliability. It means a protocol or library above UDP decides how much reliability to add, and when.
Quick check
A video call plays 50 voice frames per second and playback never pauses to wait. Which socket fits best, and why?
Quick check
HTTP/3 keeps the same HTTP semantics but changes the transport underneath. What does it run on?
Recall
Why do DNS queries use UDP while file transfers use TCP, and where does HTTP/3 fit?
Now bring sockets to the web. You type https://uni.majid.app and press Enter. The browser resolves the name, asks the operating system to open a TCP connection to port 443 (or QUIC over UDP for HTTP/3), sends the HTTP request, reads the response, and may reuse the same connection for the next requests. The page code only calls fetch("/api/...") and never sees a Socket.
The browser owns the client socket
A web application is a Network application in which the browser is the Client program and owns the client socket: when to open it, reuse it and close it. MDN puts it simply: browsers communicate with web servers using HTTP, by sending a request to the server. Every socket call from the lifecycle above is still happening, just inside the browser and inside the server runtime.
This changes the level at which you work. Frontend code for the World Wide Web is written in terms of HTTP requests, and backend code is written in terms of routes that answer them. The socket becomes infrastructure you rely on rather than an object you manage.
Recall
In a web application, who manages the client socket, and at what level does the frontend code work?
Three components: client, server, database
So what is on the other end of that connection? Picture a course portal. You submit a homework answer. The browser sends a request over the Internet. The server checks it and writes a row to the database. When you reopen the page later, the server reads that row and sends it back in a response.
The pieces have names. The Frontend is one Component: the client. The Backend is two components: the Server and the database. The client talks only to the server, never directly to the database. MDN describes the same flow: on a request, the server fetches data from the database and builds the response. This is the Request-response loop you met with network applications, and the client always starts it.
Browser: HTML, CSS, JavaScript
Checks requests, runs the logic, builds responses
Stores data that must survive between requests
Look at the server to database link closely, because it reuses everything from the first concept. It is itself a client-server connection over sockets, with the web server acting as the database client. Writes flow from server to database and reads flow back. A web application is therefore a chain of socket conversations, each with its own client and server role.
Why server and database often live apart
- They scale separately: many identical server instances can share one carefully managed database.
- Their resource needs differ: the server wants CPU, the database wants disk and memory.
- The database can be kept off the public Internet, reachable only from the servers.
This is the first placement decision on the edge-to-cloud continuum. Later parts ask where the server and the database should each live, and every placement has a cost: each read or write between separate machines adds its own network Round-trip time (RTT), which later feeds into Response time (RT).
Quick check
A user saves a form on a web app and the data must persist. Which path does it take?
Recall
Name the two backend components of a web application, and give one reason they often run on different machines.
The architecture becomes real when you read the code on each side. Consider a calculator running in the browser and the backend that serves it. Walk one request through: the user opens the About view, and the frontend sends GET /calculadora/about.
From request to handler to response
On the Backend, the framework compares the request method and path against its routes. It finds the route with method GET and path /calculadora/about and calls its handler, handler(request, h). The handler builds an object with a message field and returns it. In hapi, a returned plain object becomes the response payload, serialized as JSON. The browser receives that JSON in the HTTP response body and the Frontend renders it.
The general rule: a route is a method plus a path plus a handler. The set of routes a backend exposes is its Web API, the Interface the frontend programs against, while HTTP is the Protocol carried between them. Protocol plus interface again, one level higher than the socket API. Underneath, the Node.js runtime owns a listening socket and one connection socket per Client, exactly the welcoming door picture from the first concept.
Worked example
Tracing GET /calculadora/about
Request
The browser sends GET /calculadora/about over its connection to the server.Route match
The framework matches method GET and path /calculadora/about to the registered route.Handler
handler(request, h) runs and returns an object with a message field.Serialize
The framework turns the object into a JSON response body.Response
The response travels back over the same connection, closing the Request-response loop.Result
What the handler returns is exactly what the browser receives, as JSON.
Same language, different machines
Notice the languages. The frontend uses HTML for structure, CSS for presentation and JavaScript for behaviour, all inside the browser. The backend also runs JavaScript, but in the Node.js runtime on the Server. Same language, different processes on different machines, joined only by HTTP over a socket.
| Side | Runs where | Written in | Role |
|---|---|---|---|
| Frontend | Inside the browser on the user machine | HTML, CSS, JavaScript | Draws the calculator and sends HTTP requests |
| Backend | In the Node.js runtime on a server | JavaScript (hapi on Node.js) | Matches routes and returns data in responses |
Recall
What three things make up a backend route, and what does the client receive when the route is hit?
Recap
If you remember nothing else
- An API is how software asks other software for a service. The socket API is the door from the application layer to the transport layer.
- A socket is a file descriptor. Programs read and write it like a file, and it is addressed by IP address plus port.
- Servers call socket, bind, listen and accept. Clients call socket and connect. accept returns a new socket per client.
- TCP sockets give a reliable, ordered byte stream with connection overhead. UDP sockets send independent datagrams with minimal overhead.
- Choose by what a lost or late unit costs: files and web pages use TCP, DNS queries and video calls use UDP, and HTTP/3 runs on QUIC over UDP.
- In a web app the browser manages the client socket. The backend is a server plus a database that may run on different machines.
- A backend route is method plus path plus handler, and what the handler returns becomes the HTTP response.
Sources
- Computer Networking: A Top-Down Approach, section 2.7 Socket Programming with TCPBookKurose and Ross, UMass AmherstSocket as the door to TCP, welcoming and connection sockets(opens in a new tab)
- Computer Networking: A Top-Down Approach, 9th editionBookKurose and Ross(opens in a new tab)
- socket(): POSIX.1-2024DocsThe Open Group and IEEEReturns a socket file descriptor; SOCK_STREAM and SOCK_DGRAM(opens in a new tab)
- write(): POSIX.1-2024DocsThe Open Group and IEEEOn a socket, write equals send with no flags(opens in a new tab)
- bind(2)DocsLinux man-pages project(opens in a new tab)
- listen(2)DocsLinux man-pages project(opens in a new tab)
- accept(2)DocsLinux man-pages project(opens in a new tab)
- connect(2)DocsLinux man-pages project(opens in a new tab)
- recv(2)DocsLinux man-pages project(opens in a new tab)
- RFC 9293: Transmission Control Protocol (TCP)RFCIETF(opens in a new tab)
- RFC 768: User Datagram ProtocolRFCIETF(opens in a new tab)
- RFC 9000: QUIC, A UDP-Based Multiplexed and Secure TransportRFCIETF(opens in a new tab)
- RFC 9114: HTTP/3RFCIETF(opens in a new tab)
- RFC 9110: HTTP SemanticsRFCIETFDefault ports 80 and 443; HTTP/3 over QUIC(opens in a new tab)
- RFC 1035: Domain Names, Implementation and SpecificationRFCIETFDNS on UDP and TCP port 53(opens in a new tab)
- RFC 8835: Transports for WebRTCRFCIETF(opens in a new tab)
- RFC 959: File Transfer ProtocolRFCIETF(opens in a new tab)
- Fetch Living StandardDocsWHATWGThe browser obtains and reuses connections for fetch(opens in a new tab)
- Net module (node:net)DocsNode.js(opens in a new tab)
- hapi API reference: server.route and lifecycle methodsDocshapi.js project(opens in a new tab)
- Client-Server overviewDocsMDN Web Docs(opens in a new tab)
Part 06: Why layering: the continuum idea
What a continuum is, and how networks and the memory hierarchy already use layers to trade distance and speed.
4 concepts, slides 46-50
Why this part matters
Every design decision in COE 558 is a placement decision. Does this task run on the device, at the edge, on a fog node or in the cloud? Before the physics of latency (Part 07) and the named tiers (Part 08), you need the pattern underneath all of them.
Computing has already solved this problem twice. Networks split a long path into segments, each built for its own span. Processors stack registers, caches and RAM so that urgent data sits close to the CPU. Edge computing applies the same trade, nearness against capacity, to the whole Internet. For your research project, this analogy is the cleanest way to justify putting a component at the edge.
By the end you can
- Explain why the space between client and server is a continuum of candidate locations, and why edge and cloud are regions of one range rather than separate worlds.
- Explain why a user-to-printer path is split into LAN switching and WAN routing tiers, and never confuse tier positions with protocol stack layers.
- Explain the memory hierarchy as a placement policy driven by urgency and locality, and say why a small fast layer is enough.
- Use T = S/B + L with measured bandwidth and latency to say when latency dominates, and justify the analogy that CPU cache is to RAM as edge is to cloud.
Part 05 ended with a simple picture: a web app is a Client talking to a Server. The natural next question is where that server should actually sit, and the answer turns out to be "not in one place, but anywhere along a range".
The path is not an empty wire
Picture a student in Dhahran opening a web app whose only server is in a data center on another continent. Every click travels the full distance out as a request and the full distance back as a response. Now add three stops along the way: one in the campus building, one in the city, one in the region. Each stop is closer to the client than the far server is, so any work that can run at a nearer stop comes back sooner.
Seen this way, the space between client and server is a range of places where compute and storage can live. Latency grows as you move away from the client, and so does capacity, because far sites are larger. Placing layers along that range lets each request be answered by the nearest layer that is powerful enough to handle it.
Starts every request
Nearest, smallest
Farther, larger
Farther still
Largest capacity
This is not an invented teaching device. NIST describes fog computing as a layered model for accessing a shared continuum of computing resources, with nodes organized by their latency and distance to the end devices, and with different layers providing different services while working together as a continuum (NIST SP 500-325).
What makes a range a continuum
The word Continuum is doing real work here, so pin it down with the clearest example there is: visible light. Walk along the spectrum from 400 nm to 700 nm. Move one nanometer and the color does not visibly change. There is no single nanometer where blue stops and green starts. Yet 400 nm (violet) and 700 nm (red) are obviously different colors. That is the whole definition in one picture: neighboring points barely differ, while the two ends differ a lot. The Britannica Dictionary and Wiktionary put it the same way: a continuous series where no part is noticeably different from its neighbors, although the ends are very different from each other.
The familiar names on the wider electromagnetic spectrum (gamma, X-ray, UV, infrared, microwaves, radio) are labels people put on that smooth range because labels are useful. They are not walls that nature built.
Now map it back to computing. In the Edge-to-cloud (E2C) continuum, the two endpoints are "computing on the device itself" and "computing in a hyperscale cloud region". Everything between them is a candidate location for the layers above. The names you will meet in Part 08, Mist, Edge, Fog and Cloud, work exactly like UV and infrared: convenient labels on one range, where latency to the client and available capacity both grow as you move away from the device.
Recall
As you move from the client toward a distant server, what grows, and why would you add layers in between?
Recall
In your own words, what makes something a continuum rather than a set of separate categories?
Quick check
What best describes a continuum, as used for the edge-to-cloud continuum?
Layering along a distance is not new. The network you use every day already splits one long path into segments, each built with the technology that suits its span.
Make it concrete. A student sits in a KFUPM lab and sends a print job to a partner office in Riyadh. The first hop is the lab's Ethernet switch. The middle of the journey crosses routers in a wide area network. The last hop is the office's own switch, which delivers the job to the printer. Three segments, three tiers along the distance.
Why one technology cannot run end to end
A switch works at the Data link layer and forwards frames by MAC address. It fills its forwarding table by flooding and learning, which is cheap and automatic inside one building but does not scale to the whole world: flooding every unknown frame to every port of every network on Earth would be absurd. A router works at the Network layer and forwards by IP address, using tables computed by routing algorithms over hierarchical address blocks, which is exactly what scales across many networks (Kurose and Ross, Chapter 6). Each technology is excellent inside its own range and poor outside it, so the path is built from pieces that each fit a stretch of it.
| Segment | Span | Device | Addresses used | Stack layer |
|---|---|---|---|---|
| LAN 1 (tier 1) | One building or campus | Switch | MAC | Link |
| WAN (tier 2) | Between cities | Router | IP | Network |
| LAN 2 (tier 3) | One building or office | Switch | MAC | Link |
This is the networking Continuum: distance grows along the path, and connectivity is assembled from tiers that each serve one stretch. Keep the shape in mind, because the edge-to-cloud continuum does the same with compute instead of connectivity, and the memory hierarchy, next, does it with time instead of distance.
Recall
In the user-to-printer example, which device handles each segment, and why doesn't one technology run end to end?
Quick check
In the user-to-printer path, which standard stack layer does the first LAN's switching operate at?
The second place layering already exists is inside every computer. Here the axis is not kilometers but time: how long it takes the CPU to reach a piece of data.
Imagine a loop that sums a large array. The running total is touched on every instruction, so it lives in a register, right inside the processor. The chunk of the array being read right now is touched thousands of times a second, so it lives in cache. The rest of the dataset sits in RAM, and the file it came from sits on disk. Each step outward is slower to reach, but holds more.
This arrangement is the Memory hierarchy, a Continuum of access times. The rule is simple: the more urgently and frequently data is needed, the closer to the CPU it lives. Hennessy and Patterson describe the same picture: as you move away from the processor, each level is slower and larger, with times ranging from picoseconds up to milliseconds for magnetic disks. Notice the match with the network path: near tiers are small and fast, far tiers are large and slow.
Why a small fast layer is enough
If fast memory holds only a tiny fraction of the data, why does it help at all? Because of locality. Programs tend to reuse data they touched recently (temporal locality) and data stored near it (spatial locality). So a small, fast layer that keeps those items can answer most requests. Denning notes that this one principle shapes processor caches, disk caches, storage hierarchies and edge caches for the Web. That last item is the bridge to this course: an edge site works for the same reason a CPU cache does.
Recall
What decides which level of the memory hierarchy a piece of data lives in, and why can a small cache serve most reads?
Quick check
Why can a small CPU cache serve most of the memory reads a program makes?
A ranking tells you the order of the levels; measurements tell you how much each step costs. To measure a level you need two quantities: bandwidth (how much data per second it can move) and latency (the fixed access delay paid on every fetch). Storage is simply the size of each level.
Start with a video editing workstation. The frame on screen right now sits in cache or registers, because it is touched constantly. Today's raw clips are pulled from the SSD into RAM so they can be processed quickly. Last year's projects stay on slow archive storage and are fetched only when you open them. That is the Memory hierarchy used as a placement policy, and the question is how big the gaps between levels really are.
The measured hierarchy
The Cornell Virtual Workshop publishes figures measured on Intel Skylake server processors. Reading outward, bandwidth is roughly cut in half at each level while latency roughly quadruples. The real steps vary, so read this as a rule of thumb.
| Level | Size per socket | Read bandwidth | Latency |
|---|---|---|---|
| L1 cache | 768 KB | ~84 GB/s | 2 ns |
| L2 cache | 24 MB | ~60 GB/s | 7 ns |
| L3 cache (shared by cores) | 33 MB | ~30 GB/s | 26 ns |
| Main memory | 96 GB | ~10 GB/s | 90 ns |
The bandwidth figures come from a Sandia National Laboratories evaluation of Skylake (Hammond, Vaughan and Hughes, 2018), and the latencies were measured on a closely related Xeon model. Latency is also quoted in clock cycles, such as "4 cyc." and "14 cyc.". 2 ns in 4 cycles implies about 0.5 ns per cycle, a clock of roughly 2 GHz (our own arithmetic, not a published figure).
Turning two resources into one time
Bandwidth and latency only become useful for decisions once you combine them into the time a fetch actually takes. Cornell gives a simple model for any level:
Here T is time, S is the size of the data, B is bandwidth and L is latency. For a tiny fetch, S/B is close to zero, so latency decides everything. For a huge transfer, the S/B term dominates and bandwidth decides.
Worked example
Fetching from L1 cache vs main memory
A 16 KB fetch from L1
Treat 16 KB as 1.6 × 10⁻⁵ GB. S/B = 1.6 × 10⁻⁵ / 84 s ≈ 190 ns, plus 2 ns latency, so about 192 ns. Latency is about 1 percent.The same 16 KB from main memory
S/B = 1.6 × 10⁻⁵ / 10 s = 1.6 µs, plus 90 ns, so about 1.69 µs. Bandwidth sets the cost.A 64-byte fetch
64 bytes ≈ 6.4 × 10⁻⁸ GB. From L1: S/B ≈ 0.76 ns against 2 ns latency. From main memory: S/B = 6.4 ns against 90 ns latency. Now latency is most of the time.Result
Small fetches are latency-bound; large ones are bandwidth-bound. (Our arithmetic from the Cornell figures.)
The analogy the course is built on
Now every piece is in place. CPU cache is to RAM as Edge is to Cloud. Both put a small, fast, nearby tier in front of a large, slower, distant one. Both answer most requests from the near tier because of locality, and both fall back to the far tier on a miss. Interactive requests are small, so by T = S/B + L they are latency-bound and benefit most from a nearer tier. Bulk analytics is not interactive, so a few extra milliseconds of latency cost it little, and it needs far more compute than an edge site has. That is why Offloading it to the cloud, where capacity is, makes sense, as long as the data can be moved there or already lives there.
Recall
From the Cornell Skylake measurements, how do bandwidth and latency change from L1 out to main memory, and what are the end values?
Recall
Using T = S/B + L, why does latency matter more than bandwidth for a small interactive request?
Recall
Complete and justify: CPU cache is to RAM as ___ is to ___.
Quick check
Per the Cornell Skylake data, what happens at each step outward from L1 cache toward main memory?
Quick check
Using T = S/B + L, what dominates retrieval time for a tiny, interactive request?
Quick check
In the cache and edge analogy, what does a cache miss correspond to?
Recap
If you remember nothing else
- A continuum is a range whose neighbors barely differ but whose ends differ greatly. Device, mist, edge, fog and cloud are labels on one range.
- Layering places resources at points along the path, so each request is served by the nearest layer with enough capacity.
- Networks already do this: switches serve LANs by MAC address, routers serve the WAN by IP address. Tier positions 1, 2, 3 are not stack layers.
- The memory hierarchy trades capacity for nearness, and locality lets a small fast layer serve most reads.
- On Skylake: L1 about 84 GB/s and 2 ns, L2 60 GB/s and 7 ns, L3 30 GB/s and 26 ns, RAM 10 GB/s and 90 ns.
- Retrieval time is about S/B + L. Small interactive fetches are latency-bound, so nearness matters most.
- CPU cache is to RAM as edge is to cloud: a small, fast, near tier in front of a large, slow, far one, justified by locality.
Sources
- NIST SP 500-325: Fog Computing Conceptual Model (2018)DocsNational Institute of Standards and Technology(opens in a new tab)
- Memory Access Times (Code Optimization)DocsCornell Virtual Workshop, Cornell University Center for Advanced ComputingNew location of the page cited on slide 50(opens in a new tab)
- Memory Hierarchy (Code Optimization)DocsCornell Virtual Workshop, Cornell University Center for Advanced Computing(opens in a new tab)
- Computer Architecture: A Quantitative Approach, 6th ed., Chapter 2 Memory Hierarchy Design, Hennessy and PattersonBookElsevier (Morgan Kaufmann)(opens in a new tab)
- Computer Networking: A Top-Down Approach, 8th ed., Chapter 6 The Link Layer and LANs, Kurose and RossBookPearson(opens in a new tab)
- The Locality Principle, Peter J. Denning, Communications of the ACM 48(7), 2005PaperAssociation for Computing Machinery(opens in a new tab)
- Evaluating the Intel Skylake Xeon Processor for HPC Workloads (Hammond, Vaughan, Hughes, 2018)PaperSandia National Laboratories / U.S. DOE OSTI(opens in a new tab)
- Latency Numbers Every Programmer Should Know (after Jeff Dean and Peter Norvig)ArticleGitHub Gistcirca 2012 values, order-of-magnitude only(opens in a new tab)
- continuum (definition)DocsWiktionary(opens in a new tab)
- continuum (dictionary entry)DocsBritannica Dictionarycited on the slide(opens in a new tab)
Part 07: The physics of latency
Latency to the cloud, the speed-of-light lower bound, geodesic distance, real measurements from Dammam, and the four delays that add up.
4 concepts, slides 51-55
Why this part matters
Today’s default is to put the server in the cloud. This part prices that default in milliseconds. You will learn to turn a distance into the smallest delay physics allows, compare that floor with real measurements from Dammam, and break the gap into the four delays every packet pays.
The ideas build on each other. First, what latency is and the hard floor under it. Second, how to measure the distance that floor depends on. Third, how far above the floor real networks sit. Fourth, the four delays that explain the difference. For the exams, expect calculations: distance to bound, one way versus round trip, and the formulas L/R and d/s. For the research project and for real edge and cloud design, remember one thing above all: the d/s term is the one cost that no amount of cloud capacity removes. That is the physical argument for the edge, fog and cloud continuum in the next part.
By the end you can
- Define latency as transit time, and explain why a request-response exchange pays it twice.
- Compute the one-way and round-trip latency lower bound from a geodesic distance, and the fiber floor above it.
- Explain why latency must use geodesic rather than planar distance.
- Compare a measured RTT with the right bound, and account for the roughly threefold gap.
- Name the four packet delays, apply transmission = L/R and propagation = d/s, and predict which one dominates.
Picture a student in Dhahran opening an app whose backend runs in the AWS Frankfurt region. His phone sends a request that must physically cross about 4,400 km of Earth, and the answer must cross it again on the way back. Nothing useful can happen on his screen until both trips are done.
Latency is transit time, paid in both directions
That travel time is Latency: the time data spends in transit between two points. In the Request-response loop you met earlier, the Client always starts, so every exchange pays latency twice: once from client to Server, once from server back to client. Total network latency is the sum of the two one-way figures. They are often close, but they need not be equal, because Internet routes are not always the same in both directions.
Sends the request and waits.
Does its work. That time is processing, not latency.
Receives the answer after both transits.
Putting the server in the Cloud is the default because it is convenient, elastic and cheap to operate. The natural next question is how much time that choice costs at the very least, before any engineering detail enters the picture.
The smallest latency physics allows
Start with a number. A server 3,000 km away, reached at the speed of light, needs 3,000 ÷ 300,000 = 0.01 s, which is 10 ms one way. So every 1,000 km costs at least 3.3 ms one way, or 6.7 ms for a round trip.
Generalising this gives the Latency lower bound: divide the Geodesic distance by the speed of light . It is a lower bound for three reasons that stack. Nothing carries information faster than light. No cable between two cities can be shorter than the shortest path along the Earth. And the formula ignores every other delay a real packet meets. Because each simplification can only make the true figure larger, the result is a floor that no provider, protocol or hardware upgrade can break.
Real glass is slower than vacuum
“Speed of light” in the bound means light in a vacuum. Real links carry light through glass fiber with a refractive index of about 1.5, so the signal moves at roughly 200,000 km/s, about two thirds of (Grigorik, High Performance Browser Networking). Even a perfectly straight fiber therefore needs about 1.5× the bound. ITU-T G.114 plans 5 µs/km for optical fibre systems, which is 5 ms per 1,000 km, against 3.3 ms per 1,000 km at . Keep both numbers in mind: the vacuum bound is the absolute floor, and the fiber floor is the realistic best case for a cabled network.
Try the simulator. Pick the Frankfurt preset and read the one-way bound, then read the round-trip value to see the number doubled. The presets use the distances implied by the lecture’s own figures (for example 11,100 km for Oregon), a choice examined closely when we compare bounds with measurements below.
Light in fibre is slower than in vacuum, cables do not follow the geodesic, and routers add processing, queueing and transmission delay. The band above is only an illustration. Slide 55 notes real latency can reach up to 100× the lower bound.
Recall
What does the lower-bound formula leave out, and why is it still useful?
It leaves out the slower speed of light in fiber, cable routes longer than the geodesic, and the processing, queueing and transmission delays. It is still useful because no real network can beat it: if the bound alone already breaks your latency target, the only fix is to move the server closer.
The bound is only as good as the distance you feed it, so which distance is the right one? Stretch a string tight between London and Singapore on a real globe. It does not run across Arabia the way a ruler on a flat map suggests. It runs north, over Eastern Europe and Central Asia. Now flatten that globe into a rectangle: the string that was the shortest path turns into a curve.
Those are the two candidates. A straight line drawn on the flat map gives the planar distance. The curve that the string traces gives the Geodesic distance between the same two endpoints. Esri’s documentation defines planar distance as straight-line Euclidean distance in a 2D Cartesian coordinate system, and geodesic distance as distance across the curved surface of the world in 3D. Signals travel over the real Earth, not over a map, so latency lives in the second definition.
Map projections stretch the Earth, and the stretch grows with latitude and with the size of the area. Over a city block the two distances agree. Over continents they can disagree a lot, and near the equator they can be surprisingly close: Esri notes that for Singapore to Nairobi, about 7,440 km, the Web Mercator planar result is less than a metre longer. Because you cannot know in advance which case you are in, always use the geodesic when you compute a Latency lower bound. Using a planar distance that happens to be too short would even produce a “bound” that is not a bound at all.
| Aspect | Planar distance | Geodesic distance |
|---|---|---|
| Computed in | A flat 2D Cartesian plane (a projected map) | 3D space, across the curved surface of the Earth |
| Looks like on a flat map | A straight line | Usually a curve bending toward the pole |
| Accuracy over long range | Error grows with distance and latitude | Correct by definition (shortest surface path) |
| Use for latency | No, except for tiny local areas | Yes, always for intercity or intercontinental links |
Quick check
On a flat map, the straight London to Singapore line looks shorter than the curved one. Which path is shorter on the real Earth?
Recall
Why does the curved line on a flat map represent the shorter path?
The map projection distorts the sphere. The geodesic is the shortest path along the curved surface, and it only looks curved after the globe is flattened into a map.
With the right distance and the right formula in hand, the bound can finally meet reality. Put the client in Dammam and the server in three AWS regions: US West (Oregon, us-west-2), Europe (Frankfurt, eu-central-1) and Asia Pacific (Tokyo, ap-northeast-1). For each, compute the one-way Latency lower bound, then compare it with the Round-trip time (RTT) measured with aws-latency-test.com.
Worked example
Dammam to Frankfurt: bound against measurement
Get the distance
The geodesic from Dammam to Frankfurt is about 4,390 km. The lecture’s 14.7 ms implies about 4,410 km, which agrees.
One-way bound
4,390 ÷ 300,000 = 0.01463 s ≈ 14.6 ms.
Round-trip bound
2 × 14.6 ≈ 29.3 ms. This is the number to compare with an RTT.
Fiber floor
At 200,000 km/s the same path takes 21.9 ms one way and 43.9 ms round trip, even on a perfect cable laid exactly along the geodesic.
Compare
Measured RTT 100 ms ÷ 29.3 ms ≈ 3.4× the vacuum bound, and about 2.3× the fiber floor.
Result
Physics sets a floor of about 29 ms (vacuum) or 44 ms (fiber). The remaining ≈56 ms comes from longer cable routes, routers and queues. No provider can ever push this RTT below 29.3 ms.
The rule behind the example: compare like with like. An RTT is a round trip, so compare it with twice the one-way bound, never with the one-way figure itself. Doing that for all three regions tells the same story each time: the measurements sit about 2.9× to 3.4× above the vacuum round-trip bound. A ratio that stable across very different distances is a hint that the extra cost is systematic, not bad luck on one path.
| Region | Geodesic | Bound, one way / RTT | Fiber RTT floor | Measured RTT | Measured ÷ RTT bound |
|---|---|---|---|---|---|
| Frankfurt (eu-central-1) | ≈4,390 km | 14.6 ms / 29.3 ms | 43.9 ms | 100 ms | ≈3.4× |
| Tokyo (ap-northeast-1) | ≈8,300 km | 27.7 ms / 55.3 ms | 83.0 ms | 163 ms | ≈2.9× |
| Oregon (us-west-2) | ≈11,900 km | 39.7 ms / 79.4 ms | 119 ms | 243 ms | ≈3.1× |
Where the factor of three comes from
- About 1.5× comes from glass alone, because light in fiber moves at about two thirds of .
- Cables do not follow the geodesic. They follow coastlines and seabeds, land at a few stations, and backhaul over land to data centers.
- Each router on the way adds processing and queueing time, two of the four delays in the next concept.
Research agrees with this breakdown. Singla and colleagues measured Internet paths worldwide and found router paths about 2.3× the speed-of-light latency at the median. After removing the 1.5× fiber factor, path detours alone still leave about 1.53×.
Quick check
Dammam to Singapore is about 6,370 km along the geodesic. What is the round-trip lower bound at the speed of light in vacuum?
Quick check
The measured Dammam to Frankfurt RTT is 100 ms, about three times the 29 ms vacuum bound. What best explains the gap?
Recall
Dammam to London is about 5,020 km. Give the vacuum RTT bound and the fiber RTT floor.
5,020 ÷ 300,000 ≈ 16.7 ms one way, so 33.5 ms RTT. In fiber at 200,000 km/s it is 25.1 ms one way and 50.2 ms RTT.
Recall
The measured Dammam to Tokyo RTT is 163 ms. How much is left after perfect fiber along the geodesic?
About 83 ms is the fiber RTT floor, so roughly 80 ms comes from longer routes, router processing and queueing.
The factor of three needs a precise vocabulary. Follow one 1,500-byte packet across one router onto a 4,390 km fiber link running at 1 Gbps. Pushing its 12,000 bits onto the wire takes 12,000 ÷ 10⁹ s = 12 µs. Travelling the fiber takes about 22 ms. The router’s own work takes microseconds. Waiting behind other packets takes anything from nothing to a very long time. Now slow the link to 10 Mbps: pushing the bits out now takes 1.2 ms, and the fiber trip is still 22 ms.
Those four pieces are the four packet delays, and Kurose and Ross (section 1.4) add them up at every node a packet crosses. Processing delay happens inside switches and routers. Queueing delay happens in buffers when traffic piles up. Transmission delay happens at the network interface and depends on bandwidth. Propagation delay happens on the physical channel and depends on distance. Only the last one is the physics behind the lower bound; the other three are what real equipment and real traffic add on top.
End to end, you add the nodal delay over every hop on the path, and an RTT adds it again for the return direction. That sum is exactly the measured RTT from Dammam, which is why it sits so far above 2d/c. Hold on to the contrast that exams love: transmission depends on packet size and bandwidth and has nothing to do with distance, while propagation depends on distance and the medium and has nothing to do with bandwidth.
| Delay | Where it happens | Driven by | Formula or key factor | Shrinks when |
|---|---|---|---|---|
| Processing | Inside each router, switch or host | Header checks, bit-error checks, forwarding lookup | Device speed (usually microseconds) | Faster hardware, fewer hops |
| Queueing | In router and server buffers | Congestion: how many packets arrived ahead of yours | Traffic intensity, from zero to very large | Less traffic, more capacity, fewer hops |
| Transmission | At the network interface pushing bits onto the link | Packet size and link bandwidth | L / R | Bandwidth goes up or packets get smaller |
| Propagation | Along the physical channel (fiber, copper, radio) | Link length and the medium | d / s | The endpoints move closer |
Can it really be 100 times the bound?
The lecture claims Internet latency can reach 100× the Latency lower bound. For whole page loads, measurements support this. Singla and colleagues found that fetching just the HTML of popular web pages took 34× the round-trip speed-of-light latency at the median and 169× at the 90th percentile. Those fetches include more than the four per-hop delays: a DNS lookup, the TCP handshake and TCP’s slow start all add round trips. The lecture’s example of a 15 ms theoretical latency turning into 1,500 ms is best read the same way: an illustration of network delays stacked with protocol and application effects, not four delays on a single link.
Quick check
A file server moves from a 1 Gbps link to a 10 Mbps link on the same cable route. Which delay grows for every packet?
Recall
Which delay depends on bandwidth, and which on distance? Give both formulas.
Transmission, , depends on packet size and link rate. Propagation, , depends on link length and the signal speed in the medium.
Recap
If you remember nothing else
- Latency is time in transit. A request-response exchange pays it in both directions.
- Latency_LB = geodesic distance ÷ c (about 300,000 km/s). Double it for RTT. No network can beat it.
- Measure distance along the curved Earth (geodesic), not as a straight line on a flat map.
- Light in fiber travels at about 2/3 c, so even a perfect cable is about 1.5× the bound (about 5 ms per 1,000 km).
- Measured RTTs from Dammam (100, 163 and 243 ms) are about 3× the round-trip bound, because of slower fiber, cable detours, routers and queues.
- Total delay = processing + queueing + transmission (L/R) + propagation (d/s). Bandwidth shrinks only transmission. Only moving closer shrinks propagation.
- Full page fetches can be tens to over a hundred times the bound (Singla et al.: 34× median, 169× at the 90th percentile). This is why edge placement matters.
Sources
- Computer Networking: A Top-Down Approach, 9th edition (Kurose and Ross)BookPearsonSection 1.4, Delay, Loss, and Throughput in Packet-Switched Networks: the four delays, L/R and d/s.(opens in a new tab)
- Transmission versus Propagation Delay (interactive applet)DocsPearson, companion site for Kurose and RossPlay with packet size, rate and distance to see the two delays separate.(opens in a new tab)
- Speed of light in vacuum, CODATA valueDocsNISTExactly 299,792,458 m/s.(opens in a new tab)
- High Performance Browser Networking: Primer on Latency and Bandwidth (Ilya Grigorik)BookO’ReillyThe four delays; fiber refractive index about 1.5, signal speed about 200,000,000 m/s.(opens in a new tab)
- The Internet at the Speed of Light (Singla, Chandrasekaran, Godfrey, Maggs)PaperACM HotNets 2014HTML fetch 34× c-latency at the median, 169× at the 90th percentile; router paths 2.3×; fiber alone 1.5×.(opens in a new tab)
- ITU-T Recommendation G.114: One-way transmission timeDocsITU-TTable 1: optical fibre cable systems, 5 µs/km including repeaters and regenerators.(opens in a new tab)
- Geodesic versus planar distanceDocsEsri ArcGIS Pro documentationDefinitions of planar and geodesic distance, and the Singapore to Nairobi example.(opens in a new tab)
- Algorithms for geodesics (C. F. F. Karney)PaperJournal of Geodesy 87(1), 2013Accurate geodesics on the WGS84 ellipsoid.(opens in a new tab)
- AWS RegionsDocsAmazon Web ServicesRegion codes for Oregon, Frankfurt, Tokyo, Bahrain and UAE.(opens in a new tab)
- AWS Latency TestArticleIndependent, not an official AWS projectSource of the measured Dammam RTTs. Results vary per run and are for illustration only.(opens in a new tab)
Part 08: Edge, fog and cloud
The layers of the E2C continuum from mist to regional cloud, their latency, resources and typical roles.
4 concepts, slides 56-60
Why this part matters
Every system you design in this course, and the research project itself, starts with one question: where does each piece of work run? This part gives you the map. It is a chain of places from the device in your hand to a hyperscale data center. Each step outward buys processing power and storage, and each step costs latency. Exam questions on placement, and the mapping problems in the next part, both assume you can read this map fluently.
By the end you can
- Explain why the server side of a system is a continuum of possible locations, and why the location a user is served from sets the latency they feel.
- Name the three layers of the E2C continuum and its seven stages in order from device to cloud, with a real product or board for each.
- Explain the core trade of the continuum: resources and coverage grow outward while proximity falls, and argue it with propagation, hops and real millisecond thresholds.
- Apply the placement rule, and justify a placement with latency, processing and storage needs, and cost.
- Spot ambiguous terminology, such as near versus far edge, and a mislabelled microcontroller board.
Start with something you do every week. When you press play on Netflix in Dhahran, the video bytes do not cross an ocean from a data center in the United States. Netflix runs its own delivery network, Open Connect, and places caching appliances inside the networks of internet service providers and at Internet Exchange Points. The first bytes of your episode travel a few kilometres and a few router hops, not thousands of kilometres and dozens of hops.
In the client-server model of the earlier parts, the Server looked like one box at the far end of the line. The Netflix example shows why that picture is too simple. Between the Client and the origin data center there are several stops, and each stop is a possible home for code or data. Moving work from one stop to another moves you along a continuum of latencies. A stop closer to the client means a shorter path, so less Propagation delay and fewer routers where a packet can wait in a queue.
The consequence for users is striking. The same service, running the same code, can make one user happy and another angry, and the only difference is where the service runs relative to each of them. A viewer served from a cache in their own city sees the video start at once; a viewer served from another continent watches a spinner. Placement is therefore not an operations detail. It is part of the design of the application.
The idea is older than cloud computing. Mahadev Satyanarayanan traces edge computing back to content delivery networks (CDNs) such as Akamai in the late 1990s: a CDN uses nodes close to users to prefetch and cache web content. Edge computing generalises that idea. Instead of only caching bytes near users, you run arbitrary code there. The rest of this part names the stops on the line and the trade each one offers; the course as a whole is about deciding what to put at each one, which is the heart of the Edge-to-cloud (E2C) continuum.
Recall
Why does Netflix place Open Connect appliances inside ISP networks instead of serving every stream from a central data center?
Quick check
Why does caching a video at Layer 1 near the viewer reduce buffering?
To reason about placement you need names for the stops. Follow one request from a smart doorbell outward. The doorbell's own small chip decides that the button was pressed. The event goes to your home Wi-Fi router. From there it reaches a site run by your mobile or broadband operator nearby, for example an AWS Wavelength Zone, which places AWS compute and storage at the edge of a carrier's 5G network. Next it may reach a metro micro data center that collects events from many homes. Finally it lands in an AWS Region, inside one of its Availability Zones.
Those stops group into three coarse layers. The Cloud is farthest from the client. The Fog is the middle: mini data centers or intermediate nodes that process data partially. The Edge is closest to the client. Each Layer then splits into sublayers, which gives a seven-stage chain:
- Device: the phone, sensor or doorbell itself.
- Mist: tiny compute on or right beside the device.
- Near edge: the home router or on-premises gateway.
- Far edge: a nearby telco data center or carrier site.
- Fog: metro nodes, micro data centers and aggregation nodes.
- Regional cloud: a cloud region close to the user's country.
- Cloud: the full hyperscale infrastructure of availability zones and regions.
The chain is a spectrum, not a set of sealed boxes, which is exactly why it is called a continuum. Each stage is simply one more Server layer where you could run part of your application, and the boundaries between neighbouring stages are soft.
The standards agree on the shape and admit the edges are fuzzy. The term fog comes from Bonomi and colleagues at Cisco in 2012, who wrote that fog computing extends the cloud paradigm to the edge of the network. NIST SP 500-325 later formalised it: fog nodes sit between smart end devices and centralised cloud services, and can be physical (gateways, switches, routers, servers) or virtual (virtual switches, virtual machines, cloudlets). It describes mist as a lightweight, rudimentary form of fog computing that uses microcomputers and microcontrollers to feed fog nodes, and it does not treat mist as a mandatory layer.
The cloud sublayers, precisely
Unlike fog and mist, the cloud's own sublayers have exact definitions from the providers. Learn them from AWS, since the terms are used across the industry.
Cloud sublayers as AWS defines them
- Region
- A separate geographic area that contains several isolated availability zones.
- Availability Zone (AZ)
- One or more discrete data centers with redundant power, networking and connectivity inside a Region.
- Distance between AZs
- Within 100 km (60 miles) of each other in the same Region.
- Global footprint
- 124 AZs in 39 Regions at the time of writing. This count grows over time.
Real products at each layer
Commercial offerings make the stages concrete. Notice that the far edge alone has two flavours: sites inside a carrier's network, and sites in metro data centers near large populations.
| Layer | Product or standard | What it is |
|---|---|---|
| Far edge, telco | AWS Wavelength | AWS compute and storage deployed at the edge of carriers' 5G networks, managed from the parent Region. |
| Far edge, metro | AWS Local Zones | AWS resources placed close to large population and industry centres for low-latency applications such as real-time gaming, live streaming and AR/VR. |
| Far edge, metro | Cloudflare Workers | Code runs in lightweight isolates across 348 cities; about 95% of the connected population is within 50 ms of a Cloudflare data center. |
| On-premises edge or fog | Azure Stack Edge | 1U rack appliances with 1 to 2 NVIDIA T4 GPUs, used in Azure private multi-access edge compute (MEC, the architecture standardised by ETSI). |
| Cloud | AWS Regions and AZs | Full data centers, 124 AZs worldwide, for pooled compute and long-term storage. |
Recall
List the seven stages of the E2C chain in order from the user outward, and name the sublayers of the cloud.
Quick check
On the lecture's seven-stage chain, which example belongs to the far edge?
Names alone do not tell you where to run anything. What makes the continuum useful is that every stage offers a different bargain, and the bargain changes steadily as you move outward. Put a real part at each layer and the bargain becomes concrete. A Raspberry Pi Pico can decide "button pressed, turn on the light" instantly, and not much more; an AWS Region pools entire data centers. The steps between them fill in the gradient.
| Location | Proximity | Resources | Typical role | Example hardware | Verified spec |
|---|---|---|---|---|---|
| Mist | Closest | Very limited | Immediate, local decisions | Raspberry Pi Pico (RP2040) | 133 MHz, 264 kB SRAM, 2 MB flash |
| Near edge | Very close | Limited to moderate | Local processing | Raspberry Pi 4 Model B | 4 cores at 1.8 GHz, 1 to 8 GB RAM |
| Far edge | Close | Moderate to high | Low-latency applications | Jetson Orin Nano Super, or a Wavelength Zone instance | 67 TOPS, 8 GB RAM, 7 to 25 W |
| Fog or regional | Intermediate | High | Aggregation and coordination | Metro micro data center, or an on-premises Azure Stack Edge appliance at a large site | 1U with 1 to 2 NVIDIA T4 GPUs |
| Cloud | Distant | Very high | Large-scale computation and storage | AWS Region | 124 AZs, each one or more data centers |
Two gradients running in opposite directions
Read the table top to bottom and two things change together. Proximity falls and resources rise. Each step outward serves more users over a wider area with bigger hardware, but sits further away. Bonomi and colleagues put it in one line: the higher the tier, the wider the geographical coverage and the longer the time scale.
The role column follows from the resources column. An immediate decision needs no network at all, so it can live on a tiny chip. Aggregation needs a view of many sources, so it lives where many streams meet. Large-scale computation needs pooled capacity, so it lives in a data center. In Bonomi's smart grid example, the first tier runs control loops that react in milliseconds to sub-seconds, higher tiers run real-time analytics over seconds to minutes and transactional analytics over days, and the cloud keeps the data that must last.
Why the latency side of the trade is physics
The cost of moving outward is latency, and a head-tracked virtual reality headset shows how hard that cost can bite. Satyanarayanan reports that the display must respond to head movement in under 16 ms to feel stable. Citing Ang Li and colleagues, he gives an average round-trip time of 74 ms from 260 vantage points to their best Amazon EC2 data center, before the wireless first hop is even added. A VR application that renders in a distant cloud region therefore cannot meet its target, whatever the server does. The same application rendering on a cloudlet or far edge site one or two hops away can.
The reason comes straight from the previous part. The one-way latency to a layer can never be lower than the Latency lower bound, the distance divided by the speed of light. On top of that, every router on the way adds Queueing delay and processing. The Edge is one or a few hops away, the Fog is a metro away, and the Cloud may be a continent away. So latency, like resources, grows as you move outward: small at the edge, medium in the fog, large in the cloud.
Worked example
Propagation delay to each layer, in fibre
Pick the speed
Light in optical fibre travels at about 200,000 km/s, roughly two thirds of its speed in vacuum. The glossary's lower bound uses 300,000 km/s for vacuum; this example uses fibre because real links are fibre.Apply the formula
One-way propagation delay Edge and fog
An edge node 1 km away gives about 0.005 ms. A fog node in the same metro, 50 km away, gives 0.25 ms.Cloud
Assume a Gulf cloud region about 1,000 km of fibre away: 5 ms. Assume a US east coast region about 13,000 km of cable away (the straight-line distance is about 10,900 km, and cables do not follow straight lines): 65 ms one way, about 130 ms there and back.Result
Propagation alone spans more than four orders of magnitude across the continuum. Queueing and processing at every hop come on top, and there are more hops to the cloud, so the real gap is usually even wider.
How low is low enough? Human perception gives useful targets. Satyanarayanan cites that people take about 370 to 620 ms to recognise a face and 300 to 450 ms to recognise speech, but only about 4 ms to tell that a sound is a human voice. He concludes that an end-to-end latency of a few tens of milliseconds is a safe but achievable goal. That is what a far edge footprint buys: Cloudflare reports that 95% of the connected population is within 50 ms of one of its data centers, and most are within 20 ms.
One nuance matters at PhD level. "Close" is about the network, not just the map. Satyanarayanan defines logical proximity by low latency, low jitter and high bandwidth, and warns that the question "how close is physically close enough?" has no abstract answer. A fog node behind a congested link can respond more slowly than a well-connected cloud availability zone, so the small, medium and large ordering is typical, not guaranteed.
Recall
As you move from mist to cloud, what happens to proximity, resources, latency and the typical role?
Recall
Why can a head-tracked VR application not rely on a distant cloud region?
Quick check
Which ordering lists layers from the fewest resources to the most?
Quick check
A factory robot arm must stop within milliseconds when a person steps into its path. Where should the stop decision run?
You now have both halves of the trade: processing power and storage increase from edge to fog to cloud, in the same direction as latency. Every step toward the cloud buys capacity and costs latency. That turns placement into a real decision, and the decision has three inputs: the latency requirement of the task, its processing and storage needs, and cost.
Picture a neighbourhood of smart doorbell cameras. Each camera, at the Edge, compares consecutive frames to detect motion, which is cheap arithmetic, and sends nothing while nothing moves. A fog node in the neighbourhood combines motion events from 200 cameras and door sensors to spot a pattern across streets, which is sensor data processing. The Cloud trains and hosts the large recognition model and keeps months of clips. One application, three layers, each doing the part it is equipped for.
The pattern generalises into a rule of thumb: put each task on the layer closest to the client that still has enough resources for it, and push data outward only when you need more compute, more storage or a wider view. Bonomi and colleagues describe the fog tier doing exactly this, filtering data to be consumed locally and sending the rest to higher tiers. Filtering early also saves bandwidth, since raw video is expensive to ship; Satyanarayanan lists scalability through edge analytics as a key benefit of cloudlets. Splitting one Service pipeline across layers like this is what the next part calls a Mapping.
| Task | Layer | Why here |
|---|---|---|
| Detect motion in each frame | Edge (on the camera) | Must react instantly, needs little compute, avoids sending idle video. |
| Correlate events from many homes | Fog (neighbourhood node) | Needs a view of many sources and moderate compute, still close enough for timely alerts. |
| Train and host the recognition model, archive clips | Cloud | Needs pooled compute and very large storage; latency is not critical. |
Treat the typical roles (motion at the edge, sensor processing in the fog, AI in the cloud) as tendencies, not rules. Training and very large models belong in the cloud, but inference increasingly runs at the edge: a Jetson Orin Nano Super delivers 67 TOPS within 7 to 25 W. Edge platforms are not unlimited either. A Cloudflare Worker gets 128 MB of memory per isolate and, on the free plan, 10 ms of CPU time per request.
Cost is the third axis. AWS states that Local Zone resources are priced differently from the same resources in their parent Region. Red Hat describes the move toward the edge as an increase in the number of distributed locations but a decrease in each location's size, which means many small sites to buy, power and manage.
- T1Image resizePT 8 ms
- T2Object detectionPT 40 ms
- T3Draw boxesPT 8 ms
Formula
RT = Σ one-way hops + Σ PT
RT = (5 + 15 + 15 + 5) + (8 + 40 + 8) = 96 ms
Client→Edge 5 · Edge→Fog 15 · Fog→Edge 15 · Edge→Client 5
Each hop costs the difference between the one-way latencies of the two layers: Device 0, Edge 5, Fog 20, Cloud 70 ms from the client.
Recall
What three factors decide where a task is placed, and what is the placement rule?
Quick check
The lecture places 'running AI models' in the cloud. What is the strongest reason for that placement?
Recap
If you remember nothing else
- The server side is a continuum: device, mist, near edge, far edge, fog, regional cloud, cloud.
- Three coarse layers: cloud splits into regions and availability zones, fog into metro nodes and micro data centers, edge into mist, near and far.
- Proximity and resources pull in opposite directions: the closer the layer, the lower the latency and the smaller the hardware.
- Latency to the cloud is bounded by physics. Only moving work closer removes propagation delay.
- Place each task on the closest layer that has enough resources, and forward only what needs more.
- Real anchors: AWS Wavelength and Local Zones, Cloudflare Workers and Azure Stack Edge (ETSI MEC) at the edge; AWS Regions and availability zones in the cloud.
- Near and far edge are relative terms. Check the viewpoint.
Sources
- NIST SP 500-325: Fog Computing Conceptual Model (Iorga et al., March 2018)DocsNISTFog nodes, mist, no consensus on terms(opens in a new tab)
- ETSI GS MEC 003 V4.1.1: Multi-access Edge Computing (MEC); Framework and Reference ArchitectureDocsETSIMEC applications on virtualised edge infrastructure(opens in a new tab)
- Fog Computing and Its Role in the Internet of Things (Bonomi, Milito, Zhu, Addepalli, MCC 2012)PaperACMOrigin of fog, tiers and time scales(opens in a new tab)
- The Emergence of Edge Computing (Satyanarayanan, IEEE Computer 50(1), 2017)PaperIEEE Computer SocietyCDN history, 74 ms RTT, 16 ms VR, perception timings(opens in a new tab)
- Computer Networking: A Top-Down Approach (Kurose and Ross)BookPearsonPropagation speed in physical media(opens in a new tab)
- AWS Global Infrastructure: Regions and Availability ZonesDocsAmazon Web ServicesAZ definition, 100 km, 124 AZs in 39 Regions(opens in a new tab)
- Regions and Zones (Amazon EC2 User Guide)DocsAmazon Web Services(opens in a new tab)
- What is AWS Wavelength?DocsAmazon Web Services(opens in a new tab)
- What is AWS Local Zones?DocsAmazon Web ServicesLatency use cases and different pricing(opens in a new tab)
- Azure private multi-access edge compute (MEC)DocsMicrosoftAzure Stack Edge hardware(opens in a new tab)
- How Workers worksDocsCloudflareIsolates(opens in a new tab)
- Workers limitsDocsCloudflare128 MB memory, 10 ms CPU on free plan(opens in a new tab)
- Cloudflare global networkDocsCloudflare348 cities, 50 ms and 20 ms reach(opens in a new tab)
- Netflix Open ConnectDocsNetflixAppliances in ISP networks and at IXPs(opens in a new tab)
- Raspberry Pi 4 Model B specificationsDocsRaspberry Pi Ltd(opens in a new tab)
- Raspberry Pi Pico series documentationDocsRaspberry Pi Ltd(opens in a new tab)
- Jetson Orin Nano Super Developer KitDocsNVIDIA(opens in a new tab)
- What is edge computing and what makes it so different? (Froehlich, 2023)ArticleRed Hat DeveloperNear and far edge naming, more but smaller sites(opens in a new tab)
Part 09: Mapping tasks and response time
Four ways to place an object-detection pipeline on the continuum, how response time is computed, and why faster servers are not enough.
6 concepts, slides 61-71
Why this part matters
Every edge versus cloud argument in this course, and every placement choice in your research project, reduces to one question: which layer runs which task, and what does that do to the time the user waits?
The last two parts gave you the layers of the continuum and the four delays that make up latency. This part puts them together. You get a small vocabulary (mapping, processing time, response time), one formula you will reuse in exams and papers, and four reference designs that show the same pipeline paying very different prices depending on where it runs.
By the end you can
- Treat a mapping as a design decision that places each pipeline task on device, edge, fog or cloud.
- Compute response time as L1 + PT + L2 and read off how much of it is computation.
- Argue with Amdahl's reasoning why a faster server barely helps a distant client, and what does help.
- Weigh the four mappings against each other on latency, privacy, reach, scalability and transfer cost.
- Justify why most real systems pair edge with cloud and treat fog as an optional tier.
Picture a phone camera taking a street photo. Three things must happen before the user sees a useful result: the photo is shrunk to the size the model expects (T1, image resize), a model finds the cars and people in it (T2, object detection), and a box is drawn around each one (T3, drawing the bounding box).
Those three ordered steps form a service pipeline. The pipeline says what must happen and in what order. It says nothing about where. The edge-to-cloud continuum offers device, edge, fog and cloud, and each task can land on any of them. Choosing a Layer for every task is called a Mapping, and the rest of this part compares four such choices for the same three tasks.
The baseline: everything on one cloud server
The simplest mapping puts all three tasks on one Server in a distant Cloud data center. The Client only sends the photo and receives the finished image. Call this Mapping 1; it is the yardstick the other three are measured against.
Sends the photo, shows the result
T1 resize, then T2 detect, then T3 draw
Receives the boxed image
Measuring what the user feels
To compare mappings you need one number that captures the user's experience. Start a stopwatch when the photo leaves the phone and stop it when the boxed image arrives back. That total is the Response time (RT), and it splits into three stretches: the request travels to the server (Latency 1), the server runs T1, T2 and T3 (the Processing time (PT)), and the result travels back (Latency 2).
Neither latency term is a single number from nature. From Part 7 you know each Latency is the sum of Propagation delay, Transmission delay, Queueing delay and Processing delay along every link and router on the path (Kurose and Ross). L1 plus L2 is roughly the round-trip time you would measure with ping, and PT is extra time inserted in the middle.
The examples here treat L1 and L2 as equal only for simplicity. In practice they can differ: the request is often a small upload while the response is a large annotated image, and transmission delay grows with the number of bits pushed onto a link. Every mapping below is just a different way of making these terms bigger or smaller, which is why the formula is the lens for the whole part.
- T1Image resizePT 8 ms
- T2Object detectionPT 40 ms
- T3Draw boxesPT 8 ms
Formula
RT = Σ one-way hops + Σ PT
RT = (5 + 15 + 15 + 5) + (8 + 40 + 8) = 96 ms
Client→Edge 5 · Edge→Fog 15 · Fog→Edge 15 · Edge→Client 5
Each hop costs the difference between the one-way latencies of the two layers: Device 0, Edge 5, Fog 20, Cloud 70 ms from the client.
Recall
Write the response time formula for Mapping 1 and say which terms the client cannot shorten by buying faster server hardware.
The formula becomes a design argument once you put numbers in it. A client sits in New York and the server sits in a cloud data center in California. The request takes 70 ms to cross the country, the server works for 30 ms, and the reply takes another 70 ms.
Worked example
New York client, California cloud
Add the three terms
70 + 30 + 70 = 170 ms. This is the response time the user feels.Computation share
30 / 170 ≈ 17.6%. The other 140 / 170 ≈ 82% is spent on the two network trips.Double the server speed
PT halves from 30 ms to 15 ms, so RT becomes 70 + 15 + 70 = 155 ms.Find the ceiling
Even an infinitely fast server leaves 70 + 0 + 70 = 140 ms, a best-case speedup of only 170 / 140 ≈ 1.21x.Result
About 9% faster (170 / 155 ≈ 1.097x) for twice the compute.
Amdahl's argument, moved onto the network
The disappointing result is not a quirk of these numbers. In 1967 Gene Amdahl pointed out that the part of a job you do not speed up places an upper limit on the overall gain, so effort on one part "is wasted unless it is accompanied by achievements" of similar size in the rest (Amdahl, 1967). Amdahl wrote no equation; the usual textbook form is below, where f is the fraction of time you improve and k is how much faster that part becomes.
With f = 30/170 and k = 2, the formula gives 1 / (0.8235 + 0.0882) ≈ 1.097, the same answer as the worked example. As k grows without bound the speedup approaches 1 / (1 - f), which is the 1.21x ceiling.
In Mapping 1 the part you cannot improve is the network. Physics bounds L1 and L2, and Satyanarayanan notes that "the speed of light is an obvious physical limit on latency". The only large lever left is shortening the distance, which is exactly what Edge computing does. He concludes that relying on a distant data center is not advisable for applications that need end-to-end delays tightly controlled below a few tens of milliseconds (Satyanarayanan, 2017).
Quick check
A client sees 40 ms, 40 ms and 40 ms for L1, PT and L2. The server becomes twice as fast. What is the new response time?
So is Mapping 1 a bad design?
No. The same centralization that makes it slow for far clients is what makes it attractive. With one model in one Cloud deployment, every user gets the same model version, and an update is a single deploy. The client stays thin, since it only captures and displays. Satyanarayanan describes the benefit: "centralization exploits economies of scale to lower the marginal cost of system administration and operations" (Satyanarayanan, 2017). When demand grows, you rent more machines in the same region.
Whether distance is a problem depends on the service. For photo tagging, a few hundred milliseconds are fine and Mapping 1 is often the right call. For augmented reality or a multiplayer game, where a frame must react within milliseconds, every request paying the full Latency both ways is not acceptable.
| Strengths | Weaknesses |
|---|---|
| Every user gets the same model version | Far clients pay a large L1 and L2 on every request |
| The client only sends and receives, so it stays simple | Real-time AR and gaming miss their deadlines |
| Scaling means adding cloud machines, not touching clients | Adding machines does not shorten the path |
Quick check
A multiplayer game studio serves players worldwide from one cloud region. Which Mapping 1 weakness hurts it most?
Recall
In the New York to California example, what is the lowest RT possible if the server became infinitely fast, and what is the best possible speedup?
If the network dominates, the most radical fix is to delete it. Mapping 2 runs the whole photo pipeline on the phone. Nothing is uploaded, so there is no L1 and no L2, and the response time collapses to the processing time alone.
That sounds ideal until you look at what PT now means. The Client Device has a small battery-powered chip, not a data center accelerator. Resize (T1) and drawing (T3) are cheap, but object detection (T2) is heavy. On weak hardware it takes far longer, and the whole time the processor draws power from the battery. In the lecture's layer model this is the Device stage at the very start of the continuum, before Mist, with no edge, fog or cloud involved, and it is the case with no Offloading at all: the weakest component keeps all the work.
The simulator makes the trade visible. Its all-device preset shows 0 ms of network but 440 ms of processing, while the all-cloud preset shows 159 ms in total. Removing two terms from a sum does not help if the remaining term grows past what you removed.
What the user gains is not speed but independence. With no dependency on the network, the service keeps working with no signal. The raw data never leaves the device, which reduces privacy concerns.
Where it shows up: the smartwatch
A smartwatch that notices you have started running is Mapping 2 in a product you can buy. The motion sensors stream data constantly, and the watch must decide on the spot. Apple's documentation, for example, says that for walking, running, swimming and other workouts "your Apple Watch senses when you're moving and alerts you to start the Workout app" (Apple Support). The page does not say where the detection runs, so treat it as an example of the feature, not as proof of an on-watch model. The general point stands: running the classifier on the Device keeps raw motion data private and works without a phone nearby, but it is bounded by battery life and a small processor.
Toolkits exist precisely for this. Google's LiteRT (formerly TensorFlow Lite) runs trained models on phones and embedded boards and advertises "low latency and high privacy on billions of devices" (Google for Developers). The typical recipe is to train a large model in the cloud, then compress it until it fits the device, which attacks PT directly since it is the only term left.
- Choose Mapping 2 when connectivity cannot be assumed: a run in the mountains, a flight, a basement.
- Choose it when raw data is sensitive and should not leave the device, such as health signals.
- Choose it only when the model is small enough that PT and battery drain stay acceptable.
Quick check
What cost does a smartwatch pay under Mapping 2 that it avoids under Mapping 1?
Recall
Name two impacts of Mapping 2 on the device itself, and one benefit.
Mappings 1 and 2 sit at the two extremes: a strong server far away, or a weak processor with no network. Mapping 3 looks for the middle. It keeps a real server but moves it close, into the same building or a nearby telecom site, which runs all three tasks and sends the result back.
Both halves of the formula improve at once. Because the Server is close, L1 and L2 are small. Because it is a real server rather than a phone, PT is moderate. The simulator's all-edge preset shows 10 ms of network and 136 ms of processing, so 146 ms in total. Satyanarayanan's cloudlet idea is this design: a small data center near users that "can run arbitrary code just as in cloud computing", where "proximity of cloudlets to end users is crucial" (Satyanarayanan, 2017). The work is still offloaded from the device, but to a nearby point on the continuum instead of the far end.
Proximity has two prices, and both follow from the same fact that made the latency low. First, limited geographic reach: an Edge site is close to some users precisely because it is far from everyone else, who either get no service or a long trip. Second, limited scalability: an edge site is small, a few servers in a closet or a cabinet, so it cannot absorb a large workload the way a cloud region can. Covering a country means many small sites, each managed separately, which gives back the centralization that made Mapping 1 easy to operate.
Where it shows up: the retail store
Cameras and shelf sensors in a store feed a server in the back office. It counts footfall, notices which products customers pick up, and tracks inventory as it changes. The decisions are local and time-sensitive: restock a shelf now, open a second checkout because the queue is growing. Sending every video frame to a distant cloud would add Latency and upload a lot of footage, so the edge server answers in the store.
Now test the two weaknesses against it. Could that back-office box serve a branch in another city? No, its users are the cameras and staff in this store. That is limited reach. Could it absorb the peak traffic of a hundred stores on a sales weekend? No, it is sized for one store. That is limited scalability. The store is a good fit precisely because neither weakness matters for its workload.
Recall
Why can an edge server in one store not replace a cloud service for a national chain? And where would chain-wide sales trends be computed?
So far each mapping put the whole pipeline in one place. But the three tasks are not equally demanding, so why should they share a layer? Mapping 4 splits the pipeline: the light tasks stay near the user and the heavy task runs one tier up. T1 (resize) and T3 (draw) run in the Edge layer, and T2 (object detection) runs in the Fog layer, which has more compute.
Think of it as one logical service deployed in two physical places. The Service pipeline is now distributed, so the formula has to grow. Count the hops: client to edge, edge to fog, fog to edge, edge to client. That is four network crossings instead of two, and each task contributes its own processing term.
In the simulator's Mapping 4 preset, those crossings add up to 40 ms and processing to 56 ms, so 96 ms, the lowest of the four presets. The fog does T2 much faster than the edge, and that saving outweighs the two extra hops. NIST describes fog nodes as sitting "between smart end-devices and centralized (cloud) services" and says fog "minimizes the request-response time" (NIST SP 500-325).
What sharing a fog node buys
Speed is only part of the case. Picture ten intersections in a smart city. Each traffic camera has a small edge box that resizes frames. All ten send them to one nearby fog server, which runs a large vehicle detection model for all of them and returns detections to each box.
- Sharing: one detection model serves users of many edge nodes, instead of a copy squeezed onto each small box.
- Capacity: the fog node has more processing power and storage than any single edge site.
- Learning: the fog collects data from all the edge nodes, which keeps the model up to date, for example by retraining on new traffic patterns.
This matches how NIST frames fog. Fog nodes support "a common data management and communication system" and are organized in clusters by their "latency-distance to the smart end-devices" (NIST SP 500-325). Aggregation and coordination are the fog's job, which partly restores the centralization that Mapping 3 gave up, without going all the way to the cloud.
What splitting costs
- The fog layer introduces latency relative to Mapping 3: every boundary between layers adds its own latency, with all four delay components. The fog is still much closer than the Cloud.
- Every byte that crosses a boundary may be billed. Cloud providers already charge this way: AWS states that "there is a charge for data transfer across Regions" and that "data transfer from AWS to the internet is charged" (AWS Architecture Blog). Those quotes are about regions and the internet, not edge to fog links, but the principle carries over.
- So split where a task shrinks the data. Resizing before sending means fewer bytes cross the edge to fog link, which is exactly why T1 stays at the edge.
Quick check
Compared with Mapping 3, what new cost does Mapping 4 introduce?
Recall
List three benefits of putting T2 in the fog, and the price paid.
The four mappings are clean teaching cases. Practice is messier in one specific way: a dedicated fog tier is not always deployed, or identified, as a separate layer. In most cases, architects use two layers, Edge and Cloud.
NIST agrees that fog is optional: it "is not perceived as a mandatory layer", and "different usecase scenarios might have different architectures" (NIST SP 500-325). The claim that "most" architects choose edge plus cloud is the lecturer's observation from practice. No industry-wide statistic backs the exact share.
The resulting design has the shape of Mapping 4 with the cloud in place of the Fog: T1 and T3 at the edge, T2 in the cloud. It borrows the strength of each earlier mapping for the task that needs it. Go back to the retail store. The back-office edge server makes the live decisions, as in Mapping 3, and the cloud trains new models and computes trends across the whole chain, using the centralization that made Mapping 1 attractive. Each task gets the Layer that suits it.
| Mapping | Where T1, T2, T3 run | Strengths | Weaknesses | Example |
|---|---|---|---|---|
| 1 | All in the cloud | Consistency, simple client, easy scaling | Large L1 and L2, unfit for AR or gaming | Photo tagging in a distant data center |
| 2 | All on the device | No network, privacy, works offline | Battery, limited hardware, long PT | Smartwatch activity recognition |
| 3 | All at the edge | Low latency, data stays local | Limited reach, limited scalability | Retail store analytics |
| 4 | T1 edge, T2 fog, T3 edge | Shared model, more compute, model updates | Extra inter-layer hops and transfer cost | Smart-city traffic cameras |
Quick check
According to the lecture, which two layers do most application architects deploy?
Recap
If you remember nothing else
- A mapping assigns each pipeline task to a continuum layer. It is a design decision, not a given.
- RT = L1 + PT + L2. In the New York to California example, only 17.6% of 170 ms is computation.
- Doubling server speed gives 155 ms, and even an infinitely fast server leaves 140 ms. Shorten the distance instead.
- Mapping 2 removes the network but pays in battery, hardware limits and long PT.
- Mapping 3 cuts latency but serves only a local area with limited capacity.
- Mapping 4 shares a fog model across edge nodes but adds hops, latency and data transfer.
- Fog is optional. Most architectures combine edge and cloud.
Sources
- Validity of the Single Processor Approach to Achieving Large Scale Computing CapabilitiesPaperAFIPS Conference Proceedings, Vol. 30, 1967Gene Amdahl's original argument that the unimproved part of a job caps the overall speedup.(opens in a new tab)
- NIST SP 500-325: Fog Computing Conceptual ModelDocsNational Institute of Standards and TechnologyFog definition, placement between devices and cloud, reduced request-response time, fog as an optional layer.(opens in a new tab)
- The Emergence of Edge ComputingPaperIEEE Computer, M. Satyanarayanan, 2017Cloud economies of scale, the speed-of-light limit on latency, and cloudlets near users.(opens in a new tab)
- Computer Networking: A Top-Down Approach, 8th editionBookPearson, Kurose and RossThe four delay components that make up L1 and L2.(opens in a new tab)
- Overview of Data Transfer Costs for Common ArchitecturesDocsAWS Architecture BlogData transfer across regions and to the internet is billed, used as an analogy for inter-layer transfer.(opens in a new tab)
- Change settings in Workout on Apple WatchDocsApple SupportAutomatic workout detection. The page does not state where inference runs.(opens in a new tab)
- LiteRT: on-device AI frameworkDocsGoogle for DevelopersFramework for running models on devices with low latency and high privacy.(opens in a new tab)
- Raspberry Pi 4 Model B specificationsDocsRaspberry PiCortex-A72 system on a chip running Raspberry Pi OS: a single-board computer.(opens in a new tab)
- Raspberry Pi Pico series documentationDocsRaspberry PiA microcontroller board that does not run Linux, for contrast with the pictured board.(opens in a new tab)
- Ping time between New York and San FranciscoArticleWonderNetworkMeasured round-trip latency, about 63 ms on 2026-09-15. Measurement data, used only as a reality check.(opens in a new tab)
Part 10: Latency budget, exercises and summary
Designing against a response-time threshold, three design exercises (smart glasses, AR, VR), and the lecture in one page.
4 concepts, slides 72-77
Why this part matters
Parts 7 to 9 gave you the physics of latency, the layers of the continuum, and ways to map a pipeline onto them. One ingredient was still missing: a deadline. Without a deadline, “edge or cloud?” is a matter of taste. With one, it becomes an inequality you can check with a calculator.
This part names that deadline τ, shows where defensible values of τ come from, and then turns it into a design method you can reuse on any system. You will practise the method on three workloads that sit at the heart of edge computing research: smart glasses, warehouse augmented reality and virtual reality (Chen et al. 2017, RFC 9699). The part ends by folding the whole lecture into one loop: split the service into tasks, give each a deadline, place it, and check response time against the deadline. For the exam you will compute response times and pick feasible mappings. For the research project, any edge system you propose should state τ per task, justify it from a published source, and report measured response time at a percentile.
By the end you can
- Explain the latency budget [0, τ] and decide whether a mapping is feasible by checking RT ≤ τ, counting latency out, processing time and latency back.
- Justify τ from published human and physical limits (XR, conversation, web, wearable assistance) and check it at a percentile, not the mean.
- Apply the four-step method (tasks, τ per task, placement, RT and trade-offs) and defend a smart-glasses placement with battery, privacy and reach arguments.
- Explain why one AR or VR app needs several layers, and why motion-to-photon forces tracking and reprojection onto the device.
- State the lecture as one design loop, where τ decides the edge and cloud split, and connect it to virtualization, containers and cloud service models.
Picture a video call. When you speak, your voice should reach the other person quickly enough that the two of you do not talk over each other. The ITU-T telephony standard G.114 puts a number on “quickly enough”: if one-way delay stays below 150 ms, most applications experience essentially transparent interactivity, and 400 ms should not be exceeded for general network planning. So for one direction of that call, you have about 150 ms to spend, and every hop, queue and processing step you place in the path takes some of it.
That allowance is the Latency budget. Its upper limit is written τ (tau): the maximum acceptable end-to-end Response time (RT) of a service. The budget is the interval , and a design is inside it when . Notice who sets τ. It is not the network, and it is not the hardware you happen to own. It comes from the user and the application: human perception for a chat or a game, physics for a headset that must keep up with your head. The network and the servers only decide how much of the budget you spend.
What counts against the budget
It is tempting to compare τ with the Latency to a layer: the client reaches the edge with small latency, the fog with medium latency and the cloud with large latency. But latency is only part of the bill. Recall from part 9 that response time is the trip out, plus the Processing time (PT) at the server, plus the trip back:
A per-layer latency figure is one-way, so never compare a single arrow directly with τ. When a pipeline splits its tasks across several layers, every hop between them joins the sum as well, together with every task’s processing time. The gauge below makes the consequence visible: the same request, three placements, one deadline.
Worked example
Is the fog mapping within τ = 100 ms? (illustrative numbers)
Write the budget
The service must answer within τ = 100 ms.Sum the fog mapping
One-way latency to the fog is 20 ms and processing takes 70 ms. So RT = 20 + 70 + 20 = 110 ms. That is over budget by 10 ms.Try the edge
One-way latency to the edge is 5 ms. If the edge server can still process in 70 ms, RT = 5 + 70 + 5 = 80 ms, which fits. In practice edge servers are smaller, so check that PT does not grow enough to undo the saving.Result
Fog fails (110 > 100). Edge passes (80 ≤ 100). Always write the sum and the inequality.
The last step of that example hides the real tension of this whole part. Moving closer shrinks the latency terms, but closer sites are smaller, so the processing term can grow. Neither “always edge” nor “always cloud” is a rule; the inequality decides. The simulator lets you feel that tension. Its three tasks are an image resize, object detection and drawing boxes, and the presets map them to different layers. Set τ to 100 ms and find the one preset that stays in budget.
- T1Image resizePT 8 ms
- T2Object detectionPT 40 ms
- T3Draw boxesPT 8 ms
Formula
RT = Σ one-way hops + Σ PT
RT = (5 + 15 + 15 + 5) + (8 + 40 + 8) = 96 ms
Client→Edge 5 · Edge→Fog 15 · Fog→Edge 15 · Edge→Client 5
Each hop costs the difference between the one-way latencies of the two layers: Device 0, Edge 5, Fog 20, Cloud 70 ms from the client.
Recall
At τ = 100 ms, which simulator preset passes, and why do “all edge” and “all cloud” fail?
Where real values of τ come from
Because τ belongs to the user, you should never invent it. Studies of human perception and device physics give defensible numbers, and they span four orders of magnitude: a headset that must track your head has a couple of tens of milliseconds, a conversation has about a hundred and fifty, a recognition assistant has most of a second, and a user waiting for a report keeps attention for about ten seconds. The table collects the values this course relies on, roughly from the strictest to the most relaxed. You will use every row in the design exercises that follow.
| Interaction | τ | Source |
|---|---|---|
| VR / XR motion-to-photon | ≤ 20 ms (7 to 15 ms preferred) | RFC 9699 |
| VR motion-to-photon consensus | < 15 to 20 ms | Elbamby et al. |
| Response feels instantaneous | 0.1 s | Nielsen |
| One-way conversational delay | < 150 ms preferred, 400 ms ceiling | ITU-T G.114 |
| Web interaction rated good (INP) | ≤ 200 ms at p75 | web.dev |
| Wearable face recognition | 370 to 1000 ms | Chen et al. 2017 |
| Step-by-step instruction (Lego, Draw, Sandwich) | 600 ms tight, 2.7 s loose | Chen et al. 2017 |
| Flow of thought stays unbroken | 1 s | Nielsen |
| User keeps attention on the task | 10 s | Nielsen |
One more rule turns these numbers into a real test. A budget is a promise about the experience of users, and users remember the slow requests, not the average one. So published budgets are stated at a percentile.
Quick check
A service has τ = 120 ms. One-way latency to the fog is 25 ms and processing takes 75 ms. Is the fog mapping within budget?
Quick check
Which published figure is the standard reference for acceptable one-way conversational delay?
Knowing the inequality is not the same as designing a system. Real services are not one request and one server; they are pipelines of tasks with different needs. Design questions of this kind have no single right answer, but they do have a right method, and it is the same four steps every time: list the tasks, set τ per task, place each task on a layer, then check Response time (RT) and the trade-offs.
Smart glasses are a good first case because every constraint pulls hard. The device is small, runs on a tiny battery and sits on your face, so it cannot get hot. Yet the job, recognizing the faces and objects in front of you, is heavy vision computing. Before reading on, take a minute and list the tasks a face-recognition assistant on glasses would need. That effort is what makes the model answer stick.
Worked example
Architecting smart glasses on the E2C continuum
1. List the tasks
The Service pipeline is: capture a camera frame, compress or downscale it, detect and recognize nearby faces or objects, generate guidance text, render the overlay on the display, store history and photos, and retrain the recognition models.2. Set τ per task
The overlay must follow head motion, so it lives in a display loop of tens of milliseconds (RFC 9699 gives 20 ms for XR). Chen et al. derive 370 to 1000 ms for face recognition from how long people take to recognize a familiar face, and 600 ms tight to 2.7 s loose for step-by-step assembly guidance on Google Glass from a user study. Storage and training have no interactive deadline.3. Place each task
This is a Mapping. Capture, compression and overlay rendering stay on the glasses, the Mist layer. Detection, recognition and guidance use Offloading to a nearby Edge server (a cloudlet). Storage, analytics and model training go to the Cloud.4. Check RT and trade-offs
In Chen et al.’s Figure 17, face recognition with Glass on a WiFi cloudlet reached 435 ms at the 90th percentile. That meets the loose bound (1000 ms) but misses the tight bound (370 ms), and even the phone on the cloudlet (410 ms) misses it. With a phone client, the cloud gave 615 ms against 410 ms on the cloudlet, and the paper notes that cloud offload almost always adds 100 to 200 ms compared with a cloudlet.Result
Device for capture and display, edge for recognition, cloud for storage and training.
Step 4 is worth a second look, because it shows the method working honestly. The cloudlet design does not magically hit every bound: it meets the loose face-recognition deadline and misses the tight one. That is a real, measured result at the tail, and it is exactly the kind of statement a good design report makes. The cloud alternative misses by more, so the edge is the better placement, not a perfect one.
The trade-offs behind the placement
Latency is not the only reason for this split. Shi et al., summarizing Ha et al.’s cloudlet study for wearable cognitive assistance, report that offloading can cut energy use by 30 to 40%, which matters on a device with a battery the size of a stick of gum. They also argue that processing at the edge protects privacy better than uploading raw data to the cloud, and faces are among the most sensitive data there is.
The price is dependence. The glasses now need a working wireless link and an edge site near wherever the wearer walks. Edge sites are few and expensive, so scalability and reach are limited. The cloud has global reach and elastic capacity, but it misses the tight end of the budget. That is exactly why the design keeps both, and why every placement you propose should name what it gains and what it gives up.
| Task | τ | Layer | Reason |
|---|---|---|---|
| Capture and downscale camera frame | Every frame | Mist (glasses) | Raw video is heavy; shrink it before it touches the radio |
| Render the overlay on the display | ≈ 20 ms display loop | Mist (glasses) | Must follow head motion; no network trip fits |
| Detect and recognize faces or objects | 370 to 1000 ms | Edge (cloudlet) | Too heavy for the glasses, too slow via the cloud |
| Generate guidance text | 600 ms to 2.7 s | Edge (cloudlet) | Needs the recognition result, which already lives there |
| Store history and photos | Seconds or more | Cloud | Durable, cheap, reachable from any device |
| Retrain recognition models | Hours | Cloud | Needs large datasets and GPU pools |
Recall
Without looking: list the tasks of a smart-glasses face-recognition pipeline and place each on the continuum.
Quick check
Moving face recognition from the smart glasses to a nearby edge server mainly trades what?
Picture a warehouse worker holding a tablet. Overlays mark the next shelves to pick from, a ring counts progress, and a green arrow on the floor says the target is 12 m ahead. It looks like one app. For design purposes it is at least six tasks with deadlines that range from milliseconds to hours. That is the central idea of this concept: τ belongs to a task, not to an app, and a single app routinely needs every layer of the continuum at once.
Motion-to-photon: the tightest budget in the course
Both augmented and virtual reality are governed by one deadline, so it pays to understand it before designing either. When you move your head, or the tablet, the image must move with you. The time from the movement to the matching light leaving the display is motion-to-photon (MTP) latency. If it is too long, your eyes and your inner ear disagree. Elbamby et al. explain that high MTP sends conflicting signals to the vestibulo-ocular reflex, which can cause motion sickness. In AR the failure is also visible: a virtual label that lags behind a real shelf is noticed at once, because the real world is the reference.
The research community broadly agrees on the ceiling. Elbamby et al. report a broad consensus that MTP must stay below 15 to 20 ms. RFC 9699 uses 20 ms at most for XR as a whole, and shows how little of it is left for computing once the display has taken its share:
Estimates of the split vary. Elbamby et al. put display delay at about 10 to 15 ms, expected to fall to 5 ms, which leaves 14 ms for computing and communication. Either way, only a few milliseconds remain for any network trip, and a trip to a distant cloud does not fit even once.
The standard escape is reprojection, also called time warp: the device takes the last rendered frame and shifts it to match the newest head pose just before display. Reprojection is cheap, so it can run locally inside the MTP budget, and it means a heavy frame rendered elsewhere can arrive a little late without the image lagging behind your head. This one technique is what lets the heavy work leave the device at all.
Warehouse AR: six tasks, four layers
Worked example
Architecting warehouse AR picking
1. List the tasks
Camera and motion-sensor (IMU) pose tracking; anchoring overlays to shelves and drawing arrows; recognizing shelf labels and items; looking up the pick list and inventory for this warehouse; optimizing routes across all pickers; fleet-wide analytics and demand forecasting.2. Set τ per task
Tracking and anchoring share the 20 ms MTP budget, of which only 7 to 8 ms remain for sensor processing, rendering and the Round-trip time (RTT) to the edge. Recognition can take a few hundred milliseconds: Nielsen’s 1 s keeps the flow of thought, and 0.1 s feels instant. Inventory lookups should finish within a second, routing within seconds, and analytics can take hours.3. Place each task
4. Check RT and trade-offs
RFC 9699 is blunt: running these rendering tasks in the cloud is not feasible, because end-to-end delays must be within a few milliseconds. The same RFC names heat and battery drain as reasons to offload at all. The fog node keeps shared state close and scales to the whole site, while the cloud gives reach across sites.Result
One app, four layers: the deadline of each task picks its layer, not the name of the app.
| Task | τ | Layer | Reason |
|---|---|---|---|
| Camera and IMU pose tracking | ≤ 20 ms motion to photon | Device | Only 7 to 8 ms remain after the display |
| Anchor overlays, render arrows | Within the same 20 ms | Device, heavy parts on in-building edge | Offloading cuts heat and battery drain |
| Recognize shelf labels and items | a few hundred ms | Edge | Heavy vision work, still interactive |
| Pick list and inventory lookup | < 1 s | Fog (warehouse server) | Shared by every picker on site |
| Route optimization across pickers | Seconds | Fog | Needs site-wide state, not per-frame speed |
| Fleet analytics and demand forecasting | Hours | Cloud | Cross-warehouse data, big compute |
Notice the new role of the fog layer. Nothing in the smart-glasses design was shared between users, but a warehouse is a team. The pick list and the route optimizer serve every picker in the building, their deadline is around a second rather than a frame, and keeping them on site avoids sending every lookup to the cloud. Shared site state is the natural job of fog.
Recall
In the warehouse AR app, which task belongs on a fog node, and why?
Quick check
In the warehouse AR app, which task has the tightest latency budget?
Interactive VR: every pixel under the deadline
Virtual reality shares the same budget, but raises the stakes: in AR only the overlay depends on MTP, while in VR every pixel you see does. The same four steps apply, and the same pattern appears, with a GPU-hungry rendering task pulled between a light headset and a distant data center.
Worked example
Architecting interactive VR
1. List the tasks
Head tracking, reprojection, frame rendering, encoding and streaming frames, physics and multiplayer state, the content library, and user profiles.2. Set τ per task
Tracking through display: MTP under 15 to 20 ms. Multiplayer state: tens of milliseconds up to about 150 ms, borrowing G.114’s conversational figure as an analogy for social presence (an analogy, not a VR standard). Content downloads: seconds.3. Place each task
4. Check RT and trade-offs
Elbamby et al. report that online VR computing can take up to 100 ms and the communication delay from the network edge to a server can reach 40 ms. Even ideal 4G loopback latency is 25 ms, already above 20 ms before any rendering. Remote cloud servers suit only low-resolution, non-interactive VR where the whole 360° video can be streamed ahead.Result
Headset protects the frame deadline, edge supplies GPU power, cloud supplies content.
| Task | τ | Layer | Reason |
|---|---|---|---|
| Head tracking | < 15 to 20 ms MTP | Headset | Starts the motion-to-photon clock |
| Reprojection (time warp) | Inside the MTP budget | Headset | Fixes the pose even when a remote frame is late |
| Full-quality frame rendering | Per frame | Edge GPU | Too heavy for a light headset, too far for the cloud |
| Physics and multiplayer state | tens of ms up to ≈ 150 ms | Fog or nearest region | Shared by all players in a session |
| Content library and assets | Seconds | Cloud | Large, rarely changing, global |
| User profiles | Seconds | Cloud | Durable records, no frame deadline |
The trade-offs are sharp. An edge GPU per site is expensive, so each site serves a limited number of users. The alternative, a headset tethered by cable to a local PC, meets the budget but gives up mobility. In Latency budget terms, every design is judged the same way: does the per-frame Response time (RT) fit inside MTP, measured at the tail rather than the average?
Recall
Why can a VR headset not send every frame to a distant cloud for rendering?
Quick check
A VR headset must keep motion-to-photon latency within about 20 ms. Where should head-pose reprojection run?
Put the three designs side by side and one pattern falls out. The lecturer’s reference answers below are a compact version of the tables you built above. If you placed recognition, sensor interaction and rendering at the edge and storage and analytics in the cloud, you have the same architecture.
| Application | Edge role | Cloud role | Driving τ |
|---|---|---|---|
| Smart glasses (AR) | Recognize nearby objects or faces | Storage and complex computation | 370 to 1000 ms per recognition |
| Industrial AR | Real-time interaction with sensors | Large datasets and analytics | ≤ 20 ms MTP, < 1 s lookups |
| VR headsets | Frame rendering for smooth visuals | Processing large datasets | < 15 to 20 ms MTP |
Every app keeps the Cloud, and every app adds the Edge. What decides the Mapping of a task is a single comparison: is its deadline shorter than the cloud round trip plus the processing it needs? Recognizing a face, reacting to a sensor and drawing the next frame all have deadlines the cloud cannot meet. Storing photos, crunching datasets and training models have none, and they want exactly what the cloud is good at: scale, durability and reach.
Latency is the strongest force in that split, but not the only one. Satyanarayanan’s SEC 2017 keynote gives three reasons to use the edge, and all three appeared in the exercises. Latency, in both mean and tail, drives rendering and recognition. Bandwidth, both peak and average, explains why glasses and AR tablets should not stream raw video to a distant data center. Privacy, which he calls an exposure firewall for the IoT, explains keeping faces on a nearby server.
The whole lecture in one picture
That pattern is the lecture’s conclusion in miniature. Before, you architected an application on a single Server layer: pick a machine, deploy the backend, done. Now, because data, IoT devices and machine learning produce and consume so much close to the user, the server is spread across the Edge-to-cloud (E2C) continuum.
That changes the unit of design. You no longer place a server; you place each task of each service Service pipeline. And a placement needs a criterion. The lecture chooses response time, written as a function of Latency and Processing time (PT). Its concrete form generalizes the budget condition from the start of this part to a pipeline that crosses several layers:
Response time is the right criterion because it is what users actually experience, and because it keeps both levers in view. A faster server only shrinks the processing terms; placement is what shrinks the latency terms. Every argument in this part, from the fog example that missed by 10 ms to the headset that cannot afford one 4G round trip, was an argument about which lever was available.
The design loop
- Break the service into a pipeline of tasks.
- Set τ for each task from a published source.
- Choose a Mapping of tasks to Device, Edge, Fog and Cloud.
- Compute RT from latency and processing time, at a percentile.
- Check RT ≤ τ and weigh battery, privacy, cost and reach. If it fails, remap and repeat.
Recall
State the lecture summary in one sentence, including the design criterion.
Recall
Write the latency budget condition and name each term.
Quick check
According to the lecture summary, what is mapped to a layer of the E2C continuum?
Where the course goes next
Placing a task on a layer is still abstract. In practice it means running that task inside a virtual machine or a container on that layer. Satyanarayanan’s keynote notes that cloudlets can run lighter-weight containers such as Docker inside VMs, which is exactly what the virtualization and container lectures build.
Renting those layers means choosing a cloud model. NIST SP 800-145 defines the service models (SaaS, PaaS, IaaS) and the deployment models (private, community, public, hybrid). Those definitions are the subject of the cloud models lecture, and they decide who operates each layer in your mapping.
Recap
If you remember nothing else
- τ is the maximum acceptable end-to-end response time. A mapping is feasible only if RT ≤ τ, where RT counts latency out, processing time and latency back.
- τ comes from people and physics: about 20 ms for XR motion-to-photon, 150 ms one-way for conversation, 200 ms for web interactions, 370 to 1000 ms for face recognition.
- Check budgets at a percentile (p75, p90), not at the mean. Users feel the slow requests.
- Set τ per task, not per app. One AR app mixes 20 ms tracking with analytics that can take hours.
- Tight-deadline tasks go to the device or edge. Shared site state goes to fog. Storage, training and analytics go to the cloud.
- Offloading to the edge saves battery and keeps raw data private, but adds network dependence and needs edge sites where users are.
- Summary: the server now spans the E2C continuum. Each pipeline task is mapped individually, and response time is the guiding criterion.
Sources
- G.114: One-way transmission time (05/2003)DocsITU-TBelow 150 ms one-way most applications have essentially transparent interactivity; 400 ms planning limit.(opens in a new tab)
- RFC 9699: Use Case for an Extended Reality Application on Edge Computing InfrastructureRFCIETF / RFC EditorMTP at most 20 ms, 7 to 15 ms preferred; display 12 to 13 ms leaves 7 to 8 ms; cloud not feasible; heat and battery.(opens in a new tab)
- Towards Low-Latency and Ultra-Reliable Virtual Reality (Elbamby, Perfecto, Bennis, Doppler)PaperarXivMTP consensus below 15 to 20 ms; 4G ideal loopback 25 ms; online VR computing up to 100 ms, edge-to-server 40 ms.(opens in a new tab)
- An Empirical Study of Latency in an Emerging Class of Edge Computing Applications for Wearable Cognitive Assistance (Chen et al.)PaperACM/IEEE SEC 2017Figure 17: p90 latencies against bounds (face 370 to 1000 ms, 435 ms on cloudlet with Glass); cloud adds 100 to 200 ms.(opens in a new tab)
- Edge Computing: Vision and Challenges (Shi, Cao, Zhang, Li, Xu)PaperIEEE Internet of Things Journal 3(5), 2016Summarizes Ha et al.: cloudlet offloading cuts energy use by 30 to 40%; edge processing protects privacy better than uploading raw data.(opens in a new tab)
- Edge Computing (SEC 2017 keynote), M. SatyanarayananArticleACM/IEEE Symposium on Edge ComputingEdge value: latency (mean and tail), bandwidth, privacy; containers within VMs on cloudlets.(opens in a new tab)
- Response Times: The 3 Important LimitsArticleNielsen Norman Group0.1 s feels instant, 1 s keeps flow of thought, 10 s keeps attention.(opens in a new tab)
- Interaction to Next Paint (INP)DocsGoogle web.devGood at or below 200 ms, poor above 500 ms, measured at the 75th percentile.(opens in a new tab)
- NIST SP 800-145: The NIST Definition of Cloud ComputingDocsNISTService models (SaaS, PaaS, IaaS) and deployment models (private, community, public, hybrid).(opens in a new tab)