Mapping tasks and response time

Picture a phone camera taking a street photo. Three things must happen before the user sees a useful result: the photo is shrunk to the size the model expects (T1, image resize), a model finds the cars and people in it (T2, object detection), and a box is drawn around each one (T3, drawing the bounding box).

Those three ordered steps form a service pipeline. The pipeline says what must happen and in what order. It says nothing about where. The edge-to-cloud continuum offers device, edge, fog and cloud, and each task can land on any of them. Choosing a Layer for every task is called a Mapping, and the rest of this part compares four such choices for the same three tasks.

The baseline: everything on one cloud server

The simplest mapping puts all three tasks on one Server in a distant Cloud data center. The Client only sends the photo and receives the finished image. Call this Mapping 1; it is the yardstick the other three are measured against.

Measuring what the user feels

To compare mappings you need one number that captures the user's experience. Start a stopwatch when the photo leaves the phone and stop it when the boxed image arrives back. That total is the Response time (RT), and it splits into three stretches: the request travels to the server (Latency 1), the server runs T1, T2 and T3 (the Processing time (PT)), and the result travels back (Latency 2).

Neither latency term is a single number from nature. From Part 7 you know each Latency is the sum of Propagation delay, Transmission delay, Queueing delay and Processing delay along every link and router on the path (Kurose and Ross). L1 plus L2 is roughly the round-trip time you would measure with ping, and PT is extra time inserted in the middle.

The examples here treat L1 and L2 as equal only for simplicity. In practice they can differ: the request is often a small upload while the response is a large annotated image, and transmission delay grows with the number of bits pushed onto a link. Every mapping below is just a different way of making these terms bigger or smaller, which is why the formula is the lens for the whole part.

The formula becomes a design argument once you put numbers in it. A client sits in New York and the server sits in a cloud data center in California. The request takes 70 ms to cross the country, the server works for 30 ms, and the reply takes another 70 ms.

Amdahl's argument, moved onto the network

The disappointing result is not a quirk of these numbers. In 1967 Gene Amdahl pointed out that the part of a job you do not speed up places an upper limit on the overall gain, so effort on one part "is wasted unless it is accompanied by achievements" of similar size in the rest (Amdahl, 1967). Amdahl wrote no equation; the usual textbook form is below, where f is the fraction of time you improve and k is how much faster that part becomes.

With f = 30/170 and k = 2, the formula gives 1 / (0.8235 + 0.0882) ≈ 1.097, the same answer as the worked example. As k grows without bound the speedup approaches 1 / (1 - f), which is the 1.21x ceiling.

In Mapping 1 the part you cannot improve is the network. Physics bounds L1 and L2, and Satyanarayanan notes that "the speed of light is an obvious physical limit on latency". The only large lever left is shortening the distance, which is exactly what Edge computing does. He concludes that relying on a distant data center is not advisable for applications that need end-to-end delays tightly controlled below a few tens of milliseconds (Satyanarayanan, 2017).

So is Mapping 1 a bad design?

No. The same centralization that makes it slow for far clients is what makes it attractive. With one model in one Cloud deployment, every user gets the same model version, and an update is a single deploy. The client stays thin, since it only captures and displays. Satyanarayanan describes the benefit: "centralization exploits economies of scale to lower the marginal cost of system administration and operations" (Satyanarayanan, 2017). When demand grows, you rent more machines in the same region.

Whether distance is a problem depends on the service. For photo tagging, a few hundred milliseconds are fine and Mapping 1 is often the right call. For augmented reality or a multiplayer game, where a frame must react within milliseconds, every request paying the full Latency both ways is not acceptable.

If the network dominates, the most radical fix is to delete it. Mapping 2 runs the whole photo pipeline on the phone. Nothing is uploaded, so there is no L1 and no L2, and the response time collapses to the processing time alone.

That sounds ideal until you look at what PT now means. The Client Device has a small battery-powered chip, not a data center accelerator. Resize (T1) and drawing (T3) are cheap, but object detection (T2) is heavy. On weak hardware it takes far longer, and the whole time the processor draws power from the battery. In the lecture's layer model this is the Device stage at the very start of the continuum, before Mist, with no edge, fog or cloud involved, and it is the case with no Offloading at all: the weakest component keeps all the work.

The simulator makes the trade visible. Its all-device preset shows 0 ms of network but 440 ms of processing, while the all-cloud preset shows 159 ms in total. Removing two terms from a sum does not help if the remaining term grows past what you removed.

What the user gains is not speed but independence. With no dependency on the network, the service keeps working with no signal. The raw data never leaves the device, which reduces privacy concerns.

Where it shows up: the smartwatch

A smartwatch that notices you have started running is Mapping 2 in a product you can buy. The motion sensors stream data constantly, and the watch must decide on the spot. Apple's documentation, for example, says that for walking, running, swimming and other workouts "your Apple Watch senses when you're moving and alerts you to start the Workout app" (Apple Support). The page does not say where the detection runs, so treat it as an example of the feature, not as proof of an on-watch model. The general point stands: running the classifier on the Device keeps raw motion data private and works without a phone nearby, but it is bounded by battery life and a small processor.

Toolkits exist precisely for this. Google's LiteRT (formerly TensorFlow Lite) runs trained models on phones and embedded boards and advertises "low latency and high privacy on billions of devices" (Google for Developers). The typical recipe is to train a large model in the cloud, then compress it until it fits the device, which attacks PT directly since it is the only term left.

• Choose Mapping 2 when connectivity cannot be assumed: a run in the mountains, a flight, a basement.
• Choose it when raw data is sensitive and should not leave the device, such as health signals.
• Choose it only when the model is small enough that PT and battery drain stay acceptable.

Mappings 1 and 2 sit at the two extremes: a strong server far away, or a weak processor with no network. Mapping 3 looks for the middle. It keeps a real server but moves it close, into the same building or a nearby telecom site, which runs all three tasks and sends the result back.

Both halves of the formula improve at once. Because the Server is close, L1 and L2 are small. Because it is a real server rather than a phone, PT is moderate. The simulator's all-edge preset shows 10 ms of network and 136 ms of processing, so 146 ms in total. Satyanarayanan's cloudlet idea is this design: a small data center near users that "can run arbitrary code just as in cloud computing", where "proximity of cloudlets to end users is crucial" (Satyanarayanan, 2017). The work is still offloaded from the device, but to a nearby point on the continuum instead of the far end.

Proximity has two prices, and both follow from the same fact that made the latency low. First, limited geographic reach: an Edge site is close to some users precisely because it is far from everyone else, who either get no service or a long trip. Second, limited scalability: an edge site is small, a few servers in a closet or a cabinet, so it cannot absorb a large workload the way a cloud region can. Covering a country means many small sites, each managed separately, which gives back the centralization that made Mapping 1 easy to operate.

Where it shows up: the retail store

Cameras and shelf sensors in a store feed a server in the back office. It counts footfall, notices which products customers pick up, and tracks inventory as it changes. The decisions are local and time-sensitive: restock a shelf now, open a second checkout because the queue is growing. Sending every video frame to a distant cloud would add Latency and upload a lot of footage, so the edge server answers in the store.

Now test the two weaknesses against it. Could that back-office box serve a branch in another city? No, its users are the cameras and staff in this store. That is limited reach. Could it absorb the peak traffic of a hundred stores on a sales weekend? No, it is sized for one store. That is limited scalability. The store is a good fit precisely because neither weakness matters for its workload.

So far each mapping put the whole pipeline in one place. But the three tasks are not equally demanding, so why should they share a layer? Mapping 4 splits the pipeline: the light tasks stay near the user and the heavy task runs one tier up. T1 (resize) and T3 (draw) run in the Edge layer, and T2 (object detection) runs in the Fog layer, which has more compute.

Think of it as one logical service deployed in two physical places. The Service pipeline is now distributed, so the formula has to grow. Count the hops: client to edge, edge to fog, fog to edge, edge to client. That is four network crossings instead of two, and each task contributes its own processing term.

In the simulator's Mapping 4 preset, those crossings add up to 40 ms and processing to 56 ms, so 96 ms, the lowest of the four presets. The fog does T2 much faster than the edge, and that saving outweighs the two extra hops. NIST describes fog nodes as sitting "between smart end-devices and centralized (cloud) services" and says fog "minimizes the request-response time" (NIST SP 500-325).

What sharing a fog node buys

Speed is only part of the case. Picture ten intersections in a smart city. Each traffic camera has a small edge box that resizes frames. All ten send them to one nearby fog server, which runs a large vehicle detection model for all of them and returns detections to each box.

1. Sharing: one detection model serves users of many edge nodes, instead of a copy squeezed onto each small box.
2. Capacity: the fog node has more processing power and storage than any single edge site.
3. Learning: the fog collects data from all the edge nodes, which keeps the model up to date, for example by retraining on new traffic patterns.

This matches how NIST frames fog. Fog nodes support "a common data management and communication system" and are organized in clusters by their "latency-distance to the smart end-devices" (NIST SP 500-325). Aggregation and coordination are the fog's job, which partly restores the centralization that Mapping 3 gave up, without going all the way to the cloud.

What splitting costs

• The fog layer introduces latency relative to Mapping 3: every boundary between layers adds its own latency, with all four delay components. The fog is still much closer than the Cloud.
• Every byte that crosses a boundary may be billed. Cloud providers already charge this way: AWS states that "there is a charge for data transfer across Regions" and that "data transfer from AWS to the internet is charged" (AWS Architecture Blog). Those quotes are about regions and the internet, not edge to fog links, but the principle carries over.
• So split where a task shrinks the data. Resizing before sending means fewer bytes cross the edge to fog link, which is exactly why T1 stays at the edge.

The four mappings are clean teaching cases. Practice is messier in one specific way: a dedicated fog tier is not always deployed, or identified, as a separate layer. In most cases, architects use two layers, Edge and Cloud.

NIST agrees that fog is optional: it "is not perceived as a mandatory layer", and "different usecase scenarios might have different architectures" (NIST SP 500-325). The claim that "most" architects choose edge plus cloud is the lecturer's observation from practice. No industry-wide statistic backs the exact share.

The resulting design has the shape of Mapping 4 with the cloud in place of the Fog: T1 and T3 at the edge, T2 in the cloud. It borrows the strength of each earlier mapping for the task that needs it. Go back to the retail store. The back-office edge server makes the live decisions, as in Mapping 3, and the cloud trains new models and computes trends across the whole chain, using the centralization that made Mapping 1 attractive. Each task gets the Layer that suits it.