Course overview

The course is COE 592, Machine Learning on Embedded Systems, offered in term T261 and taught by Dr. Abdul Jabbar Siddiqui of the Department of Computer Engineering. Of everything on the title and bio slides, one line carries the most information: he leads the SERV Lab, and SERV stands for Secure, Efficient, Robust Vision. Those three adjectives are the map of the course before the syllabus arrives in Part 02.

Reading SERV as a syllabus

Look at the interests list again and notice that every item is an instance of one question: can a model run near the sensor, inside a budget of memory, compute, energy and latency, and still be trusted? Edge AI and IoT name the setting. UAV inspection and transportation systems name the platforms. Adversarial machine learning and AI for cybersecurity name the threats. MIT 6.5940, whose course description slide 11's takeaways echo almost word for word, opens with the same framing, that deep networks "demand extraordinary levels of computation, hindering its deployment on everyday devices" (Han, MIT 6.5940).

Efficient is the theme that gets the most lecture time, because it is where the arithmetic lives: how many parameters, how many operations, how many milliseconds. Robust is what happens to that efficient model when rain, noise or a new camera changes the input distribution. Secure is the adversarial case, where the change in the input is chosen on purpose: Goodfellow, Shlens and Szegedy defined adversarial examples as inputs formed by applying "small but intentionally worst-case perturbations", and traced the vulnerability to the models' linear behaviour (Goodfellow et al., 2015). A model that is only efficient can be fooled; a model that is only robust may not fit on the drone.

Slide 3 is a wall of figures with no captions, so start by describing what is visibly there. A small unmanned helicopter sits on gravel with its camera payload circled (panel A). Beside it is a rugged field case holding a ground control station (panel B). A snowy corridor with a railway line running through it has been rebuilt in 3D (panel C). Two road vehicles carry roof rigs with four labelled sensors: Visual Camera, Thermal Camera, GPS receiver and LiDAR. Three top-down strips of the same track corridor follow, two with the rails traced in blue and encroaching vegetation segmented in red, one with the vegetation segmented in purple. A photogrammetric point cloud of the corridor appears once in true colour and once coloured by elevation. Finally a ROS-style dashboard shows six panes: an IR camera looking left and right, RGB cameras looking right, forward and left, and a LiDAR output.

Read together, this appears to be a rail corridor inspection platform: a Multi-sensor platform mounted on a UAV and on vehicles, mapping the track and the vegetation around it. The slide attributes no paper and no result, so treat everything beyond the visible labels as a description of the problem class. The instructor did co-author a review of exactly this field, a literature survey of automated machine vision inspection systems in railways, prepared at the National Research Council of Canada for Transport Canada (Siddiqui, Mammeri and Liu, 2021), which is the closest citable link for the theme.

Four sensors, four different questions

Each sensor on the roof rig answers a question the others cannot. A visual camera sees colour and texture, which is what separates green vegetation from grey ballast in the orthomosaic strips. A Thermal camera images emitted infrared instead of reflected light, so it sees temperature: a hot bearing or a person on the track at night shows up with no visible illumination at all. LiDAR is, in NOAA's definition, a remote sensing method that uses light in the form of a pulsed laser to measure ranges, and those ranges become the 3D Point cloud in which clearance and elevation can be measured. GPS contributes no picture of the scene, only where and when each frame was taken, which is what lets overlapping aerial photos be stitched into a georeferenced strip and the segmentation overlays registered onto it.

Why the payload cannot carry much

The reason this slide belongs in an embedded machine learning course is the platform, not the sensors. A UAV lifts a fixed mass, and every gram of camera, LiDAR or computer is a gram less battery. Its flight time is set by that battery, and the compute it can run is set by what the battery can feed without shortening the flight. A car rig has more of all three, but still nothing like a data center. So there are exactly two ways to turn the sensor streams into an inspection result: shrink the model until it runs on board, which is the E of SERV, or ship the raw data somewhere that can process it.

Shipping the raw data does not scale. Satyanarayanan's edge computing argument gives the numbers: if 12,000 users each stream 1080p video, the ingress link into the cloud needs about 100 Gbit/s, and a million users would need 8.5 Tbit/s. The cumulative demand is "considerably lower if the raw data is analyzed on cloudlets" near the sensors and only the extracted information travels onward (Satyanarayanan, 2017). Five cameras on one vehicle are a very small version of the same problem.

A farm robot drives between rows of cotton seedlings and must find the weeds without stopping. That is an Object detection task, and the model that does it rides on the robot: no cloud, no rack, just whatever compute the vehicle's battery can feed. Slide 4 shows the instructor's answer, EcoWeedNet, described on the slide as lightweight, computationally and energy efficient, and aimed at sustainable, low-carbon, consumer-electronics-based Precision agriculture.

Efficient: what a nano detector is

The comparison on slide 4 is against YOLO11n and YOLO12n. In the YOLO family the suffix is a size class: n (nano) is the smallest, then s, m, l and x. The classes share one architecture and differ in width and depth, so the table below is the cleanest picture of what "lightweight" costs and buys. The figures are Ultralytics' own, on COCO validation at 640 px.

The EcoWeedNet paper reports 95.2% mAP@0.5 on the CottonWeedDet12 dataset with about 4.21% of YOLOv4's parameters and 6.59% of its GFLOPs, using two parameter-free attention modules (Khater et al., 2025). Whether a Lightweight model is good enough is always this pair of numbers: an accuracy metric next to a cost metric, never one alone.

What a GradCAM++ heatmap does and does not prove

The slide's Figure 5 lays GradCAM++ heatmaps from YOLO11n, YOLO12n and EcoWeedNet side by side on seedling images, and its caption says EcoWeedNet showed better emphasis and focus on weed-relevant regions. GradCAM++ computes a weighted combination of the positive partial derivatives of the last convolutional layer's feature maps with respect to a class score (Chattopadhyay et al., 2018). In plain terms it answers "which pixels pushed this score up". It does not answer "is this score right". A model can attend to the weed and still misclassify it, or attend to the soil and get lucky. The caption's claim is a qualitative observation about attention; the 95.2% is the quantitative claim about correctness, and they are different kinds of evidence.

Robust: the same detector in the rain

Slide 5 moves from a farm to a rainy sky. The task is to tell a UAV from a bird in weather that smears both into grey blobs. The baseline is YOLOv5-m, the medium class of an earlier family (21.2M parameters, 49.0B FLOPs in the Ultralytics YOLOv5 table). The instructor's model is YOLO-RAW, published in IEEE Transactions on Intelligent Transportation Systems and evaluated on three adverse test sets, rainy, additive Gaussian noise and motion blurred (Munir et al., 2025). The slide never expands RAW, and neither does the paper's title or abstract, so this course does not expand it either.

Read the second row first. YOLOv5-m looks at the centre gull and reports UAV 0.29: a False positive, and one that a drone-detection system would act on. Its only correct box is Bird 0.35. YOLO-RAW boxes all five birds with confidence scores from about 0.37 to 0.73 and reports no drone in the sky. In the bottom row, where a real drone flies over a field, both models find it, 0.90 against 0.93. The gain is not on the easy target; it is on the ambiguous ones.

A detector reports every box above a confidence threshold. Ultralytics' default is 0.25, and its documentation says raising it "can help reduce false positives" (Ultralytics predict mode). So a natural question is whether the baseline's mistake could be fixed by tuning the threshold instead of changing the model. Drag the slider and watch what each setting costs.

At 0.30 the false UAV disappears and YOLOv5-m loses nothing else, but it still sees one bird in five. At 0.60 YOLOv5-m already reports nothing and YOLO-RAW has lost three of its five correct birds. The threshold only slices whatever confidence mass the model produced. YOLO-RAW is better because it moved that mass onto the right classes in the first place, not because it was thresholded differently.

A home or factory gateway forwards packets from hundreds of Internet of Things devices, and it has a fraction of a phone's compute. Somewhere in that box a decision has to be made, per packet or per flow, about whether the traffic is an attack. That is an Intrusion detection system, and slide 6 is the first page of the instructor's PNet-IDS paper, a lightweight and generalizable Convolutional neural network for intrusion detection in the Internet of Things.

Why an intrusion detector must be lightweight

The abstract states the problem in one sentence: deep learning detectors perform well, but their "high computational and storage requirements make these impractical for most IoT devices" (Iliyasu et al., 2025). NIST's guide to intrusion detection and prevention lists four classes of system, and the relevant one here is network-based: a sensor that watches traffic on a segment and analyses it for suspicious activity (NIST SP 800-94). On an IoT gateway that sensor is the gateway itself, so the model competes for the same small CPU that forwards the packets, and it has to keep up with the link, which network engineers call running at line rate.

That is the Efficient leg. The Robust leg is the word generalizable in the title. An IDS trained on one dataset meets, in deployment, traffic and attacks it never saw, a distribution shift. PNet-IDS addresses this with a knowledge distillation framework, in which a small student network is trained to reproduce a larger teacher, and reports that robustness against distribution shifts in network traffic is enhanced as a result. The benchmarks are BoT-IoT and CIC-IDS2017, and the claimed wins are reduced parameter count, reduced FLOPs and reduced model size while maintaining high accuracy and precision (Iliyasu et al., 2025).

The last two slides are photographs with no captions, and both show a decision that has to be made before the world moves on. Slide 7 pairs a low-speed autonomous shuttle with a pedestrian crossing in front of it and a schematic of a T-intersection where a red arrow, a vehicle turning left, crosses a blue arrow, a pedestrian in the crosswalk. Slide 8 shows grayscale undercarriage images of rail cars: one component circled as defective or missing, and a crack pulled out in a zoom box.

The left-turn pedestrian conflict

The schematic is the left-turn pedestrian conflict, a named pattern in Intelligent transportation systems safety analysis. A left-turning driver scans for oncoming vehicles, accepts a gap, and turns into a crosswalk where a pedestrian is already crossing. FHWA research on left-turn phasing found that protected phasing reduced vehicle-to-vehicle injury crashes but did not significantly reduce vehicle-to-pedestrian crashes, and that under permissive phasing the turning driver must pick a gap in oncoming traffic and yield to the crosswalk at the same time, which prior studies flag as a strong conflict potential (FHWA, HRT-18-044). For an autonomous shuttle the problem is the same one the human driver has, and the detector that watches the crosswalk must answer before the vehicle enters it.

Looking under the train

Slide 8 belongs to Automated visual inspection of rolling stock: cameras placed by the track or under it photograph each wagon as it passes, and a model flags anything that does not look like an intact component. The instructor reviewed exactly this field for Transport Canada while at NRC (Siddiqui, Mammeri and Liu, 2021). The slide itself carries no citation, so the images are described here as an example of the problem class, not attributed to a particular system.

Physics sets the deadline

What unites the shuttle and the wagon is that the time available for Inference is not a design preference. It is set by how fast the scene changes. Satyanarayanan's remark that "the speed of light is an obvious physical limit on latency" (Satyanarayanan, 2017) is the reason a cloud round trip cannot be the answer here: the distance the vehicle covers during the latency of the reply is the safety margin lost.

On the first evening each student says three things out loud: their name, their educational background (previous studies), and their work background (current or previous work, industry or occupation). Then everyone opens the Pre-course survey at https://forms.cloud.microsoft/r/5ssu8gkcS9, also printed as a QR code on slide 10, and answers questions about background, skills and expectations.

It is tempting to file both under administration, but they serve the same purpose as takeaway 1 later in this part: you cannot tune what you have not measured. A graduate course on efficient deep learning draws people from computer engineering, electrical engineering, computer science and industry. Some have trained convolutional networks for years and never flashed a microcontroller; others write firmware for embedded systems daily and have never opened PyTorch. The instructor uses the room's answers to set depth and pacing, so an honest answer about a gap is worth more to you than an impressive one.

What is worth refreshing before week two

The deck does not list prerequisites, so treat the following as a hedged suggestion drawn from what the course will ask you to do rather than as a slide claim. The topics ahead (compression, pruning, quantization, deployment to boards) match the syllabus of MIT's 6.5940 TinyML and Efficient Deep Learning Computing, which lists an introductory machine learning subject and a computation structures subject as its prerequisites. Translated to tools, four skills will carry most of the weight.

None of these needs to be deep on day one. If you can train a small classifier in PyTorch, explain what a convolution computes, and compile and flash a blinking-LED program in C, you can follow the first weeks and fill in the rest as the loop demands it.

Take a small image classifier with 3 million parameters, trained in PyTorch and saved in 32-bit floating point. Its weights alone occupy 12 MB. A Raspberry Pi Pico has 2 MB of flash and 264 KB of SRAM. The model does not fit, and no amount of accuracy changes that. Everything this course teaches is what you do next.

Slide 11 states the promise as five takeaways. You will know the key Efficiency metrics for deep learning computing. You will understand how to speed up Inference and Training of neural networks on resource-constrained platforms. You will analyse the tradeoffs between different optimization techniques. You will be aware of recent research trends and industry practice. And you will get hands-on experience compressing and implementing deep learning models on resource-constrained devices. The infographic relabels the same five as Measure efficiency, Accelerate, Analyse trade-offs, Track the frontier, and Build and deploy, with a five-word footer: Measure, Optimize, Compare, Explore, Deploy.

Read the five as a loop, not a list

The order on the slide is not arbitrary. It is the order in which you actually work. You measure a baseline: size, memory, compute, latency, energy. You apply one optimization, say pruning. You compare the new numbers against the baseline and note what the step cost in accuracy. You look to the research literature and to industry for the next technique. You deploy on the real board to check that the paper numbers survive contact with hardware, because measured latency and energy routinely differ from analytical estimates. Then you measure again, and the loop turns.

Deep Compression is the canonical worked loop. Han, Mao and Dally started from AlexNet at 240 MB and VGG-16 at 552 MB (measure), pruned redundant connections, quantized the survivors from 32 to as few as 5 bits and Huffman coded the result (optimize), reported 35x and 49x reductions with no loss of accuracy (compare), and measured 3x to 4x layerwise speedup and 3x to 7x better energy efficiency on real hardware (deploy). The companion paper on learning both weights and connections had already shown pruning alone cutting AlexNet from 61 M to 6.7 M parameters. Everything after 2016 (MCUNet, TinyTL, on-device training under 256 KB) is the explore step feeding the next turn.

A preview of the efficiency metrics

Slide 11 names the metrics without listing them; the list below previews later lectures and is not slide content. The ordering follows the MIT 6.5940 sequence, which opens with the basics of efficient deep learning before pruning, quantization, neural architecture search, distillation and on-device training. Learn the names now so the loop has something concrete to measure.

The first two are the easiest to compute by hand, and the calculation is one you will repeat all term. A weight stored in b bits takes b / 8 bytes, so the size of the weights is the parameter count times bits per weight divided by eight.

Try it yourself. The calculator applies the formula for any parameter count and bit width and checks the result against generic device tiers. The device figures are datasheet capacities for one example board per tier (Arduino, Raspberry Pi and NVIDIA product pages), not what is free once the runtime and your code are loaded.

Three misconceptions the loop corrects

Slide 12 contains three words: "View on Blackboard". That is the whole syllabus section of the deck. It does not give grading weights, exam dates, project deliverables, a textbook, an office location or office hours. Anything you read elsewhere claiming to know those for this course is not coming from these slides.

Plan the project before the syllabus tells you to

Takeaway 5 promises hands-on experience compressing and implementing models on constrained devices, and the MIT counterpart of this course ends in an open-ended design project. Read that as a strong hint that a hardware Deployment is coming, whatever Blackboard eventually calls it. The students who struggle with such projects are rarely short of techniques; they are short of a baseline to compare against and a board that arrives on time. Both are fixed in week one.

1. Pick a device tier first. The memory budget of the tier decides which models are even candidates, so the device is an input to model selection, not an output of it.
2. Pick a candidate model that already fits, or is within one or two optimizations of fitting, and record its parameter count and float32 size with the formula from the previous concept.
3. Start a metrics log before the first optimization. A baseline recorded after you have started changing things is not a baseline.

The last column is a guide, not a datasheet claim. MCUNet reached over 70% ImageNet top-1 accuracy on a commercial microcontroller, so the microcontroller tier is more capable than its numbers suggest, but only after the whole loop has been applied to a purpose-built architecture. A model chosen for a GPU module and then squeezed onto a microcontroller is the hardest path in the course.

Keep the log as a plain table with one row per experiment and the same columns every time: accuracy, parameter count, size, peak memory, latency, energy if you can measure it, and a one-line note of what changed. This is the compare step made permanent, and it is the table an examiner or a reviewer will want to see.