The instructor and edge AI in practice

The course is COE 592, Machine Learning on Embedded Systems, offered in term T261 and taught by Dr. Abdul Jabbar Siddiqui of the Department of Computer Engineering. Of everything on the title and bio slides, one line carries the most information: he leads the SERV Lab, and SERV stands for Secure, Efficient, Robust Vision. Those three adjectives are the map of the course before the syllabus arrives in Part 02.

Reading SERV as a syllabus

Look at the interests list again and notice that every item is an instance of one question: can a model run near the sensor, inside a budget of memory, compute, energy and latency, and still be trusted? Edge AI and IoT name the setting. UAV inspection and transportation systems name the platforms. Adversarial machine learning and AI for cybersecurity name the threats. MIT 6.5940, whose course description slide 11's takeaways echo almost word for word, opens with the same framing, that deep networks "demand extraordinary levels of computation, hindering its deployment on everyday devices" (Han, MIT 6.5940).

Efficient is the theme that gets the most lecture time, because it is where the arithmetic lives: how many parameters, how many operations, how many milliseconds. Robust is what happens to that efficient model when rain, noise or a new camera changes the input distribution. Secure is the adversarial case, where the change in the input is chosen on purpose: Goodfellow, Shlens and Szegedy defined adversarial examples as inputs formed by applying "small but intentionally worst-case perturbations", and traced the vulnerability to the models' linear behaviour (Goodfellow et al., 2015). A model that is only efficient can be fooled; a model that is only robust may not fit on the drone.

Slide 3 is a wall of figures with no captions, so start by describing what is visibly there. A small unmanned helicopter sits on gravel with its camera payload circled (panel A). Beside it is a rugged field case holding a ground control station (panel B). A snowy corridor with a railway line running through it has been rebuilt in 3D (panel C). Two road vehicles carry roof rigs with four labelled sensors: Visual Camera, Thermal Camera, GPS receiver and LiDAR. Three top-down strips of the same track corridor follow, two with the rails traced in blue and encroaching vegetation segmented in red, one with the vegetation segmented in purple. A photogrammetric point cloud of the corridor appears once in true colour and once coloured by elevation. Finally a ROS-style dashboard shows six panes: an IR camera looking left and right, RGB cameras looking right, forward and left, and a LiDAR output.

Read together, this appears to be a rail corridor inspection platform: a Multi-sensor platform mounted on a UAV and on vehicles, mapping the track and the vegetation around it. The slide attributes no paper and no result, so treat everything beyond the visible labels as a description of the problem class. The instructor did co-author a review of exactly this field, a literature survey of automated machine vision inspection systems in railways, prepared at the National Research Council of Canada for Transport Canada (Siddiqui, Mammeri and Liu, 2021), which is the closest citable link for the theme.

Four sensors, four different questions

Each sensor on the roof rig answers a question the others cannot. A visual camera sees colour and texture, which is what separates green vegetation from grey ballast in the orthomosaic strips. A Thermal camera images emitted infrared instead of reflected light, so it sees temperature: a hot bearing or a person on the track at night shows up with no visible illumination at all. LiDAR is, in NOAA's definition, a remote sensing method that uses light in the form of a pulsed laser to measure ranges, and those ranges become the 3D Point cloud in which clearance and elevation can be measured. GPS contributes no picture of the scene, only where and when each frame was taken, which is what lets overlapping aerial photos be stitched into a georeferenced strip and the segmentation overlays registered onto it.

Why the payload cannot carry much

The reason this slide belongs in an embedded machine learning course is the platform, not the sensors. A UAV lifts a fixed mass, and every gram of camera, LiDAR or computer is a gram less battery. Its flight time is set by that battery, and the compute it can run is set by what the battery can feed without shortening the flight. A car rig has more of all three, but still nothing like a data center. So there are exactly two ways to turn the sensor streams into an inspection result: shrink the model until it runs on board, which is the E of SERV, or ship the raw data somewhere that can process it.

Shipping the raw data does not scale. Satyanarayanan's edge computing argument gives the numbers: if 12,000 users each stream 1080p video, the ingress link into the cloud needs about 100 Gbit/s, and a million users would need 8.5 Tbit/s. The cumulative demand is "considerably lower if the raw data is analyzed on cloudlets" near the sensors and only the extracted information travels onward (Satyanarayanan, 2017). Five cameras on one vehicle are a very small version of the same problem.

A farm robot drives between rows of cotton seedlings and must find the weeds without stopping. That is an Object detection task, and the model that does it rides on the robot: no cloud, no rack, just whatever compute the vehicle's battery can feed. Slide 4 shows the instructor's answer, EcoWeedNet, described on the slide as lightweight, computationally and energy efficient, and aimed at sustainable, low-carbon, consumer-electronics-based Precision agriculture.

Efficient: what a nano detector is

The comparison on slide 4 is against YOLO11n and YOLO12n. In the YOLO family the suffix is a size class: n (nano) is the smallest, then s, m, l and x. The classes share one architecture and differ in width and depth, so the table below is the cleanest picture of what "lightweight" costs and buys. The figures are Ultralytics' own, on COCO validation at 640 px.

The EcoWeedNet paper reports 95.2% mAP@0.5 on the CottonWeedDet12 dataset with about 4.21% of YOLOv4's parameters and 6.59% of its GFLOPs, using two parameter-free attention modules (Khater et al., 2025). Whether a Lightweight model is good enough is always this pair of numbers: an accuracy metric next to a cost metric, never one alone.

What a GradCAM++ heatmap does and does not prove

The slide's Figure 5 lays GradCAM++ heatmaps from YOLO11n, YOLO12n and EcoWeedNet side by side on seedling images, and its caption says EcoWeedNet showed better emphasis and focus on weed-relevant regions. GradCAM++ computes a weighted combination of the positive partial derivatives of the last convolutional layer's feature maps with respect to a class score (Chattopadhyay et al., 2018). In plain terms it answers "which pixels pushed this score up". It does not answer "is this score right". A model can attend to the weed and still misclassify it, or attend to the soil and get lucky. The caption's claim is a qualitative observation about attention; the 95.2% is the quantitative claim about correctness, and they are different kinds of evidence.

Robust: the same detector in the rain

Slide 5 moves from a farm to a rainy sky. The task is to tell a UAV from a bird in weather that smears both into grey blobs. The baseline is YOLOv5-m, the medium class of an earlier family (21.2M parameters, 49.0B FLOPs in the Ultralytics YOLOv5 table). The instructor's model is YOLO-RAW, published in IEEE Transactions on Intelligent Transportation Systems and evaluated on three adverse test sets, rainy, additive Gaussian noise and motion blurred (Munir et al., 2025). The slide never expands RAW, and neither does the paper's title or abstract, so this course does not expand it either.

Read the second row first. YOLOv5-m looks at the centre gull and reports UAV 0.29: a False positive, and one that a drone-detection system would act on. Its only correct box is Bird 0.35. YOLO-RAW boxes all five birds with confidence scores from about 0.37 to 0.73 and reports no drone in the sky. In the bottom row, where a real drone flies over a field, both models find it, 0.90 against 0.93. The gain is not on the easy target; it is on the ambiguous ones.

A detector reports every box above a confidence threshold. Ultralytics' default is 0.25, and its documentation says raising it "can help reduce false positives" (Ultralytics predict mode). So a natural question is whether the baseline's mistake could be fixed by tuning the threshold instead of changing the model. Drag the slider and watch what each setting costs.

At 0.30 the false UAV disappears and YOLOv5-m loses nothing else, but it still sees one bird in five. At 0.60 YOLOv5-m already reports nothing and YOLO-RAW has lost three of its five correct birds. The threshold only slices whatever confidence mass the model produced. YOLO-RAW is better because it moved that mass onto the right classes in the first place, not because it was thresholded differently.

A home or factory gateway forwards packets from hundreds of Internet of Things devices, and it has a fraction of a phone's compute. Somewhere in that box a decision has to be made, per packet or per flow, about whether the traffic is an attack. That is an Intrusion detection system, and slide 6 is the first page of the instructor's PNet-IDS paper, a lightweight and generalizable Convolutional neural network for intrusion detection in the Internet of Things.

Why an intrusion detector must be lightweight

The abstract states the problem in one sentence: deep learning detectors perform well, but their "high computational and storage requirements make these impractical for most IoT devices" (Iliyasu et al., 2025). NIST's guide to intrusion detection and prevention lists four classes of system, and the relevant one here is network-based: a sensor that watches traffic on a segment and analyses it for suspicious activity (NIST SP 800-94). On an IoT gateway that sensor is the gateway itself, so the model competes for the same small CPU that forwards the packets, and it has to keep up with the link, which network engineers call running at line rate.

That is the Efficient leg. The Robust leg is the word generalizable in the title. An IDS trained on one dataset meets, in deployment, traffic and attacks it never saw, a distribution shift. PNet-IDS addresses this with a knowledge distillation framework, in which a small student network is trained to reproduce a larger teacher, and reports that robustness against distribution shifts in network traffic is enhanced as a result. The benchmarks are BoT-IoT and CIC-IDS2017, and the claimed wins are reduced parameter count, reduced FLOPs and reduced model size while maintaining high accuracy and precision (Iliyasu et al., 2025).

The last two slides are photographs with no captions, and both show a decision that has to be made before the world moves on. Slide 7 pairs a low-speed autonomous shuttle with a pedestrian crossing in front of it and a schematic of a T-intersection where a red arrow, a vehicle turning left, crosses a blue arrow, a pedestrian in the crosswalk. Slide 8 shows grayscale undercarriage images of rail cars: one component circled as defective or missing, and a crack pulled out in a zoom box.

The left-turn pedestrian conflict

The schematic is the left-turn pedestrian conflict, a named pattern in Intelligent transportation systems safety analysis. A left-turning driver scans for oncoming vehicles, accepts a gap, and turns into a crosswalk where a pedestrian is already crossing. FHWA research on left-turn phasing found that protected phasing reduced vehicle-to-vehicle injury crashes but did not significantly reduce vehicle-to-pedestrian crashes, and that under permissive phasing the turning driver must pick a gap in oncoming traffic and yield to the crosswalk at the same time, which prior studies flag as a strong conflict potential (FHWA, HRT-18-044). For an autonomous shuttle the problem is the same one the human driver has, and the detector that watches the crosswalk must answer before the vehicle enters it.

Looking under the train

Slide 8 belongs to Automated visual inspection of rolling stock: cameras placed by the track or under it photograph each wagon as it passes, and a model flags anything that does not look like an intact component. The instructor reviewed exactly this field for Transport Canada while at NRC (Siddiqui, Mammeri and Liu, 2021). The slide itself carries no citation, so the images are described here as an example of the problem class, not attributed to a particular system.

Physics sets the deadline

What unites the shuttle and the wagon is that the time available for Inference is not a design preference. It is set by how fast the scene changes. Satyanarayanan's remark that "the speed of light is an obvious physical limit on latency" (Satyanarayanan, 2017) is the reason a cloud round trip cannot be the answer here: the distance the vehicle covers during the latency of the reply is the safety margin lost.