Course takeaways and getting started

On the first evening each student says three things out loud: their name, their educational background (previous studies), and their work background (current or previous work, industry or occupation). Then everyone opens the Pre-course survey at https://forms.cloud.microsoft/r/5ssu8gkcS9, also printed as a QR code on slide 10, and answers questions about background, skills and expectations.

It is tempting to file both under administration, but they serve the same purpose as takeaway 1 later in this part: you cannot tune what you have not measured. A graduate course on efficient deep learning draws people from computer engineering, electrical engineering, computer science and industry. Some have trained convolutional networks for years and never flashed a microcontroller; others write firmware for embedded systems daily and have never opened PyTorch. The instructor uses the room's answers to set depth and pacing, so an honest answer about a gap is worth more to you than an impressive one.

What is worth refreshing before week two

The deck does not list prerequisites, so treat the following as a hedged suggestion drawn from what the course will ask you to do rather than as a slide claim. The topics ahead (compression, pruning, quantization, deployment to boards) match the syllabus of MIT's 6.5940 TinyML and Efficient Deep Learning Computing, which lists an introductory machine learning subject and a computation structures subject as its prerequisites. Translated to tools, four skills will carry most of the weight.

None of these needs to be deep on day one. If you can train a small classifier in PyTorch, explain what a convolution computes, and compile and flash a blinking-LED program in C, you can follow the first weeks and fill in the rest as the loop demands it.

Take a small image classifier with 3 million parameters, trained in PyTorch and saved in 32-bit floating point. Its weights alone occupy 12 MB. A Raspberry Pi Pico has 2 MB of flash and 264 KB of SRAM. The model does not fit, and no amount of accuracy changes that. Everything this course teaches is what you do next.

Slide 11 states the promise as five takeaways. You will know the key Efficiency metrics for deep learning computing. You will understand how to speed up Inference and Training of neural networks on resource-constrained platforms. You will analyse the tradeoffs between different optimization techniques. You will be aware of recent research trends and industry practice. And you will get hands-on experience compressing and implementing deep learning models on resource-constrained devices. The infographic relabels the same five as Measure efficiency, Accelerate, Analyse trade-offs, Track the frontier, and Build and deploy, with a five-word footer: Measure, Optimize, Compare, Explore, Deploy.

Read the five as a loop, not a list

The order on the slide is not arbitrary. It is the order in which you actually work. You measure a baseline: size, memory, compute, latency, energy. You apply one optimization, say pruning. You compare the new numbers against the baseline and note what the step cost in accuracy. You look to the research literature and to industry for the next technique. You deploy on the real board to check that the paper numbers survive contact with hardware, because measured latency and energy routinely differ from analytical estimates. Then you measure again, and the loop turns.

Deep Compression is the canonical worked loop. Han, Mao and Dally started from AlexNet at 240 MB and VGG-16 at 552 MB (measure), pruned redundant connections, quantized the survivors from 32 to as few as 5 bits and Huffman coded the result (optimize), reported 35x and 49x reductions with no loss of accuracy (compare), and measured 3x to 4x layerwise speedup and 3x to 7x better energy efficiency on real hardware (deploy). The companion paper on learning both weights and connections had already shown pruning alone cutting AlexNet from 61 M to 6.7 M parameters. Everything after 2016 (MCUNet, TinyTL, on-device training under 256 KB) is the explore step feeding the next turn.

A preview of the efficiency metrics

Slide 11 names the metrics without listing them; the list below previews later lectures and is not slide content. The ordering follows the MIT 6.5940 sequence, which opens with the basics of efficient deep learning before pruning, quantization, neural architecture search, distillation and on-device training. Learn the names now so the loop has something concrete to measure.

The first two are the easiest to compute by hand, and the calculation is one you will repeat all term. A weight stored in b bits takes b / 8 bytes, so the size of the weights is the parameter count times bits per weight divided by eight.

Try it yourself. The calculator applies the formula for any parameter count and bit width and checks the result against generic device tiers. The device figures are datasheet capacities for one example board per tier (Arduino, Raspberry Pi and NVIDIA product pages), not what is free once the runtime and your code are loaded.

Three misconceptions the loop corrects

Slide 12 contains three words: "View on Blackboard". That is the whole syllabus section of the deck. It does not give grading weights, exam dates, project deliverables, a textbook, an office location or office hours. Anything you read elsewhere claiming to know those for this course is not coming from these slides.

Plan the project before the syllabus tells you to

Takeaway 5 promises hands-on experience compressing and implementing models on constrained devices, and the MIT counterpart of this course ends in an open-ended design project. Read that as a strong hint that a hardware Deployment is coming, whatever Blackboard eventually calls it. The students who struggle with such projects are rarely short of techniques; they are short of a baseline to compare against and a board that arrives on time. Both are fixed in week one.

1. Pick a device tier first. The memory budget of the tier decides which models are even candidates, so the device is an input to model selection, not an output of it.
2. Pick a candidate model that already fits, or is within one or two optimizations of fitting, and record its parameter count and float32 size with the formula from the previous concept.
3. Start a metrics log before the first optimization. A baseline recorded after you have started changing things is not a baseline.

The last column is a guide, not a datasheet claim. MCUNet reached over 70% ImageNet top-1 accuracy on a commercial microcontroller, so the microcontroller tier is more capable than its numbers suggest, but only after the whole loop has been applied to a purpose-built architecture. A model chosen for a GPU module and then squeezed onto a microcontroller is the hardest path in the course.

Keep the log as a plain table with one row per experiment and the same columns every time: accuracy, parameter count, size, peak memory, latency, energy if you can measure it, and a one-line note of what changed. This is the compare step made permanent, and it is the table an examiner or a reviewer will want to see.