AI Accelerators

Caution

This page is in notes format, and may not be of the same quality as other pages on this site.

AI accelerators (also called neural processing units, NPUs, or AI co-processors) are integrated circuits designed to run machine-learning inference workloads (e.g. neural networks) far more efficiently than a general-purpose CPU. They are commonly paired with a host processor, offloading the heavy matrix/tensor maths involved in running a model. Performance is often quoted in TOPS (tera-operations per second).

Neural networks work by having many layers of “neurons”. The value of each neuron in each layer is calculated by:

1. Taking a linear weighted sum of all the neurons in the layer before it: $z = w_1 x_1 + w_2 x_2 + \dots + w_n x_n + b$, where $b$ is a bias term (assuming a dense network in where a neuron connects to every neuron in the previous layer)
2. Applying a non-linear activation function: $y = \phi(z)$ where $\phi$ might be something like $\max(0, z)$ (output the value if the value is positive, otherwise 0, a.k.a. ReLU).

Tip

If the activation function were not applied, the entire neural network could be collapsed down into a single layer (a combination of linear operations is still just a linear operation). The activation function gives each layer non-linearity, allowing the network to approximate complex, non-linear functions.

The most computationally intensive part of running a neural network are the weighted-sum calculations. These can be easily done with matrix multiplications. An AI accelerator has dedicated hardware that is optimised for these matrix operations.

Hailo

Hailo accelerators run compiled Hailo Exexcutable Format (HEF) files. These files contain the quantized network, hardware allocation, scheduling and optional HailoRT post-processing.¹

The Hailo-8L is an entry-level AI accelerator from Hailo, delivering up to 13 TOPS. It is widely used in edge-AI and computer-vision applications (for example, on the Raspberry Pi AI Kit). It is designed to run neural networks.

Photo of the Hailo-8L BGA package.²

The Hailo-8L contains the following components³:

• NPU: Up to 13 TOPS.
• MCU: Dual ARM Cortex-M4 processors running at 200 MHz. 640 kB SRAM. FPU and MPU.
• 4-lane PCI express Gen3 endpoint with integrated PHY (this is designed to be connected to the host processor)
• 2 UART interfaces.
• 4 I2C interfaces.
• Quad SPI interface (for external flash devices)

A block diagram of the components inside the Hailo-8L AI accelerator.³

It is designed to be connected to a host processor via the PCIe link as shown below.

Typical usage block diagram showing the Hailo-8L connected to a host processor via its PCIe link.³

The Hailo-8L module is supported in Linux via the hailo-all package. On systems with apt, use:

sudo apt install -y hailo-all

Hailo-8L M.2 Module

The Hailo-8L is available as a M.2 module, which is a small form-factor board that contains the Hailo-8L chip and supporting components. The module is designed to be plugged into a host processor via a M.2 connector and it’s PCIe lanes.

The manufacturer part number is HM21LB1C2LAE.

Photo of the front and back sides of the Hailo-8L M.2 module.

Footnotes

1. Ultralytics. Hailo Export for Ultralytics YOLO Models. Retrieved 2026-08-12, from https://docs.ultralytics.com/integrations/hailo/. ↩

2. Hailo. Hailo-8L Entry-Level AI Accelerator [product page]. Retrieved 2026-06-08, from https://hailo.ai/products/ai-accelerators/hailo-8l-ai-accelerator-for-ai-light-applications/. ↩

3. Hailo (2026, Apr). Hailo-8L Datasheet [datasheet]. Retrieved 2026-06-08, from https://hailo.ai/hailo-files/hailo-8l-industrial-datasheet-en/. ↩ ↩² ↩³