Hailo-8 is an AI accelerator device based on PCIe BUS. And I have a chance to testing with Rockchip RK3588.
Installation
Installed M.2 (M-Key) Hailo-8 module on NVMe slot of NanoPi6 (RK3588). It supports PCIe 4x lanes.
Driver implementation
Hailo released version 4.18.0 driver and APIs on their github.
So I implemented kernel driver 4.18.0 for RK3588 kernel 6.1. It needs to modify some part of kernel sources. If you want to do this, follow this way.
- Download modified kernel driver source below section.
👉 hailort_kdrv_v4.18.0_for_rk3588_kernel_6.1-r1.tar.gz (1 download ) - Extract downloaded file in
driver/misc
, you can see a new driver directorydriver/misc/hailort
.
👉tar -xf ${downloaded.tar.gz} -C driver/misc
- Modify
driver/misc/Kconfig
. to includehailort/Kconfig
.- Add a line
source drivers/misc/hailort/Kconfig
beforeendmenu
line.
- Add a line
- Modify your kernel build configuration for these conditions.
- Enable “
CONFIG_PCIE_BUS_DEFAULT
” as “y
“ - Add “
CONFIG_HAILORT_PCIE
” to “m
” or “y
” (recommend as “m
“)
- Enable “
- re-configure kernel.
- Check “
.config
” file that enabled “CONFIG_HAILORT_PCIE=y|m
“. - Build your kernel or module.
- Flash new kernel image or update kernel modules.
Place firmware to “/lib/firmware/hailort”
- Download hailo-8 firmware v4.18.0 : hailo8fw.tar.gz (0 downloads )
- Make a directory
/lib/firmware/hailort
and extract downloaded firmware in there.
Create an udev rule file
- Change device permission for user, create a file
/etc/udev/rules.d/51-hailo-udev.rules
as sudoer. - And edit this with these lines.
#Change mode rules for Hailo's PCIe driver
SUBSYSTEM=="hailo_chardev", MODE="0666"
Check kernel driver load
And if you succeeded to load as like this:
Build Hailo CLI 11
Hailo supports their own CLI, and it must be installed on the system to control Hailo-8 device.
Now you need to build Hailo CLI-11 from Here.
Then check Hailo-8 device running well with this : hailortcli fw-control identify --extended
Performance test
Found some pre-compiled HEF model (from ONNX) for RaspberryPI5 series. And RPI5 module is not Hailo-8, but I tested with this model as Hailo-8L.
YOLOv5-n-seg (for Hailo-8L)
Performance marked as average 115 and 116 (for streaming) fps.
YOLOv6-n (for Hailo-8L)
Performance marked as average 355 fps.
Conclusion
Hailo-8 is a small chipset for enables HW based powerful AI acceleration for low powered ARM architecture systems.
And I impressed in their performance to handle YOLO in real-time.
But thermal may a small issue …
And Hailo officially recommend to use heat sink.
And maximum thermal is argued as :
It maybe a good choice to acquire 26 TOPs AI power for small embedded system if has good thermal solution to prevent over heating up to 100℃.