Testing Hailo-8 with rk3588

      No Comments on Testing Hailo-8 with rk3588

Hailo-8 is an AI accelerator device based on PCIe BUS. And I have a chance to testing with Rockchip RK3588.

Installation

Installed M.2 (M-Key) Hailo-8 module on NVMe slot of NanoPi6 (RK3588). It supports PCIe 4x lanes.

Driver implementation

Hailo released version 4.18.0 driver and APIs on their github.
So I implemented kernel driver 4.18.0 for RK3588 kernel 6.1. It needs to modify some part of kernel sources. If you want to do this, follow this way.

  • Download modified kernel driver source below section.
    👉 hailort_kdrv_v4.18.0_for_rk3588_kernel_6.1-r1.tar.gz (34 downloads )
  • Extract downloaded file in driver/misc, you can see a new driver directory driver/misc/hailort.
    👉 tar -xf ${downloaded.tar.gz} -C driver/misc
  • Modify driver/misc/Kconfig. to include hailort/Kconfig .
    • Add a line source drivers/misc/hailort/Kconfig before endmenu line.
  • Modify your kernel build configuration for these conditions.
    • Enable “CONFIG_PCIE_BUS_DEFAULT” as “y“
    • Add “CONFIG_HAILORT_PCIE” to “m” or “y” (recommend as “m“)
  • re-configure kernel.
  • Check “.config” file that enabled “CONFIG_HAILORT_PCIE=y|m“.
  • Build your kernel or module.
  • Flash new kernel image or update kernel modules.

Place firmware to “/lib/firmware/hailort”

Create an udev rule file

  • Change device permission for user, create a file /etc/udev/rules.d/51-hailo-udev.rules as sudoer.
  • And edit this with these lines.
    #Change mode rules for Hailo's PCIe driver
    SUBSYSTEM=="hailo_chardev", MODE="0666"

Check kernel driver load

And if you succeeded to load as like this:

Build Hailo CLI 11

Hailo supports their own CLI, and it must be installed on the system to control Hailo-8 device.
Now you need to build Hailo CLI-11 from Here.

Then check Hailo-8 device running well with this : hailortcli fw-control identify --extended

Performance test

Found some pre-compiled HEF model (from ONNX) for RaspberryPI5 series. And RPI5 module is not Hailo-8, but I tested with this model as Hailo-8L.

YOLOv5-n-seg (for Hailo-8L)

Performance marked as average 115 and 116 (for streaming) fps.

YOLOv6-n (for Hailo-8L)

Performance marked as average 355 fps.

Conclusion

Hailo-8 is a small chipset for enables HW based powerful AI acceleration for low powered ARM architecture systems.
And I impressed in their performance to handle YOLO in real-time.

But thermal may a small issue …

And Hailo officially recommend to use heat sink.

And maximum thermal is argued as :

 

It maybe a good choice to acquire 26 TOPs AI power for small embedded system if has good thermal solution to prevent over heating up to 100℃.

Leave a Reply