Face Detection on RPI5 using Hailo8L

If you are one of early adopters of the PI AI Hat with NeuralNet/AI accelerator from Hailo, you would have realized outside the basic pipeline examples ability to do a lot with it is very limited.

This article describes how one can use most of the GStreamer (GST) pipelines that work on other architectures to be used on RPI5. The other issue is the data flow compiler isn’t available on RPI5 at the time of writing this article which makes model version and Hailo API version compatibility issue aggravated.

Updating to Hail RT version to 4.18.0

Why is this required? Well if you try and download the latest models (.HEF) the are compiled to 4.18.0 version and there is no model compiler for the PI as started earlier.

If you follow the instructions for setting up the Hailo8L AI HAT on RPI5 you would end up with PICe driver and API version 4.17.0 thus we wouldn’t be able to use the models that are available from the download site. Here are the steps to run a face detection model on RPI5 using Hailo8L Hat using retinaface model.

Step 1 — Regular Hailo8L AI Hat Installation

Follow the instructions in the link here, to do the regular installation. Once you complete the installation, run the following command

$ hailortcli fw-control identify

Executing on device: 0000:01:00.0
Identifying board
Control Protocol Version: 2
Firmware Version: 4.17.0 (release,app,extended context switch buffer)
Logger Version: 0
Board Name: Hailo-8
Device Architecture: HAILO8L
Serial Number: *******
Part Number: HM21LB1C2LAE
Product Name: HAILO-8L AI ACC M.2 B+M KEY MODULE EXT TMP

Notice above the Firmware version is 4.17.0

Step 2 — Install DKMS

Update the package sources and install dkms, which would be required for the build for the new version

$ apt-get update -y && apt-get install -y dkms

Step 3 — Install latest PCIe driver, firmware & python binding

Goto the download link here, and download all the three downloads available by selecting the filters based on your installation as below

Once you have downloaded the packages install the packages with the following command (remember to choose “Y” when prompted to use “dkms” during installation of the PCIe

$ sudo dpkg --install hailort-pcie-driver_4.18.0_all.deb

$ sudo dpkg --install hailort_4.18.0_arm64.deb 

$ pip3 install  hailort-4.18.0-cp311-cp311-linux_aarch64.whl

Step 4 — TAPPAS Manual Installation

Install the per-requisites required for the tappas 3.29.0 installation

$ apt-get install -y python3-gi python3-gi-cairo gir1.2-gtk-3.0 
$ apt-get install -y python3-virtualenv

Now head back to the software download section here and select the combination for your installation

At the time of writing this blog the tappas version available is 3.29.0. Create a directory called tappas and clone the git repo from the link retrieved above. Post that would also need to clone the HailoRT source so that it can be compiled

$ mkdir tappas && cd tappas

$ git clone https://github.com/hailo-ai/tappas

$ cd tappas_v3.29.1

$ mkdir hailort

$ git clone https://github.com/hailo-ai/hailort.git hailort/sources

In my installation python3 package was managed externally which I had to fix to ensure the installation of tappas was succesful

Check if “/usr/lib/python3.11/EXTERNALLY-MANAGED” exists, if it does move it possibly to a backup file using “mv” command.

Now under the tappas_v3.29.1 directory make the following changes to the file “tools/run_app/setup.py”

lsb_release = list(filter(lambda x: 'RELEASE' in x,
                          Path('/etc/lsb-release').read_text().split('\n')))[0].split('=')[1].replace('.', '_')

required = Path(f'requirements_{lsb_release}.txt').read_text().splitlines()

Change it to

required = Path(f'requirements_20_04.txt').read_text().splitlines()

The reason for the change is that on RPI5 “/etc/lsb-release” does not exist which causes the installation to abort.

Now run the build to install using the following command

$ ./install.sh --skip-hailort

This should successfully complete the installation. Post installation run the following again to verify the updated version. Notice that the version is 4.18.0 now which would make the models work

$ hailortcli fw-control identify

Executing on device: 0000:01:00.0
Identifying board
Control Protocol Version: 2
Firmware Version: 4.18.0 (release,app,extended context switch buffer)
Logger Version: 0
Board Name: Hailo-8
Device Architecture: HAILO8L
Serial Number: *******
Part Number: HM21LB1C2LAE
Product Name: HAILO-8L AI ACC M.2 B+M KEY MODULE EXT TMP

Step 5 — Run Face Detection

Model Download

Download the retinaface model from the software download link from here. Select Hailo8L and

Scroll down to download “retinaface_mobilnet_v1”

GST Pipeline to run face recognition

Use the following GStreamer pipeline to run face recognition. Note change the file name in location attribute to the file that you want the recognition to run on:

Recognition on a local file

$ gst-launch-1.0 \
        hailomuxer name=hmux filesrc location=input.mp4 name=src_0 ! \
        queue name=hailo_preprocess_q_0 leaky=no max-size-buffers=5 max-size-bytes=0 max-size-time=0 ! \
        decodebin ! \
        videoconvert ! \
        videoscale qos=false ! \
        queue leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! \
        hailonet hef-path=/home/pi/hailo-ai/models/retinaface_mobilenet_v1.hef ! \
        queue leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! \
        hailofilter so-path=/usr/lib/aarch64-linux-gnu/post_processes/libface_detection_post.so name=face_detection_hailofilter qos=false function_name=retinaface ! \
        queue leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! \
        hailooverlay name=hailo_overlay qos=false show-confidence=false line-thickness=5 font-thickness=2 !\
        videoconvert ! \
        autovideosink

Recognition using PI Camera as source

gst-launch-1.0 hailomuxer name=hmux \
    libcamerasrc ! \
    video/x-raw,format=NV12,width=1280,height=720,framerate=30/1 ! \
    queue name=hailo_preprocess_q_0 leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! \
    videoscale qos=false n-threads=2 ! video/x-raw, pixel-aspect-ratio=1/1 ! \
    queue leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! \
    videoconvert n-threads=2 qos=false ! \
    queue leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! \
    hailonet hef-path=/home/pi/hailo-ai/models/retinaface_mobilenet_v1.hef ! \
    queue leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! \
    hailofilter so-path=/usr/lib/aarch64-linux-gnu/post_processes/libface_detection_post.so qos=false function_name=retinaface ! \
    queue leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! \
    hailooverlay qos=false ! \
    queue leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! \
    videoconvert n-threads=2 qos=false ! \
    queue leaky=no max-size-buffers=30 max-size-bytes=0 max-size-time=0 ! \
    autovideosink sync=false

To run face recognition/matching read part 2