Yolo with high-resolution cameras: Resize, Scan, or Crop? An adventure with a Raspberry Pi 5, a Hailo AI Accelerator, and a high-resolution camera, Part I.

Intro

This effort started as a weekend project. My plan was to use a Raspberry Pi 5 with a Hailo8 AI Accelerator to detect objects in frames from a 4K RTSP stream from a camera overlooking my driveway, using a Python script. As these projects tend to do, it grew from there. Since it got to be pretty long, I have broken the story into three parts. This article, the first, is my effort to understand how to create and use the Hailo architecture; the second details my efforts to tune the image capture to the detection environment; and the third is the code that integrates the detection into Home Assistant for alerts.

The Hailo8 AI Accelerator drastically decreases inference time on the Pi, but the tutorials are challenging to get working and even more difficult to fix when they break. So, I built my own implementation from scratch to understand what was actually happening under the hood. This series documents my implementation. Sometimes, the only way to understand something is to build it yourself.

Hardware

There are numerous tutorials on how to attach the Hailo8 accelerator to a Pi, so I won’t repeat that here. Mine is attached through a Geekworm x1004 PCIe to Dual M.2 HAT. That HAT provides 2 M.2 2280 slots, so I can run an SSD and the Hailo8 simultaneously. I have set the dtparam-pciex1_gen=3 parameter in my firmware/config.txt.

Software

The code is located at https://github.com/TeigLevingston/Hailo_Engine_Example. This article will highlight the section of the code marked ‘Hailo configuration start’ in the motion_detection.py file. The script has other functions that open the stream, pull frames, detect motion, and send regions of interest to the engine. If you set the constant for the VIDEO_PATH to a local file, you can walk through the code with the VSCode debugger and watch the parameters being set. These functions let me test the engine inputs a bit and are expanded on in Part 2. Note: don’t forget to create your python environment with the — system-site-packages flag. I always forget and have to start over in order to get the Hailo imports to work.

Use Case

I have been able to run the Hailo tutorials for the most part, but like others, I found them too obtuse for a hello-hailo project. The examples provided skipped over some details that I felt were important to understand if I planned on integrating the accelerator into my custom environment. Some of the details require a bit of digging. For instance, the hailo SDK 4.2x requires numpy v 1.23.3. There is a note somewhere in the engineering docs explaining it, but that is not the most current version of numpy, and it tripped me up for a while.

To that end, I wrote my own implementation and heavily commented it. After I was finished, the process details were pretty evident, which may be why they were skipped in the tutorials. It really is pretty straightforward. I just don’t like running code I didn’t write myself, or at least understand thoroughly. For this example, I put everything in a single file to make it simple to walk through the code and the comments. I have tried to detail what is moving and where it is moving.

Seeing what is going on

Once you have set up your environment according to Hailo’s instructions, you will also need the .hef file for the model you want to use. I recommend starting with models provided by the Hailo-Zoo, since training a model and converting it to the hef format is an entirely different process. I used the YOLO11m.hef (located here, https://hailo-model-zoo.s3.eu-west-2.amazonaws.com/ModelZoo/Compiled/v2.17.0/hailo8/yolov11m.hef) for my initial project. YOLO is trained on the Common Objects in Context data set, so I included the names in this file for ease of debugging. If I decide to do more with them later, that list will need to become a parameter.

YOLO models come in different versions, each with different sizes and purposes. The size is indicated by the letter following the version number, and the purpose is indicated by a short description following the size. Detection is the default purpose, so the version 11 medium model for detection is YOLO11m. The latest model version in the HAILO Zoo, as of this writing, is 11 with several sizes available. For edge devices, some research shows that the best trade-off between detection and performance is the m size (Plants 2025, 14(6), 881; https://doi.org/10.3390/plants14060881), among others.

Import the Hailo Platform into the script, and you will now have several modules and objects available. The first object you work with is the vdevice. The vdevice is the virtual device you will configure, and it represents the Hailo device. Once we create the device, we make the configuration object based on parameters in the Hef file and the physical device’s bus type (PCIe or USB). The vdevice takes several other parameters, but all of them come from the model stored in the hef file for your model. You can review the model parameters using hailortcli parse <path to the hef file>.

Next, we create an object for the Network_group using the hef and the config we just created. The vdevice stores the configs in an array, so you could create multiple configurations and network_groups with different models in a vdevice. The config we created is the first in the list so the index is 0. From that object, we create another to hold the network group parameters, which is used later to activate the network group for inference.

Now that the device is configured, we return to the model to retrieve references to the input and output stream information, as well as the input shape. The input shape is a tuple, which was a bit confusing at first because the model expects four members: batch size, height, width, and channels (NHWC), but the input stream information only returns 3: height, width, and channels. At this point in the configuration, the batch size parameter isn’t used, so you leave it out. Later, in the inference function call, we will set the batch size. You can set a batch size and send multiple images or streams to the model in a single call.

Now we are almost to a completely configured vdevice. What is left is to create an object to hold our input and output stream parameters. Creating these objects requires passing the network group we created earlier, the quantized state of the data we will be sending, and the datatype of the objects to be inferred. The datatypes are found using hailortcli run <path to the hef file>, though they are also returned with the parse command. The Quantized state is only returned in the run data as either True or False. Quantization is the technique of reducing the precision of a model’s weights and activations. When done correctly, it reduces the model’s size, speeds up inference, and makes it more energy-efficient. That makes it more compatible with edge devices. It should be true for Hef models.

Getting something from the device

Now that we have the parameters configured on the vdevice, we can activate it and create the infer-pipe. The activation calls are pretty self-explanatory, the direct calls to the activation.enter and infer_pipe.enter methods are used to activate a device when it is assigned to a class-level object.

The purpose of all of that is to get to the infer function. The infer function takes an image to infer, and the confidence threshold you want to apply to the detections. It does some sanity checking on the image and converts it to RGB format, which is what the model expects, from the BGR format that our cv2 captures create. Finally, it adds the missing batch-size dimension mentioned above to the image array and passes the resulting array to the inference engine. It returns the detections after running them through a parsing function that enforces the confidence threshold and formats them in the form I want so I can manage them later.

Closing for now

I feel pretty comfortable that I know who is who in the zoo now (pun sort of intended). The next step in the adventure is to refactor this into a separate module so I can use it to assess performance across the different image processing techniques: full frame, tiling, and motion-detection-based regions of interest.