Cross-Compiling TensorFlow Lite for ZYNQ 7000 (ARMv7-A + VFPv3)

If you’re like me — wrestling with the challenge of deploying TensorFlow Lite on a ZYNQ 7000 board (or any platform without out-of-the-box support) — and you’ve spent frustrating hours digging through forums and GitHub issues, unsure of what to try next — this blog post is for you.

In this guide, I’ll walk you through the exact steps I used to build TensorFlow Lite from source for the ARMv7-A architecture (specifically targeting the vfpv3 floating-point unit), which powers the PYNQ-Z1 board.

Target Device Specifications:

1. Development Board: PYNQ-Z1
2. CPU: ARM Cortex-A9
3. Arch: ARMv7-A (32-bit)
4. FPU: vfpv3

How It Started

When I ran interpreter.invoke() using the default tflite-runtime wheel, it crashed with:

Illegal instruction (core dumped)

Root Cause of the Issue

The error occurs because the prebuilt TensorFlow Lite wheels are compiled for newer ARM configurations, like those using vfpv4 or ARMv8-A instruction sets. Unfortunately, the Cortex-A9 in the ZYNQ 7000 series does not support these instructions, leading to an immediate crash when unsupported ops are executed.

To fix this, we need to cross-compile TensorFlow Lite with the correct compiler flags for ARMv7-A and vfpv3, ensuring binary compatibility with the ZYNQ 7000’s CPU.

Verifying target device specifications:

cat /proc/cpuinfo

Host Environment Specifications

This is the setup I used to build the TensorFlow Lite wheel inside a Docker container:

1. CPU: 8 cores × Intel(R) Xeon(R) CPU E5–2640
2. Architecture: x86_64
3. RAM: 32 GB
4. Operating System: Ubuntu 22.04.1 LTS

Packages

1. Python: 3.10
2. Numpy: 1.26.4
3. TensorFlow: v2.15.0

💡 Note: While this setup worked for me, the build process can be memory-intensive. If you’re using a VM or a system with limited resources, consider increasing your swap size or reducing parallel build jobs (e.g., using make -j1).

⚙Building TFLite wheel

• Install Required Packages

sudo apt update
sudo apt install -y cmake python3-pip
pip3 install pybind11

• Clone TensorFlow source from git, and check out to v2.15.0

git clone https://github.com/tensorflow/tensorflow.git tensorflow_src
cd tensorflow_src
git checkout v2.15.0

• Change the vector floating-point version in download_toolchains.sh file. The file is located in tensorflow/lite/tools/cmake/download_toolchains.sh.

## Before

case $1 in
  armhf)
    if [[ ! -d "${TOOLCHAINS_DIR}/gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf" ]]; then
      curl -LO https://storage.googleapis.com/mirror.tensorflow.org/developer.arm.com/media/Files/downloads/gnu-a/8.3-2019.03/binrel/gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf.tar.xz >&2
      tar xvf gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf.tar.xz -C ${TOOLCHAINS_DIR} >&2
    fi
    ARMCC_ROOT=${TOOLCHAINS_DIR}/gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf
    echo "ARMCC_FLAGS=\"-march=armv7-a -mfpu=neon-vfpv4 -funsafe-math-optimizations \
      -isystem ${ARMCC_ROOT}/lib/gcc/arm-linux-gnueabihf/8.3.0/include \
      -isystem ${ARMCC_ROOT}/lib/gcc/arm-linux-gnueabihf/8.3.0/include-fixed \
      -isystem ${ARMCC_ROOT}/arm-linux-gnueabihf/include/c++/8.3.0 \
      -isystem ${ARMCC_ROOT}/arm-linux-gnueabihf/libc/usr/include \
      -isystem \"\${CROSSTOOL_PYTHON_INCLUDE_PATH}\" \
      -isystem /usr/include\""
    echo "ARMCC_PREFIX=${ARMCC_ROOT}/bin/arm-linux-gnueabihf-"
    ;;

## After

case $1 in
  armhf)
    if [[ ! -d "${TOOLCHAINS_DIR}/gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf" ]]; then
      curl -LO https://storage.googleapis.com/mirror.tensorflow.org/developer.arm.com/media/Files/downloads/gnu-a/8.3-2019.03/binrel/gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf.tar.xz >&2
      tar xvf gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf.tar.xz -C ${TOOLCHAINS_DIR} >&2
    fi
    ARMCC_ROOT=${TOOLCHAINS_DIR}/gcc-arm-8.3-2019.03-x86_64-arm-linux-gnueabihf
    echo "ARMCC_FLAGS=\"-march=armv7-a -mfpu=neon-vfpv3 -funsafe-math-optimizations \
      -isystem ${ARMCC_ROOT}/lib/gcc/arm-linux-gnueabihf/8.3.0/include \
      -isystem ${ARMCC_ROOT}/lib/gcc/arm-linux-gnueabihf/8.3.0/include-fixed \
      -isystem ${ARMCC_ROOT}/arm-linux-gnueabihf/include/c++/8.3.0 \
      -isystem ${ARMCC_ROOT}/arm-linux-gnueabihf/libc/usr/include \
      -isystem \"\${CROSSTOOL_PYTHON_INCLUDE_PATH}\" \
      -isystem /usr/include\""
    echo "ARMCC_PREFIX=${ARMCC_ROOT}/bin/arm-linux-gnueabihf-"
    ;;

“ARMCC_FLAGS=\”-march=armv7-a -mfpu=neon-vfpv3 -funsafe-math-optimizations \

• Build the wheel using docker

make -j1 -C tensorflow/lite/tools/pip_package docker-build \
  TENSORFLOW_TARGET=armhf PYTHON_VERSION=3.10

💡make -j1: Make the build system run only one job (build task) at a time. You can increase for faster build with high memory usage

Build takes 1–2 hrs. After build is completed the wheel files are locate in /tensorflow/tensorflow/lite/tools/pip_package/gen/tflite_pip/python3.10/dist directory.

📦Installing on TFLite on Target:

• Copy the built wheel into the target device.
• Check wheel for compatibility: Unzip the wheel and cross-check the architecture it is built to.

unzip tflite_runtime-2.15.0-cp310-cp310-linux_armv7l.whl -d tflite_unziped
cd tflite_unziped/tflite_runtime
readelf -A _pywrap_tensorflow_interpreter_wrapper.so

💡readelf -A lets you verify that the compiled binary matches your target CPU’s expected instruction set (in this case: ARMv7-A + VFPv3).

💻Output:

Attribute Section: aeabi File Attributes Tag_CPU_name: “7-A” Tag_CPU_arch: v7 Tag_CPU_arch_profile: Application Tag_ARM_ISA_use: Yes Tag_THUMB_ISA_use: Thumb-2 Tag_FP_arch: VFPv3 Tag_Advanced_SIMD_arch: NEONv1 Tag_ABI_PCS_wchar_t: 4 Tag_ABI_FP_denormal: Needed Tag_ABI_FP_exceptions: Needed Tag_ABI_FP_number_model: IEEE 754 Tag_ABI_align_needed: 8-byte Tag_ABI_align_preserved: 8-byte, except leaf SP Tag_ABI_enum_size: int Tag_ABI_VFP_args: VFP registers Tag_CPU_unaligned_access: v6

If this is how your output looks, then you have cross-compiled your TFLite for ARM Cortex-A9 architecture (ARMv7, vfpv3).

• Install the wheel using pip

python3.10 -m pip install ./tflite_runtime-2.15.0-cp310-cp310-linux_armv7l.whl

🧩Conclusion:

Successfully running TensorFlow Lite on the ZYNQ 7000 takes a bit of low-level elbow grease — especially when prebuilt binaries aren’t compatible. But once you tailor the build to your target architecture, it opens the door to efficient ML on embedded platforms like the PYNQ-Z1.

If you found this helpful or ran into any issues, feel free to connect or drop a comment below. I’d love to hear how it went for you. Cheers!

🔗 LinkedIn