Vision Models Speed Test, Hosted on Tesla on AWS

This article is for people looking to run vision-related tasks on large amount of models. Instead of using an LLM endpoint like Anthropic and incurring costs, you can choose to deploy your own smaller models and only incur the (ever-decreasing) instance cost.

Ever since they came out, I wanted to deploy an open vision model and see their performance for real-life tasks. I appreciate the self-reported metrics for programming tasks and answers to general questions, but they are not always relevant to what a business might need.

This is a bit of an older article, it is actually a few months old. Right now, for the same task, I am using Gemma4 E2B (“Edge”, 2 Billion Parameters). But there is an important lesson here: smaller models are faster, and in fact, Qwen2.5-VL is even faster (despite taking more place in memory) than Qwen 3-VL. The lesson: once you have a reasonable level of accuracy, you can increase speed by choosing the right model, or even by downgrading. This is probably worth the effort for larger or continuous tasks.

Here, the task is to read the calories for the photo of a nutrition label. These labels are standard in most of North America.

How fast does the vision model read the amount of calories in one serving?

The only models that work reasonably well in this task require at last 16 GB of VRAM. I first tried some other models on 6GB VRAM, they worked fine for one photo, but failed with others. I then switched to a cloud deployment with 32GB of VRAM (EKS on gd4n.2xlarge instances with Tesla T4 chips). This lets me use some larger models. All models I am listing did fairly good on this task — I’ll have another article about their accuracy. I used 12 different photos.

Here are the response time collected from model server logs (ollama) from common models on 32 GB of VRAM, on Tesla T4:

MiniCPM-V 8B: 317 ms (avg), 239 ms (median), < 249 ms 95% of the time. (N=635)

Llama3.2-Vision 11B: 1.54 s (avg), 854 ms (median), < 1.01 sec 95% of the time. (N=762)

Qwen2.5-VL 7B: 1.01 s (avg), 895 ms (median), < 1.17 sec 95% of the time. (N=663)

Qwen3-VL 2B: 1.85 s (avg), 1.74 s (median), < 2.06 sec 95% of the time. (N=663)

Qwen3-VL 4B: 2.36 s (avg), 2.27 s (median), < 2.62 sec 95% of the time. (N=1326)

Qwen3-VL 8B: 3.18 s (avg), 2.98 s (median), < 3.71 sec 95% of the time. (N=663)

So, MiniCPM-V is a clear winner, and Qwen2.5-VL, despite being an older generation, is a good contender.