Bigger Isn’t Always Better: Gemma 31B vs Ministral 14B on a Pivot Table

In the last article I tested Sparrow’s Standard mode (Ministral 14B) against a genuinely hard case: a Solvency II regulatory pivot table from Aviva’s 2022 Solvency and Financial Condition Report. Ministral 14B handled it well, correctly mapping every line-of-business column against its row categories.

This time I ran the same table, same generic “*” query, no schema, through Sparrow’s Advanced mode — backed by Gemma 4 31B Dense, over twice the parameter count of Ministral 14B.

The result was worse.

Take the “Premiums written — Net” row (R1500). Ministral 14B correctly placed the Health insurance value under column C0210:

"R1500": {
  "C0210": 441387
}

Gemma 31B nulled that same column, then shifted the real value one column over — into C0220, which it mislabeled as “Insurance with profit participating feature”:

"C0210": null,
"C0220": "441,387"

This wasn’t a one-off. The pattern repeated down every row: C0210 came back null throughout the entire table, and the whole column mapping shifted right by one, with column IDs and their labels no longer matching the source document. A missing column (C0230) also silently disappeared from the output rather than showing up as null.

Ministral 14B didn’t make this mistake. It’s a smaller model, running locally via MLX-VLM in Sparrow’s Standard tier, yet it stayed faithful to the actual grid structure where the larger model drifted.

The lesson: model size isn’t a proxy for fit. On dense tabular layouts, what matters is how well a model sticks to structure rather than how many parameters it has. “Advanced” doesn’t automatically mean “more accurate” — it’s worth testing both tiers against your own documents before assuming the bigger model wins.

Video: https://www.youtube.com/watch?v=w9ovXGBxA70

GitHub: https://github.com/katanaml/sparrow

Live instance: https://sparrow.katanaml.io