Telling the Model How It’s Graded: NTU + Alibaba’s MeRF

Telling the Model How It’s Graded: NTU + Alibaba’s MeRF

In this ICLR 2026 paper from Dacheng Tao’s group at Nanyang Technological University and Alibaba’s Tongyi Lab, the authors do something deceptively simple when training reasoning models with reinforcement learning: they write the grading rules directly into the prompt and just tell the model how it’s going to be scored.

The mainstream recipe for training reasoning models today is RLVR — reinforcement learning with verifiable rewards. The answers come in formats you can check with rules, so scoring can be automated. But the whole process is trial-and-error. At the start, the model has no idea what kind of response earns points. It just keeps generating, hoping to stumble onto something, and slowly extracts patterns from the scattered reward signals it gets back.

The authors put the bind nicely: you’re being asked to learn something you don’t even know exists. If you never once luck into the right answer, that high-scoring behavior is something you can never learn.

So here’s the move. Since the reward is defined by rules, those rules can obviously be written out in plain language. And large models are natural in-context learners — they read the instructions in a prompt and follow them. So why not just tell the model the scoring rules outright? That’s MeRF: take the text description of the reward function and drop it into the system prompt as a kind of motivation. Something like “correct answer earns 2 points, wrong answer loses 1.5, get the format right and earn 1 more.”

Now the model is pulled by two forces at once: the real external reward, and its own internal understanding of the rules.

One contrast I find telling: if you show the base model these rules but don’t do any reinforcement learning, it doesn’t get better. So the prompt hint alone does nothing. What actually works is the pairing — “rule hint” plus “RL training” together. Another nice detail: at validation time the motivation passage isn’t even in the prompt, and the model performs just as well. That means it internalized the rules into a capability, rather than leaning on the hint.

On results, across a stack of tasks — from Qwen2.5’s 7B and 14B to DeepSeek-R1-Distill’s 1.5B and 8B — MeRF consistently beats vanilla RLVR, and it opens up the gap in just a few hundred steps.

On the ablation side, the authors set the motivation to three settings: one that perfectly matches the true reward, one that only mentions right-versus-wrong without the formatting bonus, and one that deliberately flips all the scores to mislead the model. The result: the more the motivation agrees with the true reward, the better it works. And even when fed the wrong rules, the model — after a few rounds of oscillation — somehow learns on its own not to trust the misleading version, even reads it as reverse-speak, and still beats the baseline.

| Find papers faster on arXivSub with AI summary (CVPR/ICCV/ICML/ICLR/NeurIPS/AAAI/MICCAI)