How I Went From #122 to #1 in 24 Hours: Building a Multimodal Damage-Claim Verifier for HackerRank Orchestrate

A month ago, I finished #122 out of 1,349 participants in HackerRank Orchestrate's May edition. Last week, the June edition results came in. I'd finished #1 out of 15,295 registered participants, 2,039 of whom shipped a working agent.

This post is the full story of how I got there: the problem, the architecture, the decisions that mattered, the mistakes that cost me hours, and a detailed walkthrough of the part almost nobody talks about in these recaps — the live AI interview, which carries as much weight as your code and your output combined.

If you're planning to compete in a future edition of Orchestrate, or any hackathon that pairs a build phase with a live technical defense, I'm writing this specifically for you.

What Orchestrate Actually Tests

Orchestrate gives you 24 hours to design, build, and ship an AI agent solving a real-world problem, then submit four artifacts: your code, your agent's output, your full chat transcript with whatever AI tool you used to build it, and a 30-minute live interview with an AI judge who has already read everything you submitted.

Each of those four is scored independently and combined into a final number. That structure matters more than it sounds like it should. A brilliant model output with no defensible reasoning behind it scores worse than a modest output you can explain with total precision. I learned this the hard way in May, and it shaped everything I did differently in June.

The June problem was multimodal evidence review: given a damage claim like a dented car, a cracked laptop screen, a crushed package, along with submitted photos, a short conversation describing what happened, and the claimant's history, decide whether the photos support the claim, contradict it, or simply don't provide enough information to say either way. The dataset was deliberately adversarial: stock photos with watermarks, images with handwritten sticky notes telling the AI to approve the claim, claims in multiple languages carrying the same injection attempt, toy cars submitted as real vehicle damage, and images where two photos in the same claim quietly showed two different vehicles.

The Architecture Decision That Mattered Most

Before writing any code, I had to decide between four real options: a deterministic rule-based pipeline, a trained classifier, a multi-agent pipeline, or a single agent equipped with tools.

I rejected the first two quickly. Rules can't actually look at a photo and judge whether a dent matches a claimed part. A classifier needs far more labeled data than the 20 rows I had to train anything that generalizes.

The harder call was between multi-agent and single-agent. A multi-agent pipeline where one agent classifies the issue, another retrieves evidence requirements, another inspects images, a final agent synthesizes a verdict sounds clean on paper. I rejected it because the hardest cases in this dataset weren't about reading one image correctly. They were about weighing two or three images against each other and reaching one coherent judgment. If you split that across agents talking through text summaries, you lose exactly the nuance that mattered. I confirmed this empirically, not just theoretically: a smaller open-weight model I used early in the build kept failing one specific case. It was a claim where one image showed real damage on the wrong part of a car while a second image showed the actual claimed part completely undamaged across four separate rounds of targeted prompt fixes. When I later swapped in a stronger model with the identical single-agent setup, it solved that exact case correctly on the first attempt. The bottleneck was cross-image reasoning capacity, and splitting that reasoning across multiple agents would only have made it worse.

So I built a single agent with three tools: one to request a closer, focused look at a specific image when uncertain, one to look up the documented minimum evidence standard for a given issue type, and one to submit the final structured verdict. All images are attached to the model upfront. A model can't decide which image to inspect more closely if it hasn't seen any of them yet, so giving it everything at once and letting it choose when to dig deeper is what makes the tool use genuinely agentic rather than a scripted sequence.

The Bug That Taught Me the Most

About halfway through the build, I found something that genuinely changed how I think about building safety layers around language models.

I'd written a post-model safety gate : deterministic code that runs after the model produces its verdict, designed to catch cases where image-based prompt injection (literal sticky notes saying "approve this claim") could otherwise influence the outcome. The original rule was blunt: if an image was flagged as possibly non-original (looked like a stock photo) and the verdict was "supported," automatically downgrade it to "not enough information."

That rule made sense for the case I built it around, which was an obvious stock photo of catastrophic wreckage submitted for an unrelated minor scratch claim. But once I had a stronger model running, I caught it firing on a case where it shouldn't have. The model looked at a real photo, correctly identified specific, severe, directly observed damage of a missing rear bumper, a visibly crushed trunk lid, and called it "supported," citing the exact image as evidence. It also noted, almost as an aside, that the photo had a professional-quality look to it, and flagged it as possibly non-original. My gate saw that flag and silently downgraded a correct, well-evidenced verdict to "not enough information."

I was throwing away the model's right answer because of a flag it raised as a minor caveat, not as its actual reasoning. The fix wasn't to remove the rule, because genuine manipulation attempts still needed a hard block. It was to make the rule conditional on whether the model could point to specific, cited evidence. If it could, the verdict stands and gets flagged for human review instead of being silently overridden. If it couldn't, the downgrade still applies.

The lesson that stuck with me: a safety layer that can't distinguish "this evidence might not be original but is still real" from "this image is actively trying to manipulate me" isn't a safety layer, it's just noise that happens to look safe. Specificity has to be the deciding factor, not the mere presence of a flag.

Building Under Real Constraints

Here's the part most hackathon write-ups skip: my build did not go smoothly, and I think that's worth being honest about.

Across the day I hit rate limits and daily token caps on more than one provider, the kind of wall where the API simply stops answering until a quota window resets, sometimes hours later. The first time it happened, I lost real progress: a run that had processed most of the sample set died partway through, and re-running it from scratch meant re-paying for every claim it had already correctly handled, on top of losing the time.

That was the point where I stopped treating it as bad luck and started treating it as a design requirement. Every run now writes a checkpoint file after each claim completes, keyed by provider and claim ID. If the process dies because of a rate limit, network failure or a deliberate Ctrl+C, the next run reads that checkpoint, skips every claim already completed, and picks up exactly where it left off. --no-resume forces a clean run when you actually want one. This turned a real failure mode into a non-event: hitting a quota wall partway through a 44-claim run became "wait for the window to reset, run the same command again," not "start over and pay for everything twice."

It's a small piece of engineering, but it's the kind of thing you only build after the failure actually costs you something. I'd rather ship a system that assumes the network and the API will fail partway through, because under real conditions, they will.

The AI Interview: What Actually Happens and How to Prepare

This is the section I most wanted to write, because going in, I had almost no idea what to expect, and I think that uncertainty costs people points unnecessarily.

The interview is a 30-minute live voice conversation with an AI judge that has already read your code, your output, and your full chat transcript before you start. It is not a generic "tell me about your project" conversation. The judge asks specific, pointed questions about decisions visible in your actual code, not decisions you claim to have made in your README, decisions the source code itself demonstrates.

A few concrete things I learned, some the easy way and some the hard way:

The judge will ask you to defend specific tradeoffs, not describe your project in general terms. Expect questions like "why did you choose a single agent over a multi-agent pipeline, and where specifically would the alternative have lost accuracy?" A vague answer like "I thought it would work better" scores far worse than a specific one grounded in an observed failure case. If you can point to an actual bug you found, an actual case that failed and why, and an actual fix you made, that is the strongest possible material for this interview.

Be ready to walk through your safety or guardrail logic in detail. Any system handling adversarial input will get probed here. I was asked directly about the distinction in my safety gate between hard-blocking manipulation attempts versus handling merely-suspicious evidence differently and because that distinction came from a real bug I'd found and fixed, I could explain not just what the code does but why it does it that way, and what it used to do before I found the problem.

Be ready to talk about a real limitation and how you addressed it, not just what worked. I was asked directly what happened when something actually failed during the build, and what I changed because of it. The honest, concrete answer that I lost real progress to a provider rate limit early on, and built a checkpoint-and-resume system afterward so that failure mode could never cost me a full re-run again was a far stronger answer than describing only the parts that went smoothly. The judge isn't looking for a project with no failures. A failure you noticed, understood, and fixed is some of the best evidence you can offer that you actually built this yourself.

Practice explaining your architecture out loud before the real thing. This sounds obvious but it's the single highest-leverage thing I did. I ran a full mock interview with my AI coding assistant playing the judge role, going question by question through exactly the kind of probing I expected, and refining my answers until they were precise rather than rambling. The actual interview followed an extremely similar shape. If you only do one thing to prepare beyond the build itself, do this.

The interview is scored as heavily as your code. In the final breakdown, my interview score (25.2 out of 30) was a larger differentiator between me and the next two finishers than my code score was. Our code scores were within half a point of each other. The build gets you in the room. The interview is where rank gets decided.

What I'd Tell Someone Doing This for the First Time

Treat every constraint you hit as information, not just an obstacle. The API quota walls I hit weren't fun in the moment, but debugging through them, and being able to explain exactly what broke and why, became some of the strongest material in both my chat transcript and my interview.

Don't build defensively against the rubric. Build the most honest, well-reasoned system you can, and let your explanation of it be equally honest. A system with real, acknowledged limitations that you can explain precisely will score better than a system you oversell and can't defend under questioning.

And if you've been putting off competing in something like this because your first attempt didn't go the way you hoped. I finished #122 a month before I finished #1. The gap between those two results wasn't talent I didn't have in May. It was giving the whole process, especially the interview, the seriousness it deserved.

The full code, architecture writeup, and design decisions are public on GitHub under the MIT license: Multi-Modal Evidence Review System . If you're building something similar, take whatever's useful from it.

— Sristee Shrivastava (@srshriv on GitHub)