What we built instead: a kernel that pays for itself

We did not build a chatbot that knows about buildings. We built a harness tuned end to end for commercial real estate, and we let the commodity models plug into it. One rule drives every decision I make about this system: for every question and every workflow, the most accuracy, the best performance, and the fewest tokens. Here is how it wins that (Some of the sauce. Not all of it.)

1. A real data model underneath. Every property and lease is a typed entity. Lease terms are structured, confidence-scored fields, not free text the model re-reads every turn. The model is not re-deriving the lease each time. It is reasoning over a clean, queryable model of the portfolio. We standardized the data before scaling the AI, which is what every serious industry report has been asking people to do exactly (PRODA 2026 data trends).
2. Retrieval that’s been to war (v1 → v3). Generic RAG dumps chunks and hopes, multi-dimension vectors, score cutoffs, chunk-vs-sub-chunk strategies tuned to leases, invoices, and financials. It pulls golden chunks with the proof snippet attached. Fewer, better passages means fewer tokens and higher accuracy at the same time. The point of the version history is the lesson in it: you cannot shortcut your way to a mature retrieval layer. You earn it by watching the earlier versions fail in specific ways.
3. Memory and context engineering. We keep a structured session memory, a rolling summary instead of raw transcript, active scope (which entity/portfolio you’re in) with a confidence score, remembered golden chunks so it can reuse evidence instead of paying to re-fetch it, data-room memory that switches to statistics (min/max/avg, distinct counts) when the table gets big instead of dumping raw rows, generated-file memory that keeps small outputs inline and large ones in cold storage, and compaction telemetry that measures token cost before and after and flags when memory degrades. Warm digests, cold raw evidence. The result: a follow-up often skips a brand-new search entirely because the harness already remembers where the answer came from. That’s the single biggest lever on the bill. This is a harness capability, not a model one.
4. Tools that are commercial real estate verbs, not demo toys. Natural language to SQL over the data room, accounting report generation, anomaly detection, completeness validation, due-diligence kickoff, and a CAD and DXF construction takeoff pipeline that reads drawings, clusters the labels, and builds an evidence-backed report. These are the verbs of the industry, with verification built into each one, not a generic “code interpreter” you have to babysit.
5. Your expertise becomes the agent’s. Customers encode their own domain knowledge as playbooks and structured skills. Your asset manager’s process for auditing a base-year stop becomes a capability the harness runs the same way, every time, across the whole portfolio, instead of overwriting it with a generic prior.
6. The right model for the job, not the biggest one. We route across a tiered model matrix, a fast default for most turns and a heavier reasoning model only for deep research, with timeouts and retries. The model is a swappable part. When the next frontier model ships, it plugs into the same kernel, everything improves, and we rewrite nothing.
7. Trust as a first principle. Human-in-the-loop confirmation gates (cryptographically signed, single-use, replay-protected) on high-impact and bulk actions, prompt-injection shielding on every conversation before it reaches the model, full cost accounting per run, and no model-log retention. In CRE, “oops” isn’t acceptable, so verification isn’t a feature bolted on, it’s part of the kernel.
8. We measure all of it. An offline evaluation harness with LLM-as-judge, answer confidence, runtime, and success across retrieval versions and search modes, on every release. We don’t believe the harness is better. We regression-test that it is, every release. This is the lab bench. Without it, every “improvement” is a guess.

That is the kernel. Not a demo, not a wrapper, not a subscription to someone else’s intelligence. Every layer exists because we measured what happened without it. The harness pays for itself because it earns it, in tokens saved, in answers that are actually right, in workflows that run without a human checking the output and hoping. The benchmark post is coming. Same models, same workloads, very different results. That will be the receipts.