rp1 on EmDash, Part 3: Hunting a Race Condition You Can’t Reproduce in a Test

rp1 Build Series · EmDash · Part 3: Bug Fix

*← Part 2: Onboarding*

Ask an AI agent to find a bug and you usually get one of two outcomes: a confident answer that points at the wrong file, or a long exploration that circles back to “it could be any of these things.” Neither is useful when someone is staring at a production incident. The problem is not the model’s capability — it is the absence of structure. Debugging is a hypothesis-driven process. Without explicit hypotheses, explicit verification steps, and a record of what was found where, you get plausible-sounding guesses instead of root causes.

Part 2 built a feature from scratch: requirements, design, implementation, review. The workflow was generative — we were adding something that did not exist.

Part 3 is the opposite. We start with something broken and work backward to find out why.

The bug is issue #446 in the EmDash GitHub repository. The symptom: a user edits a field inline using the visual editor, clicks Publish immediately after, and the change does not appear in the published version. No error. No warning. The save indicator clears normally. The content published is just not the content they typed.

This class of bug — a timing race between two concurrent network requests — is among the hardest to reproduce reliably in tests. The window is typically 50 to 500 milliseconds. In a local dev environment with a fast database, it usually closes before publish fires. In production, under load, it does not.

The right tool here is not grep and guess. It is a systematic investigation workflow.

/code-investigate: Hypothesis-Driven Fault Tracing

Before looking at a single line of code, /code-investigate structures the investigation as a set of explicit hypotheses. Each hypothesis is a specific, testable claim about where the bug lives. The agent confirms or refutes each one with direct code evidence: file paths, line numbers, the actual code, and the reasoning.

This is not the same as asking Claude “where is the bug?” in a chat session. That produces a plausible narrative. This produces a structured investigation report with line-level citations for every claim, which you can verify independently by reading the same lines.

The workflow ends with a structured report: root cause with evidence, a step-by-step description of how the bug manifests, a proposed fix with effort estimate, and identified caveats. Not a guess. A dossier.

The Issue: A Race with Three Cooperating Parts

Issue #446 describes the symptom. The investigation needed to explain the mechanism. Three hypotheses were formed and tested in order.

Hypothesis 1: handleBlur() discards the save Promise

The visual editor allows inline editing by clicking directly on rendered content on the page. When the user finishes and tabs away or clicks elsewhere, a blur event fires. The handleBlur() function at toolbar.ts:760 captures the edited text, then calls saveField() to persist it.

saveField() returns a Promise — it fires a PUT request to the server. The question is: what does handleBlur() do with that Promise?

toolbar.ts:768  var newValue = (element.textContent || "").trim();
toolbar.ts:769  if (newValue !== originalText.trim()) {
toolbar.ts:770    saveField(annotation.collection, annotation.id, annotation.field, newValue);
toolbar.ts:771  }

The return value of saveField() is discarded at line 770. The Promise is created, the network request begins, and the reference to the pending operation is immediately dropped. There is no way for any other code to know a save is in flight.

Hypothesis 1: confirmed.

Hypothesis 2: publish() fires immediately with no in-flight check

The publish() function at toolbar.ts:626 handles the Publish button click. If it checks whether a save is pending before firing its own POST request, the race cannot occur.

toolbar.ts:626  function publish(collection, id) {
toolbar.ts:627    publishBtn.disabled = true;
toolbar.ts:628    publishBtn.textContent = "Publishing…";
toolbar.ts:630    ecFetch("/_emdash/api/content/.../publish", {
toolbar.ts:631      method: "POST",
                    ...

No check. The POST fires on line 630, immediately after the button state updates. There is already a saveState variable in scope — declared at toolbar.ts:531 as var saveState = "idle", set to "saving" during a saveField() call. publish() never reads it.

Hypothesis 2: confirmed.

Hypothesis 3: The server publishes whichever revision the database holds at that moment

On the server, publish() in content.ts reads the current state of the content row and promotes the best available revision to live. The question is how it decides which revision to use.

content.ts:903  const existing = await this.findById(type, id);
content.ts:909  let revisionToPublish = existing.draftRevisionId || existing.liveRevisionId;

findById() reads from the database at the moment the publish POST arrives. If the save PUT has not yet committed its transaction, draftRevisionId is still null (or points to an older draft). The fallback on line 909 then selects liveRevisionId — the last published version — and promotes it again. The user's edit is not in the published result.

Hypothesis 3: confirmed.

How the Race Unfolds

With all three components confirmed, the investigation report described the full race sequence:

The window is the network round-trip for the save PUT. On a fast local connection this is 10 to 50 milliseconds. On a production server with a loaded database, it stretches to 200 to 500 milliseconds — well within reach of a user who clicks Publish immediately after finishing an edit.

The investigation report also noted an additional nuance: collections with usesDraftRevisions = false (which write directly to content columns rather than creating revision rows) are affected by the same race. The server's findById() call reads whatever the database holds at that instant, whether or not the PUT transaction has committed. The fix is client-side and collection-type agnostic.

What Made This Possible Without Prior Knowledge

The investigation started with a GitHub issue description and a codebase the agent had not read. Two things made it fast.

The first was the knowledge graph from Part 1. The architecture.md file in .rp1/context/ documents the middleware chain and the visual editing subsystem. The modules.md file references toolbar.ts as the browser-side visual editor and content.ts as the content repository. The agent loaded these files before reading source, which meant it knew where to look before forming its first hypothesis.

The second was the .then() pattern that already existed in toolbar.ts. Lines 948 and 1114 — both in image editing paths — correctly chain onto saveField()'s return value:

toolbar.ts:948   saveField(collection, id, field, null).then(function() {
toolbar.ts:949     if (imgEl) { imgEl.style.display = "none"; }

handleBlur() is the only call site that discards the Promise. The fix was already demonstrated, twice, in the same file. The investigation report cited these lines explicitly: "The pattern already exists at lines 948 and 1114. handleBlur() is the only callsite that drops the Promise."

This kind of observation — noticing what is done correctly elsewhere and identifying the single place where the pattern is violated — is exactly what systematic code tracing produces. It is harder to see in a grep session because you are looking for the bug, not for the correct usages that surround it.

The Fix: Three Changes, One File, Ten Lines

The fix requires no server-side changes. The race is a client-side coordination failure: publish() does not know a save is in progress because no one told it.

The solution is a single pendingSavePromise variable. The toolbar already manages one content entry at a time, so one variable is sufficient. No map, no counter, no locking mechanism.

Change 1 — declare the variable near saveState:

// toolbar.ts:531 — near existing var saveState = "idle";
var pendingSavePromise = null;

Change 2 — store the Promise in handleBlur() and clear it on settle:

// toolbar.ts:770 — was: saveField(...);
pendingSavePromise = saveField(annotation.collection, annotation.id, annotation.field, newValue).then(function() {
  pendingSavePromise = null;
}, function() {
  pendingSavePromise = null;
});

The two-argument .then(onFulfilled, onRejected) form is used because toolbar.ts is ES5 browser JavaScript embedded as a raw string inside a TypeScript file. No const, no arrow functions, no async. The rejection handler clears the reference without re-throwing, which means pendingSavePromise always resolves. This is intentional: a failed save does not permanently block publish.

Change 3 — chain publish() onto pendingSavePromise if set:

// toolbar.ts:626 — at the top of function publish(collection, id)
if (pendingSavePromise) {
  pendingSavePromise.then(function() { publish(collection, id); });
  return;
}

If a save is in flight when the user clicks Publish, publish() returns early and queues itself on the pending save's resolution. Once the save PUT commits and pendingSavePromise resolves, publish() calls itself again — this time with pendingSavePromise === null — and fires the POST normally.

The fix is 13 lines across three locations in one file. No server changes. No new dependencies. No migration.

The Second PR Review: Catching What Slipped Through

After committing the fix, /pr-review ran again — this time covering both the race condition fix and the word-count plugin from Part 2 together. The review surfaced three findings against the two commits.

Finding 1 (Medium) — Dismissed as intentional. The reviewer flagged that pendingSavePromise always resolves (because the rejection handler returns normally rather than re-throwing), so publish() fires whether the save succeeded or failed. This is correct behavior: if the save failed, the user has already seen an error indicator in the UI and their explicit Publish click should still be honored. The visualizer that ran as part of the review confirmed this in its state diagram note: "a failed save does not permanently block publish." Dismissed.

Finding 2 (Medium) — Fixed. The WordCountWidget React component from Part 2 included onChange in the useEffect dependency array. If the parent component passes a new onChange reference on each render — common when the callback is defined inline — the effect fires, which updates form state, which triggers a re-render, which creates a new reference, which fires the effect again. Infinite loop. Fixed by holding onChange in a useRef and depending only on [count]:

const onChangeRef = React.useRef(onChange);
onChangeRef.current = onChange;
React.useEffect(() => {
  onChangeRef.current(count);
}, [count]);

Finding 3 (Low) — Fixed. The content:afterSave hook derived contentId with a fallback to empty string (?? ""). An empty string passed to ctx.content.update() would execute a WHERE id = '' predicate that silently matches zero rows. Added a one-line guard: if (!contentId) return;.

Two fixes, one commit. The review’s judgment moved from request_changes to effectively approved once the changes landed.

What /code-investigate Changes About Debugging

Debugging is mostly not about cleverness. It is about not skipping steps. You form a hypothesis, you check the code, you record what you found, you move to the next hypothesis. The bugs that take days to find are the ones where someone skipped the middle step — checked the code quickly, did not find what they expected, and moved on assuming the hypothesis was wrong.

/code-investigate enforces the steps. Every hypothesis is stated explicitly before the code is read. Every finding is recorded with a file path, a line number, and the exact code as evidence. The investigation report is a document you can hand to someone who was not in the session and they can follow the reasoning end to end.

The difference is not speed on the first investigation. It is what happens on the second investigation, six months later, when a different engineer opens the same file and wonders why pendingSavePromise exists. The investigation report is in .rp1/work/issues/446/. The reasoning is on disk.

Closing the Loop on the Series

Part 1 built the map. Part 2 used it to build a feature end to end. Part 3 used it to trace a bug the test suite cannot reproduce.

In each case, the workflow is the same: structured inputs, specialized agents, persistent artifacts, explicit checkpoints. The output is not just code or a fix. It is a record of how the decision was made, traceable back to the evidence that justified it.

*rp1.run/getting-started · AI Engineer Melbourne, June 3–4 2026*