Part 16: Human and Agent: Interaction Design and the Practice of Building Agents

A series on Generative AI Agents and Applications: Table of Content

By this point in the series the lit-review agent is a capable machine. It can plan a survey, fan out across sub-agents, retrieve and verify claims against a knowledge graph, hold a budget, observe its own runs, and abstain when it is unsure. Hand it to a working researcher and it can still fail completely. Not because the engine is weak, but because the person cannot tell what it is doing, cannot stop it before it writes forty files to the wrong folder, does not know whether to trust the citation count it just printed, and assumed it could read paywalled PDFs when it never could.

None of those are model failures. They are interaction failures. Capability lives in the model and the harness, everything the previous fifteen parts built. Usability lives somewhere else: in the seam between the agent and the person using it. That seam is a design surface with its own decisions, when the agent acts on its own and when it stops to ask, what it shows of its work, how it sets expectations, and whether a person can calibrate their trust in it. This article is about that surface, and it doubles as the practitioner capstone: every discipline the series practiced shows up here one last time, as an interaction decision.

1. Where the human sits relative to the loop

Part 1 defined the agent as a model in a control loop: sense, decide, act, observe, repeat. The first interaction question is simply where the human stands relative to that loop. There are three stances.

Human-in-the-loop. The agent pauses before a consequential action and cannot proceed without explicit approval. The person is a required step inside the cycle. Human-on-the-loop. The agent acts autonomously, but a person monitors and can interrupt. The human is alongside the loop, not inside it. Human-out-of-the-loop. The agent runs to completion and the person sees only results.

Figure 1. The same control loop, three stances. The human moves from outside the loop (sees only results), to alongside it with an interrupt, to inside it as a required approval gate. The right stance is chosen per action by reversibility and stakes, not set once for the whole agent.

The engineering point, and it echoes the agents-versus-workflows distinction from Part 1 and Part 7, is that the stance is not a global setting. It is per-action, and it follows from reversibility and stakes. Reading a paper is out-of-the-loop: cheap, reversible, no reason to interrupt. Drafting a summary is on-the-loop: the person glances at it and can stop a bad run. Deleting a file, overwriting prior work, or sending an email is in-the-loop: irreversible enough that a person should sign off first. Almost every real agent is a mixture, and the design work is drawing the line action by action, not picking one stance for everything.

2. Approval and confirmation flows

The in-the-loop stance has concrete mechanics, and getting them right is its own small discipline. A good approval gate does three things. It fires on the right actions, the consequential and irreversible ones, not every step. It shows enough context to decide: the actual file path, the actual recipient, the actual diff, not a vague “the agent wants to do something.” And it offers a real choice, approve, edit, or reject, rather than a rubber stamp the person learns to click through.

This is not a new primitive. An approval gate is a pre-tool hook from Part 8 with a human placed in the decision. The hook already intercepts the call before it fires and can return a structured block; the only addition is that the block now waits on a person instead of resolving from a rule. Everything Part 8 said about hooks as guarantees rather than wishes applies directly: a prompt line asking the model to “please confirm before deleting” is a wish, and an approval gate wired into the harness is a guarantee.

Figure 2. An approval gate is a Part 8 pre-tool hook with a person in the decision. It fires only on consequential actions, shows the concrete context needed to judge (recipient, path, diff), and offers approve, edit, or reject. A rejection returns to the agent as a structured block, exactly as any hook would.

Two failure modes bracket the design, and naming them is half the work. Too many confirmations produce alert fatigue: a person asked to approve every trivial step stops reading and approves blindly, which is worse than no gate at all because it manufactures the appearance of oversight. Too few let the agent do something irreversible the person would have stopped. The cure for both is the same axis from Section 1: gate most insistently exactly where the action is irreversible or the stakes are high, and nowhere else. For the lit-review agent, save_to_file inside output/ is auto-approved, while writing outside that directory, or overwriting an existing file, requires confirmation. The hook from Part 8 already drew that boundary for safety; here the same boundary becomes an interaction decision.

3. Mixed-initiative interaction

The three stances describe where control sits at a given moment. Mixed-initiative interaction describes the more interesting case where it moves. Neither party holds the wheel the whole time: the agent proposes a plan and the person edits it, the person starts a task and the agent asks a clarifying question, the agent handles the routine steps and escalates the genuine judgment call. The design question is who holds initiative at each step, and whether the handoff is explicit rather than silent.

The plan from Part 7 is the canonical mixed-initiative artifact. It was introduced as structured todo sub-goals, working memory the harness re-surfaces near the end of a long window. Make that same plan visible and editable and it becomes a place where initiative changes hands: the agent drafts the survey plan, the person reorders or removes a subtopic, the agent executes the revised version. The plan was already an object in the system; the interaction design is just choosing to show it and let a hand reach in.

The clarifying question is the other key primitive, and it is harder than it looks because it has a cost. Every interruption spends the person’s attention, so the judgment is whether the question is worth more than a reasonable guess. The restraint principle cuts both ways. Do not ask when you can reasonably proceed: an agent that confirms every inference trains the person to ignore it, the under-initiative failure. Do not proceed when a wrong guess is expensive: an agent that silently assumes “the transformer paper” means the 2017 one, and surveys the wrong literature for an hour, has over-stepped. The same reversibility-and-stakes axis that placed the human in or out of the loop also decides when a clarifying question earns its interruption.

4. Progressive disclosure of agent actions

An agent that does real work generates a lot of it. The orchestrated lit-review run from Parts 7 and 8 plans, spawns three workers, makes a dozen tool calls each, retrieves and verifies, synthesizes, and saves. Part 11 captured all of that as a trace. The mistake is to believe the person wants the trace. Dump the full tool-by-tool record in front of a researcher and the one fact that matters, the worker that returned zero papers, is buried under forty lines of routine calls.

This is lost in the middle from Part 2 and Part 6, now aimed at human attention instead of the model’s context window. The same phenomenon, the signal in the middle of a long sequence gets the least attention, applies to a person scrolling a log. The answer is the same one Part 8 used for skills, where only what is needed enters context: show only what is needed, and let the person drill down for the rest.

Figure 3. Progressive disclosure has altitudes. The default is a one-line summary. One level down is the structured plan with the current step marked, the natural top of the hierarchy because Part 7 already produced it. The full per-worker trace from Part 11 is there for anyone who wants it, but it is the bottom of the ladder, not the front door.

So the lit-review agent’s default view is a single line: “Surveying 3 subtopics, 12 papers retrieved, synthesizing now.” One level down is the plan with the current step highlighted, the structured todo from Part 7 doing exactly the job it was shaped for. The full per-worker trace is one more click down, available to anyone debugging or auditing but never forced on a person who just wants the survey. Legibility at the user's altitude is the goal, with depth on request, not depth by default.

5. Transparency, explanation, and provenance

These three words get used interchangeably and they should not be. Transparency is the agent showing what it did, the disclosure of Section 4. Explanation is the agent saying why it did it. Provenance is the agent showing where a fact came from. They are not the same, and they are not equally trustworthy.

Provenance is the most concrete and the most underrated, and it traces straight back to Part 5 and Part 14. A tool’s structured error contract and a graph’s CITES edge were introduced as signals the agent reasons over internally. Promoted to the surface, they become provenance the person can inspect. Every claim the agent presents should carry its source, a retrieved chunk, a tool result, a specific graph edge, or be marked plainly as the model's own generation. The lit-review agent should never print "this paper has 1,200 citations" without the person being able to see whether that number came from the Semantic Scholar API or from the model's prior.

explanation (weaker):

“I included this paper because it is highly influential and central to the topic.”

This is generated text. It is a plausible-sounding rationale produced by the same next-token mechanism that produces everything else, and it can be fluent and wrong at once. It points at the model’s own narration, which is not evidence.

points at the model’s narration

provenance (stronger):

“Citation count: 1,204 (source: Semantic Scholar API, fetched 2026–06–17). Cited-by edge to your seed paper: verified in graph.”

This points at an external artifact the person can check. It is not a story about why; it is a pointer to where. A wrong provenance is falsifiable in a way a wrong explanation is not.

points at a checkable external artifact

The honesty note, in the series register: explanations are themselves generated text. Part 2 established that, and Part 7 sharpened it, the model’s stated “Thought” is not a window into its true reasoning, it is more text predicted to look like reasoning. So a post-hoc rationale is not proof of anything. Provenance, which points at an artifact outside the model, is structurally stronger than explanation, which points back at the model’s own output. Show both, but trust them differently, and never let a confident explanation stand in for a missing source.

6. Calibrating trust

All of this serves one goal that is easy to state and easy to get wrong: appropriate trust. Not maximal trust. The failure is two-sided. Over-reliance is the person accepting a fluent, confident, wrong answer because the agent gave no signal it should be doubted, the automation-bias failure that the invented citation from Part 11 was waiting to cause. Under-reliance is the person distrusting a correct agent and redoing its work by hand, throwing away the value the agent was built to deliver. Both come from the same root: a mismatch between the agent’s displayed confidence and its actual reliability.

Figure 4. The trust matrix. The diagonal is appropriate trust: the person leans in when the agent is right and checks when it is wrong. The off-diagonal cells are the paired failures. The levers that move a person toward the diagonal are the same two from Section 5, calibrated confidence and provenance, which is why displayed confidence has to track actual reliability.

This is where Section 5 pays off. Surfacing provenance and calibrated confidence is precisely what lets a person sit on the diagonal, leaning in where the agent is reliable and checking where it is not. The design goal is to match the signal to the truth. Connect the two failure modes directly: an agent that hedges everything trains under-reliance, because the person learns the hedges are noise and starts ignoring them, including the ones that matter. An agent that asserts everything trains over-reliance, because nothing ever signals doubt. Appropriate trust is built by an agent whose confidence means something, which is why the uncertainty discipline of Part 17 is the natural sequel to this one.

7. Setting accurate expectations

The expectation gap is set before the first interaction, by how the agent is framed. An agent described as “review any literature” produces a user who is surprised and burned the first time it cannot open a paywalled PDF. The capability did not change; the promise was wrong, and the broken promise costs trust that the agent then has to earn back.

Accurate expectation-setting is the user-facing form of rules-not-wishes from Part 4. The same discipline that says a system prompt should state what the agent does, not wish for what it might do, says the interface should state what the agent does, what it does not do, and where its edges are. And it should state it in the interface, where the person actually meets the agent, not in a launch post they will never read. An overpromising prompt produces an overpromising agent; an overpromising product page produces a disappointed user, by the same mechanism one layer out.

There is a real-time version of this, and the series already built it. An agent that abstains well is an agent that sets expectations well. Part 10 introduced abstention as a safety move, declining to act when the action is unsafe. Read as interaction design, “I cannot reliably determine this from the sources I have” is expectation-setting in the moment: it tells the person exactly where this agent’s edge is, on this query, right now. The lit-review agent should state its scope plainly at the surface, open-access papers, citation data from a single source, summaries rather than peer review, so that the person’s trust is built on a true model of what the agent can do rather than a hopeful one.

8. Best practices: the synthesis

Pull the series together as a practitioner’s checklist. Each line is one principle and a pointer to where the series earned it. This is the take-away card, not a summary of the sections above, so it stays tight.

Designing the agent. Default to a tool, then a skill, then a hook, then a sub-agent (Part 8 decision ladder). Match the structure to the failure; do not reach for the heaviest orchestration by reflex (Part 7). Write rules, not wishes, in prompts, tool descriptions, and user-facing copy alike (Parts 4 and 5). Design tool outputs as carefully as tool inputs; the result is both the model’s next-step signal and the user’s provenance (Part 5).

Designing the seam. Keep the human in the loop where actions are irreversible or high-stakes, out of it where they are cheap and reversible (Sections 1 and 2). Make the agent’s work legible at the user’s altitude; disclose progressively (Section 4). Carry provenance on every surfaced claim, and mark generation as generation (Section 5). Aim for appropriate trust, not maximal trust; calibrate displayed confidence to actual reliability (Sections 5 and 6). Set true expectations at the surface, and let the agent abstain out loud (Sections 6 and 7).

Keeping it alive. Instrument everything; a trace is a debugging artifact, a user-facing explanation, and a future eval case at once (Parts 11, 12, 15). Build a learning loop, not a prompt landfill; route each fix to the layer that owns it (Parts 13 and 15).

(For the links to the Parts, refer to Table of Content)

Read top to bottom, the list is the argument of the whole series compressed: capability is built deliberately, the seam to the human is designed deliberately, and both are kept honest by instrumentation and a learning loop. None of it is exotic. All of it is discipline applied consistently to a system that is, by its nature, fluent enough to hide its own failures.

9. Where this leaves the agent

The series began by insisting that building an agent is an engineering exercise, not a research project. This part adds the other half: it is a design exercise too. The model gives you capability. The harness gives you control. The interaction surface gives you usability, and without the third, the first two reach no one. A person never experiences your tool schemas or your orchestration topology. They experience when the agent asks, what it shows, and whether they can trust it.

That seam is where all the discipline of the previous fifteen parts either pays off or leaks away. A capable agent with a badly designed surface is an unusable agent, and an unusable agent delivers none of the capability it cost so much to build. The work is to design the surface on purpose, action by action, claim by claim, with the same care the engine got.

This is the practitioner capstone of the build-and-practice arc. Parts 1 through 15 build an agent that is capable, safe, observable, and able to learn; Part 16 is the discipline that puts it in human hands. One thread runs underneath it all and was never quite named: an agent’s confidence has to mean something before a person can calibrate their trust in it. The final part is the coda that names it. Agents Under Uncertainty: knowing when you do not know.

Table of Content

If you found this helpful, clap 👏 to help others discover it, and follow for more!