It doesn’t have to be all or nothing with Al agents.

Photo by Anatol Rurac on Unsplash

Ever since I got my first agent up and running, I've been trying to find ways to see it in action without giving away the kitchen sink. I wasn't about to let it run around my computer, but there was only so much it could do in an isolated VPS looking up public websites.

So over the recent public holiday, I tried to use a Hermes-based agent to consolidate my notes across Evernote and Capacities into a single Obsidian vault. The idea was for it to rationalise the different formats and sort them into logical folders.

I used Syncthing to mirror a designated folder with copies of two markdown vaults to a sandboxed environment so the agent has no direct access to my computer. The agent works only within the VPS, but I get to see what happens as its actions are mirrored in the designated folder.

What I got to observe was an overconfident employee who assured me multiple times that it would preserve image links when moving markdown files, only to fail repeatedly. It took several rounds of feedback before it found a reliable way to locate the right image and link it back to a broken file I identify - and even then, a global fix still remains elusive.

Unlike a person, the agent has no sense of accountability. It will confidently get things wrong, again and again, until you understand the pattern well enough to correct it. You're figuring out its shortcomings, but you're also figuring out how to give better instructions. Trust, if it develops at all, comes through iteration, not orientation.

Whether you're an individual exploring Al tools or a firm deciding how to integrate them into real workflows, the questions are somewhat similar. How much access is enough for it to be useful? What's the blast radius if it does something terribly wrong? What do you need to understand before extending more access, not after?