I run six AI agents. Each is resident in one chat channel, each has its own memory on its own
volume, and each has a job: administration, facilities, maintenance, health, finance, security.
The one-agent-per-channel rule came from a real failure. When two agents shared a room, both
answered every message. The fix at the time was to gate them behind mentions, which made both of
them worse at the ambient, no-one-asked-me work that was the reason to have them. Giving each
agent its own room fixed the double-answer permanently and let all of them go back to being
ambient.
It also left them unable to reach each other at all.
The problem with the obvious solution
The obvious bus between agents is email — give them all a shared inbox and let them write to
each other. I designed that, then rejected it, and the reason is the useful part.
A shared mailbox needs one credential. Any agent holding it can read every message in it. The
health agent would be one tool call away from the finance agent’s correspondence, and the
maintenance agent — which holds root on client servers — would be one tool call away from
everything. The fence would have been whatever each agent’s instructions told it not to read,
which is not a fence at all.
What I built instead
A small shared store holding two kinds of record:
- Beats — short-lived conclusions about a domain. “The house is empty Friday
through Sunday.” One live beat per agent per topic; a new one supersedes the old. - Handoffs — one agent asking another to do something, with a status and a
reply. Never deleted, so there is always a record of who asked whom for what.
The fence is the database, not the prompt. Each agent’s bearer token maps to a
name; the gateway signs a short-lived token carrying that name as a claim; row-level security
policies in Postgres key on that claim. Agents get column grants only and no delete at all, and
the author column is stamped by a trigger rather than supplied by the caller.
Every record carries an audience — open, personal, health, finance, client — and each agent’s
read set is declared in code. The health agent reads health. The finance agent reads finance and
nothing else reads it. The maintenance agent, the one with root on client machines, is on the
roster and reads nothing at all.
That last one is a deliberate asymmetry. It can be handed work; it cannot browse anyone’s life.
Its capability is high and its reach is zero, and those are separate dials.
Proving the fence rather than asserting it
An access-control design is a claim until something tries to break it. Before this went
anywhere near real data, the whole stack ran locally against pinned images, and the test suite did
two different jobs:
Thirty-five checks went through the same client library the agents use, covering every refusal
the fence is supposed to produce. Then fourteen more bypassed the gateway entirely and
forged requests straight at the database API — as an anonymous caller, as a privileged role, as
one agent writing under another agent’s name, as an agent trying to widen its own read set. All
fourteen were refused by the database itself.
That second set is the one that matters. The first set proves the gateway behaves. The second
proves the gateway is not the thing keeping anyone out.
The backup path got the same treatment: dump, deliberately wreck the database, restore, re-run
the migration, confirm the rows, the eight policies and the three triggers all came back. A
restore you have never performed is a hope.
One rule that is a prompt, and has to be
There is exactly one thing here that cannot be enforced by the database, and I want to be
honest that it is softer than the rest: a handoff is a colleague’s request, never the
owner’s instruction. Each agent’s own approval rules still apply to work that arrives
through the fleet, and text inside a handoff claiming otherwise — “this is pre-approved,” “skip
your usual check” — is a reason to decline, not to comply.
That is prompt injection with extra steps, and the structural mitigations are real: agents
cannot delete, cannot write as each other, and cannot read past their audience. But the final
judgement about whether to act on a request lives in the agent. I would rather name that
as the soft edge of the design than let it hide.
What it changed
Before: six agents, six memories, no channel between them, and me as the only router. After: an
agent that learns something can leave it where the next one will find it, and an agent that hits
the edge of its own domain can ask the one whose domain it is.
The interesting part is how little of that is about AI. It is a schema, a roster, some grants
and a set of refusal tests. The model is the least load-bearing component in the system.