I keep years of receipts, statements, and scanned documents in a self-hosted document archive — the kind of system that’s genuinely useful once it’s populated, and genuinely useless if you can’t ask it questions. "What did I pay for that appliance repair last spring" should be answerable in one sentence, not a manual search through a filing system. That’s a job for an AI assistant with access to the archive — but the archive lives entirely inside my home network, on purpose, and the assistant runs in the cloud. Those two facts are in direct conflict, and resolving them safely turned into its own small project.
The bridge has to be narrow, not the whole house
The archive server itself has no business being reachable from the public internet — it holds financial records. But a cloud-based AI assistant can never reach into a home network directly; there’s no tunnel it can join, no VPN it can dial into on its own. The tempting shortcut — just expose the whole application to the internet behind a login — was never really on the table. A single leaked session, a single unpatched vulnerability in a large web application, and everything in that archive is exposed at once.
The alternative is a narrow bridge: a small, purpose-built service that speaks exactly one protocol (the standard one AI assistants use to call tools), authenticates every request with its own login flow, and reaches the archive over a private, internal-only connection that never leaves the home network. If that bridge is ever compromised, the blast radius is "whatever this one credential can read" — not "the whole archive." The credential behind it, by design, can only read; it can’t yet write, because permission to write is a separate, deliberate step I haven’t taken.
What a security review actually found
Before connecting anything real to it, I had the bridge reviewed adversarially — treating it the way an attacker would, not the way I’d hoped it worked. That review returned 45 potential issues; nine survived independent verification as real. Two of those were the kind that matter most on a small server: ways to knock it offline for everyone, not just steal data.
The bridge, like most authentication flows, has to hold onto some request state — a code, a session — for a short window while a login completes. Nothing had bounded how large that state could be, or how much of it could accumulate at once. A verifier proved that a modest, achievable stream of malformed requests could balloon that held state to hundreds of megabytes on a server that only had a couple hundred to spare — enough to crash the process with nothing more than a few dozen requests per second from one machine, no valid credentials required. A second, related gap let an oversized request body get fully read into memory before anything checked whether it was even well-formed, which is its own way to exhaust a small server for free.
Both got fixed the standard way — hard caps on request and state size, checked before any expensive work happens, plus a rate limit on the endpoints that don’t yet require a credential to hit. None of this was theoretical: the bridge sits on the open internet, deliberately, because that’s the whole point of it existing. Findings like these are exactly what an adversarial review before launch is for.
A "working" integration that had never actually returned valid data
Weeks after that hardening pass, the assistant’s calls to the archive were still failing — every single one, silently retried for minutes before giving up. The bridge itself was healthy: correct authentication, correct routing, a 200 response every time. The bug was one layer deeper, in the open-source tool server the bridge was wrapping. It had been built against an older, looser interpretation of the protocol, and simply returned raw data from the archive’s own API on every call — never once wrapping it in the small envelope the protocol actually requires. A strict client reads that as malformed and keeps retrying, which is exactly the slow, confusing failure that had been happening.
The reason it went unnoticed at first is a good lesson on its own: the original verification had checked "does a response come back," using a generic command-line tool, rather than "is the response valid," using something that actually speaks the protocol the same way the real client does. Once diagnosed — from the bridge’s own audit log, which recorded the real failing calls as they happened — the fix was swapping to a better-maintained fork of the same tool that returns properly-formed responses, plus one more bug caught in the same pass: the bridge had been quietly forwarding its own visitor’s login token straight through to the archive’s internal API, which understandably rejected it as invalid. That’s also a security fix in its own right — a credential that authenticates someone to the front door should never be handed to what’s behind it.
The archive nobody could actually get documents into
None of that mattered much while the archive itself was badly under-filled. A pass through years of email receipts found that seventy-one percent of a large batch — every receipt that arrived as plain text or an HTML email rather than a PDF attachment — had been silently unreachable by the import rules the whole time. The archive simply had no way to read the body of an email, only its attachments, so anything without a PDF attached vanished without a single visible error. Whether an import rule was configured to skip that mail outright or to attempt and fail at it, neither path actually imported anything — one just happened to fail loudly and the other didn’t, which made the quieter failure mode look like the healthier configuration.
Fixing that meant adding two small internal services purely to make email bodies legible: one to convert HTML and office documents into something extractable, one to pull text out of what’s left. Getting ten thousand-plus backlogged documents through that new pipeline surfaced two more problems that only show up at volume — the archive had been processing documents one at a time on a multi-core machine, turning what should have been an hour of backfill into more than a day, and a routine deployment mid-import wiped out the temporary files a few thousand queued documents were waiting on, because they’d been sitting on the server’s disposable scratch space rather than anything durable. Both got fixed — more parallel workers, queued work moved to storage that survives a restart — but the second one cost a real afternoon of reprocessing, twice, before the fix stuck.
The part that’s easy to forget
None of the interesting engineering here was really about the AI assistant. It was about the boring, load-bearing plumbing underneath it: a narrow, reviewed entry point instead of a wide one; a protocol implementation that’s actually correct instead of merely responsive; an ingestion pipeline that can’t silently drop most of what it’s fed. An assistant that can answer "what did I pay for that repair" in one sentence is a nice result. Getting there was mostly about making sure the boring parts underneath it were actually true.