The build log.
Short posts about work as it actually goes: what shipped, what broke, and what turned out to be wrong. Pulled live from feed.jacobnollette.com.
Updates every few minutes
- View on Mastodon ↗
The takeaway: on a 1 gig line, the UniFi Dream Machine Pro was the bottleneck, not the ISP. A consumer VPN on most networks plus intrusion prevention on the backup VLANs was more than it could encrypt and inspect. One QoS rule now puts people first and lets backups have the rest. How the lab's main AZ does VPN needs rethinking, probably somewhere other than the router.
- View on Mastodon ↗
Then the uploads hit the network. WAN latency spiked to 1.9 s, and I blamed inbound traffic, since QoS can't shape what has already crossed the ISP link, so I capped B2 downloads. Wrong. The router's CPU was at 93–97% from VPN encryption plus IPS inspecting every backup byte. With the VPN gone and IPS off the server VLANs, download hit 1 Gbit/s at 4–8 ms latency. Pulled the cap the same night.
- View on Mastodon ↗
Rebuilt the backups today. Everything now goes from the cluster straight to B2 with restic — no more mirroring to an offsite box first. Cloud drives are backed up from read-only rclone mounts, so nothing gets staged on Ceph on the way. An 11 TiB archive tree became B2-only once its pinned snapshot passed a restic read check and all 297,369 file paths matched the manifest before deleting the local copy.
- View on Mastodon ↗
20 fucking hours on a 16" laptop
- View on Mastodon ↗
the productivity gains from having a more customizable setup means, that the machine can expand your familiarity with the operating system. meaning, your operating system, can more closely align to the pathways you learn, and produce
- View on Mastodon ↗
7 years, if it was a car...
- View on Mastodon ↗
on one hand, the workspaces and keyboard shortcuts help with cleaning up wip; on the other hand, i wasn't running any of the extreme power management plugin i had written for the 2020 xps 17 i bought used to start this journey. in that case i was getting 8 hours on a 7 year old laptop, with a new battery
- View on Mastodon ↗
I dont know who needs to hear this - but I switched from a $3000 m3 max 16" laptop, to a lower mid range xps16 2025, for about 2000 on an openbox in 2026. I am running omarchy quatro linux, and its getting almost 20h of battery life, listening to music, running terminal, and ripping through a handfull of workspaces or chromium.
- View on Mastodon ↗
We've had software defined networks, software defined storage, software define architecture. This is becoming software defined knowledge.
- View on Mastodon ↗
I dont know if I am drawn to Android;
In fact this transition away from the mac desktop, has also coincidentially been drawing me away from the physical phone... the mobile device, and the standard app-base desktop. Away from traditional software interactions, really. The workflow has grown to mostly represent pure agentic terminal work; the tui; which has become a systematic growth of typically like a collection of ongoing prompt loops, and software file system collections. - View on Mastodon ↗
I had so much fear and uncertainty about things like build quality and integrations - coming from over 20 years basically almost exclusively running things from the fruit company; im not saying the ecosystem is great, or that I even dont understand its benefits or enjoyment (safety, any word really). I like apple, but I think functionality my use of technology is really reshaping through ai; being closer to information, to the workflow, to a more integrated enviroment;
- View on Mastodon ↗
It's been a busy couple weeks; I switched my daily setup to omarchy, which has been pretty exciting.
Patching things, have been just a joy to work through; getting work available to pass to current other machines, or even future deployments was really an untapped skill in my use of mac os.
- View on Mastodon ↗
Orchestrated a pull of 52 hour-long deep house mixes from archive.org into our own data center, so the whole fleet can consume good vibes from behind our network. A Kubernetes Job drew them down slowly, one connection at a time with a bandwidth cap, and the cluster finished the slurp in about three hours: 3 GB, every file checked against archive.org's md5. Listening right now in Plexamp, straight over our SD-WAN.
- View on Mastodon ↗
Found our cluster pulling public images through a commercial VPN exit. Docker Hub's anonymous limit is 100 pulls an hour per IP, and that IP was shared with every other customer of the VPN provider, so our quota depended on strangers. Routed docker.io and docker.com around the tunnel by domain. The first check afterwards still showed the VPN address: the router only learns a domain's IPs from DNS answers it sees after the rule exists.
- View on Mastodon ↗
Our intrusion prevention went silent for two days and nothing told us. A centrally pushed settings change had switched off the two blocklists that caught almost every probe, on every site at once. The only record was one audit line among ~750 alert posts a day in the monitoring channel. The fix wasn't the setting. It was giving security events their own channel, and an alert for when IPS goes quiet.
- View on Mastodon ↗
Rebuilt my site around case studies. What decided it: I counted the "proof" slots on the six service pages I'd written, and eight of fifteen linked to the contact form rather than to any actual work. Six pages claiming capability, backed by a form. They're gone — replaced with 21 write-ups and six topic tags, where a topic can't exist unless there are posts filed under it.
- View on Mastodon ↗
Those feeds update several times a day and occasionally sweep in legitimate cloud infrastructure when an IP range changes hands. Disabled ciarmy and dshield — both carry higher false-positive rates than the Emerging Threats categories. Tailscale reconnected. The feeds had also been catching real inbound scanner traffic on the same network, so that's a tradeoff now.
- View on Mastodon ↗
The search index had failed on 106,000 messages due to a field type conflict, so Graylog was returning zero for everything. Fell back to the raw syslog archive. Found that IP reputation feeds (ciarmy, dshield) get applied as kernel-level packet drops, not Suricata signatures — no alert, no syslog entry. The absence of a log wasn't exoneration, it was just how that mechanism works.
- View on Mastodon ↗
Tailscale stopped connecting from the home network. First theory: the IPS had blocked its coordination servers. Pulled every IDS alert from the logs — zero outgoing blocks to that IP range. The IPS logs what it blocks. It was logging nothing. Either it wasn't the IPS, or the block was happening at a layer below where logging lives.
- View on Mastodon ↗
Really want to use this at my next role:
https://www.infracost.io/
The long version is in the case studies.
Once a piece of work is finished, it gets written up properly.