A dedicated container on my Proxmox host runs a coding agent that writes, reviews, and ships changes to this site — through the same git-push pipeline I use. The interesting engineering isn't the model; it's the guardrails around it.
VLANs decide which networks can talk. They say nothing about which people can. Part 5 of the homelab series: a central identity service in its own VM, Tailscale as the front door, and why the break-glass path matters more than the lock.
The plan was to consolidate everything onto one Proxmox box. Then a perf tool needed real PMU counters, a firmware flash needed a real serial port, and testing on ARM meant a phone running a custom kernel. Meet the fleet — and what each machine taught me.
Wake-on-LAN only works from inside the LAN — so I put a $5 ESP32 on the LAN. It subscribes to an MQTT topic and broadcasts magic packets on demand. The build, the failure modes, and what a hardware watchdog is really for.
The lab started as services sprawled across whatever machine was awake — until one bad upgrade took them all down at once. This is what that failure taught me: one Proxmox host, ZFS snapshots as the undo button, and a network segmented by trust.
No VPS, no database, no CMS bill. This site is a git repo that compiles to static files — search index, social cards, and fonts included — and Cloudflare Pages serves it for free. Here's the whole pipeline, with the actual configs.
A language model generates one token at a time, and each token re-reads gigabytes of weights and a growing cache. Understanding the KV cache and the memory-bandwidth wall explains latency, context limits, and every serving optimization from the last two years.
mmap is sold as the fast way to read files, but it loses as often as it wins. The deciding factor is page faults versus syscalls — and whether you touch the data once or many times.
Every language prints 0.30000000000000004 and every newcomer files a bug report. It's not a bug — it's binary fractions doing exactly what the IEEE 754 standard says. Here's the why, and what to do about it.
Git feels like a pile of special commands until you see the data model underneath: four object types in a key-value store keyed by hash. Learn the model and every confusing command suddenly makes sense.
A container isn't a lightweight VM — there's no guest kernel, no virtual hardware. It's just a normal Linux process wearing two kernel features as a disguise. Strip away Docker and the magic is surprisingly small.
A packet arrives, a key is pressed, a timer fires — and the CPU drops everything, mid-instruction-stream, to run kernel code. How interrupts work, why handlers are split in half, and how NAPI killed the interrupt storm.
Pull the power mid-write and a naive filesystem corrupts. A journal is the write-ahead log that lets ext4 come back consistent — by promising to do work before actually doing it.
malloc looks like one function, but underneath it's a memory allocator juggling free lists, size classes, and the kernel. Knowing how it works explains fragmentation, allocator contention, and why your RSS never goes down.
One round trip now negotiates keys, proves identity, and encrypts everything after the first message. What actually happens in a TLS 1.3 handshake, why it's structurally safer than 1.2, and where the remaining sharp edges live.
Replicate data across five machines and you'd think any one of them could answer. But to stay consistent through failures, a majority has to agree on every decision. Here's the arithmetic behind quorums.
A classic stack smash overwrites the return address and hijacks execution. Stack canaries are the cheap, clever defense your compiler inserts automatically — and understanding them explains a whole class of exploits.
Raft won because you can hold the whole protocol in your head: terms, a leader, and one log-matching rule. The mechanics of election and replication — and the operational corners (fsync, membership changes, pre-vote) where deployments actually bleed.
C's volatile keyword is the most misunderstood tool in concurrent programming. It stops the compiler from caching a value — but it does nothing about the reordering that actually breaks your lock-free code.
Every database picks one of two ways to put data on disk. The choice between B-trees and LSM-trees is really a choice about whether you optimize for reads or writes — and you can't have both for free.
Text feels like a solved problem until an emoji breaks your database, a filename won't delete, or 'MÜLLER' and 'müller' won't compare equal. The layered history of character encoding explains every mojibake bug you'll ever hit.
Strip the frameworks away and an LLM agent is about 35 lines: call the model, if it asks for a tool run the tool and feed the result back, repeat until it stops. I built one from scratch to prove it — and the systems concepts underneath are all familiar.
nginx, Node, Redis, and every async runtime you use sit on one Linux primitive. Why select died at scale, how epoll inverts the cost model, the level- vs edge-triggered trap, and the thundering herd that took years to fix.
Postgres never updates a row — it writes a new version and lets old transactions keep seeing the old one. Multiversion concurrency is why reads don't block writes, why VACUUM exists, and why 'the database is bloated' is a real sentence.
An index is a sorted structure with physical properties, not a magic 'go faster' flag. Leftmost prefixes, covering indexes, selectivity, and the honest reasons the planner chose a sequential scan over the index you lovingly created.
Every deadlock requires four conditions to hold simultaneously, and every practical cure breaks exactly one of them. Why lock ordering is the fix that survives contact with real codebases, and how to find the cycle when it happens anyway.
Before the Model Context Protocol, every agent-to-tool integration was bespoke M×N glue. MCP is the USB-C of agent tooling: a standard client-server protocol so any model can discover and call any capability. What it is, and why it's a systems standard, not an AI one.
Exposing one service teaches you TLS. Exposing ten teaches you reverse proxies, DNS, network segmentation, and the uncomfortable truth that your smart bulbs are on the same wire as your NAS. A field guide to the plumbing.
A naive file-to-socket server copies every byte four times and crosses the kernel boundary twice per chunk. sendfile collapses it to zero user-space copies — and the lineage from there (splice, MSG_ZEROCOPY, io_uring) is the story of modern fast I/O.
For 15 years Linux picked the next task with a red-black tree of virtual runtimes. In 6.6 that changed for the first time since 2007. What fairness actually means, how nice values work, and why latency-sensitive tasks forced a redesign.
The gap between -O0 and -O2 is often 10x, and it's not magic — it's a pipeline of individually simple transformations over an intermediate representation. SSA, inlining, the passes that matter, and why your careful micro-optimization was already done for you.
A core router matches each packet against ~1M CIDR prefixes and must pick the most specific — billions of times per second. The data structures behind that (tries, TCAM, poptrie lineage) and why 'most specific wins' shapes everything from BGP hijacks to your homelab VPN.
Every database, filesystem, and message broker converges on the same move: append the intention to a sequential log, fsync, then update the real structures whenever. WAL is one idea wearing a dozen names — and its failure modes repeat too.
A function call costs ~2ns. A syscall costs ~100ns before doing any work — and that's after decades of optimization plus a Spectre-era tax. Where the time goes, why vDSO exists, and what that means for API design.
Between 'sum the bytes' and SHA-256 lies the workhorse category: non-cryptographic hashes. What avalanche means, how hash quality turns into hash-table throughput, and why every language runtime switched to seeded hashing.
NAT rewrites addresses; connection tracking is the memory that makes the rewrite reversible. Together they're why the internet survived IPv4 exhaustion — and why your idle SSH session dies, your table overflows, and P2P needs hole-punching.
Every packet's last hop is decided not by routing but by a 1982 protocol with no authentication, a cache with quirks, and failure modes — gratuitous ARP, proxy ARP, spoofing — that explain a shocking fraction of 'the network is weird' tickets.
Signals interrupt your program between any two instructions and run your handler in the wreckage. Why almost everything is illegal inside a handler, what async-signal-safe really means, and the self-pipe/signalfd escape hatches.
The fastest malloc is a pointer bump; the fastest free is resetting one offset. Arenas trade individual deallocation for speed, locality, and the death of use-after-free within a lifetime — a trade half your workloads should take.
A powered-off machine has no OS, no IP stack, no running software — yet a single crafted broadcast frame boots it. How the NIC stays half-awake watching for its own MAC, why WoL is a layer-2 trick, and the homelab plumbing to make it reliable.
Fixed windows let 2x through at the boundary, token buckets are three lines of arithmetic, and the distributed version is a consistency problem wearing an ops costume. The algorithms, their failure modes, and what to return when you say no.
Half of all confusing C build errors — undefined reference, multiple definition, static initialization order — are linker concepts wearing compiler costumes. Symbols, relocations, and the archive rule that surprises everyone.
C++ virtual calls, Linux file_operations, and Go interfaces are the same machine code: load a pointer, call through it. What a vtable really is, what an indirect call costs, and how devirtualization claws it back.
A superloop is the right architecture more often than embedded tutorials admit — until the day it isn't, and you need preemption, priorities, and blocking. What FreeRTOS actually buys you on an ESP32, and what it costs.
Reorder three fields and a struct shrinks from 24 bytes to 16 — a 33% memory cut with zero code changes. Where padding comes from, how to see it with pahole, and when packing is a trap.
UB doesn't mean 'returns garbage' — it means the optimizer may assume it never happens and reason backwards from that. How signed overflow checks get deleted, why time-travel is real, and the sanitizer workflow that catches it.
Flash can't overwrite in place, erases in giant blocks, and wears out per cell — so every SSD runs a translation layer that remaps, garbage-collects, and levels wear behind your back. Write amplification, TRIM, and the cliff at 90% full, explained.
Signed integers could have been encoded a dozen ways. Hardware settled on two's complement because it makes the adder ignorant of sign — one circuit for everything. The same choice explains INT_MIN, sign extension bugs, and why abs() can return a negative number.
Between pressing enter and your first line of code, the kernel parses an ELF file, maps segments, loads an interpreter you didn't know existed, and that interpreter relocates half your program. A tour of the machinery under ./a.out.
A 'simple' name lookup traverses your language runtime, nsswitch, a stub resolver, a recursive resolver, and up to three tiers of authoritative servers — with caches at every layer that disagree. Map the path once and DNS bugs become findable.
A 'load balancer' is anywhere from a NAT rewrite to a full HTTP proxy with retry logic — and the algorithm matters less than people think while the health checking matters more. The mechanics, from ECMP and Maglev down to power-of-two-choices.
Every queue advertises delivery guarantees; every distributed-systems engineer eventually learns the ack-crash window makes duplicates inevitable. Why exactly-once is really exactly-once *processing*, and the idempotency patterns that make it true.
RAID levels are three ideas — stripe, mirror, parity — composed. The interesting parts are the failure math: why RAID 5 rebuilds fail at modern disk sizes, what the write hole is, and why RAID is not backup, mathematically.
The same loop over the same data runs 5x faster when the array is sorted. The instructions are identical. The difference is a gambler inside your CPU that's either winning or losing.
At reset, an x86 CPU wakes up pretending it's 1978 — 16-bit mode, 1 MB address space. Everything after that is bootstrapping: firmware, boot sector or EFI application, protected mode, and finally a kernel. I wrote a bootloader to learn the path; here's the map.
The Linux kernel's list_head puts the container inside the object instead of the object inside the container — one implementation, zero allocations, and a container_of macro that looks illegal but isn't. I built a library around the pattern; here's why it wins.
NIC queues, io_uring, audio pipelines, logging, SPSC channels — wherever a fast producer meets a consumer on a budget, there's a fixed array with two indices. The design decisions that matter: power-of-two sizing, the full/empty problem, and who waits.
Every process thinks it has a private, contiguous address space starting at zero. None of them do. Virtual memory is the translation layer that makes the lie hold — and it's why fork, mmap, and swap can exist at all.
Databases read disk through fixed-size pages cached in a buffer pool they manage themselves — duplicating what the kernel's page cache already does. The reasons why (eviction control, WAL ordering, O_DIRECT) are a masterclass in when to bypass the OS.
DRAM is ~100ns away; your CPU can do ~400 arithmetic operations in that time. Caches exist to hide that gap, and whether they succeed is decided by how your code lays out and walks its data.
Everyone knows floating point is 'imprecise.' Fewer know that the same expression can give different results on different machines, that summation order changes your answer, and why associativity — the thing algebra promised you — is a lie in float.
MQTT moves a temperature reading in about 30 bytes over a connection that survives flaky cellular links and sleepy microcontrollers. The publish/subscribe model, the three QoS levels, and the retained-message and last-will tricks that make it click.
fork clones a 10 GB process in under a millisecond by copying nothing — just page tables, marked read-only. The first write to any page pays the real cost. Elegant, and the source of Redis's most famous operational trap.
A single unbounded copy can hand an attacker your instruction pointer. The classic stack smash, then the 25-year arms race — stack canaries, NX, ASLR — that turned a reliable exploit into a hard research problem.
hash(key) % N reshuffles nearly everything when N changes — a cache stampede with a math degree. Consistent hashing moves only 1/N of keys per node change, and virtual nodes fix the balance problem the elegant version hides.
UTF-8 was sketched on a placemat in 1992 and now carries ~98% of the web. It won because of five design properties — self-synchronization, ASCII compatibility, sort-order preservation — that no committee would have dared require.
C has no destructors, no defer, no exceptions — just early returns that leak and nested ifs that march off the screen. The kernel settled this decades ago: goto-based cleanup is the correct idiom, and it's more disciplined than its reputation.
Eleven states, two of which cause most production incidents. Implementing TCP taught me the state diagram isn't trivia — TIME_WAIT port exhaustion, SYN backlogs, and orphaned FIN_WAIT_2 are all right there in the RFC 793 figure.
Every recursive call is a frame pushed onto the call stack — return address, saved registers, locals. See the frame layout once and stack overflows, tail calls, and the recursion-to-iteration transformation all become mechanical.