Two servers each timestamp an event. Server A says 10:00:00.003, server B says 10:00:00.001. Did B’s event happen first? You genuinely cannot know: NTP keeps commodity clocks within single-digit milliseconds when healthy (and lies much harder when not — VM migrations, leap-second smears, a dying oscillator), while your systems do meaningful work in microseconds. Every “sort by timestamp across machines” design contains this bug. Lamport’s 1978 paper — arguably the founding document of distributed systems — dissolved the problem by replacing the question: not when did events happen, but could one have caused the other?

Happened-Before: Causality as the Only Real Order

Define a → b (a happened-before b) if: they’re in one process and a came first; or a is a message send and b its receive; or transitivity chains them. Crucially, some pairs have neither a → b nor b → a — they are concurrent, and no amount of clock precision changes that; it’s a statement about information flow, not physics trivia. Everything in this field is machinery for tracking →.

Lamport clocks are the minimal version: a counter per process — increment on each local event, stamp messages, and on receive set counter = max(local, received) + 1. The guarantee is one-directional: a → b ⇒ L(a) < L(b). The converse fails — smaller timestamp does not mean earlier/causal — and that asymmetry is exactly what people forget. What Lamport clocks are good for: a total order consistent with causality (order by (L, process-id) tiebreak), which is enough for distributed mutual exclusion, log ordering in consensus protocols (Raft’s terms are Lamport-flavored), and “newest wins” schemes where you accept arbitrary-but-consistent tiebreaks.

Vector clocks buy the converse. Each process keeps a vector of counters, one slot per participant; merge element-wise-max on receive:

text
  V(a) = [2,1,0]   V(b) = [3,1,0]  → V(a) < V(b): a → b (causal)
  V(a) = [2,1,0]   V(c) = [1,2,0]  → incomparable: CONCURRENT

Now V(a) < V(b) ⟺ a → b, and incomparability detects concurrency — which is the operationally valuable power: Dynamo-lineage stores used vector clocks to distinguish “this write supersedes that one” (discard the old) from “these writes are concurrent” (keep both as siblings, make the reader reconcile) — the mechanism behind Riak’s sibling values, and the thing you silently lose with last-write-wins-by-wall-clock, which resolves every conflict by destroying one side (Cassandra’s LWW is exactly this trade, chosen with eyes open). The cost is the vector: O(participants) per stamp, awkward with churning membership (dotted version vectors and client-ID pruning are the patch literature), which is why real systems spend it only per-key, not per-message.

The Compromise That Won: Hybrid Logical Clocks

Pure logical time answers causality but can’t answer “give me yesterday’s state” — humans and SQL live in wall time. HLC (2014) fuses them: a timestamp that stays within bounded distance of physical NTP time but ticks logically (max+increment) on message exchange, so it never violates causality even when clocks skew. That’s what CockroachDB runs on, and MongoDB’s cluster times are the same idea — you get →-consistent ordering and meaningful timestamps in one 64-bit-ish value. The maximalist alternative is Spanner’s TrueTime: GPS+atomic clocks tighten uncertainty to a few ms, and transactions wait out the uncertainty interval before committing — buying external consistency with specialized hardware plus deliberately spent latency. Both designs concede the same premise: wall clocks alone are unfit to order events; the question is only how much machinery you deploy around them.

The practical residue for ordinary backends: never merge/sort cross-machine events by raw wall timestamps when correctness matters; use per-entity version counters (a one-slot vector clock — usually all you need); treat “concurrent” as a real state your data model must represent or explicitly LWW away; and reserve wall time for what it’s for — humans, TTLs, and metrics.

Takeaways

  • Clock skew vs event rate makes wall-time ordering fiction; happened-before (information flow) is the only order that exists.
  • Lamport clocks: causality implies smaller timestamp, never the reverse — good for consistent total orders, blind to concurrency.
  • Vector clocks make concurrency detectable — the difference between merging conflicts and silently deleting one side (LWW).
  • HLC = causal correctness + approximate wall time; TrueTime = pay hardware and latency for bounded uncertainty. Same admission, different budgets.
  • In practice: per-entity versions, explicit conflict states, wall time for humans only.