Two threads want the same lock. Thread B finds it taken. Now what? Everything about lock design flows from the two possible answers: keep asking (spin) or go to sleep and get woken (block). One wastes CPU to save latency; the other pays latency to save CPU. Picking wrong in the kernel means wasted cores; picking wrong in userspace means your latency-optimized spinlock performs worse than the mutex you replaced — for reasons that only make sense once you see what the scheduler does to spinners.

What Spinning Really Does to the Machine

A naive spinlock hammers an atomic exchange in a loop. The problem isn’t just burned cycles — it’s the cache coherence traffic. An atomic RMW needs the line in exclusive state, so N spinners bounce the lock’s cache line between cores continuously, and that traffic slows down the lock holder too — the one thread you want running fast. Hence the canonical shape:

c
while (atomic_exchange(&lk, 1)) {          /* one RMW attempt */
    while (atomic_load(&lk)) cpu_relax();  /* then spin read-only on our copy */
}

Test-and-test-and-set: spinners wait on a shared cached copy (quiet), storm the line only when it’s freed. cpu_relax() is the pause instruction — politeness to the pipeline and to SMT siblings. Past a dozen cores even this storms too hard, which is why serious kernels moved to ticket locks (FIFO, bounded unfairness) and then MCS/qspinlocks, where each waiter spins on its own cache line and the queue hands the lock off point-to-point. Lock design is cache-line choreography wearing a concurrency costume.

The Decision Arithmetic

Blocking has a fixed toll: parking a thread and waking it later costs a couple of syscalls and a context switch — call it 1–10 µs end to end. Spinning costs (waiters × hold-time) of CPU. So:

  • Hold time ≪ context-switch cost (nanoseconds: update a counter, unlink a node) → spinning wins; sleeping would cost 100× the wait.
  • Hold time ≥ switch cost (I/O, allocation, anything unbounded) → blocking wins; spinning burns a core-second per second per waiter.

The kernel uses spinlocks precisely where code cannot sleep (interrupt context) and holds are provably tiny; mutexes elsewhere. So far, symmetric. The asymmetry — the trap — is userspace.

Why Userspace Spinning Backfires

A kernel spinlock holder has preemption disabled: it will finish promptly. Your userspace thread enjoys no such promise. The scheduler can preempt the lock holder mid-critical-section — and now every spinner is burning its full timeslice waiting for a thread that isn’t running. Worse, the spinners look like busy, CPU-hungry threads, so they outcompete the holder for CPU. Add priority differences and you get priority inversion with a blowtorch. Malte Skarupke’s benchmarks of this (“measuring mutexes, spinlocks and how bad the Linux scheduler really is”, 2020) plus Linus Torvalds’ memorable response made the canonical point: in userspace, pure spinlocks are almost always wrong, and your benchmark won’t show it until the machine is loaded.

The Grown-Up Answer: Futex and Friends

Linux mutexes aren’t “syscall every time” — that’s a 2003 problem. A glibc mutex is a futex: an atomic word in userspace, and the kernel is involved only on contention:

text
  lock:   CAS 0→1 succeeds?           done — zero syscalls
          contended? set to 2, futex_wait(&word, 2)   → sleep
  unlock: set 0; was it 2? futex_wake(&word, 1)

Uncontended lock/unlock is a few nanoseconds of pure userspace atomics — the fast path costs what a spinlock costs. And the real cure for the holder-preempted problem is adaptive locking: spin briefly while the holder is running on another CPU, then block (PTHREAD_MUTEX_ADAPTIVE_NP, kernel rt-mutex optimistic spinning, most language runtimes’ locks). You get spin latency when holds are short and blocking’s citizenship when they aren’t — the dichotomy dissolved into a policy knob.

Practical defaults follow: use the platform mutex (it’s already adaptive-ish and futex-fast); reach for explicit spinning only for provably-tiny critical sections on dedicated/pinned threads (DPDK-style polling designs, where the core is spent anyway); and if a lock is hot enough that any of this matters, the winning move is usually to shard or remove the shared state rather than tune the lock guarding it.

Takeaways

  • Spin when expected wait < context-switch cost (~µs); block when holds are long or unbounded. Kernel interrupt context spins because it must.
  • Real spinlocks are cache-line engineering: TTAS, ticket, MCS — each step reduces coherence storms.
  • Userspace spinning breaks when the scheduler preempts the holder; spinners then starve the one thread that matters.
  • Futexes make uncontended mutexes nearly free; adaptive mutexes spin-then- block, taking both wins.
  • A truly hot lock is a design smell — shard the state before micro-optimizing the lock.