Object storage only makes single files atomic. So how does a lakehouse table commit a write that touches hundreds of files?
It uploads all the new data files first, where no reader can see them yet, then writes one small manifest file that points at them. That single write is the commit: before it, readers see the old table; after it, the new one.
That leaves the hard case: two writers both try to write version 002 at once.
Keeping numbered versions (MVCC) gives readers a stable snapshot, but it doesn't
say which writer wins. The rule that does is: each writer names the version its
work was based on, and the commit is a compare-and-swap. If you started from
001 and the table is already at 002, your write fails. The loser isn't
wasted, either. Its data files are still valid, so it re-reads 002, rebases,
and tries 003.
This is optimistic concurrency control. It bets that nobody else is writing, does all the expensive uploading without a lock, and checks only at the end. When conflicts are rare the bet usually wins, and the cost of rebasing is rarely paid.
The same shape shows up elsewhere. In speculative decoding, a small model drafts tokens and the large model verifies them. In a CPU, the branch predictor guesses which way a branch goes and runs ahead; a wrong guess is thrown away. In all three there is an exact check before anything becomes visible, so a wrong bet costs time, never correctness.
I worked out the "name your base version" check before I knew it was called OCC, and the resemblance to speculative decoding was what I noticed; the CPU branch predictor, the older instance of the pattern, I learned afterwards.
The original OCC paper is Kung and Robinson, "On Optimistic Methods for Concurrency Control" (1981).