TIL
Short notes on things I learned, each linking the best source I know.
Only
2026
- 10/06Mocks, renderers and kernels are all effect handlersalgebraic-effectsEach lets a program make requests it does not carry out and answers them from outside. The theory of algebraic effects names the parts.
- 09/26Planning can't fix a wrong world model; only learning from experience canrlPlanning works out what a model implies, mistakes included; only real experience corrects the model. A reasoning model's chain of thought is planning against its own weights.
- 09/25Reward is enough only if you know which rewardrlGrounded rewards lift the ceiling that human raters put on an agent, but they leave the question of which reward, and for most goals that is the hard part.
- 09/24Memorizing is the right compression of a fact stated only one wayinformation-theoryWhy a model that can recite every biography can't answer questions about them, and why rewording fixes it.
- 09/24Model collapse is the loss of the tails, and keeping the real data bounds ittraining-dataModels trained on their own output forget rare events first. Replacing the data compounds the loss; keeping the real data bounds it.
- 09/24Weak labels elicit what a strong model already knows, so the student beats its teachertraining-dataWhy GPT-4 trained on GPT-2-level labels doesn't just copy GPT-2's mistakes.
- 08/29A compare-and-set needs an owner, so Cassandra elects one for each writecassandraQuorums make the latest write visible. Making check and write one atomic step takes one place that puts operations in order.
- 08/29Consistent hashing needs virtual nodes, and they even out load slowlypartitioningA uniform hash spreads the points evenly, but not the gaps between them. More points per machine help slowly, so production systems place them deliberately.
- 08/29Every atomic commit protocol can block; they differ in whenconsensusCommit can't give up safety. Two-phase commit blocks when its coordinator fails; Paxos Commit only without a majority of acceptors or a stable leader.
- 08/28Spanner's commit wait caps throughput only for serialized writesqueueingBy Little's law a wait on every write costs latency, not rate, except where only one write can be in flight.
- 08/24Near full utilization, load turns randomness into long waitsqueueingWhy idle time is what keeps work from waiting.
- 08/23W-TinyLFU lets new cache entries prove themselves in a small LRU windowcachingA small LRU window lets new keys earn their way into the cache.
- 08/14Before asking how a system works, ask what problem it solvescognitive-scienceMarr's three levels, and why measuring every feather won't explain flight.
- 08/11A cut-off Raft leader can keep appending, but it can never commitraftWhy Raft's vote rule checks the last term before log length.
- 08/11In Raft, an entry stored on a majority of nodes can still be erasedraftWhy a Raft leader only counts replicas for its own term's entries.
- 08/11The end-to-end argument repeats at every layermessagingEvery middle layer has its own two ends and its own retry loop.
- 08/05Games like chess are already decided; they just haven't been computedgame-theoryBackward induction fixes the outcome of perfect play in any finite game with nothing hidden. Checkers has been computed; chess is too large.
- 08/05Simple isn't cheap: Kolmogorov complexity has no clockinformation-theoryKolmogorov complexity counts a program's length, not its running time, so a simple world can still be expensive to predict.
- 07/31A database page fragments because rows aren't deleted in the order they were writtendatabase-internalsRows are allocated like a stack but freed like a heap.
- 07/31A join gets expensive when the filter needs the join's outputdatabase-internalsThe hard case is when the filter needs the join's own result.
- 07/31A slotted page lets rows move, but only within the pagedatabase-internalsWhy databases address rows as (page, slot), and the limit that leaves.
- 07/30Fencing keeps you correct; a worker aborting itself only saves moneyconcurrency-controlTwo responses to a lost lease, and why only one is about safety.
- 07/18Saying which hypothesis wastes bits, so a Bayes mixture beats the two-part codeinformation-theoryThe two-part code is MAP in bits, and it gives every sequence several codewords; a code agreed in advance for the whole model class spends that waste.
- 07/18Scientific realism is idle on settled science and decisive at the frontierphilosophyRealists and anti-realists do the same science, until the science isn't settled yet. Quarks today, atoms in 1900.
- 07/18Solomonoff's predictor can't dodge its bound by oscillatinginformation-theoryThe expected daily error can't go negative, so it has to shrink.
- 07/16Crick's comma-free genetic code was elegant only for a noiseless channelinformation-theoryIt spent every spare triplet on finding the reading frame and none on surviving mutations; the real code does the reverse.
- 07/16Outcome RL gets at most one bit per episode, and it works because each bit is aimed at the objectiverlWhy a pass/fail reward can move a model in ten steps.
- 07/08"Be true to yourself" needs things that matter beyond your choicephilosophyWhy the ideal of authenticity depends on standards it seems to reject.
- 07/05Bureaucracy keeps its clerks stateless: the case lives in the filesorg-theoryWhy Weber put the files into his definition of a bureau.
- 07/04Optimistic concurrency control makes the same bet as speculative executionconcurrency-controlGuess, then check: the same bet in databases, CPUs and LLM decoding.
- 06/17Pain is the brain's estimate of danger, and damage is only one inputhealthWhy a twig can hurt more than a snakebite, and pain-free spines look worn on a scan.
- 05/23GRPO's group mean is a Monte Carlo baseline that replaces the criticrlGRPO's group mean is the control-variate baseline PPO learns with a critic, estimated by sampling instead.
- 05/02Small PRs and frequent deploys only speed you up once CI/CD is fastqueueingBatch size is an output of what each batch costs, so the lever for small batches is a cheap pipeline, and it works by a square root.
- 05/01Lowering a language model's loss means compressing the text betterinformation-theoryA probability distribution and a code are the same object, so a model's loss is the size of the text compressed with it, and training shrinks that size.
- 05/01MDL counts the model's length because a fitted model has to be sentinformation-theoryA code agreed in advance costs nothing to share; a code chosen after seeing the data has to be sent, and that is the whole two-part code.
- 01/24Why Bitcoiners retell the 1933 gold seizure: collect at the old price, then redefine the dollareconomicsAmericans turned in their gold at $20.67 an ounce. Once the Treasury held all of it, the price became $35.
2025
- Going Bayesian doesn't escape the reference class problemprobabilityChoosing which class to count someone in becomes choosing which conditional probability to trust.
- The outside view works only when your case is another draw from the same barrelprobabilityBase rates beat plans when past cases are exchangeable with yours. For AI risk no class is, and forecasts don't converge.
2024
- 11/11Gramine's fast path runs through libc, so Go programs trap on every syscallteeWhy an unmodified Go binary in an SGX enclave pays two extra enclave exits per syscall.