Solomonoff's universal predictor ends up predicting like whatever computable source is generating the data. The key bound says its total excess log-loss over the true source stays under a constant, at every horizon. But one day's excess can be negative: on a given day the predictor may give the outcome that actually happened more probability than the truth did. The bound only caps the net sum. So what stops a predictor from overshooting one day, undershooting the next, staying under budget forever, and never settling down?
Look at a single day in expectation. Say the truth gives two outcomes probability 0.7 and 0.3, and the predictor gives them m and 1 − m. The day's excess is the log of the truth's probability over the predictor's, for whichever outcome occurs. Averaged over how often the truth actually produces each outcome:
0.7 · log(0.7 / m) + 0.3 · log(0.3 / (1 − m))
Minimize over m. The minimum is at m = 0.7, where both log terms are log 1, and the minimum value is zero. You can beat the truth on a day, but you can't expect to, because the truth decides how often your lucky days come up. In general this expected daily excess is the KL divergence, and its never being negative is Gibbs' inequality.
Non-negativity alone doesn't finish the argument; a sum of non-negative terms can still have large terms. It has to be combined with the budget. Averaged over what the truth could do, the bound still holds: the expected daily excesses sum to at most a constant at every horizon. Now none of them are negative, so there's nothing left to cancel against, and a convergent sum of non-negative terms forces the terms to shrink toward zero.
I did the two-outcome minimization before I knew it was Gibbs' inequality; the names came afterward. The step I added was that non-negativity only closes the loophole once it's combined with the budget.
Rathmanner and Hutter's "A Philosophical Treatise of Universal Induction" covers the convergence result properly.