eth_estimateGas was non-deterministic. One goroutine, cancelling the wrong request, was quietly turning a failure into a "success".
rpc/transactions/call.go · +83 / −11 · fixes #22870The one-liner
Ethereum's eth_estimateGas is supposed to return the smallest gas limit at which a transaction still succeeds. In Erigon it sometimes returned a value below that: use the estimate, and your transaction reverts out of gas.
The cause was not the gas math. It was a cancellation race on a shared EVM: a leftover goroutine from one probe cancelling the EVM in the middle of the next probe. The aborted probe reported no error, so a gas limit that should have failed got counted as a pass.
The symptom
eth_estimateGas runs a binary search. It calls your transaction over and over with different gas limits, looking for the boundary: the lowest limit that still executes without running out of gas.
Each of those trial calls is a probe. For speed, Erigon does not build a fresh EVM per probe. It keeps one EVM and calls Reset() between probes. That sharing is exactly where the bug lived.
The classification a probe returns is binary: the call either failed (out of gas, revert) or succeeded. If even one probe near the boundary is mislabelled as a success, the search happily walks below the true minimum and returns a limit that cannot actually pay for the call.
The mechanism
To enforce a timeout, each probe spawned a small watcher goroutine. It waited on two channels: cancel the EVM if the request's context is done, or exit quietly when the probe signals it is finished.
On a normal, fast return both of those became ready at almost the same instant. Go's select then picks one at random. Roughly half the time it picked the cancel branch and called evm.Cancel() anyway, on the EVM it shares with every other probe.
evm.Cancel() late, after Probe N+1 has already Reset() and started. That cancel sets the abort flag on the shared EVM. Probe N+1 is quietly aborted, and an aborted frame reports err == nil, so the search treats a gas limit that failed as one that passed. Tap to zoom.err == nil. To the binary search that is indistinguishable from a genuine success, and it converges below the real minimum.The select coin-flip, plus the timing of whether the stale cancel lands before or after the next probe's Reset(), is what made the result non-deterministic. Same call, same head, same node: 200 serial requests produced 20 different (all too-low) answers.
The fix
Drop the hand-rolled goroutine and two-channel select. Register the cancel as a context callback with context.AfterFunc, and defer its stop().
Defers run last-in-first-out, so stop() runs before the context is cancelled on the normal path. It deregisters the callback first, so on a clean return the cancel can never fire. A real timeout still fires it mid-execution and returns the timeout error, exactly as before.
- // done is closed on return to stop the watcher goroutine. - done := make(chan struct{}) - defer close(done) - go func() { - select { - case <-ctx.Done(): timedOut.Store(true); r.evm.Cancel() - case <-done: - } - }() + cancelled := make(chan struct{}) + stop := context.AfterFunc(ctx, func() { + defer close(cancelled) + timedOut.Store(true); r.evm.Cancel() + }) + defer func() { + if !stop() { <-cancelled } // join a callback that already started + }()
select with two ready branches, so nothing to pick wrong on the happy path.stop() returns false and the defer waits on cancelled, so any Cancel() completes inside this probe and cannot bleed into the next.setupEVMTimeout in eth_callMany.go, so it is a pattern Erigon already trusts.Why it matters
A gas estimate is a promise: pay this much and the call goes through. When the estimate lands below the true minimum, the promise breaks on-chain, the transaction reverts, and the fee is spent for nothing. Worse, it was intermittent, so it slipped past any check that runs the estimate once and trusts it.
The regression test makes the property explicit: 200 identical serial requests at a fixed head must return one value. It reproduced the old behaviour reliably (20 distinct values) and pins the new one at exactly one.