Erigon PR #22877 ● merged go · concurrency

Same call, different gas, every time.

eth_estimateGas was non-deterministic. One goroutine, cancelling the wrong request, was quietly turning a failure into a "success".

erigontech/erigon · rpc/transactions/call.go · +83 / −11 · fixes #22870

The one-liner

Ethereum's eth_estimateGas is supposed to return the smallest gas limit at which a transaction still succeeds. In Erigon it sometimes returned a value below that: use the estimate, and your transaction reverts out of gas.

The cause was not the gas math. It was a cancellation race on a shared EVM: a leftover goroutine from one probe cancelling the EVM in the middle of the next probe. The aborted probe reported no error, so a gas limit that should have failed got counted as a pass.

20
distinct gas values from 200 identical serial calls, before the fix
1
distinct value after the fix, fully deterministic
1
shared EVM, reused across every probe of the binary search

The symptom

A binary search that lands too low

eth_estimateGas runs a binary search. It calls your transaction over and over with different gas limits, looking for the boundary: the lowest limit that still executes without running out of gas.

Each of those trial calls is a probe. For speed, Erigon does not build a fresh EVM per probe. It keeps one EVM and calls Reset() between probes. That sharing is exactly where the bug lived.

The classification a probe returns is binary: the call either failed (out of gas, revert) or succeeded. If even one probe near the boundary is mislabelled as a success, the search happily walks below the true minimum and returns a limit that cannot actually pay for the call.

The mechanism

One goroutine, cancelling the wrong request

To enforce a timeout, each probe spawned a small watcher goroutine. It waited on two channels: cancel the EVM if the request's context is done, or exit quietly when the probe signals it is finished.

On a normal, fast return both of those became ready at almost the same instant. Go's select then picks one at random. Roughly half the time it picked the cancel branch and called evm.Cancel() anyway, on the EVM it shares with every other probe.

time shared EVM · Reset() between probes abort flag = set Probe N gas = G · returns OK Probe N+1 gas = G′ · truly runs out of gas Reset() → stale evm.Cancel() fires after probe N is done, lands inside probe N+1 aborted frame → err == nil Probe N+1 was cut short, but the frame-entry check reports no error… …so it is counted as a success.
The race in one picture. Probe N finishes fine. Its leftover watcher fires evm.Cancel() late, after Probe N+1 has already Reset() and started. That cancel sets the abort flag on the shared EVM. Probe N+1 is quietly aborted, and an aborted frame reports err == nil, so the search treats a gas limit that failed as one that passed. Tap to zoom.
Why "no error" is the trap. In the EVM, cancelling a call aborts the current frame on entry. An aborted frame is not an execution error, so it comes back with err == nil. To the binary search that is indistinguishable from a genuine success, and it converges below the real minimum.

Why it looked random

The select coin-flip, plus the timing of whether the stale cancel lands before or after the next probe's Reset(), is what made the result non-deterministic. Same call, same head, same node: 200 serial requests produced 20 different (all too-low) answers.

The fix

Deregister before you can cancel

Drop the hand-rolled goroutine and two-channel select. Register the cancel as a context callback with context.AfterFunc, and defer its stop().

Defers run last-in-first-out, so stop() runs before the context is cancelled on the normal path. It deregisters the callback first, so on a clean return the cancel can never fire. A real timeout still fires it mid-execution and returns the timeout error, exactly as before.

rpc/transactions/call.goDoCallWithNewGas
- // done is closed on return to stop the watcher goroutine.
- done := make(chan struct{})
- defer close(done)
- go func() {
-   select {
-   case <-ctx.Done(): timedOut.Store(true); r.evm.Cancel()
-   case <-done:
-   }
- }()

+ cancelled := make(chan struct{})
+ stop := context.AfterFunc(ctx, func() {
+   defer close(cancelled)
+   timedOut.Store(true); r.evm.Cancel()
+ })
+ defer func() {
+   if !stop() { <-cancelled }  // join a callback that already started
+ }()

Why it matters

An estimate you can build on

A gas estimate is a promise: pay this much and the call goes through. When the estimate lands below the true minimum, the promise breaks on-chain, the transaction reverts, and the fee is spent for nothing. Worse, it was intermittent, so it slipped past any check that runs the estimate once and trusts it.

The regression test makes the property explicit: 200 identical serial requests at a fixed head must return one value. It reproduced the old behaviour reliably (20 distinct values) and pins the new one at exactly one.

The wider lesson. Reusing one stateful object across calls is a fine optimisation, right up until a cleanup path from the previous call can still touch it. Shared mutable state plus a racing canceller is the whole bug, in any language.