Add prefetch trie nodes #1871

Raneet10 · 2025-11-06T10:03:15Z

Description

This PR enables prefetching of trie nodes before execution of block. It adds a waitforwarm config which if enabled will block execution of block until the warming finishes.

Changes

Bugfix (non-breaking change that solves an issue)
Hotfix (change that solves an urgent issue, and requires immediate attention)
New feature (non-breaking change that adds functionality)
Breaking change (change that is not backwards-compatible and/or changes current functionality)
Changes only for a subset of nodes

Checklist

I have added at least 2 reviewer or the whole pos-v1 team
I have added sufficient documentation in code
I will be resolving comments - if any - by pushing each fix in a separate commit and linking the commit hash in the comment reply
Created a task in Jira and informed the team for implementation in Erigon client (if applicable)
Includes RPC methods changes, and the Notion documentation has been updated

Testing

I have added unit tests
I have added tests to CI
I have tested this code manually on local environment
I have tested this code manually on remote devnet using express-cli
I have tested this code manually on amoy
I have created new e2e tests into express-cli

…flag for txpool prefetch

This reverts commit 6b1c3f1.

This reverts commit 8ef31e3.

…warm + rm disklayer read

…est-prefetch-trie-nodes

marcello33 · 2025-11-06T12:43:27Z

core/txpool/legacypool/legacypool.go

+	header := pool.currentHead.Load()
+	pool.mu.RUnlock()
+
+	if header == nil || len(txs) == 0 {


Shall we move the txs len check on top of the function?
Just an optimization to reduce lock contention. Since txs is an input and doesn’t depend on pool state, you can fast-return before touching any shared data.

marcello33 · 2025-11-06T12:45:08Z

core/blockchain.go

 	storageCacheHitMeter  = metrics.NewRegisteredMeter("chain/storage/reads/cache/process/hit", nil)
 	storageCacheMissMeter = metrics.NewRegisteredMeter("chain/storage/reads/cache/process/miss", nil)

 	accountCacheHitPrefetchMeter  = metrics.NewRegisteredMeter("chain/account/reads/cache/prefetch/hit", nil)


Shall we remove all the unused metrics here? Lint's complaining

marcello33 · 2025-11-06T12:46:41Z

core/blockchain.go

 	defer cancel()

 	if followupInterrupt == nil {
 		followupInterrupt = &atomic.Bool{}


is linter right here?

Not sure. I didn't make this change.

marcello33 · 2025-11-06T12:47:53Z

core/blockchain.go

-		vmCfg := bc.cfg.VmConfig
-		vmCfg.Tracer = nil
-		bc.prefetcher.Prefetch(block, throwaway, vmCfg, followupInterrupt)
+	// go func(start time.Time, throwaway *state.StateDB, block *types.Block) {


Is this worth deleting?
Or - alternatively - have a comment to explain why it's disabled

I commented it out so that we don't run multiple prefetches together. I'll probably remove this once we finalize all the optimizations.

marcello33 · 2025-11-06T12:49:21Z

core/blockchain.go

+		return nil, 0
+	}
+	vmCfg := bc.cfg.VmConfig
+	vmCfg.Tracer = nil


why is this required? Same for PrefetchFromTxpool

marcello33 · 2025-11-06T12:51:48Z

core/blockchain_insert.go

+		// Append cache hit/miss percentages
+		if bc != nil {
+			if total := bc.lastProcAccHit + bc.lastProcAccMiss; total > 0 {
+				pct := int64(100 * bc.lastProcAccHit / total)


seems like this (and others in the same func) conversion is pleonastic

Yeah right. I don't think these will make it to develop anyway.

marcello33 · 2025-11-06T12:54:44Z

go.mod


 // Note: Change the go image version in Dockerfile if you change this.
-go 1.24.6
+go 1.24.8


worth updating in the CI yml fles as well.
Also, since we're bumping, maybe we can use 1.24.9 to solve the govulncheck issues (I have that on my upstream PR as well)

Could do yeah but anyway this isn't getting merged to develop just yet.

cffls · 2025-11-06T19:44:15Z

core/analyzer.sh

@@ -0,0 +1,102 @@
+#   ./log_analyzer.sh "3 days ago" # Scans the last 3 days


This should be excluded from this PR I guess?

cffls · 2025-11-06T19:53:28Z

core/state_prefetcher.go

+			// Aggregate touched pre-state from stateCpy and populate the process reader cache directly
+			acc, stor := stateCpy.PrefetchTouched()
+			for a := range acc {
+				reader.Account(a)
+			}
+			for a, bucket := range stor {
+				for k := range bucket {
+					reader.Storage(a, k)
+				}
+			}


What's the rationale of this change? IntermediateRoot also fetches touched account/storages.

Yes but it also calculates state root which we strictly don't need. Pre-execution should fill up the accounts/storages. Do you think it still should be there ?

It should be there. Simply loading account/storage isn't sufficient, because in order to calculate the final state root hash, some sibling nodes will need to be retrieved, even though they aren't related to touched accounts.

cffls · 2025-11-06T20:17:38Z

core/blockchain.go

+		vmCfg.Tracer = nil
+		synthetic := types.NewBlockWithHeader(header).WithBody(types.Body{Transactions: txs})
+		var noop atomic.Bool
+		bc.prefetcher.Prefetch(synthetic, warmdb, vmCfg, &noop)


This call execute transactions without taking fees from transactions, which introduces a DoS vector. Malicious users can submit resource-expensive transaction and replace them with a bit higher gas continuously without paying the gas fee.

For example, if the current base gas price is 50Gwei, I can submit a transaction with 32M gas of SLOAD, with a initial gas price of 1Gwei, and continuously replace the existing tx by bumping up the gas by 10% for 41 times before reaching the 50Gwei base price (1Gwei * 1.1^41 = 49Gwei). This will result in the node to fully execute this transaction 41 times before the transaction is actually included in a block.

Good catch. Though I am planning to remove PrefetchFromTxpool altogether as I didn't see any impact of prefetching from txpool. Perhaps running it in worker (which is present currently) is more deterministic ?

codecov · 2025-11-07T10:03:03Z

Codecov Report

❌ Patch coverage is 49.66667% with 151 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (src-optimizations@a63c799). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
core/blockchain.go	49.45%	39 Missing and 7 partials ⚠️
core/state/reader.go	2.70%	36 Missing ⚠️
core/state/statedb.go	0.00%	22 Missing ⚠️
core/blockchain_insert.go	57.50%	13 Missing and 4 partials ⚠️
miner/worker.go	63.41%	10 Missing and 5 partials ⚠️
core/state_prefetcher.go	86.53%	7 Missing ⚠️
eth/ethconfig/gen_config.go	0.00%	6 Missing ⚠️
eth/backend.go	0.00%	1 Missing ⚠️
miner/fake_miner.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@                 Coverage Diff                  @@
##             src-optimizations    #1871   +/-   ##
====================================================
  Coverage                     ?   47.92%           
====================================================
  Files                        ?      844           
  Lines                        ?   143880           
  Branches                     ?        0           
====================================================
  Hits                         ?    68959           
  Misses                       ?    70402           
  Partials                     ?     4519

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

sonarqubecloud · 2025-11-20T08:49:51Z

Quality Gate passed

Issues
5 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.9% Duplication on New Code

See analysis details on SonarQube Cloud

cffls · 2025-12-04T04:35:06Z

core/state/reader.go

+
+// SeedCaches inserts the supplied accounts and storage slots directly into the
+// local caches.
+func (r *ReaderWithCacheStats) SeedCaches(accounts map[common.Address]*types.StateAccount, storage map[common.Address]map[common.Hash]common.Hash) {


How do we know/ensure the data provided by seeded cache is always correct?

Well the aim is to warm the cache with unmodified account/storage slots that are touched during the tx. I presume the data should be correct for most cases except maybe if there's a tx dependency.
During testing on mainnet ndoe, however, I never saw a failure due to this.

Could you help me understand the lifecycle of the cache? For example, when is value written, read, or deleted? Do they all happen for the same block? What if a new block read into an old value that should've been updated?

Sure, sorry for not clarifying earlier. So basically:

We start warming process for the individual block here.

This would create a cached state reader and calls warmReaderCache.

warmReaderCache calls the Prefetch routine.

The Prefetch method does the seeding.

The warmed reader is used for actual processing of that block.

The statedb is updated and we go again for the next block.

Thanks for the explanation!

cffls · 2025-12-04T04:40:35Z

miner/worker.go

-			return err
+			// start warm reader cache
+			tasks, _ := core.NewWarmExecTasks(w.chain, w.chainConfig, env.header, env.state, env.coinbase, txs, vm.Config{})
+			numProcs := max(1, 4*runtime.NumCPU()/5)


How is this calculation derived?

I think this was the same value being used in state prefetcher from upstream.

cffls · 2025-12-04T04:43:17Z

core/parallel_state_processor.go

+	blockCtx := NewEVMBlockContext(header, bc, &coinbase)
+	shouldDelayFeeCal := false
+
+	for i, tx := range txs {


There could be large amount of transactions in txs, which may cause unnecessary warm up. For example, if there are 10k pending txs, but only 20% of them will be included in the next block, 80% of compute would be wasted. One way of limiting is to include transactions until the sum of their gas limit reaches the block gas limit.

True but before the callsite we are filtering the txs.

I see. On the other hand, with this filter, we might not be able to fill the block. For example, if a malicious user submit 2 transactions with high gas price and each transaction has gas limit of 30M, we will only mine these two transactions. However, if each transactions actually consumes only 1M gas, the block will be only filled with 2M gas in total.

Correct but I think the only way to accurately determine the gas used is to execute the txs and this is a time consuming process itself.

* bump version * stable version * V2.5.6 candidate (#1941) * Revert recorded write operations when tx fails * Revert selfdestruct on failure * Added test for selfdestruct revert write * Add HasSelfDestructed check when applying to final db * p2p: added functionality to temporarily ban peers to prevent connections (#1922) * p2p: added functionality to temporarily ban peers to prevent connections * p2p: cleanup * chore: bump kurtosis-pos (#1923) * chore: bump kurtosis-pos * chore: nits * chore: install go * fix(miner): delete pending task after successful block sealing to prevent node stall (#1929) * fix(miner): delete pending task after successful block sealing to prevent node stall * chore: nits * chore: set staleThreshold to 0 * chore: update comment * Remove Recommit and Fix Parent Actual Time track on Prepare (#1938) * remove recommit and track parent actual time * applying pr comments * fix actual time cache store place * chore: update deps (#1939) chore: update deps * bump version * bump version * stable version --------- Co-authored-by: Angel Valkov <[email protected]> Co-authored-by: Marcello Ardizzone <[email protected]> Co-authored-by: Pratik Patil <[email protected]> Co-authored-by: Krishang <[email protected]> Co-authored-by: kamuikatsurgi <[email protected]> * Unstuck * Added block range check * bump version 2.5.7 * bump to 2.5.6-beta3 --------- Co-authored-by: Lucca Martins <[email protected]> Co-authored-by: Marcello Ardizzone <[email protected]> Co-authored-by: Pratik Patil <[email protected]> Co-authored-by: Krishang <[email protected]> Co-authored-by: kamuikatsurgi <[email protected]>

…block building work (#1943) * bump version * stable version * V2.5.6 candidate (#1941) * Revert recorded write operations when tx fails * Revert selfdestruct on failure * Added test for selfdestruct revert write * Add HasSelfDestructed check when applying to final db * p2p: added functionality to temporarily ban peers to prevent connections (#1922) * p2p: added functionality to temporarily ban peers to prevent connections * p2p: cleanup * chore: bump kurtosis-pos (#1923) * chore: bump kurtosis-pos * chore: nits * chore: install go * fix(miner): delete pending task after successful block sealing to prevent node stall (#1929) * fix(miner): delete pending task after successful block sealing to prevent node stall * chore: nits * chore: set staleThreshold to 0 * chore: update comment * Remove Recommit and Fix Parent Actual Time track on Prepare (#1938) * remove recommit and track parent actual time * applying pr comments * fix actual time cache store place * chore: update deps (#1939) chore: update deps * bump version * bump version * stable version --------- Co-authored-by: Angel Valkov <[email protected]> Co-authored-by: Marcello Ardizzone <[email protected]> Co-authored-by: Pratik Patil <[email protected]> Co-authored-by: Krishang <[email protected]> Co-authored-by: kamuikatsurgi <[email protected]> * fix: add pendingWorkBlock to prevent duplicate work * fix: if check * nit: comment * chore: bump version * chore: add pending work check in new head event as well * chore: read veblop timeout --------- Co-authored-by: Lucca Martins <[email protected]> Co-authored-by: Angel Valkov <[email protected]> Co-authored-by: Marcello Ardizzone <[email protected]> Co-authored-by: Pratik Patil <[email protected]>

github-actions · 2025-12-26T00:27:20Z

This PR is stale because it has been open 21 days with no activity. Remove stale label or comment or this will be closed in 14 days.

…/test-prefetch-trie-nodes

sonarqubecloud · 2026-01-05T14:58:19Z

Quality Gate passed

Issues
7 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.7% Duplication on New Code

See analysis details on SonarQube Cloud

praetoriansentry and others added 23 commits October 21, 2025 15:29

feat: adding pre exec

07daffb

core/txpool: add trie prefetch

ccab3fe

core: track state root calc vs block exec

25c5569

core/txpool: start prefetch service

b4b035c

core/txpool: try avoiding copying of state

04544cf

core/txpool: temp disable prefetcher

0d0128b

core: add global prefetch reader

c7ac291

core: store trie root

7f58d8f

core,internal/cli: add prewarming prior to execution and log stats + …

1c88a2a

…flag for txpool prefetch

core: rm timeout from prewarmStateForBlock

f0d1c50

core: try caching readerWithCache

653424c

core: make seedReaderCache async

8ef31e3

core: return semi-warmed reader

6b1c3f1

Revert "core: return semi-warmed reader"

46529c1

This reverts commit 6b1c3f1.

Revert "core: make seedReaderCache async"

fa10617

This reverts commit 8ef31e3.

core: log validation time + re-enable throwaway

59ea31f

core: log time for seedReaderCache

e37cd47

core,triedb: try skipping to disklayer while prefetch

0031a7e

triedb: do hash check for disklayer-first fetch

86198b6

core,eth,internal/cli,miner,triedb: add config to optionally wait on …

31d812c

…warm + rm disklayer read

Merge branch 'develop' of github.com:maticnetwork/bor into raneet10/t…

624867a

…est-prefetch-trie-nodes

core,triedb: rm uncommittedStorage from cache + some cleanup

aef9546

core: more cleanup

e263992

Raneet10 requested a review from a team November 6, 2025 10:03

core: fix test

ca67765

marcello33 reviewed Nov 6, 2025

View reviewed changes

cffls reviewed Nov 6, 2025

View reviewed changes

Raneet10 added 2 commits November 7, 2025 11:11

core: try directly writing to reader cache

854e5e2

core: try adding touched acc/storage directly in reader cache

ab0993b

Raneet10 added 6 commits November 10, 2025 10:09

core,miner: rm PrefetchFromTxPool

4bdc1f4

core: rm analyzer script + unused metrics

10fb3df

core: re-enable IntermediateRoot in Prefetch

821e07e

core,miner: use blockstm executor for warming in worker

bda9e9d

core,eth,internal/cli,miner: add warminworker

8ba0702

miner: filter txs in worker warmup

5069130

cffls reviewed Dec 4, 2025

View reviewed changes

lucca30 and others added 4 commits December 16, 2025 14:47

bump version

41f1ba8

chore: bump version to v2.5.6-beta4

3e9fc69

github-actions bot added the Stale label Dec 26, 2025

Raneet10 removed the Stale label Dec 26, 2025

Raneet10 changed the base branch from src-optimizations to develop December 30, 2025 06:56

Raneet10 added 3 commits December 30, 2025 12:28

Merge tag 'v2.5.6-beta4' of github.com:maticnetwork/bor into raneet10…

beccf3d

…/test-prefetch-trie-nodes

miner: fix build

c7545cd

miner: filter warmup txs based on historic contract usage

4a02b78

		@@ -0,0 +1,102 @@
		# ./log_analyzer.sh "3 days ago" # Scans the last 3 days

Add prefetch trie nodes #1871

Are you sure you want to change the base?

Add prefetch trie nodes #1871

Uh oh!

Conversation

Raneet10 commented Nov 6, 2025

Description

Changes

Checklist

Testing

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Nov 7, 2025

Codecov Report

Uh oh!

sonarqubecloud bot commented Nov 20, 2025

Quality Gate passed

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Dec 26, 2025

Uh oh!

sonarqubecloud bot commented Jan 5, 2026

Quality Gate passed

Uh oh!

Reviewers

Assignees