Reddio’s Optimization of EVM through Multi-threaded Parallelization

The Ethereum Virtual Machine (EVM) is widely recognized as both the "execution engine" and the "smart contract execution environment" of Ethereum, making it one of the blockchain's most critical components. A public blockchain consists of an open network with thousands of nodes, each potentially differing in hardware specifications. To ensure that smart contracts produce identical results across all nodes, achieving "consistency," it is essential to establish a uniform environment across various devices. Virtual machines make this possible.

The EVM enables smart contracts to run uniformly across different operating systems, such as Windows, Linux, and macOS. This cross-platform compatibility ensures that every node, regardless of its system, achieves the same results when executing a contract. A prime example of such technology is the Java Virtual Machine (JVM).

The smart contracts we typically see on block explorers are first compiled into EVM bytecode before being stored on the blockchain. When the EVM executes a contract, it reads the bytecode sequentially. Each instruction in the bytecode (opcode) has an associated gas cost. The EVM tracks the gas consumption of each instruction during execution, with the consumption depending on the complexity of the operation.

Furthermore, as the core execution engine of Ethereum, the EVM processes transactions in a serial manner. All transactions are queued in a single line and executed in a specific order. Parallel processing is not used because the blockchain must strictly maintain consistency. A batch of transactions must be processed in the same order across all nodes. If transactions were processed in parallel, it would be difficult to accurately predict the transaction order unless a corresponding scheduling algorithm is introduced, which would add complexity.

In 2014-15, due to time constraints, the Ethereum founding team chose a serial execution method because it was simple to design and easy to maintain. However, as blockchain technology has evolved and the user base has grown, the demand for higher TPS (transactions per second) and throughput has increased. With the emergence and maturity of Rollup technology, the performance bottleneck caused by EVM's serial execution has become increasingly apparent on Ethereum Layer 2.

As a key component of Layer 2, the Sequencer handles all computation tasks as a single server. If the efficiency of external modules working with the Sequencer is sufficiently high, the final bottleneck will depend on the efficiency of the Sequencer itself. At this point, serial execution becomes a significant obstacle.

The opBNB team once achieved extreme optimization of the DA layer and data read-write modules, allowing the Sequencer to process up to around 2000 ERC-20 transfers per second. While this number may seem high, if the transactions being processed are much more complex than ERC-20 transfers, the TPS value will inevitably decrease. Therefore, parallelization of transaction processing will be an inevitable trend in the future.

Next, we will delve into more specific details to explain the limitations of the traditional EVM and the advantages of a parallel EVM.

Two core components of Ethereum transaction execution

At the code module level, besides the EVM, another core component related to transaction execution in go-ethereum is stateDB, which is used to manage account states and data storage in Ethereum. Ethereum uses a tree structure called Merkle Patricia Trie as the database index (or directory). Each time a transaction is executed by the EVM, certain data stored in stateDB is modified, and these changes are eventually reflected in the Merkle Patricia Trie (hereafter referred to as the global state tree).

Specifically, stateDB is responsible for maintaining the state of all Ethereum accounts, including both EOA (Externally Owned Account) accounts and contract accounts. The data it stores includes account balances, smart contract code, and more. During transaction execution, stateDB performs read and write operations on the relevant account data. After the transaction execution is complete, stateDB needs to submit the new state to the underlying database (such as LevelDB) for persistence.

In summary, the EVM interprets and executes smart contract instructions, altering the blockchain’s state based on the computation results, while stateDB acts as the global state storage, managing all account and contract state changes. Together, they build Ethereum’s transaction execution environment.

The Process of Serial Execution

There are two types of transactions in Ethereum: EOA transfers and contract transactions. EOA transfers are the simplest transaction type, which is ETH transfers between ordinary accounts. These transactions do not involve contract calls and are processed very quickly. Due to the simplicity of the operation, the gas fee charged for EOA transfers is very low.

Unlike simple EOA transfers, contract transactions involve calling and executing smart contracts. When processing contract transactions, the EVM interprets and executes each bytecode instruction in the smart contract. The more complex the contract logic means the more instructions involved, which ultimately means the more resources consumed.

For example, processing an ERC-20 transfer takes about twice as long as an EOA transfer. For more complex smart contracts, such as transactions on Uniswap, it takes even longer, potentially being several times slower than an EOA transfer. This is because DeFi protocols require handling complex logic such as liquidity pools, price calculations, and token swaps during transactions, requiring very complex calculations.

So, in the serial execution model, how do the EVM and stateDB work together to process transactions?

In Ethereum’s design, transactions within a block are processed sequentially, one by one. Each transaction (tx) has an independent instance used to perform the specific operations of that transaction. Although each transaction uses a different EVM instance, all transactions share the same state database, which is stateDB.

During transaction execution, the EVM continuously interacts with stateDB, reading relevant data from stateDB and writing the modified data back to stateDB.

Let's take a look at how the EVM and stateDB collaborate to execute transactions from a code perspective:

1. The processBlock() function calls the Process() function to handle the transactions within a block.

// processBlock attempts to process and commit a block into the blockchain.
func (bc *BlockChain) processBlock(block *types.Block) error {
    //...
    // Initialize the statedatabase
    statedb := bc.state.NewBlockchainState()

    // Process the block's transactions and retrieve receipts
    receipts, logs, usedGas, err := bc.processor.Process(block, statedb, bc.vmConfig)
    if err != nil {
        return err
    }

    // Commit state changes and block to the database
    return bc.writeBlockWithState(block, receipts, logs, statedb, true)
    //...
}

1. A for loop is defined in the Process() function, where transactions are executed one by one.

func (p *StateProcessor) Process(block *types.Block, statedb *state.StateDB, cfg vm.Config) (*ProcessResult, error) {
    // Iterate over and process the individual transactions
    for i, tx := range block.Transactions() {
        msg, err := TransactionToMessage(tx, signer, header.BaseFee)
        if err != nil {
            return nil, fmt.Errorf("could not apply tx %d [%v]: %w", i, tx.Hash().Hex(), err)
        }

        statedb.SetTxContext(tx.Hash(), i)

        receipt, err := ApplyTransactionWithEVM(msg, p.config, gp, statedb, blockNumber, blockHash, tx, usedGas, vmenv)
        if err != nil {
            return nil, fmt.Errorf("could not apply tx %d [%v]: %w", i, tx.Hash().Hex(), err)
        }

        receipts = append(receipts, receipt)
        allLogs = append(allLogs, receipt.Logs...)
    }

    //...
}

1. After all transactions are processed, the processBlock() function calls the writeBlockWithState() function, which then calls the statedb.Commit() function to commit the state changes.

// writeBlockWithState writes a block's state to the chain database.
func (bc *BlockChain) writeBlockWithState(block *types.Block, receipts types.Receipts, logs []*types.Log, statedb *state.StateDB, cfg vm.Config) error {
    //...

    // Commit the state changes to the blockchain
    root, err := statedb.Commit(true)
    if err != nil {
        return err
    }

    // Write the block and state root to the chain database
    if err := bc.db.WriteBlockWithState(block, root, receipts, logs, statedb.IntermediateRoot(false).Bytes()); err != nil {
        return err
    }
    if emitHeadEvent {
        // Emit a head event if this is the canonical head
        bc.chainHeadFeed.Send(events.ChainHeadEvent{Block: block})
    }

    return nil
    //...
}

Once all transactions in a block have been executed, the data in stateDB is committed to the global state tree (Merkle Patricia Trie) mentioned earlier, generating a new state root (stateRoot). The state root is an important parameter in each block, recording the "compressed result" of the new global state after the block execution.

It’s easy to understand the bottleneck of the EVM's serial execution model: transactions must be queued and executed sequentially. If there is a time-consuming smart contract transaction, other transactions can only wait until it is completed, which clearly cannot fully utilize CPU and other hardware resources, significantly limiting efficiency.

Multi-threaded Parallel Optimization of EVM

To compare serial execution and parallel execution with a real-life example, the former is like a bank with only one counter, while parallel EVM is like a bank with multiple counters. In a parallel mode, multiple threads can be started to process multiple transactions simultaneously, resulting in several times the efficiency improvement. However, the tricky part lies in the issue of state conflicts.

If multiple transactions attempt to modify data for the same account and are processed simultaneously, conflicts may arise. For example, if only one NFT can be minted and both Transaction 1 and Transaction 2 request to mint it, fulfilling both requests would obviously result in an error. Such situations require coordination. In practice, state conflicts occur more frequently than the example mentioned, so if transaction processing is to be parallelized, measures to handle state conflicts are essential.

Reddio's Parallel Optimization of EVM

As a ZKRollup project for EVM, Reddio’s parallel optimization approach involves allocating one transaction to each thread and providing a temporary state database in each thread, calledpending-stateDB. The specific details are as follows:

1. Multi-threaded Parallel Transaction Execution: Reddio sets up multiple threads to process different transactions simultaneously, with each thread operating independently. This approach can enhance transaction processing speed several times over.
2. Allocating a Temporary State Database for Each Thread: Reddio assigns each thread an independent temporary state database (pending-stateDB). During transaction execution, each thread does not directly modify the global stateDB but instead temporarily records state changes in the pending-stateDB.
3. Synchronizing State Changes: Once all transactions within a block have been executed, the EVM sequentially synchronizes the state changes recorded in each pending-stateDB to the global stateDB. If no state conflicts occurred during transaction execution, the records in the pending-stateDB can be successfully merged into the global stateDB.

We optimized the handling of read and write operations to ensure that transactions can correctly access state data and avoid conflicts.

Write Operations: All write operations (i.e., state modifications) are not directly written to the global stateDB but are first recorded in the WriteSet of the pending-state. After transaction execution is completed, the state changes are then merged into the global stateDB through conflict detection.

Read Operation: When a transaction needs to read a state, the EVM first checks the ReadSet of the pending-state. If the ReadSet indicates that the required data is available, the EVM directly reads the data from the pending-stateDB. If the corresponding key-value pair is not found in the ReadSet, it retrieves the historical state data from the global stateDB of the previous block.

The key issue in parallel execution is state conflicts, which become particularly prominent when multiple transactions attempt to read or write the state of the same account. To address this, Reddio introduces a conflict detection mechanism:

• Conflict Detection: During transaction execution, the EVM monitors the ReadSet and WriteSet of different transactions. If it detects that multiple transactions are attempting to read or write the same state item, it considers this a conflict.
• Conflict Handling: When a conflict is detected, the conflicting transactions are marked for re-execution. To prevent repeated conflicts and indefinite re-execution, transactions are re-queued with adjusted priority or in a sequence that reduces the likelihood of recurring conflicts. Additionally, conflict resolution mechanisms—such as lock-based access controls or transaction isolation strategies—are implemented.

After all transactions are completed, the change records from multiple pending-stateDBs are merged into the global stateDB. If the merge is successful, the EVM commits the final state to the global state tree and generates a new state root. The performance improvement from multi-threaded parallel optimization is evident, especially when handling complex smart contract transactions.

According to research on parallel EVM, in low-conflict workloads (where there are fewer conflicting transactions or transactions occupying the same resources in the transaction pool), benchmarked TPS shows an improvement of about 3–5 times compared to traditional serial execution. In high-conflict workloads, theoretically, if all optimization methods are applied, the difference in improvement can even reach up to 60 times.

Summary

Reddio’s multi-threaded parallel optimization scheme for the EVM significantly enhances transaction processing capacity by allocating a temporary state database for each transaction and executing transactions in parallel across different threads. By optimizing read and write operations and introducing conflict detection, EVM-based blockchains can achieve large-scale parallelization of transactions while ensuring state consistency, addressing the performance bottlenecks of traditional serial execution. This lays an important foundation for the future development of Ethereum Rollups.