Custom blockchain virtual machine development

Development of Custom Blockchain Virtual Machine

EVM is not the only possible architecture. It was designed in 2014 for specific compromises of that time: 256-bit stack machine with expensive storage, no parallelism, synchronous execution. Over ten years, problems emerged where EVM is suboptimal choice: game engines with thousands of states, ZK-friendly computation, specialized financial primitives. Custom VM is not reinventing the wheel, but engineering solution for specific performance/expressivity requirement.

Why Create VM From Scratch

Before designing custom VM, need to honestly answer: is task not solved by existing options?

When existing is sufficient:

• EVM + Solidity + Foundry cover 95% of DeFi tasks
• Solana SBF (sBPF) — high-performance DeFi, gaming
• CosmWasm — custom L1 with Cosmos SDK without VM writing
• Move (Aptos/Sui) — resource-oriented programming for asset safety

When custom VM is needed:

• ZK-proof generation: existing VMs (EVM, WASM) not optimized for finite field arithmetic. ZK-friendly VM built around Plonky2, Halo2 or STARK-compatible operations.
• Domain-specific language needed at execution level: e.g., chess engine VM where each opcode — chess position operation, state transitions verified on-chain.
• Parallel execution: EVM is sequential. Solana Sealevel executes transactions in parallel, knowing accounts in advance. Similar architecture for EVM-incompatible L1.
• Minified execution environment: embedded systems, IoT — minimal bytecode interpreter for embedded devices with on-chain verification.

Architectural Decisions When Designing VM

VM Type: Stack-based vs Register-based

Stack-based VM (EVM, WASM, JVM): operands taken from stack, result placed on stack. Simpler compiler, compact bytecode, but harder optimization — no explicit registers.

Register-based VM (Lua VM, Dalvik, SBF): operands — named registers. Faster interpretation (fewer push/pop ops), easier JIT compilation, bytecode slightly larger.

For blockchain usually choose stack-based VM: simpler to formally verify (important for audit), bytecode more compact (smaller calldata), and most compilers well-studied.

Arithmetic: 256-bit vs Field Arithmetic

EVM uses 256-bit integers — convenient for Ethereum addresses and hashes, wasteful for ZK-proof systems. ZK-friendly VM built around arithmetic in prime field F_p, where p — prime modulus of specific proof system:

• Plonky2: F_{2^64 - 2^32 + 1} (Goldilocks field)
• Groth16 / Plonk on BN254: p ≈ 2^254
• STARKs: F_{2^{64}-2^{32}+1} or F_{2^{31}-1}

Operations in these fields natively cheaper for proof generation. If goal is verifying VM execution via ZK proof, arithmetic field must match proof system.

Instruction Set Architecture (ISA)

When designing ISA, minimize number of opcodes while keeping Turing-completeness. Minimal viable set:

Arithmetic:   ADD, SUB, MUL, DIV, MOD
Bitwise:      AND, OR, XOR, NOT, SHL, SHR
Comparison:   EQ, LT, GT
Memory:       LOAD, STORE (or PUSH/POP for stack-based)
Control flow: JUMP, JUMPI, HALT
Stack ops:    PUSH_N, POP, DUP, SWAP
Crypto:       HASH (keccak256 or poseidon for ZK)

Specialized opcodes for domain: e.g., for financial VM — MULPERCENT (multiplication with percentages without overflow), SQRT (for AMM formulas). Each new opcode — additional complexity in interpreter implementation, JIT compiler, and formal verification.

Interpreter Implementation

Dispatch Loop

Basis of any VM — dispatch loop. Three approaches by performance:

Switch-case (simplest):

loop {
    let opcode = bytecode[pc];
    pc += 1;
    match opcode {
        0x01 => { // ADD
            let a = stack.pop()?;
            let b = stack.pop()?;
            stack.push(a.wrapping_add(b));
        }
        0x60 => { // PUSH1
            let val = bytecode[pc];
            pc += 1;
            stack.push(val as u64);
        }
        0x00 => break, // HALT
        _ => return Err(InvalidOpcode(opcode)),
    }
}

Computed goto / threaded dispatch (C/C++, GCC extension): instead of switch each opcode handler ends with jump to next handler directly. ~20-30% faster switch in hot loop.

JIT-compilation: compiling bytecode to native machine code before execution. Cranelift (Rust, used in Wasmtime) or LLVM backend. Gives 5–10x speedup for compute-intensive contracts. Adds significant complexity: need JIT compiler, security sandbox for generated code.

For blockchain VM JIT rarely used — execution environment must be deterministic and sandboxed. WebAssembly (Wasmtime/Wasmer) takes this ready-made.

Memory Model

EVM has three memory types: stack (1024 elements), memory (linear, grows on demand, paid for), storage (persistent, expensive). Custom VM can simplify to two:

Execution memory — local function memory, not persistent. Flat array with bounds checking:

struct Memory {
    data: Vec<u8>,
    gas_used: u64,
}
impl Memory {
    fn load(&mut self, offset: u32, size: u32) -> Result<&[u8]> {
        self.expand_to(offset + size)?;
        Ok(&self.data[offset as usize..(offset + size) as usize])
    }
    fn expand_to(&mut self, size: u32) -> Result<()> {
        if size as usize > self.data.len() {
            let gas = memory_expansion_cost(self.data.len(), size as usize);
            self.gas_used += gas;
            self.data.resize(size as usize, 0);
        }
        Ok(())
    }
}

State storage — persistent key-value. Behind blockchain — usually Merkle Patricia Trie (Ethereum) or Jellyfish Merkle Tree (Aptos/Diem) for state commitment. Implementation on top of RocksDB or LevelDB.

Gas Metering and Determinism

Determinism is mandatory requirement for blockchain VM. Same bytecode + state + input must give same result on any node. This means:

• No floating point arithmetic (IEEE 754 can give different results)
• Deterministic hash map traversal order (in Rust use BTreeMap, not HashMap)
• Versioned opcodes: new opcodes introduced only through hard fork
• Bounds for all operations: max stack size, max memory, max gas

Gas metering — mechanism preventing DoS through infinite loops. Each opcode has fixed gas cost. Before execution check and subtract:

fn execute_opcode(&mut self, opcode: u8) -> Result<()> {
    let cost = GAS_TABLE[opcode as usize];
    self.gas_remaining = self.gas_remaining
        .checked_sub(cost)
        .ok_or(OutOfGas)?;
    // ... execute opcode
}

Gas table for custom VM designed based on benchmarking real opcode execution times. Expensive operations (HASH, STORE) cost proportionally more.

ZK-Compatible VM: Specifics

If goal is proving VM execution via ZK proof (zkEVM or custom zkVM), architecture significantly more complex.

Execution trace: need to generate execution trace — table of all state transitions for each step. Proof system verifies trace correct relative to ISA.

Step | PC | Opcode | Stack[0] | Stack[1] | Memory | ...
-----|-----|--------|----------|----------|--------|----
0    | 0   | PUSH 5 | —        | —        | []     | ...
1    | 2   | PUSH 3 | 5        | —        | []     | ...
2    | 4   | ADD    | 5        | 3        | []     | ...
3    | 5   | HALT   | 8        | —        | []     | ...

Constraint system: for each opcode write polynomial constraints proving transition correctness. E.g., for ADD: stack_next[0] = stack[0] + stack[1]. All this — algebraic circuit for proof system.

Reference projects: RISC Zero (zkVM on RISC-V ISA), Valida (specifically for ZK-proof friendly ISA), Polygon Miden (STARK-based zkVM). Studying their open sources — mandatory step before designing own zkVM.

Development Stages

Development of production-ready custom VM — 12–24 months for team of 3–5 engineers. Most projects claiming "custom VM in a quarter" implement thin wrapper over WASM or EVM, not genuinely custom architecture.