DeFi Strategy Backtesting System Development

Developing DeFi Strategy Backtesting System

"Strategy showed 200% APY on backtest" — often means backtest was written wrong. Most common error: strategy uses closing price of candle it's also making entry decision on. This is look-ahead bias — in real time you don't know current candle's closing price. Second variant: backtest ignores gas costs and slippage, turning unprofitable strategy into profitable on paper.

DeFi-specific backtesting is harder than traditional finance because you need on-chain data: historical pool states, real lending protocol rates, historical gas prices, liquidation and flash loan events. Everything changes per-block.

Sources of Historical On-Chain Data

The Graph and Subgraph Archives

The Graph indexes on-chain events from deployment block. Most major DeFi protocols (Uniswap v2/v3, Aave v2/v3, Compound, Curve) have official subgraphs with history of all swap, deposit, borrow events.

Problem: hosted service Graph has rate limits and periodically loses data on reindexing. For serious backtesting need either own Graph Node with Ethereum archive node, or commercial sources (Dune Analytics, Flipside Crypto, Goldsky).

Dune Analytics — SQL interface to decoded on-chain data. Write queries against any contract's events. Limitation: API for programmatic access expensive, but one-time dataset exports free.

Archive Nodes

Some data can't be gotten from events — need to read specific block state. Example: balanceOf of address in historical block, token totalSupply, AMM pool price at specific moment.

For this need archive node (archive node) — full state history. Infura, Alchemy, QuickNode provide archive access via eth_call with blockNumber parameter. Own Ethereum archive node takes 12+ TB.

from web3 import Web3

w3 = Web3(Web3.HTTPProvider(ARCHIVE_RPC_URL))

def get_pool_reserves_at_block(pool_address: str, block_number: int) -> tuple:
    """Get Uniswap v2 pool reserves at specific block"""
    pool = w3.eth.contract(address=pool_address, abi=UNISWAP_V2_PAIR_ABI)
    reserves = pool.functions.getReserves().call(block_identifier=block_number)
    return reserves[0], reserves[1]

Backtesting System Architecture

Data Layer

Load and normalize historical data into local PostgreSQL database. Schema:

-- Historical Uniswap v3 swap events
CREATE TABLE uniswap_v3_swaps (
    block_number    BIGINT NOT NULL,
    block_timestamp TIMESTAMPTZ NOT NULL,
    tx_hash         BYTEA NOT NULL,
    pool_address    VARCHAR(42) NOT NULL,
    amount0         NUMERIC(78, 0),
    amount1         NUMERIC(78, 0),
    sqrt_price_x96  NUMERIC(78, 0),
    tick            INTEGER,
    liquidity       NUMERIC(78, 0),
    PRIMARY KEY (tx_hash, pool_address)
);

-- Historical lending rates (Aave)
CREATE TABLE aave_rate_history (
    block_number        BIGINT NOT NULL,
    block_timestamp     TIMESTAMPTZ NOT NULL,
    asset               VARCHAR(42) NOT NULL,
    liquidity_rate      NUMERIC(40, 0),  -- RAY
    variable_borrow_rate NUMERIC(40, 0),
    utilization_rate    NUMERIC(20, 18),
    PRIMARY KEY (block_number, asset)
);

-- Historical gas price
CREATE TABLE gas_price_history (
    block_number    BIGINT PRIMARY KEY,
    block_timestamp TIMESTAMPTZ NOT NULL,
    base_fee_gwei   NUMERIC(20, 9),
    priority_fee_p50 NUMERIC(20, 9)
);

Simulation Engine

Engine iterates blocks sequentially, calls strategy with available data per block:

class BacktestEngine:
    def __init__(self, strategy: Strategy, start_block: int, end_block: int):
        self.strategy = strategy
        self.db = DataLayer()
        
    def run(self) -> BacktestResult:
        portfolio = Portfolio(initial_capital=self.strategy.config.initial_capital)
        
        for block_data in self.db.iter_blocks(self.start_block, self.end_block):
            # Only data before current block — no look-ahead
            context = MarketContext(
                block=block_data,
                prices=self.db.get_prices_at(block_data.number),
                lending_rates=self.db.get_rates_at(block_data.number),
                gas_price=block_data.base_fee + block_data.priority_fee_p50,
            )
            
            signals = self.strategy.generate_signals(context, portfolio)
            
            for signal in signals:
                # Apply realistic execution
                execution = self.simulate_execution(signal, context)
                portfolio.apply(execution)
                
        return BacktestResult(portfolio=portfolio, metrics=self.compute_metrics(portfolio))
    
    def simulate_execution(self, signal: Signal, ctx: MarketContext) -> Execution:
        """Account for slippage, gas, partial fills"""
        slippage = self.estimate_slippage(signal.size, ctx.pool_liquidity)
        gas_cost_usd = ctx.gas_price * signal.estimated_gas * ctx.eth_price / 1e18
        executed_price = signal.direction * slippage
        
        return Execution(
            price=executed_price,
            gas_cost=gas_cost_usd,
            timestamp=ctx.block.timestamp,
        )

Slippage Calculation

For Uniswap v2: price_impact = trade_size / (pool_reserve × 2). For v3 — more accurate through concentrated liquidity math, but v2 formula works for quick estimates.

For lending protocols slippage doesn't apply, but there's utilization impact: large deposit reduces utilization rate and accordingly APR for following periods.

Backtest Metrics

Insufficient to look only at P&L. Important metrics:

Gas-adjusted APY is key metric specifically for DeFi. Strategy with 50% APY and weekly rebalancing on Ethereum mainnet may have 30% gas-adjusted APY. Same strategy on Arbitrum — 45% (gas 50x cheaper).

Stack

Python — pandas for time series, sqlalchemy for PostgreSQL. PostgreSQL — historical data storage (TimescaleDB for time series optionally). The Graph / Dune — data export. Plotly / Dash — interactive P&L, drawdown, equity curve charts. Celery — queues for parallel backtests (parameter sweep).

Timeline Guidelines

System for one protocol and one strategy with basic metrics — 1-2 weeks. Full multi-protocol platform with parallel parameter sweep and dashboard — 4-6 weeks. Time on collecting historical data depends on archive depth. Cost calculated individually.