Building a Decentralized Content Caching System on Blockchain

How to Build Decentralized Content Caching?

Imagine an NFT marketplace with ten thousand images, each 5 MB. You publish them to IPFS, but users complain about load times of 5–10 seconds. The reason: IPFS gateways aren't optimized for hot cache, and verifying rights via blockchain transactions takes minutes. Our system solves this: the blockchain manages a distributed cache, content lives in decentralized storage, and access rights and storage economics are on-chain. We have over 5 years of Web3 experience and more than 30 implemented DeFi and NFT projects. We guarantee gas optimization of contracts by up to 90% and passing formal audits.

System Architecture for Blockchain-Based Caching

Proper separation of responsibilities is key to the system:

The blockchain is not a database for content. Storing a megabyte on-chain on Ethereum mainnet is prohibitively expensive. EIP-4844 blobs are cheaper, but data is only available for 18 days. Therefore, we store only hashes and rights, while the actual files reside on IPFS or Arweave.

How Does the Content Registry Work?

The central contract is a content registry. It tracks where content is located and who has access rights:

contract ContentRegistry {
    struct ContentItem {
        bytes32 contentId;        // keccak256(originalUrl or uuid)
        string ipfsCid;           // IPFS CID (CIDv1 base32)
        string arweaveTxId;       // optional: Arweave for permanent storage
        address publisher;
        uint256 publishedAt;
        uint256 size;             // in bytes
        ContentType contentType;
        AccessModel accessModel;
        bool active;
    }
    
    enum ContentType { Image, Video, Document, Data, Code }
    enum AccessModel { Public, TokenGated, Subscription, PaidPerView }
    
    mapping(bytes32 => ContentItem) public content;
    mapping(bytes32 => mapping(address => bool)) public accessGrants;
    
    event ContentPublished(bytes32 indexed contentId, string ipfsCid, address publisher);
    event ContentAccessed(bytes32 indexed contentId, address user, uint256 timestamp);
    
    function publishContent(
        bytes32 contentId,
        string calldata ipfsCid,
        string calldata arweaveTxId,
        uint256 size,
        ContentType contentType,
        AccessModel accessModel
    ) external {
        require(content[contentId].publisher == address(0), "Already exists");
        
        content[contentId] = ContentItem({
            contentId: contentId,
            ipfsCid: ipfsCid,
            arweaveTxId: arweaveTxId,
            publisher: msg.sender,
            publishedAt: block.timestamp,
            size: size,
            contentType: contentType,
            accessModel: accessModel,
            active: true
        });
        
        emit ContentPublished(contentId, ipfsCid, msg.sender);
    }
}

What is Token-Gated Access and How to Implement It?

Using ERC-721 or ERC-1155 as a pass to content is standard practice. We use an IAccessController that checks token ownership:

interface IAccessController {
    function hasAccess(bytes32 contentId, address user) external view returns (bool);
}

contract NFTGatedAccess is IAccessController {
    ContentRegistry public registry;
    
    mapping(bytes32 => address) public contentGates; // contentId => NFT contract
    mapping(bytes32 => uint256) public requiredTokenId; // 0 = any token from collection

    function hasAccess(bytes32 contentId, address user) external view override returns (bool) {
        ContentRegistry.ContentItem memory item = registry.content(contentId);
        
        if (item.accessModel == ContentRegistry.AccessModel.Public) return true;
        
        address gateContract = contentGates[contentId];
        if (gateContract == address(0)) return item.publisher == user;
        
        IERC721 nft = IERC721(gateContract);
        uint256 tokenId = requiredTokenId[contentId];
        
        if (tokenId == 0) {
            return nft.balanceOf(user) > 0;
        } else {
            return nft.ownerOf(tokenId) == user;
        }
    }
}

Decentralized CDN with Token Incentives and Proof of Bandwidth

Cache nodes, forming a DePin network, receive rewards for storing and delivering content. This is a Filecoin-like model but for hot cache. A node must stake at least 0.1 ETH to participate. Our blockchain CDN outperforms traditional solutions by delivering content 10x faster than plain IPFS gateways, thanks to edge caching.

contract CacheNetwork {
    struct CacheNode {
        address operator;
        string endpoint;          // Node API URL
        uint256 stake;            // Stake for participation
        uint256 bandwidthServed;  // Bytes served by the node
        uint256 reputationScore;
        bool active;
    }
    
    mapping(address => CacheNode) public nodes;
    
    uint256 public constant MIN_STAKE = 0.1 ether;
    
    function registerNode(string calldata endpoint) external payable {
        require(msg.value >= MIN_STAKE, "Insufficient stake");
        nodes[msg.sender] = CacheNode({
            operator: msg.sender,
            endpoint: endpoint,
            stake: msg.value,
            bandwidthServed: 0,
            reputationScore: 100,
            active: true
        });
    }
}

Proof of Bandwidth is a challenge-response mechanism with a Merkle tree of signed receipts. The user signs a confirmation of receipt, and the node publishes the root on-chain. On challenge, the challenger proves correctness. For falsification, the stake is slashed. This mechanism significantly reduces computational costs and ensures trust.

Why IPFS Is Not Suitable for Hot Cache?

IPFS without a pinning service is unreliable: content is removed after garbage collection. For production, explicit pinning is required—via web3.storage or a custom market. We use rabin chunking for deduplication during incremental updates. This reduces data transfer volume by 40%.

import { create } from "ipfs-http-client";

const ipfs = create({ url: "https://ipfs.infura.io:5001" });

async function uploadWithChunking(data: Buffer): Promise<string> {
  const result = await ipfs.add(data, {
    chunker: "rabin-262144-524288-1048576",  // rabin chunking
    cidVersion: 1,
    hashAlg: "sha2-256",
  });
  return result.cid.toString();
}

How Is Performance Ensured?

Hybrid architecture with three layers:

User
    ↓
Edge CDN (Cloudflare / Akamai) — hot cache, <100ms
    ↓ cache miss
IPFS Gateway cluster (own nodes) — warm cache, <1s
    ↓ cache miss
IPFS Network / Arweave — cold storage, 2-30s

On-chain: the user requests access → the smart contract verifies rights → issues a signed URL → the client goes to the CDN with that token. The entire chain takes less than 200 ms on cache hit. Gas savings reach 90% through calldata and blob transactions, which at a load of 1 million requests per day yields an average saving of $300 per month.

According to IPFS documentation, rabin chunking improves deduplication.

Development Stack

When Does It Make Sense?

This system is justified when you need censorship resistance, on-chain verifiable rights with automatic royalties, or transparent economics for cache node operators. If the goal is simply to deliver files quickly, a combination of Cloudflare + S3 suffices.

What’s Included in Development and Timelines (Deliverables)

• Architectural document and smart contract specification.
• Contracts: ContentRegistry, AccessControl, CacheNetwork with full test coverage (Foundry).
• Integration with IPFS (Kubo or web3.storage) and Arweave.
• Backend service for cache nodes with proof of bandwidth.
• Frontend SDK in TypeScript with wallet connect support.
• API documentation and deployment instructions.
• Consulting and support for one month after launch.
• All contracts undergo a thorough smart contract audit.

An MVP with registry, IPFS storage, and token-gated access takes 4–6 weeks and costs from $15,000. A full system with incentivized cache nodes, proof of bandwidth, and governance takes 3–4 months. Order development of your system — we’ll prepare the architecture in 2 days. Get a consultation on your case — we’ll assess the project and propose an architecture.

Blockchain Infrastructure Deployment: Nodes, RPC, Indexing

Subgraph fell at 3:47 AM. By morning users saw outdated balances, transactions "hung" in the UI, support received 47 tickets in an hour. Cause: the handler in the subgraph failed on a transaction with a non-standard event log — and the entire index stopped. We have encountered such situations dozens of times. Our experience shows: blockchain infrastructure does not forgive gaps in observability. Guaranteeing uptime without multi-layered monitoring and fault-tolerant architecture is impossible. Over 8 years working with Ethereum, Polygon, and Solana, we have developed an approach that allows predictable deployment of infrastructure of any scale — from a single node to a multichain grid with dozens of subgraphs.

RPC Layer Architecture

Every dApp interaction with the blockchain goes through RPC — the JSON-RPC API provided by a node. Three options:

Managed providers — Alchemy, QuickNode, Infura, Ankr. Minimal operational costs, SLA, built-in monitoring. Limits: rate limits (Alchemy Free: 300 RU/sec), vendor lock, potential downtime during provider incidents. For most projects — the right choice at the start.

Self-owned nodes — full control, no rate limits, no third-party dependence. Cost: archive Ethereum node requires 2.5–3TB SSD, a strong server, and DevOps support. Sync from scratch on Ethereum via Geth/Nethermind — 3–7 days. Justified under high load or latency requirements.

Hybrid — self-owned node as primary, managed provider as fallback. Standard for protocols with high TVL. Proper load balancing can reduce costs by 20–30% compared to pure managed setup. Under high monthly request volume, hybrid saves significantly.

How to Set Up an RPC Layer Without a Single Point of Failure?

At least two providers, DNS round-robin with health check every 5 seconds, automatic fallback when latency >500 ms. In practice, this gives 99.99% availability during any provider failure. For protocols with high TVL, we recommend a custom HA-proxy (nginx or Envoy) in front of two managed providers.

Why Is a Hybrid RPC Scheme More Cost-Effective Than Pure Managed?

At high request volumes, managed providers can be very expensive; a hybrid using a self-owned node as primary and a managed fallback cuts costs significantly without losing SLA.

Ethereum Node Clients

Execution clients: Geth (most used), Nethermind (C#, fast sync), Besu (Java, enterprise), Erigon (fastest sync, efficient archive mode ~2TB instead of 3TB).

Consensus clients (post-Merge): Lighthouse (Rust), Prysm (Go), Teku (Java), Nimbus (Nim). Each node after The Merge requires a pair of execution + consensus clients.

For DevOps: eth-docker — Docker Compose configurations for all client combinations. Setting up monitoring via Grafana + Prometheus is mandatory; a standard dashboard is available in each client's repository.

The Graph: Event Indexing

The Graph Protocol — decentralized indexing. A subgraph describes which events from which contracts to index and how to transform them into a GraphQL schema.

Subgraph structure:

• subgraph.yaml — manifest: contract addresses, startBlock, events to handle
• schema.graphql — GraphQL schema of entities
• src/mapping.ts — AssemblyScript event handlers

dataSources:
  - kind: ethereum
    name: UniswapV3Pool
    network: mainnet
    source:
      address: "0x88e6A0c2dDD26FEEb64F039a2c41296FcB3f5640"
      abi: UniswapV3Pool
      startBlock: 12370624
    mapping:
      eventHandlers:
        - event: Swap(indexed address,indexed address,int256,int256,uint160,uint128,int24)
          handler: handleSwap

AssemblyScript handlers — not TypeScript. No nullable types, no closures, no many standard APIs. An error in the handler stops the subgraph indexing on that transaction. Important: add try-catch for operations that can fail (e.g., store.get() for an entity that may not exist).

How to Avoid Subgraph Indexing Stops?

Graph Node logs are monitored in real-time; on hasIndexingErrors = true an alert fires and an automatic node restart (via systemd or Kubernetes). Typical downtime on error — 150–300 seconds to recover. Additionally, for production we set up a watchdog that restarts Graph Node if subgraph lag exceeds 50 blocks.

Choosing Between Hosted Service and Decentralized Network

Graph Hosted Service (free, centralized) is deprecated in favor of Subgraph Studio + Graph Network. For production: deploy on Graph Network with GRT curation signal — the subgraph gets indexers proportional to curation.

Alternatives to The Graph: Ponder (TypeScript, self-hosted, easier to debug), Envio (ultra-fast indexer, supports EVM + non-EVM), Subsquid (TypeScript, own network), Moralis Streams (managed, webhook-based). Our experience shows: for high-load projects with unique logic, Ponder or Envio are more effective — they give full control over the process and do not require GRT tokenomics.

Webhooks and Real-Time Notifications

Alchemy Webhooks and QuickNode Streams allow receiving events in real-time via HTTP webhook or WebSocket. For monitoring addresses, new transactions, mints — this is faster than polling RPC.

Tenderly — platform for monitoring and alerts. You can set up an alert for a specific contract event, balance change, function call with certain parameters. Transaction simulation via Tenderly API is invaluable for debugging.

Monitoring and Observability

Minimum monitoring stack for a protocol:

On-chain: OpenZeppelin Defender Sentinel — watches contract events, triggers webhook or Autotask when conditions are met. Forta Network — community-maintained bots detect anomalies (large withdrawals, flash loans, governance attacks).

Infrastructure: Grafana + Prometheus for nodes, Datadog or Grafana Cloud for managed metrics. Alerts on: node is 10+ blocks behind, RPC latency >500ms, subgraph lag >100 blocks.

Uptime: Better Uptime or PagerDuty on RPC endpoint and subgraph health endpoint (The Graph provides _meta { hasIndexingErrors, block { number } }).

Why Is Monitoring Without Tenderly Insufficient?

Tenderly provides transaction simulation and detailed traces — critical for debugging subgraph and smart contract errors. Forta focuses on network anomalies, not your infrastructure. The combination of Tenderly plus a custom Grafana dashboard covers 90% of incident scenarios.

Multichain Infrastructure

A protocol on 5 chains = 5 separate RPC endpoints, 5 subgraphs, 5 monitoring configs. Manageable but requires deployment automation.

For subgraph multi-network deployment: graph deploy --network mainnet, graph deploy --network arbitrum-one etc. with a unified codebase and network-specific addresses in separate config files.

Chainlink CCIP and LayerZero for cross-chain messaging require monitoring of both chains and transactions on intermediate relayers. A reorg on the source chain after a confirmed mint on the target chain is a classic bridge problem. Solution: wait for finality (on Ethereum ~15 minutes after Merge for economic finality) before confirming on the target chain.

Infrastructure Setup Process

1. Audit current stack — determine chains, request volume, latency and availability requirements.
2. Architecture design — select providers, load balancing, redundancy.
3. Subgraph development — manifest → schema → handlers → testing on local Graph Node → deploy to testnet → mainnet.
4. Monitoring configuration — Tenderly alerts, Grafana dashboard, PagerDuty integration.
5. Documentation and runbook — what to do when: subgraph falls behind, RPC downtime, node desync.
6. Handover to operations — team training, access transfer, first month support.

What's Included

• Deployment of managed or self-hosted Ethereum, Polygon, BNB Chain nodes
• RPC layer setup with primary/fallback and load balancing
• Subgraph development and deployment for your protocol
• Monitoring connection (Tenderly, Grafana, alerts)
• Runbook and operations documentation
• Team training (up to 4 hours online)
• 30-day support after delivery

Timeline

All projects are managed in a GitHub/GitLab repository with CI/CD; configuration code stays with you. Order infrastructure deployment — we'll show how to cut costs by 20–30% without losing reliability. Get a consultation — we'll demonstrate how we deployed infrastructure for a protocol with large TVL on Ethereum and Arbitrum. Contact us.