A trading bot stumbles on candle data — queries time out, the backtester takes a minute to load a page. With a dataset of 100 pairs over a year, the database falls over, and offset pagination spawns duplicates. Sound familiar? We build REST APIs for historical market data that withstand 1000 concurrent requests and return candles in 50 ms. Below are the engineering decisions we bake into every project: from database selection to caching strategy.
Problems We Solve
Typical trading API pitfalls — slow queries on large datasets, inflexible filtering, lack of proper pagination. We've seen projects where fetching a year of OHLCV for 100 pairs kills the database. Another common issue is data inconsistency with offset pagination: if new candles are inserted between requests, the client gets duplicates or gaps. Load testing of our solutions shows performance gains of up to 200% thanks to time‑based sharding and aggressive caching.
Endpoint Design
Basic set of endpoints for market data:
GET /v1/ohlcv/{exchange}/{symbol} ?from=2023-01-01T00:00:00Z &to=2023-01-31T23:59:59Z &interval=1h &limit=1000 GET /v1/trades/{exchange}/{symbol} ?from=1704067200000 &to=1704153600000 &limit=10000 GET /v1/orderbook/{exchange}/{symbol}/snapshot ?timestamp=1704067200000 &depth=20 GET /v1/tickers/{exchange}/{symbol}/history ?from=2023-01-01 &to=2023-01-02 &fields=close,volume We use ISO 8601 for human interfaces and Unix timestamps (milliseconds) for programmatic access. Both formats are supported via automatic detection.
Parameters and Validation
from fastapi import FastAPI, Query from datetime import datetime from typing import Optional @app.get("/v1/ohlcv/{exchange}/{symbol}") async def get_ohlcv( exchange: str, symbol: str, interval: str = Query("1h", regex="^(1m|5m|15m|1h|4h|1d|1w)$"), from_time: datetime = Query(..., alias="from"), to_time: datetime = Query(..., alias="to"), limit: int = Query(1000, ge=1, le=50000), ): if (to_time - from_time).days > 365: raise HTTPException(400, "Date range cannot exceed 365 days") data = await candle_service.get_candles( exchange, symbol, interval, from_time, to_time, limit ) return {"data": data, "count": len(data)} Pagination for Large Datasets
Cursor‑based pagination outperforms offset for time‑series data:
{ "data": [...], "cursor": { "next": "eyJ0aW1lc3RhbXAiOiAxNzA0MDY3MjAwMDAwfQ==", "has_more": true } } The cursor is a base64‑encoded JSON containing the last timestamp in the current page. On the next request the client passes ?cursor=... instead of ?from=....
| Parameter | Cursor | Offset |
|---|---|---|
| Consistency under inserts | Guaranteed | Duplicates/gaps possible |
| Performance on large sets | O(log n) | O(n) with large offsets |
| Sort support | Ascending only (timestamp) | Any |
| Implementation complexity | Medium | Simple |
Caching Strategies
| Strategy | TTL | Applicability |
|---|---|---|
| HTTP Cache‑Control (public, max-age=3600) | 1 hour | Data older than 24 hours |
| Redis Cache | 60 seconds | Frequently requested ranges (last 30 days) |
| Query Cache (TimescaleDB/ClickHouse) | 5 minutes | Heavy aggregations |
Historical candles are immutable — a perfect caching candidate. If the requested range is fully closed, we cache for 24 hours. If it includes the current moment, we cache for 60 seconds.
How to Design a REST API for Market Data?
When designing we use REST with uniform endpoints, support for multiple timeframes and formats. The key principle is resource‑oriented design: /v1/ohlcv/{exchange}/{symbol}. Filters via query parameters, cursor pagination. We document with OpenAPI — clients can test directly in Swagger UI.
Why Is Proper Caching of Historical Data Important?
Proper caching reduces latency by 3–5x and cuts database load. Our configurations account for request frequency and data staticity. For popular pairs with deep history we use ClickHouse with materialized views — giving up to 40% performance boost.
Rate Limiting and Authentication
from slowapi import Limiter from slowapi.util import get_remote_address limiter = Limiter(key_func=get_remote_address) @app.get("/v1/ohlcv/{exchange}/{symbol}") @limiter.limit("100/minute") async def get_ohlcv(...): ... For commercial APIs we offer tiered plans via API keys with different limits: free (10 req/min, 30 days history), paid (1000 req/min, full history).
Documentation via OpenAPI
FastAPI auto‑generates an OpenAPI schema. We additionally include request/response examples, format descriptions, error codes. Swagger UI and ReDoc are built‑in — clients can test the API right in the browser without extra tools.
Example error response
{ "error": { "code": "INVALID_INTERVAL", "message": "Interval must be one of: 1m, 5m, 15m, 1h, 4h, 1d, 1w" } } How We Work
- Analysis: we review your data, peak load, use cases.
- Design: define endpoint schema, response format, pagination.
- Implementation: build with FastAPI, integrate TimescaleDB/ClickHouse, configure caching.
- Testing: load testing (k6 + locust), stress test up to 10 000 RPS.
- Deployment: deploy in your cloud or on‑premise, set up monitoring (Prometheus + Grafana).
Timeline — from 2 to 4 weeks depending on complexity and data volume. Cost is estimated individually.
What's Included
- REST API with the functionality described above (OHLCV, trades, order book, tickers).
- OpenAPI documentation (Swagger/ReDoc).
- Integration examples in Python, JavaScript, cURL.
- Deployment (Docker, Kubernetes, CI/CD).
- 99.9% uptime guarantee (SLA).
- One month of free support after launch.
Why Choose Us?
5+ years of experience in Web3 infrastructure, 30+ implemented APIs for trading systems on Ethereum, Solana, and Binance Smart Chain. Our solutions handle peak loads of up to 50 000 requests per minute. We use Tenderly, Slither, Mythril for security audits.
Contact us to discuss your project. Order turnkey development with performance guarantees — get an engineer's consultation within 2 business days.