Crypto Data Collection: CoinGecko and CoinMarketCap Integration
Parsing Data from CoinGecko / CoinMarketCap
When integrating a DeFi application with external price feeds, developers face rate limits, update lags, and incomplete data. Parsing data from CoinGecko and CoinMarketCap APIs—the two primary approaches for collecting crypto data—each has its own limitations and coverage quality. We will walk through building a resilient crypto data collection system using both sources with fallback, Redis caching, and PostgreSQL storage. Reach out to us for a consultation—we'll help you choose the optimal architecture for your project.
Why Combine CoinGecko and CoinMarketCap?
The CoinGecko API is preferable for DeFi tokens and long-tail assets: it has a more generous free tier and better coverage. CoinMarketCap provides more accurate volumes from major CEXs. For production price feeds, we use both with fallback logic—this reduces the risk if one source fails. CoinGecko covers DeFi tokens 1.5 times better than CoinMarketCap, especially on Ethereum and Polygon. According to the CoinGecko API documentation, the base data update frequency is 1–10 seconds.
| Parameter | CoinGecko | CoinMarketCap |
|---|---|---|
| Key required? | Optional (free without key) | Mandatory even for basic requests |
| Free limit | 10–30 req/min (without key) | 10,000 credits/month |
| Max IDs per request | 250 | 100 |
| Historical data | Up to 5 years (Pro) | Up to 1 year (paid plan) |
| Data delay | ~1–10 s for prices | ~1–5 s |
Setting Up Stable Data Collection Under API Limits
We use Redis for caching with a TTL of 1–2 minutes—this reduces API load by 5–10 times. Example client with automatic retry on 429 status:
const COINGECKO_BASE = 'https://api.coingecko.com/api/v3'
// Pro: 'https://pro-api.coingecko.com/api/v3'
class CoinGeckoClient {
constructor(private apiKey?: string) {}
private async request<T>(path: string, params?: Record<string, string>): Promise<T> {
const url = new URL(`${COINGECKO_BASE}${path}`)
if (params) Object.entries(params).forEach(([k, v]) => url.searchParams.set(k, v))
if (this.apiKey) url.searchParams.set('x_cg_pro_api_key', this.apiKey)
const res = await fetch(url.toString())
if (res.status === 429) {
const retryAfter = res.headers.get('Retry-After')
await sleep((parseInt(retryAfter || '60') + 1) * 1000)
return this.request(path, params) // retry
}
if (!res.ok) throw new Error(`CoinGecko ${res.status}: ${await res.text()}`)
return res.json()
}
async getSimplePrice(
ids: string[],
vsCurrencies: string[] = ['usd'],
includeMarketCap = false,
include24hVol = false,
include24hChange = false
) {
return this.request<Record<string, Record<string, number>>>('/simple/price', {
ids: ids.join(','),
vs_currencies: vsCurrencies.join(','),
include_market_cap: String(includeMarketCap),
include_24hr_vol: String(include24hVol),
include_24hr_change: String(include24hChange),
})
}
async getMarkets(page = 1, perPage = 250) {
return this.request<CoinMarketData[]>('/coins/markets', {
vs_currency: 'usd',
order: 'market_cap_desc',
per_page: String(perPage),
page: String(page),
sparkline: 'false',
})
}
async getMarketChart(coinId: string, days: number | 'max') {
return this.request<MarketChart>(`/coins/${coinId}/market_chart`, {
vs_currency: 'usd',
days: String(days),
interval: days === 'max' || days > 90 ? 'daily' : 'hourly',
})
}
}
Getting the Full List of Coins with Contract Addresses
To match contract address to CoinGecko ID, use the /coins/list?include_platform=true endpoint. We cache this data for 24 hours since it rarely changes:
async function buildTokenAddressIndex(): Promise<Map<string, string>> {
const coins = await client.request<CoinWithPlatforms[]>(
'/coins/list',
{ include_platform: 'true' }
)
const index = new Map<string, string>() // 'chain:address' → coingecko_id
for (const coin of coins) {
for (const [platform, address] of Object.entries(coin.platforms || {})) {
if (address) {
index.set(`${platform}:${address.toLowerCase()}`, coin.id)
}
}
}
return index
}
CoinMarketCap API
CoinMarketCap API requires a key even for basic requests. The free plan is 10,000 credits per month (1 credit ≈ 1 request). Example request for latest quotes:
class CoinMarketCapClient {
private headers = {
'X-CMC_PRO_API_KEY': process.env.CMC_API_KEY!,
'Accept': 'application/json',
}
async getLatestQuotes(symbols: string[]): Promise<CMCQuoteResponse> {
const res = await fetch(
`https://pro-api.coinmarketcap.com/v1/cryptocurrency/quotes/latest?symbol=${symbols.join(',')}`,
{ headers: this.headers }
)
const data = await res.json()
if (data.status.error_code !== 0) {
throw new Error(`CMC error: ${data.status.error_message}`)
}
return data
}
}
Architecture and Stack
For a production-grade system, we use a microservice on Node.js/TypeScript that collects data from both APIs in the background. Redis acts as a first-level cache with TTL, and PostgreSQL as a long-term store. For monitoring and alerts, we set up Grafana + Prometheus, tracking request counts, latencies, and error rates.
CoinGecko Pricing Tiers
| Tier | Requests/min | Price | Historical Data |
|---|---|---|---|
| Free | 10–30 | $0 | Up to 1 year (limited) |
| Pro | 500 | $129/month | Up to 5 years |
| Enterprise | custom | custom | Full access |
Caching and Storage
For a price feed updated every minute, we use Redis with a TTL of 120 seconds and batch updates of 250 IDs:
class PriceCache {
constructor(private redis: RedisClient, private client: CoinGeckoClient) {}
async getPrice(coinId: string): Promise<number> {
const cached = await this.redis.get(`price:${coinId}`)
if (cached) return parseFloat(cached)
const prices = await this.client.getSimplePrice([coinId])
const price = prices[coinId]?.usd
if (price) await this.redis.setEx(`price:${coinId}`, 60, String(price))
return price
}
async refreshPrices(coinIds: string[]): Promise<void> {
const chunks = chunk(coinIds, 250)
for (const ids of chunks) {
const prices = await this.client.getSimplePrice(ids, ['usd'], true, true, true)
const pipeline = this.redis.pipeline()
for (const [id, data] of Object.entries(prices)) {
pipeline.setEx(`price:${id}`, 120, JSON.stringify(data))
}
await pipeline.exec()
}
}
}
Historical data goes to PostgreSQL with an index on (coin_id, timestamp). For intensive time-range queries, we use TimescaleDB.
Process
- Analysis — assess number of tokens, update frequency, API budget.
- Design — choose stack (Node.js, Redis, PostgreSQL), design database schema and cache architecture.
- Implementation — write clients with retry, rate limiting, caching; set up batch updates.
- Testing — check under load (simulate rate limits, connection drops).
- Deployment — deploy in Docker on your server or cloud.
- Monitoring — set up a Grafana dashboard with metrics: latency, cache hit ratio, error rate.
- Support — for one month after launch, assist with incidents and fine-tuning.
Typical Integration Mistakes
- Ignoring rate limits → IP block. Solution: use a queue with delays.
- No fallback when one API fails → data loss. Solution: combine both sources with priority.
- Storing all data in one table without partitioning → slow queries. Solution: TimescaleDB for time series.
- Caching without TTL → stale prices. Solution: Redis TTL of 60–120 seconds.
What's Included
- Architecture tailored to your data volume (from 100 to 10,000 tokens)
- Implementation of API clients with retry, rate limiting, logging
- Redis cache with optimal TTL
- PostgreSQL/TimescaleDB for history
- Background workers for automatic updates
- Operation documentation and a Grafana dashboard
- One month of post-launch support
- Training your team on how to use the system
Our proven architecture guarantees high availability and accurate data. Order a price feed setup — we will find a solution for your task. Get a consultation on integration today. Our experience: more than 5 years in crypto development, 30+ projects. Guaranteed support and reliable delivery.