SEMrush API Integration for SEO Site Analytics
Imagine managing SEO for 10 domains. Every week you manually open SEMrush, export reports, merge CSVs in Excel. That takes half a day, and data often contains typos. For an online store with 100,000 products, manual position collection across all categories is virtually impossible—data becomes outdated faster than you gather it. The SEMrush API solves this once and for all: programmatically pull positions, organic keywords, backlinks, and competitor metrics, store them in your database, and build a dashboard. Automation via the API is 24 times faster than manual collection—saving 90% of time. We have implemented such integrations for 8 projects, including daily monitoring of 50+ domains. Our stack: Python 3.11, PostgreSQL 16, Docker, Grafana. The code is delivered as a repository with documentation.
Why Automate Data Collection?
Manual collection from SEMrush means hours of routine and risk of copy errors. The API gives access to thousands of keywords in one request, automatic metric updates on schedule, and the ability to compare 20+ domains in a unified dashboard. SEMrush API vs manual scraping:
| Criterion | Manual Collection | Automation via API |
|---|---|---|
| Time for 5 domains | 4 hours | 10 minutes |
| Accuracy | Copy errors | 0% |
| Update frequency | Once a week | Daily |
| Competitor comparison | Laborious | Built-in |
Savings on manual data collection can reach 400,000 rubles per year for a team of 3—by freeing up time for strategic analysis.
How to Manage API Units Costs?
Unit consumption directly depends on request volume. Optimization: for daily monitoring, request only key metrics (top-200 positions, traffic, domain rating). Run a full backlink audit once a week. With a Business plan (10,000 units/month), you can cover 5 domains daily. Use caching—do not request the same data twice in a day.
What to Do on Integration Errors?
The API may return errors due to rate limits, invalid keys, or temporary issues. The client code must handle them: on ERROR status, retry with exponential backoff. Set up alerts in Telegram or Slack on collection failures. This ensures no metric gaps.
How to Configure a Client for SEMrush API?
SEMrush uses an API key as a query parameter. Response defaults to CSV, but JSON is available for some endpoints. A basic Python client implementation looks like this:
import requests
import csv
import io
from typing import Literal
class SemrushClient:
BASE_URL = 'https://api.semrush.com'
ANALYTICS_URL = 'https://api.semrush.com/analytics/v1'
def __init__(self, api_key: str):
self.api_key = api_key
self.session = requests.Session()
def _request(self, params: dict) -> list[dict]:
params['key'] = self.api_key
resp = self.session.get(self.BASE_URL, params=params, timeout=30)
resp.raise_for_status()
if resp.text.startswith('ERROR'):
raise ValueError(f'SEMrush API error: {resp.text}')
reader = csv.DictReader(io.StringIO(resp.text), delimiter=';')
return list(reader)
Main API Endpoints
For collecting organics, competitors, and backlinks, we use the following methods. Unit costs summary:
| Method | Endpoint | Units (100 rows) | Typical Data |
|---|---|---|---|
| Organic keywords | domain_organic |
10 | Positions, traffic, URL |
| Organic competitors | domain_organic_organic |
10 | Overlapping keywords |
| Backlinks | backlinks |
40 | Sources, Authority Score |
| Domain ranks | domain_ranks |
10 | General metrics |
Example of getting organic keywords for a domain:
def get_organic_keywords(self, domain: str, database: str = 'ru', limit: int = 1000) -> list[dict]:
params = {
'type': 'domain_organic',
'domain': domain,
'database': database,
'display_limit': limit,
'display_sort': 'tr_desc',
'export_columns': 'Ph,Po,Pp,Nq,Tr,Ur',
}
return self._request(params)
The response contains key fields: Ph — keyword, Po — position, Nq — monthly search volume, Tr — estimated traffic, Ur — page URL.
For backlink audit:
def get_backlinks(self, target: str, limit: int = 1000) -> list[dict]:
params = {
'type': 'backlinks',
'target': target,
'target_type': 'root_domain',
'display_limit': limit,
'display_sort': 'page_ascore_desc',
'export_columns': 'source_url,target_url,anchor,page_ascore,domain_ascore,nofollow,first_seen',
}
return self._request(params)
Daily Data Collection and Storage
We set up a pipeline: on schedule (e.g., cron), a script collects domain metrics, top-200 keywords, and saves everything into PostgreSQL. Example schema:
CREATE TABLE semrush_domain_metrics (
id SERIAL PRIMARY KEY,
domain TEXT NOT NULL,
snapshot_date DATE NOT NULL,
organic_keywords INTEGER,
organic_traffic INTEGER,
semrush_rank INTEGER,
UNIQUE(domain, snapshot_date)
);
CREATE TABLE semrush_keyword_positions (
id SERIAL PRIMARY KEY,
domain TEXT NOT NULL,
keyword TEXT NOT NULL,
position INTEGER,
search_volume INTEGER,
url TEXT,
snapshot_date DATE NOT NULL,
UNIQUE(domain, keyword, snapshot_date)
);
Step-by-Step Integration Setup
To automate data collection, follow these steps:
- Obtain API key from SEMrush panel (API section).
- Install dependencies:
pip install requests psycopg2-binary. - Implement the
SemrushClientclass as shown above. - Create tables in PostgreSQL per the schema above.
- Configure a cron job to run the script daily.
- Integrate metrics into Grafana for visualization.
What’s Included in the Work
Our integration includes:
- Python client code with error handling and pagination.
- Scripts for scheduled data collection.
- PostgreSQL schema for metric storage.
- Documentation for setup and launch.
- Optional Grafana dashboard configuration.
- Post-deployment support for bug fixes.
API Units Calculation
For daily monitoring of 5 domains (metrics + 200 keywords), consumption is approx 500–700 units per day. With a Business plan (10,000 units/month), this fits within limits. Optimization: do not request full backlink lists daily, only key metrics. Full backlink audit—once a week.
Timeframes and Cost
Basic integration with daily metric collection for one domain and top-200 keywords — 2-3 working days. Extended version with competitor analysis, backlink audit, and Grafana dashboard — 5-7 days. Cost is calculated individually after auditing your tasks. Get a consultation — contact us to evaluate the project. Order integration today and start saving time and resources.
We guarantee quality: code is tested, alerts are configured on collection errors, and post-deployment support is provided. Our experience: 5+ years in web development and SEO integrations, 8 projects implemented with total monitoring of 50+ domains.