Setting Up Automatic Position Monitoring in Google and Yandex
Search positions are not static: Google and Yandex algorithm updates, competitor activity, site changes—all affect ranking. Knowing about changes a week or two later means reacting too late. We configure automatic monitoring that sends a signal within 24 hours. Our practice includes automating tracking for 50+ projects, and with over 10 years of experience, we guarantee reliability and accuracy.
Why Manual Monitoring Is Inefficient
Manual position checking is a labor-intensive process that gives only an instant snapshot. For daily control, it takes at least 30 minutes per 100 keywords. Automation works without human intervention and captures even short-term dips caused by test algorithm updates. Compare: manual collection of 500 keywords takes 2.5 hours per day (60+ hours per month), while automatic setup takes 5 minutes and zero time for collection. Manual entry errors occur in 5-10% of records; the system eliminates them completely. One client lost 30% of traffic because they didn't notice a drop in positions for a week—automatic alerts would have prevented the loss.
How We Solve the Data Delay Problem
We use API sources that return current positions with a delay of no more than a day. Combining Google Search Console (free, historical data) and DataForSEO (real-time check) gives the full picture. For Yandex, we use the official XML API, which is also free within its quota.
| Parameter | Manual Check | Google Search Console | DataForSEO API | Yandex XML API |
|---|---|---|---|---|
| Cost | Free (time) | Free | From a few cents per query | Free (quota) |
| Freshness | Instant | 2–3 days | Real-time | Real-time |
| Check Volume | 10–50 queries/hour | Up to 1000 queries/day | Unlimited | Quota 1000–10000 queries/day |
| History | None | 28 days | From connection | Not stored |
DataForSEO beats Google Search Console in speed, and manual checking in scalability. For Yandex, we use the official XML API, which is free within quota.
| Frequency | Manual | Automatic | Recommendation |
|---|---|---|---|
| Daily | 2.5 hours for 500 keywords | 10 minutes setup + 0 collection | For competitive niches |
| Weekly | 30 minutes for 100 keywords | 0 minutes | For stable queries |
| Monthly | 10 minutes for 50 keywords | 0 minutes | For brand queries |
Automatic monitoring saves up to 30 times the time for daily checks.
What's Included in Monitoring Setup
- API integration (DataForSEO, Google Search Console, Yandex XML) — registration, key acquisition, traffic routing.
- Daily collection scripts — in Python with error handling, retries, logging.
- PostgreSQL database — structure for storing position history with indexes for fast queries.
- Alert system — comparing current with previous positions, sending to Telegram when threshold is exceeded.
- Documentation — architecture description, instructions for adding and removing keywords.
- Guarantee — free support for 30 days after launch, bug fixes under warranty.
How It Works: Step by Step
- Analysis — collect all target keywords, define regions and languages.
- API setup — connect data sources, test queries.
-
Database deployment — create tables
tracked_keywordsandposition_history. - Script implementation — daily scheduled run (cron or Airflow).
- Alert integration — set up Telegram bot, define thresholds.
- Launch and monitoring — verify correctness, provide dashboard access.
Request monitoring setup right now to avoid missing position changes.
Example Integration with DataForSEO
import requests, json, base64
class DataForSEOClient:
BASE_URL = 'https://api.dataforseo.com/v3'
def __init__(self, login: str, password: str):
creds = base64.b64encode(f'{login}:{password}'.encode()).decode()
self.headers = {
'Authorization': f'Basic {creds}',
'Content-Type': 'application/json',
}
def check_positions(self, keyword, target_domain, location_code=2840, language_code='en', depth=100):
payload = [{
'keyword': keyword,
'target': target_domain,
'location_code': location_code,
'language_code': language_code,
'depth': depth,
}]
resp = requests.post(f'{self.BASE_URL}/serp/google/organic/live/advanced',
headers=self.headers, data=json.dumps(payload), timeout=60)
resp.raise_for_status()
return resp.json()
def parse_position(self, response, target_domain):
tasks = response.get('tasks', [])
if not tasks:
return None
items = tasks[0].get('result', [{}])[0].get('items', [])
for item in items:
if item.get('type') == 'organic' and target_domain in item.get('domain', ''):
return {'position': item.get('rank_absolute'), 'url': item.get('url'),
'title': item.get('title'), 'featured_snippet': item.get('rank_absolute') == 0}
return None
Database Structure for Position History
CREATE TABLE tracked_keywords (
id SERIAL PRIMARY KEY,
keyword TEXT NOT NULL,
target_domain TEXT NOT NULL,
search_engine VARCHAR(20) DEFAULT 'google',
location_code INTEGER,
language_code VARCHAR(10),
active BOOLEAN DEFAULT true,
created_at TIMESTAMP DEFAULT NOW()
);
CREATE TABLE position_history (
id SERIAL PRIMARY KEY,
keyword_id INTEGER REFERENCES tracked_keywords(id),
position INTEGER,
url TEXT,
checked_at DATE NOT NULL,
UNIQUE(keyword_id, checked_at)
);
CREATE INDEX idx_positions_keyword_date ON position_history(keyword_id, checked_at DESC);
Daily Monitoring Run and Alerts
import psycopg2
from datetime import date
def run_daily_check(db_conn, dfs_client, target_domain):
today = date.today().isoformat()
with db_conn.cursor() as cur:
cur.execute('SELECT id, keyword, search_engine, location_code, language_code FROM tracked_keywords WHERE active = true')
keywords = cur.fetchall()
for kw_id, keyword, engine, loc_code, lang_code in keywords:
try:
response = dfs_client.check_positions(keyword=keyword, target_domain=target_domain,
location_code=loc_code or 2840, language_code=lang_code or 'en')
result = dfs_client.parse_position(response, target_domain)
position = result['position'] if result else None
url = result['url'] if result else None
with db_conn.cursor() as cur:
cur.execute('''
INSERT INTO position_history (keyword_id, position, url, checked_at)
VALUES (%s, %s, %s, %s)
ON CONFLICT (keyword_id, checked_at) DO UPDATE
SET position = EXCLUDED.position, url = EXCLUDED.url
''', (kw_id, position, url, today))
db_conn.commit()
except Exception as e:
print(f'Error checking {keyword}: {e}')
def detect_significant_changes(db_conn, threshold=5):
with db_conn.cursor() as cur:
cur.execute('''
WITH ranked AS (
SELECT k.keyword, p.position, p.checked_at,
LAG(p.position) OVER (PARTITION BY p.keyword_id ORDER BY p.checked_at) AS prev_position
FROM position_history p
JOIN tracked_keywords k ON k.id = p.keyword_id
WHERE p.checked_at >= CURRENT_DATE - INTERVAL '2 days'
)
SELECT keyword, prev_position, position,
(COALESCE(prev_position, 101) - COALESCE(position, 101)) AS change
FROM ranked
WHERE prev_position IS NOT NULL
AND ABS(COALESCE(prev_position, 101) - COALESCE(position, 101)) >= %s
ORDER BY ABS(change) DESC
''', (threshold,))
return [{'keyword': row[0], 'prev': row[1], 'current': row[2], 'change': row[3],
'direction': 'up' if row[3] > 0 else 'down'} for row in cur.fetchall()]
Telegram Notifications
import httpx
def send_telegram_alert(bot_token, chat_id, changes):
if not changes:
return
lines = ['*Position changes for the day:*\n']
for ch in changes[:20]:
arrow = '↑' if ch['direction'] == 'up' else '↓'
prev = ch['prev'] or '100+'
curr = ch['current'] or '100+'
lines.append(f"{arrow} `{ch['keyword']}`: {prev} → {curr}")
text = '\n'.join(lines)
httpx.post(f'https://api.telegram.org/bot{bot_token}/sendMessage',
json={'chat_id': chat_id, 'text': text, 'parse_mode': 'Markdown'})
Timelines and Cost
Setting up monitoring for one domain with PostgreSQL storage and Telegram alerts takes 2–3 business days. Adding visualization (Grafana/Metabase), support for multiple sites, and automatic keyword import from Search Console takes 4–6 days. Cost is calculated individually based on keyword volume and required integrations. Get a consultation for an accurate estimate.
DataForSEO SERP API documentation
Additionally: Google Search Algorithms — understanding ranking helps better interpret monitoring data.