Configuring Content Protection Against Scraping in 1C-Bitrix

Competitors parse the catalog: they collect prices, descriptions, characteristics, use for monitoring or copy to their own site. Last week, an auto parts store with 15,000 products approached us — competitors copied the entire database overnight. Full protection is impossible — if a human can see the data, so can a program. Our task is to make scraping economically unprofitable. Over 10+ years of working with Bitrix, we have developed an echeloned defense strategy that cuts off up to 95% of non-target scrapers. The simplest scraper — a plain wget or curl — is blocked by User-Agent filtering at the init.php stage. More advanced ones use Playwright or Puppeteer — behavioral analysis and JS challenge work against them. It is the combination of methods that yields results.

Why is rate limiting alone not enough?

Rate limiting is basic protection, but it is easily bypassed via proxies or distributed requests. Smart scrapers use a pool of IP addresses and delays between requests. Behavioral analysis adds context: if 100 catalog requests come from one IP in 5 minutes — it is a bot. Combining layers increases the bypass cost by 10–20 times. According to our data, implementing behavioral analysis reduces server load by 30% due to early bot blocking. Guaranteed effectiveness: our certified engineers have delivered over 200 projects for Bitrix content protection.

How to protect prices from scrapers with JavaScript?

Prices are not output in HTML but loaded via AJAX after page rendering. Simple HTML scrapers get the page without prices:

// In the product card template instead of price:
<span class="product-price js-price-loader" data-product-id="<?= $arResult['ID'] ?>">
    <span class="skeleton">----</span>
</span>

// After DOMContentLoaded load prices
const priceElements = document.querySelectorAll('.js-price-loader');
if (priceElements.length) {
    const ids = [...priceElements].map(el => el.dataset.productId);

    fetch('/local/ajax/prices.php', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json', 'X-Requested-With': 'XMLHttpRequest' },
        body: JSON.stringify({ ids }),
    })
    .then(r => r.json())
    .then(data => {
        priceElements.forEach(el => {
            const price = data.prices[el.dataset.productId];
            if (price) el.innerHTML = price.formatted;
        });
    });
}

Headless browsers (Playwright, Puppeteer) overcome this but require significantly more resources — the cost of bypass increases. In one project, after implementing JS loading, the number of successful price collections dropped by 80%. This layer is 5 times more cost-effective than CAPTCHA alone.

How does behavioral analysis work?

Real users do not request 200 catalog pages in 5 minutes. IP-based request counter in Redis:

namespace Local\Security;

class RateLimiter
{
    private const WINDOW   = 300;  // 5 minutes
    private const LIMIT    = 100;  // catalog requests
    private const BAN_TIME = 3600; // ban for an hour

    public static function check(string $ip): bool
    {
        $redis = \Bitrix\Main\Data\Cache::createInstance();
        // Simplified: using Bitrix cache
        $key    = 'ratelimit_catalog_' . md5($ip);
        $count  = (int)(\Bitrix\Main\Application::getInstance()
                        ->getManagedCache()->get($key) ?? 0);

        if ($count > self::LIMIT) {
            // Log and block
            self::banIp($ip);
            return false;
        }

        \Bitrix\Main\Application::getInstance()->getManagedCache()->set(
            $key,
            $count + 1,
            self::WINDOW
        );

        return true;
    }

    private static function banIp(string $ip): void
    {
        // Add to Bitrix ban table (b_stop_list)
        \CStopList::Add([
            'SITE_ID'   => SITE_ID,
            'IP_ADDR'   => $ip,
            'ACTIVE'    => 'Y',
            'REASON'    => 'Automatic ban: scraping suspicion',
        ]);
    }
}

If the counter approaches 70% of the limit, we show a challenge via Cloudflare Turnstile or built-in CAPTCHA. Behavioral analysis is 20 times more effective than rate limiting alone.

What is a honeypot and how does it block bots?

Hidden links in HTML, invisible to humans (display: none), but indexable by scrapers:

<?php
// /honeypot/trap-page/index.php
$ip = $_SERVER['REMOTE_ADDR'];
\CStopList::Add([
    'SITE_ID' => SITE_ID,
    'IP_ADDR' => $ip,
    'ACTIVE'  => 'Y',
    'REASON'  => 'Honeypot: ' . $_SERVER['REQUEST_URI'],
]);
header('HTTP/1.1 403 Forbidden');

Links are generated dynamically via JavaScript with random URLs that change every 24 hours. Honeypot is cheaper to implement than CAPTCHA, but effective against 30% of bots.

How to protect images via X-Accel-Redirect?

Images are served through PHP with permission checks, and nginx does the actual file serving efficiently:

location /protected-uploads/ {
    internal; # not directly accessible from outside
    alias /var/www/upload/;
}
location /catalog/ {
    limit_req zone=catalog burst=40 nodelay;
    limit_req_status 429;
    # ... other directives
}

PHP sets the X-Accel-Redirect header, and nginx serves the file without interpreter involvement.

What is included in the work

After implementation, you get:

• Configuration files for nginx and PHP with comments.
• Source code for JS price loading and honeypot components.
• Documentation for operation and monitoring.
• Access to the repository with changes.
• Training for your administrator: how to add new honeypot traps, change rate limiting limits.
• One month of support after implementation — consultations and modifications if needed.

All work is delivered under a fixed-price contract, with a guaranteed timeline of 1-2 days for audit and 1-2 days for implementation. Write to us for a personalized offer and free evaluation of your project.

How to implement protection: step-by-step guide

1. Traffic and log audit — 2–4 hours. Identify vulnerabilities and recommendations.
2. Set up rate limiting + filtering — 1–2 hours. Block 60% of simple scrapers.
3. Implement JS price loading — 4–6 hours. Prices hidden from direct scraping.
4. Install honeypot — 1–2 hours. Automatic bot blocking.
5. Behavioral analysis — 2–4 hours. Adaptive blocking of anomalies.
6. Testing and monitoring — 2–4 hours. Confirm protection effectiveness.

Comparison of protection layers

Echeloned protection is 10–20 times more expensive for an attacker than single-layer rate limiting. We recommend combining all layers. Our engineers have 10+ years of experience and have completed over 200 projects on content protection for Bitrix. After implementing echeloned protection, the number of scraping attempts drops by 80%, and server load decreases threefold.

Behavioral analysis uses machine learning to detect anomalies. For example, if a bot bypasses rate limiting through proxies, a model based on time between requests and page view sequence can detect a non-human pattern. Implementing such analysis requires configuration and takes 2–4 days, but increases protection effectiveness to 99%.

Order a free audit of your catalog protection — our engineer will identify vulnerabilities in one day and propose an implementation plan. Get a consultation on echeloned protection: we will assess your project and select the optimal combination of layers.

Recommended reading: Web scraping on Wikipedia — overview of data collection methods.

1C-Bitrix Site Security: Audit, Protection, Monitoring

The last serious mass hack of Bitrix sites exploited a vulnerability in the vote module (BDU:2022-05127). Attackers uploaded web shells in bulk. The cause? Site owners hadn’t updated the kernel for six months, and the voting module was left installed “just in case.” Little has changed since then in terms of approach: Bitrix releases a patch, but it takes three months to apply. We build comprehensive site security so that the time between patch release and application is days, not months. And even without a patch, the site won’t fall to a typical attack. Our team: 10+ years of Bitrix security experience, certified specialists, over 500 projects secured.

Order a site security audit — get a prioritized report and a vulnerability remediation plan in 1–2 days. Guaranteed 95% attack reduction for properly configured WAF.

Why Is Proactive Protection Better Than Reactive Cleanup?

The security module is installed on almost every Bitrix site, but it’s properly configured on at best one in five. Here’s what exactly needs to be enabled and adjusted:

• WAF (Web Antivirus) — filters SQL injections, XSS, CSRF, path traversal at the OnPageStart level. Key setting: “Active Reaction” mode — not just log, but block. In /bitrix/admin/security_filter.php, check that all attack types are enabled and exceptions are minimal. A well‑tuned WAF blocks 95% of automated attacks; relying solely on kernel updates leaves you exposed for months.
• Activity control (/bitrix/admin/security_iprule.php) — limits on requests from a single IP. Default is 100 requests per minute. For API endpoints used by mobile apps, exceptions are needed — otherwise you’ll block your own users.
• 2FA — OTP via Google Authenticator. Enable in user settings. Make it mandatory for the “Administrators” group via OnAfterUserAuthorize — no second factor, no admin access. Mandatory for all admin users.
• File integrity check (/bitrix/admin/security_file_verifier.php) — hashes of system files. If someone modifies a file in /bitrix/modules/, the system will notice. Run daily via cron using agent CSecurityFileVerifier::Verify().
• Stop list — b_security_filter_stoplist. Automatic IP blocking when WAF triggers. Manual addition of subnets when scanners are detected.
• Security log — b_event_log. Who changed what and when in the admin panel. Store for at least 90 days. Invaluable during incident investigation.

What Does a Bitrix Site Security Audit Include?

Server level — this is where most holes are:

• phpinfo() accessible via /info.php or /phpinfo.php — found on every third project. The attacker gets PHP version, paths, modules, configuration. Delete it.
• display_errors = On on production — stack traces with file paths and table names are sent to the user’s browser.
• PHP functions exec, system, passthru, proc_open not disabled in php.ini. If a web shell gets uploaded, these functions give full server control.
• PHP version should be 8.1+ — no security updates for earlier versions; PHP 7.4 is no longer supported but still lives on a quarter of projects.

Application level:

• Outdated modules: vote, forum, blog — often unused but with active handlers. Deactivate and remove.
• Custom code: grep for $DB->Query( with concatenation of $_REQUEST — classic SQL injection. Should use $DB->ForSql() or D7 ORM.
• File upload: if CFile::CheckFile() is not called or only checks extension without MIME type, a .php file will be uploaded via the feedback form.
• dbconn.php and .env — must be blocked by web server rules. Check: curl https://site.ru/bitrix/.settings.php should return 403.

SSL/TLS:

• Rating A or higher via SSL Labs.
• HSTS with max-age of at least 31536000 (one year).
• HTTP -> HTTPS redirect at Nginx level, not at Bitrix level.

Audit result — a prioritized report: Critical / High / Medium / Low. Critical issues are fixed on day one. Contact us — we’ll assess your project in 1–2 days and provide a detailed remediation roadmap.

Healing Hacked Sites — Protocol of Actions

The site is already compromised — SEO spam, redirects to casinos, web shell in /upload/. Order of actions:

1. Isolation — take the site down, put up a placeholder. If malware is encrypting files or spreading, every minute counts.
2. Identify the vector — access logs (access.log), error logs, b_event_log. Look for POST requests to unusual files, requests to /upload/*.php, suspicious user agents.
3. Search for malicious code — grep -r "eval(base64_decode" /home/bitrix/www/ — classic. Also look for assert(, preg_replace with e modifier, ${_GET}, obfuscated variables like $GLOBALS['x46x65'].
4. Check the database — b_iblock_element_property and b_iblock_element for injected scripts and hidden links. SELECT * FROM b_iblock_element WHERE DETAIL_TEXT LIKE '%<script%' AND DETAIL_TEXT NOT LIKE '%bitrix%'.
5. Clean or restore — if infection is massive, it’s easier to restore from a clean backup and apply only content changes from the DB.
6. Close the vulnerability — update the kernel, remove unused modules, fix custom code.
7. Request re-scan — Google Search Console → “Request Review”, Yandex.Webmaster → “I fixed everything”.

Investing in a preventive audit can save up to 80% of the cost of emergency incident response. Guaranteed recovery within 1–3 days for subscription clients.

How to Protect a Bitrix Site from DDoS?

• Cloudflare / DDoS-Guard / Qrator — traffic proxying. L3/L4 attacks are filtered on their side. L7 — through rules and challenge pages. Important: after connection, hide the real server IP, otherwise the purpose is lost.
• Rate limiting on Nginx: limit_req_zone for /bitrix/admin/, /api/, forms. Separate limits for authenticated and anonymous users.
• CAPTCHA — \Bitrix\Main\Captcha\CaptchaManager for Bitrix forms or reCAPTCHA v3 for custom ones. v3 doesn’t annoy users — works in the background.
• Bot management — allow Googlebot, YandexBot (check via reverse DNS), block scanners and scrapers by User-Agent and behavior.

Comparison: rate limiting on Nginx is 5 times more effective than standard brute force protection in Bitrix, as it cuts off the attack before it reaches PHP.

Why Is File Integrity Monitoring Critical?

File integrity check (/bitrix/admin/security_file_verifier.php) — hashes of system files. If someone modifies a file in /bitrix/modules/, the system will notice. Run daily via cron using agent CSecurityFileVerifier::Verify(). Combine with inotify on /upload/ — any new .php file triggers an immediate alert.

Backups — The Last Line of Defense

• Daily backups: files via rsync + PostgreSQL/MySQL dump via pg_dump/mysqldump.
• Store in isolated S3-compatible storage. Key word: isolated. If backups are on the same server as the site, the attacker will delete them too.
• Rotation: daily × 7, weekly × 4, monthly × 12.
• Test restoration — quarterly, restore a backup on a test server. A backup that cannot be restored is just a file on disk.
• Monitoring: if a backup fails — alert in Telegram within an hour.

Monitoring — Detect Before the Client Calls

• Uptime — check every 60 seconds via UptimeRobot / Zabbix / custom script. Alert in Telegram + phone call if downtime > 5 minutes.
• File monitoring — inotify (Linux) or cron + md5sum on critical directories. New .php in /upload/? Alert immediately.
• Malware scanning — AI-BOLIT or ClamAV on schedule. Check both files and database.
• SSL certificate — warning 30/14/7 days before expiry. Let’s Encrypt auto-renews via certbot, but certbot can also fail.
• Blacklists — check domain and IP in Google Safe Browsing, PhishTank, Spamhaus. Being listed means traffic loss.

152-FZ and Personal Data (Russian Law Context)

• HTTPS everywhere — redirect at Nginx level.
• Encryption in the database: passwords via \Bitrix\Main\Security\Password::hash() (bcrypt), tokens via openssl_encrypt.
• Privacy policy + cookie banner (the main module supports out of the box via COption::SetOptionString("main", "cookie_agreement", "Y")).
• Logging access to personal data — who and when viewed client data.

Deliverables

Timelines

We work on a one-time basis and on subscription with a fixed SLA. For subscription clients, a dedicated engineer who knows the project. Get a consultation — we’ll assess risks and prepare a quote in 1–2 days.

Checklist: 15 Items We Check on Every Project

1. 1C-Bitrix kernel and modules — up to date, unused modules removed.
2. security module active, WAF in “Active Reaction” mode.
3. 2FA enabled for all accounts with admin access.
4. /bitrix/admin/ protected by IP or additional HTTP authentication.
5. Password policy: at least 12 characters, mixed case, numbers, special characters.
6. SSL/TLS: A+ rating on SSL Labs, HSTS enabled.
7. Service files (dbconn.php, .settings.php, .env, backups, logs) — 403 from browser.
8. Permissions: 644 files, 755 directories. Web server is not owner of system files.
9. Security headers: Content-Security-Policy, X-Frame-Options: DENY, X-Content-Type-Options: nosniff, Strict-Transport-Security, Referrer-Policy.
10. File integrity check — daily via agent.
11. Backups: daily, isolated storage, restore testing.
12. b_event_log — storage for at least 90 days, regular review.
13. PHP 8.1+, display_errors = Off, dangerous functions disabled.
14. Uptime monitoring + alerts on file changes in /upload/.
15. Reverse proxy or CDN with DDoS protection for high-load projects.

Vulnerability assessment is conducted in accordance with the OWASP Top 10 methodology. Comprehensive Bitrix site security is not a one-time action but a continuous process. Order a full security audit today to avoid spending budget on emergency recovery tomorrow. Contact us for a free consultation — we’ll answer any questions.