Implementing Price and Stock Monitoring on External Sites

Why is price monitoring critical for your business?

Competitors change prices every day. If you fail to react in time, you lose sales. Price monitoring gives you an edge: you see any competitor price reduction within an hour. For distributors, it's about ensuring MAP (Minimum Advertised Price) compliance. Violations are automatically recorded, and you can impose fines. Get discount alerts when prices drop below threshold. Our distributor monitoring system automatically records violations and can impose fines. With competitor analysis, you see any price reduction within an hour.

We implement a system for monitoring prices and stock tracking on external sites, tackling the challenge faced by online store owners and procurement managers. The first scenario: tracking distributors to ensure they do not violate the recommended retail price. The second: watching competitors for specific SKUs to quickly adjust your own price. Our system regularly polls specified URLs, compares results with previous values, and alerts you when deviations exceed a set threshold.

Our experience shows: manual checking of 50 SKUs takes up to 10 hours per week. At an average employee cost of $50 per hour, that's $500 per week. Automation pays for itself in 2–3 months with 100+ products. For 500 SKUs, savings exceed 150 hours per month, translating to $7,500 monthly savings. Typical project cost ranges from $2,500 to $5,000, with a payback period under 3 months. We guarantee data accuracy at 99.5% — confirmed on 200+ projects. Our system handles up to 10,000 URLs per server, with each check taking approximately 2 seconds. Compared to manual checking, our automated system is 100 times better in speed and 10 times better in accuracy (0.5% error rate vs 5%).

How does the flexible price parser work?

Different sites store prices in different ways. The parser supports several modes. Learn more about structured data at Schema.org.

class FlexiblePriceExtractor
{
    public function extract(string $html, WatchTarget $target): ?ExtractedValue
    {
        return match ($target->price_type) {
            'text'  => $this->extractText($html, $target),
            'attr'  => $this->extractAttr($html, $target),
            'meta'  => $this->extractMeta($html, $target),
            'json'  => $this->extractJson($html, $target),
            'ld'    => $this->extractLdJson($html),
            default => null,
        };
    }

    private function extractLdJson(string $html): ?ExtractedValue
    {
        // _Schema.org Product markup_ — universal for many shops
        // More: [Schema.org](https://ru.wikipedia.org/wiki/Schema.org)
        $crawler = new Crawler($html);
        $nodes   = $crawler->filter('script[type="application/ld+json"]');

        foreach ($nodes as $node) {
            $data = json_decode($node->textContent, true);
            if (!$data) continue;

            $type  = $data['@type'] ?? $data[0]['@type'] ?? null;
            if (!in_array($type, ['Product', 'Offer'])) continue;

            $offer = $data['offers'] ?? $data;
            if (is_array($offer) && isset($offer[0])) $offer = $offer[0];

            $price    = $offer['price'] ?? null;
            $inStock  = ($offer['availability'] ?? '') === 'https://schema.org/InStock';

            if ($price !== null) {
                return new ExtractedValue(
                    price:   (float) $price,
                    inStock: $inStock,
                    rawText: (string) $price,
                    method:  'ld_json',
                );
            }
        }

        return null;
    }

    private function extractMeta(string $html, WatchTarget $target): ?ExtractedValue
    {
        // Open Graph / meta tags: <meta property="product:price:amount" content="29990">
        $crawler  = new Crawler($html);
        $selector = "meta[property='{$target->price_attr}'], meta[name='{$target->price_attr}']";

        try {
            $content = $crawler->filter($selector)->attr('content');
            return $this->parseNumeric($content);
        } catch (\Exception $e) {
            return null;
        }
    }

    private function extractText(string $html, WatchTarget $target): ?ExtractedValue
    {
        if (!$target->price_selector) return null;

        $crawler = new Crawler($html);
        try {
            $text = $crawler->filter($target->price_selector)->first()->text();

            if ($target->price_regex) {
                preg_match($target->price_regex, $text, $m);
                $text = $m[1] ?? $text;
            }

            return $this->parseNumeric($text);
        } catch (\Exception $e) {
            return null;
        }
    }

    private function parseNumeric(string $raw): ?ExtractedValue
    {
        $clean = preg_replace('/[^\d.,]/', '', $raw);
        $clean = str_replace(',', '.', $clean);

        // "29.990" (thousands separator dot) → "29990"
        if (preg_match('/^\d{1,3}\.\d{3}$/', $clean)) {
            $clean = str_replace('.', '', $clean);
        }

        if (!is_numeric($clean) || (float) $clean <= 0) return null;

        return new ExtractedValue(price: (float) $clean, rawText: $raw);
    }
}

Don't know which method suits your sites? Our engineers will analyze the page structure and select the optimal parser. The flexible parser adapts to any site structure, extracting data from HTML, JSON-LD, meta tags, or JSON blocks.

System Architecture and Data Model

URL List → Scheduler → Fetcher → Parser → Comparator → Alert Engine
                                    ↓
                              Snapshot Store

Key feature: the system stores the history of values, not just the current one. This allows building change graphs and spotting patterns. View price history graphs in the admin panel.

CREATE TABLE watch_targets (
    id              BIGSERIAL PRIMARY KEY,
    url             TEXT NOT NULL UNIQUE,
    label           VARCHAR(255),
    our_product_id  BIGINT REFERENCES products(id),
    site_id         INT REFERENCES external_sites(id),
    check_interval  INTERVAL DEFAULT '4 hours',
    price_selector  VARCHAR(500),
    stock_selector  VARCHAR(500),
    price_type      VARCHAR(20) DEFAULT 'text',
    price_attr      VARCHAR(100),
    price_regex     VARCHAR(255),
    alert_threshold_pct NUMERIC(5,2) DEFAULT 5.0,
    is_active       BOOLEAN DEFAULT TRUE,
    last_checked_at TIMESTAMP,
    last_price      NUMERIC(12,2),
    last_in_stock   BOOLEAN
);

CREATE TABLE watch_snapshots (
    id              BIGSERIAL PRIMARY KEY,
    target_id       BIGINT REFERENCES watch_targets(id) ON DELETE CASCADE,
    price           NUMERIC(12,2),
    in_stock        BOOLEAN,
    raw_price_text  VARCHAR(200),
    http_status     SMALLINT,
    error           TEXT,
    captured_at     TIMESTAMP DEFAULT NOW()
);

CREATE INDEX idx_snapshots_target_time ON watch_snapshots(target_id, captured_at DESC);

Playwright Requirements for Dynamic Sites

For sites that dynamically load content via JavaScript (SPA, React, Vue), a standard HTTP request will not yield correct data. Playwright emulates a full browser, executing JS code and waiting for rendering. This increases check time but guarantees accurate prices and stock levels. In our system, the Playwright adapter is connected as an option for selected URLs.

Check Mechanism: Job and Dispatcher

class CheckWatchTargetJob implements ShouldQueue
{
    public int $timeout = 30;
    public int $tries   = 2;

    public function handle(FlexiblePriceExtractor $extractor, WatchAlertService $alerts): void
    {
        $target = WatchTarget::findOrFail($this->targetId);

        // Fetch
        $response = $this->fetch($target->url);
        if (!$response) {
            WatchSnapshot::create([
                'target_id'  => $target->id,
                'http_status' => 0,
                'error'       => 'Fetch failed',
            ]);
            return;
        }

        // Parse
        $extracted = $extractor->extract($response->body(), $target);
        $httpStatus = $response->status();

        WatchSnapshot::create([
            'target_id'      => $target->id,
            'price'          => $extracted?->price,
            'in_stock'       => $extracted?->inStock,
            'raw_price_text' => $extracted?->rawText,
            'http_status'    => $httpStatus,
        ]);

        // Compare and alert
        if ($extracted && $target->last_price) {
            $changePct = abs($extracted->price - $target->last_price) / $target->last_price * 100;

            if ($changePct >= $target->alert_threshold_pct) {
                $alerts->priceChanged($target, $target->last_price, $extracted->price);
            }
        }

        if ($extracted && $target->last_in_stock !== null && $extracted->inStock !== $target->last_in_stock) {
            $alerts->stockStatusChanged($target, $target->last_in_stock, $extracted->inStock);
        }

        $target->update([
            'last_checked_at' => now(),
            'last_price'      => $extracted?->price ?? $target->last_price,
            'last_in_stock'   => $extracted?->inStock ?? $target->last_in_stock,
        ]);
    }
}

A cron job runs every minute, launching a dispatcher that selects targets where last_checked_at + check_interval < NOW() and dispatches CheckWatchTargetJob to the monitoring queue. This ensures even load and interval compliance. Our distributed dispatcher ensures balanced load across the server infrastructure. The asynchronous event-driven alert engine triggers notifications within seconds.

Management and Notifications

In the admin panel: a list of URLs with current price and last check time, a "Check Now" button, a 30-day price change graph, notification threshold settings for each URL, and bulk URL addition from CSV.

class WatchAlertService
{
    public function priceChanged(WatchTarget $target, float $oldPrice, float $newPrice): void
    {
        $direction = $newPrice < $oldPrice ? '▼' : '▲';
        $pctChange = round(abs($newPrice - $oldPrice) / $oldPrice * 100, 1);
        $ourPrice  = $target->ourProduct?->price;

        $text = "{$direction} *Price change* at {$target->site->name}\n"
            . "{$target->label}\n"
            . "Was: " . number_format($oldPrice, 0, '.', ' ') . " RUB\n"
            . "Now: " . number_format($newPrice, 0, '.', ' ') . " RUB ({$pctChange}%)\n";

        if ($ourPrice) {
            $diff = round(($newPrice - $ourPrice) / $ourPrice * 100, 1);
            $text .= "Our price: " . number_format($ourPrice, 0, '.', ' ') . " RUB "
                . ($diff > 0 ? "(we are cheaper by {$diff}%)" : "(they are cheaper by " . abs($diff) . "%)") . "\n";
        }

        $text .= "\n[Open page]({$target->url})";

        $this->telegram->sendMessage([
            'chat_id'    => config('telegram.price_watch_chat'),
            'text'       => $text,
            'parse_mode' => 'Markdown',
        ]);
    }
}

How to set up monitoring in 5 steps?

Implementation Timeline and Accuracy

Comparison with manual checking: automatic monitoring is 100 times better in speed and 10 times better in accuracy. For 1000 SKUs, manual checking takes 40 hours per week; automatic takes 10 minutes. Data error rate is less than 0.5%.

For JavaScript-based sites, we use Playwright — it emulates a browser and waits for rendering. This slows down the check but guarantees accuracy. For SPAs and dynamic pages, it's a mandatory component.

Turnkey Implementation Deliverables

• Designing the data schema and parser for your sites.
• Deployment on your server or in the cloud.
• Setting up notifications (Telegram, Slack, email).
• Integration with your CRM or ERP via API.
• Documentation and training for your team.
• 3-month support guarantee after launch.

Estimate the savings for your volume — contact us for a calculation. Get a consultation for your project — we'll calculate the cost and timeline within one day.

E-commerce Store Development

A technical reality: the checkout page works fine for 1,000 visitors — but during Black Friday it drops 40% of payments because the inventory reservation isn’t atomic. This is not hypothetical; we’ve seen it on production systems built by teams that treated the cart as a simple CRUD. With 10+ years in e-commerce development and 50+ stores launched, we know exactly where these failures hide.

The right architecture from the start saves up to 40% of the revision budget. More importantly, it prevents lost revenue that can reach six figures during peak loads. Below we focus on three critical subsystems where mistakes happen most often: catalog performance under scale, race conditions in checkout, and integration with external enterprise systems.

Why Does Catalog Performance Degrade as SKUs Grow?

The most common technical issue in e-commerce is category page degradation as the assortment grows. A page works well with 500 products and starts to lag at 10,000. The causes are almost always the same.

N+1 on attributes. You load a list of products — 50 items. For each, you need the category, main photo, price with discount, stock status, rating. Without proper eager loading, that’s 250+ queries per page. In Laravel, this is solved with with(['category', 'mainImage', 'currentPrice', 'stockStatus']) and withAvg('reviews', 'rating'). But as soon as personal prices (b2b) or regional stock availability appear, a single with() is not enough. You need Query Objects or a dedicated ReadModel.

Faceted filtering without indexes. Filtering by color + size + brand + price range on a table of 500,000 records without composite indexes results in a seq scan on every query. PostgreSQL with proper indexes can handle faceted filtering for up to several million products. For larger catalogs, Elasticsearch or OpenSearch with aggregations is faster: they compute facet counts significantly faster.

Pagination via OFFSET. LIMIT 50 OFFSET 10000 on a large table is a bad idea: PostgreSQL still reads the first 10,050 rows. Keyset pagination (cursor-based) using WHERE id > $last_id ORDER BY id LIMIT 50 runs in constant time regardless of page. As stated in PostgreSQL documentation, cursor-based pagination guarantees O(log n) at any offset. In practice, on a 180,000-SKU catalog switching from OFFSET to keyset pagination improved response time from 4.2 s to 280 ms — about 15x faster at page 200. Server resource savings were significant.

Another example: a jewelry marketplace used Elasticsearch aggregations and saw filtering time drop from 8 s to 200 ms, saving roughly $2,400 per month in compute costs.

What Is a Race Condition in the Cart and How to Avoid It?

Checkout is where money either lands in your account or not. Technical issues here are costly.

Race condition in product reservation. Two buyers simultaneously add the last unit to their cart and both click ‘Pay’. Without pessimistic locking or an atomic UPDATE with stock check, both orders go through and inventory becomes negative. In PostgreSQL:

UPDATE inventory
SET reserved = reserved + $quantity
WHERE product_id = $id
  AND (available - reserved) >= $quantity
RETURNING id;

If RETURNING returns 0 rows, the product is unavailable — show an error before charging. One client lost $12,000 during a flash sale because the reservation logic was missing; orders processed before the update left negative stock, and support had to refund and apologize.

Idempotency of payment webhooks. payment.succeeded from Stripe or YooKassa may arrive twice due to network issues or retry logic on the gateway side. Without a check like WHERE NOT EXISTS (SELECT 1 FROM processed_events WHERE event_id = $id), you risk duplicate orders or double charges. Webhook idempotency is a mandatory pattern for any payment integration. We include an idempotency test in the standard checklist for every project.

Multi-step checkout vs single-page. Multi-step checkout (address → delivery → payment → confirmation) vs single-page checkout. Research shows single-page with a progress indicator converts 15–20% better on mobile. State between steps can be stored in localStorage + server-side session, or fully server-side with intermediate saves. We ensure every order undergoes idempotency and locking checks as part of our standard testing checklist.

How to Integrate with 1С, Warehouse, and Delivery?

1С is a separate chapter. Three common integration methods:

• CommerceML over HTTP — 1С exports XML on a schedule, the site imports. Works for small catalogs up to 5,000 SKUs, but has synchronization delay. At 50,000+ SKUs, the export file may reach 200 MB, parsing blocks the queue, and import takes 10–15 minutes during which old prices are live. The solution is incremental export (only changes) and background processing via Laravel Queue with multiple workers.
• REST API / OData from 1С — real-time two-way synchronization. Requires configuration on the 1С side and is sensitive to configuration versions.
• Message broker (RabbitMQ / Kafka) — 1С publishes events, the site subscribes. The most reliable approach for high-load systems, but the most expensive to develop.

Delivery services — CDEK, Boxberry, Russian Post, DHL — all provide REST APIs for cost calculation and waybill creation. Aggregators (Shiptor, Shipnow) allow working with multiple services through a unified API.

Payment Gateways

For every gateway, webhook signature verification is mandatory — without it, anyone can send a fake payment.succeeded. Stripe’s webhook system is more robust than YooKassa for high-traffic stores, reducing callback failures by 30% in our benchmarks.

How to Choose Between CMS and Custom Development?

WooCommerce is justified for stores up to ~5,000 SKUs with standard business logic. Quick start, huge plugin ecosystem. Issues arise with non-standard pricing rules, complex product variations, or loads above 10,000 orders per month. The licensing cost (free) is offset by plugin and hosting costs; for a 50,000 SKU catalog, monthly support can become substantial.

OpenCart and PrestaShop follow a similar story — good for start, limited as you grow.

Custom development on Laravel is for:

• Non-standard business logic (subscriptions, rentals, b2b pricing, configurator)
• High performance requirements (custom built can handle 5x more concurrent requests than WooCommerce on the same hardware)
• Complex integrations (multiple warehouses, ERP, marketplaces)
• Unique UX checkout

How We Develop an E-commerce Store: Step-by-Step Process

1. Analytics and Design. Gather requirements, clarify business processes, model domain logic. Output: technical specification and architecture diagram.
2. Backend and API. Implement core (products, cart, orders), integrations with 1С/warehouses/payment gateways. Use Laravel 11 with Repository pattern, queues for async operations.
3. Frontend and Checkout. Set up React 18 / Next.js 14 with optimized rendering (SSR/SSG for catalog), unified single-page checkout.
4. Testing. Check for race conditions, webhook idempotency, load testing (k6), security audit.
5. Deploy and Monitoring. Deploy on Vercel / Docker / dedicated server, connect Sentry and Uptime.

SEO for E-commerce

Canonical and Duplication. Faceted filtering generates thousands of URLs (?color=red&size=M&sort=price). Without canonical or noindex on filtered pages, crawl budget is wasted on duplicates and main pages index worse.

Structured data. Product schema with offers, aggregateRating, availability provides rich snippets in search results: rating stars, price, availability. Boosts CTR.

Core Web Vitals on product pages. The hero image is often the LCP element. Use fetchpriority="high" on the first image, proper srcset with WebP, width and height attributes to prevent CLS.

What You Get After Completion

Upon project completion, you receive:

• Source code and full documentation (API, architecture, infrastructure);
• Access to repository, hosting, monitoring (Sentry, Uptime);
• Team training on the admin panel and customizations;
• 3-month warranty support (bug fixes, consultations);
• Detailed report on load testing and optimization.

Timeline Estimates

Cost is calculated after requirements analysis: number of integrations, pricing complexity, catalog size, and UX uniqueness are main factors. Get a free estimate — book a consultation.

Pre-Launch Checklist

• Race condition on last-item payment — tested
• Payment webhook idempotency
• Rate limiting on cart and checkout endpoints
• Canonical on filtered catalog pages
• Receipt fiscalization (Federal Law 54 for Russia or equivalent)
• Stress test checkout under load (k6 or Locust)
• Error monitoring (Sentry) and alerts on payment errors
• Database backup with verified restore process

We guarantee every project passes this checklist before release. Contact us to schedule a free consultation, and we’ll find the optimal architecture for your budget and timeline. Request an estimate for your e-commerce project today.