A manager spends an hour on manual product import from multiple suppliers: download XML from supplier A, CSV from B, API from C, match SKUs, correct prices, remove duplicates. Result — a product with the same SKU appears twice with different stock levels, price from one supplier, stock from another — inconsistent product card. With five or ten suppliers, manual processing becomes endless routine with inevitable errors. Imagine: 10 suppliers, 50,000 products, new prices and stock daily. If loading manually, one employee spends 40 hours a week just on import, and errors lead to lost orders. Our multi-supplier import system solves this with a single pipeline: adapters normalize data, deduplicator removes repeats, merger assembles the final card by priorities. Automation cuts processing time to 5 minutes per 10,000 products (according to Invesp research), errors drop from 5–10% to <0.1%. Cost savings: a typical three-supplier implementation costs $3,500 and pays back within 3 months, saving up to $6,000 per month on manual labor. With over 10 years of experience and 150+ successful projects, we guarantee a seamless integration. Operational cost savings — up to 80% per supplier. Our proven track record ensures reliable, certified imports.
Why Multi-Supplier Import is Better Than Manual?
Automation outperforms manual import by 10 times in speed and 50 times in accuracy. Automation also eliminates the human factor during SKU matching, especially critical for products with many variations. Comparison of key parameters:
| Parameter | Manual Import | Automated Import |
|---|---|---|
| Time for 10,000 products | ~50 minutes | ~5 minutes |
| Error rate | 5–10% | <0.1% |
| Skip unchanged | No | Yes (hashing) |
| Supplier priority handling | Manual with errors | Automatic by rules |
| Monthly cost for 10 suppliers | $6,000 | $1,200 |
How to Build an Import Pipeline?
- Identify suppliers and their data formats.
- Implement adapters that implement
SupplierAdapterInterface. - Configure normalization: convert data to a unified DTO.
- Run deduplication by external IDs and SKU.
- Set up merge rules and priorities.
- Connect monitoring and alerts.
Each phase of a typical project takes 1–2 days, total 5–7 working days for three suppliers.
Architecture and Implementation
Supplier A (XML) ─┐
Supplier B (CSV) ─┤─► Normalizer ─► Deduplicator ─► Merger ─► Catalog DB
Supplier C (API) ─┘
Each supplier is a separate adapter implementing SupplierAdapterInterface. The pipeline is unified: normalizer converts to DTO, deduplicator matches by external ID and SKU, merger applies priority rules.
Deduplication Details
Deduplication is performed by external ID and SKU. If a product already exists but data changed (checked by hash), the record is updated. If unchanged, it is skipped. This allows processing millions of records without overwrites.Conflict Resolution Mechanism
Key solution — separate storage of raw supplier data and final product card. Raw data is stored in supplier_products with a unique key (supplier_id, external_id). Final data is in products with a link to the primary supplier.
CREATE TABLE supplier_products (
id BIGSERIAL PRIMARY KEY,
supplier_id INT NOT NULL REFERENCES suppliers(id),
external_id VARCHAR(255) NOT NULL,
sku VARCHAR(255),
barcode VARCHAR(50),
name TEXT NOT NULL,
price NUMERIC(12,2),
stock INT DEFAULT 0,
attributes JSONB DEFAULT '{}',
raw_data JSONB,
imported_at TIMESTAMP NOT NULL,
hash VARCHAR(64),
UNIQUE(supplier_id, external_id)
);
CREATE TABLE products (
id BIGSERIAL PRIMARY KEY,
master_sku VARCHAR(255) UNIQUE,
name TEXT,
price NUMERIC(12,2),
stock INT,
primary_supplier_id INT REFERENCES suppliers(id),
merged_from INT[],
updated_at TIMESTAMP
);
XML Adapter Example
interface SupplierAdapterInterface {
public function fetch(): Generator;
public function getSupplierId(): int;
}
class SupplierProductDTO {
public function __construct(
public readonly string $externalId,
public readonly string $name,
public readonly float $price,
public readonly int $stock,
public readonly ?string $sku = null,
public readonly ?string $barcode = null,
public readonly array $attributes = []
) {}
}
class XmlSupplierAdapter implements SupplierAdapterInterface {
public function fetch(): Generator {
$reader = new XMLReader();
$reader->open($this->feedUrl);
while ($reader->read()) {
if ($reader->nodeType === XMLReader::ELEMENT && $reader->name === 'item') {
$node = new SimpleXMLElement($reader->readOuterXML());
yield new SupplierProductDTO(
externalId: (string) $node->id,
name: (string) $node->name,
price: (float) $node->price,
stock: (int) $node->quantity,
sku: (string) $node->article ?: null
);
}
}
}
public function getSupplierId(): int {
return $this->supplierId;
}
}
Import Service and Queues
class SupplierImportService {
public function import(SupplierAdapterInterface $adapter): ImportResult {
$supplierId = $adapter->getSupplierId();
$result = new ImportResult();
DB::transaction(function () use ($adapter, $supplierId, $result) {
foreach ($adapter->fetch() as $dto) {
$hash = hash('sha256', serialize($dto));
$existing = SupplierProduct::where(['supplier_id' => $supplierId, 'external_id' => $dto->externalId])->first();
if ($existing && $existing->hash === $hash) {
$result->skipped++;
continue;
}
SupplierProduct::updateOrCreate(
['supplier_id' => $supplierId, 'external_id' => $dto->externalId],
['name' => $dto->name, 'price' => $dto->price, 'stock' => $dto->stock, 'sku' => $dto->sku, 'imported_at' => now(), 'hash' => $hash]
);
$result->upserted++;
}
SupplierProduct::where('supplier_id', $supplierId)->where('imported_at', '<', now()->subMinutes(30))->update(['stock' => 0]);
});
return $result;
}
}
class ImportSupplierJob implements ShouldQueue {
public int $timeout = 1800;
public int $tries = 3;
public function handle(SupplierImportService $service): void {
$adapter = SupplierAdapterFactory::make($this->supplier);
$result = $service->import($adapter);
Log::info("Supplier {$this->supplier->name} imported", $result->toArray());
if ($this->isLastActiveImport()) {
MergeProductsJob::dispatch();
}
}
}
Each supplier is triggered by a separate scheduled job. Normalization uses streaming with PHP Generators, saving memory for tens of thousands of products.
Monitoring and Alerts
Key metrics: number of imported/skipped records, percentage of price anomalies (change >20% — signal of supplier error), import execution time. When thresholds are exceeded, a notification fires. Key metrics: number of new, updated, deleted products; parsing error percentage; import execution time; price anomalies (change >20% in one import). When thresholds are exceeded, an alert is sent to Telegram or Slack.
Scope of Work and Results
We deliver a fully configured pipeline with adapters for your suppliers, operational documentation, team training (up to 3 hours), and 30 working days of support. Optionally, monitoring and alerts integration.
| Phase | Duration | Result |
|---|---|---|
| Pipeline design | 1 day | Data schema, adapter list |
| Implementation of 2 adapters (XML+CSV) | 2 days | Working adapters with normalization |
| Merge rules configuration | 1–2 days | Prioritization and merging logic |
| Queues, logs, alerts | 1 day | Monitoring and notifications |
| Documentation and training | 1 day | Admin guide |
A typical project with three suppliers takes 5–7 working days. Each additional adapter takes 0.5 to 2 days. Savings per supplier — up to 80% of employee time.
With 10+ years of experience, 150+ successful projects, and a 100% satisfaction guarantee, we are the trusted partner for multi-supplier import automation. Estimate savings for your business — contact us for a consultation. Order an import system implementation — forget about routine product loading.