Zero Duplicate Orders: Designing an Idempotent Shopify-to-ERP Sync

When a Shopify store is fulfilled through an ERP or a 3PL system, the integration between them is the part that quietly makes or breaks the operation. Orders that sync twice, inventory that drifts, webhooks that arrive out of order — each of these turns into a support ticket and a picking error. On a recent integration we built an order-sync pipeline whose central design goal was idempotency: process the same event any number of times and the result is always correct exactly once.

Why “just call the API” is not enough

Shopify delivers events through webhooks, and webhooks make one guarantee that surprises people: at-least-once delivery. The same orders/create event can arrive twice. A retry after a timeout can arrive after you have already processed the original. If your handler naively creates an ERP order on every webhook, duplicates are not a possibility — they are a certainty.

Idempotency keys

The foundation was a persisted mapping between Shopify’s identifiers and our ERP records, enforced by a unique constraint at the database level rather than in application code:

CREATE UNIQUE INDEX UX_SyncedOrder_ShopifyId
    ON dbo.SyncedOrder (ShopDomain, ShopifyOrderId);

Before creating anything, the handler checks for an existing mapping. If the order was already synced, the webhook is acknowledged and discarded. The unique index is the real guardrail — even under a race between two concurrent deliveries, the database rejects the second insert and we handle the conflict gracefully instead of writing a duplicate.

if (await _repo.ExistsAsync(shopDomain, shopifyOrderId))
    return Results.Ok();          // already processed, acknowledge

try
{
    await _erp.CreateOrderAsync(mappedOrder);
    await _repo.RecordSyncAsync(shopDomain, shopifyOrderId, erpId);
}
catch (DuplicateKeyException)
{
    return Results.Ok();          // lost a race, another worker won
}

Verifying the webhook is really from Shopify

Every incoming webhook is verified with its HMAC signature before a single byte of the body is trusted. This is computed over the raw request body with the app’s shared secret — validate first, deserialise second:

var computed = Convert.ToBase64String(
    new HMACSHA256(secretBytes).ComputeHash(rawBody));
if (!CryptographicOperations.FixedTimeEquals(
        Encoding.UTF8.GetBytes(computed),
        Encoding.UTF8.GetBytes(hmacHeader)))
    return Results.Unauthorized();

Handling failure without losing orders

The ERP is not always available, and it should never be able to lose an order. We decoupled receipt from processing: the webhook is validated, stored, and acknowledged fast, then a background worker performs the actual ERP write with retry and exponential backoff. If the ERP is down, events queue up and drain automatically when it recovers. Nothing is dropped because a downstream system had a bad five minutes.

Reconciliation catches what webhooks miss

Webhooks are the fast path, not the source of truth. A scheduled reconciliation job periodically pulls orders updated since the last checkpoint and compares them against what we have synced, healing any gap from a missed or failed webhook. Belt and braces.

The payoff

• Zero duplicate orders reaching the ERP after go-live — enforced at the database, not just in code.
• Orders reach the ERP within seconds of checkout via the webhook fast path, while the write itself happens safely in the background.
• Automatic recovery from downstream outages, with reconciliation healing anything a webhook missed — no manual replay.

Takeaway

A reliable e-commerce-to-ERP integration is not about the happy path — that part is easy. It is about duplicates, out-of-order delivery, downstream outages, and the events that slip through. Design for idempotency from the first line and those failure modes stop being incidents. We build and rescue Shopify-to-ERP and 3PL integrations of exactly this kind; let us know if your sync is causing duplicates or drift.