Webhook System with Logging and Retry: Turnkey Setup

You launched an integration with a CRM partner. The first events go through, but an hour later the client complains that half the notifications were never received. Logs are silent — the receiver returned 200, but didn't process the data. Without per-attempt details, figuring out the cause is guesswork. We build Webhook systems with full logging and retry: not just 'fire and forget', but control over every event. 10+ years in B2B integrations, 100+ projects — we guarantee transparency and reliability.

A webhook is an outgoing HTTP request that you don't fully control. The receiver can return 200 without processing the data. It can crash 9 seconds after receiving. It can miss an event without a trace. Without detailed logging of all attempts and the ability to manually resend, debugging integration issues is nearly impossible.

Why logging every attempt is the foundation of a reliable webhook system

Suppose an event fails on the third attempt after a timeout. If you don't store history, you only see the final status. With full logging, you get the complete picture: first attempt failed with 500, second with an 8-second timeout, third with 502. This immediately points to issues on the receiver's side. Systems without per-attempt logging force developers to spend hours reproducing. Our approach reduces debugging time by an average of 5x compared to traditional monitoring.

What data we log and how it's implemented

Minimum data set for each delivery attempt:

We store attempts separately from deliveries — one delivery can have up to 8 attempts. This allows viewing the full history and understanding at which step things went wrong.

CREATE TABLE webhook_attempts (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  delivery_id UUID NOT NULL REFERENCES webhook_deliveries(id) ON DELETE CASCADE,
  attempt_number INTEGER NOT NULL,
  started_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  duration_ms INTEGER,
  request_body JSONB,
  request_headers JSONB,
  response_code INTEGER,
  response_headers JSONB,
  response_body TEXT,           -- truncated to 2000 characters
  error_message TEXT,
  success BOOLEAN NOT NULL DEFAULT false
);

CREATE INDEX idx_attempts_delivery ON webhook_attempts(delivery_id);
CREATE INDEX idx_attempts_started ON webhook_attempts(started_at DESC);

Logging implementation:

class WebhookAttemptLogger
{
    public function log(
        WebhookDelivery $delivery,
        int $attempt,
        WebhookAttemptData $data
    ): WebhookAttempt {
        return WebhookAttempt::create([
            'delivery_id'      => $delivery->id,
            'attempt_number'   => $attempt,
            'started_at'       => $data->startedAt,
            'duration_ms'      => $data->durationMs,
            'request_body'     => $delivery->payload,
            'request_headers'  => $data->requestHeaders,
            'response_code'    => $data->responseCode,
            'response_headers' => $data->responseHeaders,
            'response_body'    => $data->responseBody
                ? mb_substr($data->responseBody, 0, 2000)
                : null,
            'error_message'    => $data->errorMessage,
            'success'          => $data->success,
        ]);
    }
}

class SendWebhookJob implements ShouldQueue
{
    public function handle(
        WebhookAttemptLogger $logger
    ): void {
        $startedAt = now();
        $requestHeaders = $this->buildHeaders();

        try {
            $response = Http::timeout(15)
                ->withHeaders($requestHeaders)
                ->post($this->delivery->subscription->endpoint_url, $this->delivery->payload);

            $durationMs = (int)(microtime(true) * 1000 - $startedAt->timestamp * 1000);

            $logger->log($this->delivery, $this->delivery->attempt_count, new WebhookAttemptData(
                startedAt: $startedAt,
                durationMs: $durationMs,
                requestHeaders: $requestHeaders,
                responseCode: $response->status(),
                responseHeaders: $response->headers(),
                responseBody: $response->body(),
                success: $response->successful(),
            ));

            if ($response->successful()) {
                $this->delivery->markDelivered();
            } else {
                $this->delivery->scheduleRetry();
            }

        } catch (\Throwable $e) {
            $durationMs = (int)(microtime(true) * 1000 - $startedAt->timestamp * 1000);

            $logger->log($this->delivery, $this->delivery->attempt_count, new WebhookAttemptData(
                startedAt: $startedAt,
                durationMs: $durationMs,
                requestHeaders: $requestHeaders,
                errorMessage: get_class($e) . ': ' . $e->getMessage(),
                success: false,
            ));

            $this->delivery->scheduleRetry();
        }
    }
}

How manual retry is organized

An administrator or developer must be able to resend any event without changing code. This is critical for debugging integrations and recovering from failures.

class WebhookDeliveryController extends Controller
{
    // Resend a specific delivery
    public function resend(WebhookDelivery $delivery): JsonResponse
    {
        abort_if(
            $delivery->status === 'delivered',
            422,
            'Delivery already succeeded'
        );

        $delivery->update([
            'status'          => 'pending',
            'attempt_count'   => 0,
            'next_attempt_at' => now(),
        ]);

        SendWebhookJob::dispatch($delivery);

        return response()->json(['queued' => true]);
    }

    // Resend all failed deliveries for a subscription
    public function resendFailed(WebhookSubscription $subscription): JsonResponse
    {
        $count = WebhookDelivery::where('subscription_id', $subscription->id)
            ->where('status', 'failed')
            ->count();

        WebhookDelivery::where('subscription_id', $subscription->id)
            ->where('status', 'failed')
            ->update([
                'status'          => 'pending',
                'attempt_count'   => 0,
                'next_attempt_at' => now(),
            ]);

        WebhookDelivery::where('subscription_id', $subscription->id)
            ->where('status', 'pending')
            ->each(fn($d) => SendWebhookJob::dispatch($d));

        return response()->json(['requeued' => $count]);
    }

    // Attempt history for a specific delivery
    public function attempts(WebhookDelivery $delivery): JsonResponse
    {
        return response()->json(
            $delivery->attempts()
                ->orderBy('attempt_number')
                ->get(['attempt_number', 'started_at', 'duration_ms',
                       'response_code', 'response_body', 'error_message', 'success'])
        );
    }
}

Step-by-step implementation plan for a webhook system with logging

Log retention strategy: successful attempts — 30 days with body, then only metadata. Failed attempts — 90 days for audit. Error response body — max 2 KB, binary data not stored.

Why our system saves up to 80% of debugging time

A typical log file lacks context: you see an error but not what led to it. Our system stores the full chronology of each delivery, linking all attempts. This cuts investigation time from hours to minutes. Built-in aggregates (average attempts, P95 delivery time) allow early detection of problematic integrations. Compare: without logging — manual server log search, guesswork, restarting integrations. With our system — open the dashboard, filter by status, see the history of each attempt. Click "resend" and failed events are re-queued. We implement this on Laravel queues with PostgreSQL. The result is up to 80% savings in debugging time.

What's included and timeline

• Development of the attempt logging and retry module
• Queue and retry policy configuration
• Dashboard creation with filtering and aggregates
• API and data schema documentation
• Team training on using the system
• One month of technical support

Timeline: attempt logging and manual retry system — 3 to 5 days. With dashboard, aggregates, filtering, and retention policy — 1 to 1.5 weeks.

Order a turnkey system — contact us for an accurate estimate of your project. Get a consultation on implementation.

API Development with REST, GraphQL, WebSocket, and tRPC

A client comes to us with a Postman collection of 200 endpoints and says: 'Everything works, but the frontend is slow.' We open the Network tab — 47 sequential requests to load one dashboard page. Each one waits for the previous. This is not a server speed issue — it's an API architecture problem. With 10 years on the market, we've redesigned dozens of such integrations, and we guarantee: the right protocol and contract solve the problem at its root.

When REST stops being enough

REST works well for simple CRUD operations. But as soon as a mobile app appears alongside the web interface, over-fetching begins: the mobile app requests /api/users/123 and gets a 4KB object, but only needs name and avatar. Multiply that by a list of 50 users — 200KB traffic instead of 8KB.

GraphQL solves this with selection sets. The client describes exactly the fields it needs, and the server returns only those. On a project with React Native + Next.js, we migrated from REST to Apollo Server: payload size on the main screen dropped from 340KB to 28KB — a 92% traffic savings. Our certified engineers confirm: the typical pain when adopting GraphQL is N+1 query. A resolver for the author field on a post calls SELECT * FROM users WHERE id = ? for each post in the list. On a page with 20 posts — 21 database queries. Solved with DataLoader — it batches queries and turns them into one SELECT * FROM users WHERE id IN (...).

What is tRPC and how is it better than REST/GraphQL?

If the entire stack is TypeScript (Next.js + Node/Bun), tRPC removes a whole layer of problems. You define a procedure on the server — the client gets full type-safety automatically, without code generation and without Swagger. Renamed a field in the Zod schema — TypeScript highlights all places on the frontend where it's used. tRPC reduces code by 2 times compared to REST + Swagger + openapi-typescript: no need to maintain a separate specification and generate types — everything is inferred from runtime validators. However, tRPC is not suitable if the API is consumed by third-party clients or mobile apps in other languages — in such cases we use GraphQL or REST with OpenAPI specification.

WebSocket and real-time: when SSE, when WS?

HTTP polling every 5 seconds is an illusion of real-time with up to 5 seconds delay and useless server load. For chats, live notifications, collaborative editing — WebSocket or Server-Sent Events. SSE is a one-way stream from server to client, works over ordinary HTTP, automatically reconnects. Suitable for notifications, data streaming, progress bars. WebSocket is bidirectional, needed for chats and collaborative features. Experience shows: 80% of 'real-time' tasks are solved with SSE, not WebSocket — fewer infrastructure complexities.

A typical mistake: opening a WebSocket connection for each page component. On one project, the dashboard opened 12 parallel WS connections. The correct approach is one connection manager at the application level, subscriptions through it. In our work results, we always transfer the connection scheme and a ready solution.

Swagger / OpenAPI as a contract

Documentation written after the fact becomes outdated the day after release. We write the OpenAPI 3.1 specification before development starts; it becomes the contract between frontend and backend. The frontend generates types via openapi-typescript, the backend validates incoming data using generated schemas. Contract deviation from implementation is caught on CI, not during review. For Laravel — l5-swagger or dedoc/scramble. For Node.js — @fastify/swagger or Zod + zod-to-openapi.

How to properly authenticate an API?

JWT with long-lived access tokens without rotation is a source of problems when compromised. The correct scheme: access token for 15 minutes, refresh token for 30 days with rotation on each use. Refresh token stored in an httpOnly cookie, access token in memory (not in localStorage). For inter-service communication — API Keys with scope limitations or mTLS. OAuth 2.0 with PKCE for public clients (SPA, mobile).

How to handle versioning and backward compatibility?

Breaking changes in an API without versioning break clients. Three approaches we use in projects:

We guarantee backward compatibility through automated checks (oasdiff) on CI.

How we develop APIs: step-by-step plan

1. Analysis — audit of current integrations, data schema compilation, protocol selection (REST/GraphQL/tRPC/WebSocket).
2. Contract design — OpenAPI or SDL (GraphQL) before the first line of code.
3. Development — implementation per contract, unit tests for each endpoint.
4. Load testing — k6: 500 virtual users, 10 minutes, p95 latency ≤ 200ms.
5. Deployment — CI/CD with backward compatibility check, automatic documentation publication.
6. Team training — handover of Postman collection or Playground, connection instructions.

What is included (deliverables)

• OpenAPI 3.1 specification (or SDL for GraphQL).
• Generated client types for TypeScript / Dart / Kotlin.
• Set of automated tests covering all endpoints (unit + integration).
• Load tests (k6) and report (p50/p95/p99 latency, RPS).
• Documentation in Swagger UI / Redoc / GraphiQL.
• Team training (2–4 hour workshop).
• Support for 30 days after delivery (per contract).

Our experience

• 10+ years in the API development market.
• 200+ completed projects (REST, GraphQL, WebSocket, tRPC).
• 50+ certified engineers (AWS, Kubernetes, API Design).
• Traffic savings averaging 85% when migrating from REST to GraphQL for mobile apps.
• 100% backward compatibility — not a single broken client in the last 3 years.

Timeline

API development for a typical SaaS project with 30–50 endpoints: from 3 to 8 weeks depending on business logic complexity and number of external integrations. Migration of an existing REST API to GraphQL: from 2 to 6 weeks. Adding a WebSocket layer to an existing backend: from 1 to 3 weeks. Cost is calculated individually after an audit. Get a consultation — contact us to discuss your project.