Executive Summary & Key Takeaways
Key Insights- Authenticate raw bytes before parsing. Follow provider-specific signing rules. Validate signed timestamps. Persist before acknowledgement. Deduplicate events and make business actions idempotent. Test recovery and key rotation.
Quick Definition / Direct Answer
Direct SummaryVerify webhook signatures against the original request bytes, using the provider’s documented signing format and endpoint secret. Validate signed timestamps when supported, reject malformed signatures, and persist verified events before acknowledging delivery. Process events through an idempotent worker so retries, concurrent deliveries, and failures cannot repeat business actions.
A webhook connects two systems by delivering an HTTP request when an event occurs. A payment provider might announce a settled invoice; a deployment service might report a finished build. The receiving application must decide whether that request is authentic before it changes business state.
Direct answer: Verify webhook signatures against the original request bytes, using the provider’s documented signing format and endpoint secret. Validate signed timestamps when supported, reject malformed signatures, and persist verified events before acknowledging delivery. Process events through an idempotent worker so retries, concurrent deliveries, and failures cannot repeat business actions.
This technical guide designs a receiving pipeline and provides an executable Node.js verifier for a deliberately defined HMAC protocol. It then explains how to connect that verifier to Express and a PostgreSQL inbox. The custom protocol is an educational reference: use a provider’s maintained SDK when it offers verification, and implement its exact protocol rather than substituting this example.
What you will build
The proposed architecture is:
Provider → HTTPS endpoint → bounded raw body → signature + freshness checks
→ payload validation → durable inbox transaction → HTTP acknowledgement
→ background worker → authorized, idempotent business action
The endpoint should authenticate and durably accept a delivery. A separate worker should perform slower actions. This separates the provider’s response deadline from database updates, remote API calls, email delivery, and downstream failures.
Prerequisites and scope
Use a supported Node.js release, Express with raw-body middleware, and a transactional database. Configure a webhook secret through your deployment secret manager. The examples use ES modules and built-in node:crypto and node:test; Express and a PostgreSQL driver are separate application dependencies.
The guide does not supply a complete worker, database adapter, secret-management integration, or provider-specific event schema. Those boundaries are defined explicitly so the example cannot be mistaken for a finished payment or account-provisioning system.
Key takeaways
- Authenticate exact bytes before parsing or using the payload.
- Use the provider’s documented signature protocol; the example below defines a custom protocol.
- Combine signed delivery freshness with durable event deduplication.
- Commit accepted events before acknowledging them, then process through a recoverable worker.
- Protect business actions with operation-level idempotency and reconciliation.
Understand the threats before writing the handler
Treat every inbound request as untrusted until verification succeeds. An attacker may forge a request, alter a signed payload, replay a captured delivery, send oversized bodies, or race duplicate events across application instances. A valid delivery can also arrive twice or arrive after a later event.
| Failure or threat | Required control | What remains to solve |
|---|---|---|
| Forged or altered request | Verify authenticated bytes | Protect the secret and endpoint configuration |
| Replayed signed delivery | Signed timestamp freshness | Deduplicate events within the accepted window |
| Concurrent duplicate requests | Database uniqueness | Make external side effects idempotent |
| Worker crash after an action | Recovery state and operation key | Reconcile uncertain downstream results |
| Older event arriving late | Resource/version-aware handling | Define permitted business transitions |
| Resource exhaustion | Body limits, timeouts and rate controls | Size controls for real provider traffic |
Signature verification establishes that a request matches a configured key and signed message. It does not establish that the event is appropriate for every tenant, that the business action has not already happened, or that the payload should be trusted as executable instructions.
Define exactly what is signed
For this example, the sender and receiver agree on the following protocol:
X-Webhook-Timestampcontains an integer Unix timestamp in seconds.X-Webhook-Signaturecontains exactly 64 hexadecimal characters: the HMAC-SHA256 digest.- The authenticated message is the timestamp string, a literal period, then the raw HTTP body bytes.
- The secret is a UTF-8 string with at least 32 bytes generated from strong random material.
- Requests use JSON without content compression.
The timestamp is part of the authenticated message. An attacker cannot replace an old timestamp with a recent one without invalidating the signature. A separate unsigned timestamp would not provide that property.
For third-party webhooks, headers, encoding, algorithms, multiple signatures, and secret representations may differ. Stripe, for example, documents a dedicated verification flow; follow that flow or its SDK instead of sending Stripe deliveries through this custom parser. Stripe webhook documentation
Preserve the original bytes
Do not compute the signature from JSON.stringify(req.body). Parsing and serializing JSON can change whitespace, escape sequences, or key order. A semantically identical object can therefore produce a different digest. Bind raw-body middleware to the webhook route before general JSON middleware. Express documents express.raw() as producing a Buffer body and exposes size and inflation controls. Express API reference
Also inspect gateways, serverless adapters, and observability middleware. A component that normalizes or decompresses the body can change the representation your verifier sees. Determine whether the provider signs compressed wire bytes or an uncompressed representation before enabling transformations.
Implement a strict HMAC verifier
Save the following as verify.mjs:
import { createHmac, timingSafeEqual } from 'node:crypto';
export function verifyWebhook({
body, timestamp, signature, secrets,
nowSeconds = Math.floor(Date.now() / 1000),
toleranceSeconds = 300
}) {
if (!Array.isArray(secrets) || secrets.length < 1 || secrets.length > 2 ||
secrets.some(s => typeof s !== 'string' || Buffer.byteLength(s) < 32)) {
throw new Error('Webhook keys are not configured correctly');
}
if (!Number.isSafeInteger(nowSeconds) ||
!Number.isSafeInteger(toleranceSeconds) || toleranceSeconds < 1) {
throw new Error('Invalid verification clock configuration');
}
if (!Buffer.isBuffer(body) || body.length > 1024 * 1024) return false;
if (typeof timestamp !== 'string' || !/^[0-9]{1,12}$/.test(timestamp)) return false;
if (typeof signature !== 'string' || !/^[a-fA-F0-9]{64}$/.test(signature)) return false;
const sentAt = Number(timestamp);
if (!Number.isSafeInteger(sentAt) ||
Math.abs(nowSeconds - sentAt) > toleranceSeconds) return false;
const supplied = Buffer.from(signature, 'hex');
let matched = false;
for (const secret of secrets) {
const expected = createHmac('sha256', secret)
.update(timestamp + '.', 'utf8')
.update(body)
.digest();
const equal = timingSafeEqual(expected, supplied);
matched = equal || matched;
}
return matched;
}
Both digests have the same length because the parser validates the incoming hex string before decoding it. This avoids the unequal-length exception in timingSafeEqual. Comparing digest bytes through that API is preferable to comparing hexadecimal strings with ordinary equality. It does not make the whole HTTP handler constant-time; surrounding parsing, error handling, and application logic still matter. Node.js crypto documentation
The example accepts a five-minute window in both directions. That is a design choice for this protocol, not a universal provider default. Synchronize application clocks and choose a tolerance consistent with your sender’s delivery behavior. If a provider signs each retry with a new delivery timestamp, validate that timestamp rather than rejecting an old business event merely because its event-creation time is old.
Connect verification to an Express route
The following wiring shows the boundary between verification and storage. inbox.accept is an application-owned adapter described in the next section; it must not be replaced by an in-memory Set in production.
import express from 'express';
import { verifyWebhook } from './verify.mjs';
export function buildApp({ inbox, secrets }) {
const app = express();
app.post('/webhooks/example',
express.raw({ type: 'application/json', limit: '1mb', inflate: false }),
async (req, res, next) => {
try {
const valid = verifyWebhook({
body: req.body,
timestamp: req.headers['x-webhook-timestamp'],
signature: req.headers['x-webhook-signature'],
secrets
});
if (!valid) return res.status(400).json({ error: 'Invalid delivery' });
let event;
try { event = JSON.parse(req.body.toString('utf8')); }
catch { return res.status(400).json({ error: 'Invalid payload' }); }
if (!event || Array.isArray(event) ||
typeof event.id !== 'string' || !/^[A-Za-z0-9_-]{1,128}$/.test(event.id) ||
typeof event.type !== 'string' || event.type.length > 128 ||
!['invoice.settled', 'subscription.changed'].includes(event.type)) {
return res.status(400).json({ error: 'Unsupported event envelope' });
}
// This endpoint configuration determines source/account identity.
// Validate event.data against the schema for its type inside the adapter.
await inbox.accept({
source: 'example-provider', account: 'configured-account',
event, rawBody: req.body
});
return res.status(200).json({ received: true });
} catch (error) { next(error); }
});
app.use(express.json());
app.use((error, req, res, next) => {
if (res.headersSent) return next(error);
const status = error.status === 413 ? 413 : error.status === 415 ? 415 : 503;
res.status(status).json({ error: 'Delivery could not be accepted' });
});
return app;
}
Startup must fail if secrets are missing or incorrectly configured; validate them before the application begins accepting traffic. The verifier also checks configuration defensively. Add a request timeout and resource controls at the server or gateway. Return a generic error to callers, while recording a sanitized internal reason for operators.
This route deliberately supports only two event types. Decide whether other authentic event types should be acknowledged and ignored or rejected, based on the sender’s retry semantics. Restrict subscriptions at the provider where possible; avoid retry storms caused by rejecting a harmless but newly introduced event indefinitely.
Persist an inbox before acknowledging delivery
A durable inbox is a database record of an authenticated delivery awaiting work. The unique key should combine a configured source, a configured account or tenant, and the provider event ID. An attacker-controlled tenant field must not choose which secret or customer account authorizes an action.
A reference PostgreSQL schema is:
CREATE TABLE webhook_inbox (
source text NOT NULL,
account_id text NOT NULL,
event_id text NOT NULL,
event_type text NOT NULL,
payload jsonb NOT NULL,
payload_sha256 char(64) NOT NULL,
received_at timestamptz NOT NULL DEFAULT now(),
status text NOT NULL DEFAULT 'pending'
CHECK (status IN ('pending', 'processing', 'retry', 'processed', 'dead')),
attempts integer NOT NULL DEFAULT 0 CHECK (attempts >= 0),
available_at timestamptz NOT NULL DEFAULT now(),
lease_until timestamptz,
last_error_code text,
PRIMARY KEY (source, account_id, event_id)
);
Implement inbox.accept as a parameterized transaction. Validate the complete event-type schema, then insert with ON CONFLICT DO NOTHING. If insertion succeeds, commit before responding with 200. If the unique record already exists, load the stored record and classify the delivery as a duplicate. Database unavailability must produce a retryable failure, not a successful acknowledgement.
A payload digest provides a diagnostic signal if an event ID is reused with changed bytes. Treat that as a protocol-specific discrepancy. Whitespace changes may alter a raw digest even when the event is semantically identical; do not blindly overwrite the accepted payload or automatically execute it again. Investigate whether your provider guarantees immutable event representations.
The worker can poll this table directly, avoiding a database-plus-queue dual write. If you use a broker, create an outbox record in the same transaction as the inbox insert. A separate dispatcher publishes pending outbox records, tolerating repeated dispatch. Writing to a database and then to a broker without that coordination creates a failure window between the two systems.
Make worker recovery and business actions idempotent
A worker should claim a bounded batch atomically, establish a lease, and release the database transaction before slow external calls. Another worker may reclaim an expired lease after a crash. Use database time for lease calculations, record attempt counts, and make lease duration consistent with task timeouts. Renew leases for genuinely long tasks and prevent stale workers from overwriting newer completion state with a fencing token or claim identifier.
Inbox uniqueness prevents simultaneous insertion of the same delivery. It does not make business actions exactly once. Consider a worker that sends a remote request successfully and crashes before marking the inbox row processed. The next worker cannot infer from the local status alone whether the remote action occurred.
Use a stable business operation key for each action. For a payment update, that might include account, invoice, and transition rather than only the webhook ID. Supply an idempotency key to downstream APIs when supported, enforce uniqueness in your own operation table, and reconcile uncertain outcomes before repeating irreversible actions. Distinct provider events can describe the same underlying business operation.
Do not assume delivery order. A subscription cancellation might arrive before an earlier activation. Apply a documented state-transition policy, check resource versions where available, or retrieve current authoritative state from the provider. Fetch that state using your authenticated server integration; do not follow arbitrary URLs supplied by event payloads.
Plan retries, dead letters and retention
Retry transient database, network, and downstream availability failures with bounded exponential backoff and jitter. Give permanent schema or business-rule errors a distinct terminal state. After a configured attempt budget, move work to a dead-letter state with enough sanitized context for investigation.
Do not acknowledge an event merely because it was placed in process memory. Likewise, do not retry every error immediately: a worker that repeatedly encounters an unavailable API can amplify an outage. Limit concurrency and expose backlog age so operators can distinguish normal load from stalled processing.
Retain deduplication records for at least the supported provider retry and operator replay window, plus an operational margin. The signature freshness window and the event-retention window solve different problems. A fresh signed retry can refer to an event accepted days earlier. Document payload retention separately because payloads may contain personal or commercially sensitive information.
Rotate keys and observe the pipeline
Use separate keys for development and production and, where the provider supports it, for each endpoint/account. During a planned rotation, accept the active and retiring key for a bounded overlap. Remove the old key after the sender has switched and the overlap has ended. The sample computes comparisons for both keys rather than exiting after the first match.
A suspected key compromise needs incident handling: rotate promptly, review accepted events, reconcile affected business operations, and inspect how the secret was exposed. A valid HMAC generated with a stolen key cannot establish legitimate sender intent.
Monitor verification failures by reason, acceptance latency, duplicate counts, worker backlog age, retries, dead letters, and business-operation conflicts. Log event IDs and configured source/account identifiers according to your logging policy. Avoid logging secrets, full signature headers, raw bodies, payment data, or unrestricted exception payloads. Alert on sudden verification failures and growing oldest-pending age; thresholds should reflect your own traffic baseline.
Test the actual failure boundaries
Before deployment, test the verifier and the complete route with exact bytes:
- Accept a valid signature and reject one changed body byte.
- Reject missing, malformed, truncated, and unsupported signature formats.
- Reject stale and excessively future timestamps, including exact boundary tests.
- Verify Unicode, escaped characters, whitespace, empty bodies, and unusual valid JSON.
- Confirm earlier middleware does not parse the webhook body.
- Reject oversized or compressed requests according to the documented protocol.
- Send simultaneous duplicates to multiple application instances and confirm one inbox insertion.
- Stop the database and confirm that the endpoint does not return success.
- Crash a worker immediately before and after a downstream action; verify safe recovery.
- Replay an old event with a fresh authenticated delivery timestamp and confirm deduplication.
- Deliver events in reverse order and confirm permitted state transitions.
- Exercise key rotation and removal, backlog draining, and dead-letter recovery.
Run cryptographic unit tests locally with fixed secrets and timestamps. Use the provider’s official testing mechanism for end-to-end protocol validation. Never use production secrets in fixtures or commit them to your repository.
Run the verifier tests
Save the companion test as verify.test.mjs and run node --test verify.test.mjs. Fixed timestamps make these tests deterministic. The included tests cover tampering, stale and future deliveries, malformed headers, UTF-8 bodies, key rotation, the tolerance boundary, and missing configuration. These tests validate the custom verifier; they do not validate your provider integration or database adapter.
import {test} from 'node:test';
import assert from 'node:assert/strict';
import {createHmac} from 'node:crypto';
import {verifyWebhook} from './verify.mjs';
const key='x'.repeat(32),now=1700000000,body=Buffer.from('{"text":"Café 🧪"}');
const sign=(t,b=body,k=key)=>createHmac('sha256',k).update(t+'.').update(b).digest('hex');
const args=(t=String(now))=>({body,timestamp:t,signature:sign(t),secrets:[key],nowSeconds:now});
test('valid raw UTF-8 bytes',()=>assert.equal(verifyWebhook(args()),true));
test('tampered bytes',()=>assert.equal(verifyWebhook({...args(),body:Buffer.from('{}')}),false));
test('stale and future deliveries',()=>{for(const offset of [-301,301])assert.equal(verifyWebhook(args(String(now+offset))),false)});
test('inclusive tolerance boundary',()=>assert.equal(verifyWebhook(args(String(now-300))),true));
test('malformed signatures and timestamps',()=>{for(const signature of ['', 'zz'.repeat(32),'00',undefined])assert.equal(verifyWebhook({...args(),signature}),false);assert.equal(verifyWebhook({...args(),timestamp:'1e9'}),false)});
test('key rotation',()=>assert.equal(verifyWebhook({...args(),secrets:['y'.repeat(32),key]}),true));
test('missing key is configuration failure',()=>assert.throws(()=>verifyWebhook({...args(),secrets:[]})));
test('wrong body type and oversized payload rejected',()=>{assert.equal(verifyWebhook({...args(),body:'{}'}),false);assert.equal(verifyWebhook({...args(),body:Buffer.alloc(1024*1024+1)}),false)});
Related engineering guides
For the surrounding delivery contract, read the API integration release checklist. To connect ingestion to a larger architecture, see Product Scaling: Enterprise Architecture Strategies. The webhook-specific signature and inbox design here complements those broader guides.
Frequently asked questions
Is HTTPS enough to secure a webhook?
HTTPS protects transport and authenticates the server to its client, but an accessible endpoint can still receive requests from an unintended sender. Signature verification adds application-level message authentication. Keep HTTPS, verify the provider’s protocol, and enforce payload and resource limits together. Optional IP restrictions can be additional controls when the provider publishes maintained ranges, but they should not substitute for signature verification.
Why does a correct signature fail after JSON parsing?
The digest authenticates bytes, while parsing produces an object. Re-serializing that object can create different bytes even when values appear unchanged. Capture the body before JSON middleware, then parse it only after verification. Check the endpoint secret, exact signing format, gateway transformations, and timestamp configuration as separate troubleshooting steps.
Does a timestamp prevent every replay?
No. A captured request can remain valid during the permitted window. The signed timestamp limits the lifetime of that captured delivery, while durable event deduplication prevents repeated handling within and beyond that window. If the provider does not authenticate a timestamp, adding an arbitrary timestamp check on an unsigned header does not create trustworthy freshness protection.
Can I use Redis or an in-memory Set for deduplication?
An in-memory Set cannot coordinate independent servers and is lost on restart. Redis can support coordination, but you must design persistence, expiry, failover, and the relationship between a reservation and completed processing. For this architecture, a database inbox makes acceptance and durable pending work one transaction. Any alternative must preserve the same crash-recovery guarantees.
When should the endpoint return a successful response?
After verification, validation, and the durable acceptance transaction commit. A known duplicate can also receive success when the previously accepted record remains durable. You do not need to wait for slow background business processing. If acceptance fails before commit, return a failure appropriate to the provider’s retry behavior. Confirm its documented timeout and status-code semantics rather than assuming them.
Does a unique event ID guarantee exactly-once processing?
It guarantees that your unique database key admits one inbox record. It does not eliminate worker redelivery, uncertain external outcomes, or distinct events that trigger the same operation. Combine event deduplication with operation-level idempotency, explicit worker recovery, and reconciliation. Describe those guarantees precisely instead of advertising exactly-once behavior across unrelated systems.
How should I select the timestamp tolerance?
Base it on the signed delivery timestamp, expected transport delay, and achievable clock accuracy for your protocol. The five-minute value in the sample is an explicit example. Test both old and future boundaries and monitor rejection reasons. Increasing tolerance can mask clock failures and widens the captured-request acceptance period, so investigate persistent clock skew before changing the policy.
How do I safely support multiple tenants?
Bind secrets to a configured endpoint/account identity or a documented authenticated routing scheme. An unsigned tenant name supplied in a request must not authorize cross-account access. After verification, validate that resource IDs belong to the account whose key authenticated the event. Deduplication and business-operation keys should include that account identity so unrelated tenants cannot collide.
What should happen to authenticated but unsupported event types?
Define an explicit policy. If the event is irrelevant and safe to ignore, acknowledge it and record a bounded diagnostic signal. If it violates a required integration contract, quarantine or reject it according to the provider’s retry rules. Avoid endlessly rejecting new benign event types while subscribing to every possible event. Manage provider subscriptions and versioned schemas together.
How do I replay failed events without weakening verification?
Keep public ingress verification intact. Replay previously verified, durably stored events through an authenticated internal recovery tool with authorization, audit logs, and the same idempotent worker path. A provider-driven resend should go through normal signature verification. Do not add a public “skip verification” flag for operator convenience, and do not delete deduplication history simply to force a repeated action.
Definitions and deployment checklist
HMAC: A keyed message authentication code. In this example it authenticates a timestamp and payload using SHA-256.
Freshness window: The permitted difference between a signed delivery timestamp and the receiver’s clock.
Durable inbox: A transactional record proving that an event was accepted and remains available for processing.
Idempotency: A repeated operation has the same intended business effect as one operation, enforced through application and downstream mechanisms.
Outbox: A database record committed alongside application state and later dispatched to another system.
Lease: A time-limited worker claim that permits recovery after a worker becomes unavailable.
Ship only after verifying raw-body preservation, supported signature formats, clock behavior, secret rotation, database uniqueness, safe retry policies, operation-level idempotency, and observability. Document which parts of the implementation are guaranteed by your database and which depend on the provider or downstream API.
For help designing the surrounding integration, contact Acadify Solution. For reliability testing and evaluation work, explore Acadify AI.
Primary references
- Node.js crypto: HMAC and timing-safe comparison
- Express API: raw-body middleware
- Stripe webhook integration documentation
The inbox, outbox, leasing, and business-operation design in this guide is a proposed application architecture. Adapt and test it against your own database isolation, provider contract, and downstream guarantees.
Glossary & Key Architecture Definitions
- • HMAC: keyed message authentication code. Freshness window: accepted signed timestamp range. Durable inbox: transactional event acceptance record. Idempotency: repeated operations preserve intended effect. Outbox: transactional dispatch record. Lease: time-limited worker claim.
Engineering Research & Citations
- [1] Node.js crypto: https://nodejs.org/api/crypto.html
- [2] Express API: https://expressjs.com/en/api/
- [3] Stripe webhooks: https://docs.stripe.com/webhooks
No perspectives submitted yet. Be the first to start the discussion.