Skip to content

Drains

Export project logs, Web Analytics and Web Vitals to an HTTPS endpoint.

View as Markdown

Open your project's Settings → Drains to send new observability events to your own HTTPS endpoint. Team owners and admins can create, edit, pause, resume, test and delete drains. Each project can have five drains, including paused ones.

Choose Logs, Web Analytics, or Web Vitals for each drain. Logs include application, middleware and workflow runtime records and the saved build log after a build succeeds, fails or is stopped. Analytics includes accepted browser and server events and the existing PostHog summaries. Vitals exports accepted performance metrics.

Configure a destination

Enter a public DNS HTTPS URL, choose JSON or NDJSON, and optionally add authentication headers as a JSON object. Stored header values are encrypted and are never returned. Leaving the header field blank preserves existing headers; entering {} removes them.

Arkor platform domains, application URLs and registered custom domains cannot be destinations because delivery requests could generate logs that feed back into the same drains. Destinations are checked again before delivery; a previously external host that becomes registered on Arkor is rejected without contacting it or retrying.

The default is JSON, all environments and sources, and 100% sampling of available input. Environment filters exclude legacy server analytics events that have no environment information. Sampling does not recover events already dropped or sampled upstream.

Copy the signing secret after creating the drain. It is shown once; generating a replacement invalidates the previous secret. Send test event queues synthetic data, and its result is shown separately from normal delivery results. Multiple accepted tests remain queued, while the page displays only the latest test result. A successful test confirms the endpoint accepted the request, not that it stored the events.

Receive events

JSON deliveries contain an array of event objects. NDJSON has one object per line. Every event includes schemaVersion, id, timestamp, type, source, projectId, nullable deploymentId and environment, and a data object. Runtime records identify the application, middleware or workflow role inside their data. Build records include chunk ordering information.

Runtime invocation, console and exception records include optional data.customContext when that Worker emitted Custom Context. Keys and values are masked before export. Internal context records are hidden. data.customContextIncomplete: true indicates that context is ambiguous, truncated or could not be preserved safely.

The optional data.correlationId links middleware and application Workers serving the same HTTP request. Drains export the context of each invocation; they do not wait for another Worker's context or copy it between independently delivered invocations. Your receiver can group records by projectId, deploymentId and data.correlationId. Delivery order and complete groups are not guaranteed, and a correlation ID is not the event ID used for deduplication.

The x-arkor-drain-batch-id header identifies a delivery batch. The x-arkor-drain-signature header contains sha256=<hex>, an HMAC-SHA256 of the exact UTF-8 request body using your signing secret. Verify it before accepting events:

import { createHmac, timingSafeEqual } from 'node:crypto';

const body = await request.text();
const expected = Buffer.from(
  `sha256=${createHmac('sha256', signingSecret).update(body, 'utf8').digest('hex')}`,
);
const received = Buffer.from(request.headers.get('x-arkor-drain-signature') ?? '');
if (expected.length !== received.length || !timingSafeEqual(expected, received)) {
  return new Response(null, { status: 403 });
}
// Persist events durably before responding with a 2xx status.

Delivery and lifecycle

Respond within ten seconds with a 2xx status. Network errors, timeouts, 408, 429 and 5xx responses receive up to three retries. Redirects and other non-2xx responses are terminal failures. Retry-After can delay a retry by up to one hour. Pending messages expire after 24 hours. Delivery allows five HTTP requests globally and one per project-owning team and one across all teams owned by the same authenticated account. Waiting for an available slot does not consume HTTP retries or extend the original deadline. A slow receiver can delay other drains belonging to the same owner account; Queue order and a fixed delivery latency are not guaranteed.

Queue delivery can duplicate events and change their order. Deduplicate using each event's id; do not rely on the batch ID alone. Collection before Queue acceptance remains best effort. Existing source size bounds and truncation remain in effect. All sources across teams owned by the same account share a collection budget of 600 final Queue messages, 30,000,000 UTF-8 bytes and 60,000 events per server-clock minute. The entire split emission must fit before any message is queued. Exceeding a limit drops that emission before acceptance, even at 100% sampling; saved build logs instead remain pending within their original 24-hour deadline. Fixed minute windows can allow twice a window’s limits across a boundary.

Paused or deleted drains, changed settings and project ownership changes invalidate queued deliveries. A request already sent to the receiver cannot be recalled. Every saved settings change or resume starts a new event window. Earlier collected events and completed build logs still awaiting collection are discarded; there is no historical replay. Normal events timestamped in the same millisecond as drain creation, a settings change or resume are also discarded because their order is ambiguous. Runtime log connections are applied asynchronously to existing deployments, and the page shows connecting, connected, disconnecting, disconnected or a connection-update failure. Completed removal is shown as disconnected.

Project suspension and deletion schedule removal of runtime log connections. Resuming or restoring a project schedules reconnection when an enabled drain includes runtime logs. The periodic synchronizer applies these changes to existing Workers without a redeploy. Connection changes are asynchronous; delivery checks the project's current state before sending.

Application response time

Runtime collection uses a Cloudflare Tail Worker after application execution. External HTTP delivery runs in a separate Queue Worker. Document and static-asset serving acquire no extra database reads, destination requests, or response-body copies. Analytics and Vitals check enabled settings and enqueue their accepted events after the collector response handoff. Projects without a matching enabled drain do not use the source Queue. Actual Tail overhead should be measured on Cloudflare before enabling collection broadly.