Harness Worker

Cloudflare Worker composition for Ze Harness. Each server Durable Object (HarnessServerObject, bound via HARNESS_SERVERS) hosts any number of Sessions sharing one SQLite database, one SQLite-backed Computer Workspace, one attached Container, and the alarm that coordinates Turn recovery with post-command filesystem synchronization.

Servers and Sessions

A Session is a context boundary: its turns, history, permission grants, and overlay state. A server is the files-and-state home: the SQLite database, the workspace files, the container, the host MCP definitions, and the shared host OAuth credentials. One named server holds N sessions; deleting a Session never disturbs its siblings.

Every session route carries both identities: /v1/servers/:server/sessions/:sessionId/.... Server-level routes carry only the server: GET /v1/servers/:server/sessions lists the server's Sessions oldest-first, GET /v1/servers/:server/events streams the server's activity (below), and DELETE /v1/servers/:server deletes the server (below). The gateway forwards by server name and sets x-zephyr-server (plus x-zephyr-session-id on Session routes); the object rejects mismatched identities with 404.

Server activity stream

GET /v1/servers/:server/events?afterServerCursor=<n> is the SSE feed the client's watchServer consumes: every Session creation, title change, and deletion plus every Turn change on any Session of the server, each stamped with the server-wide cursor, replayed from afterServerCursor and then live. The gateway streams the body through unchanged. The route is observational: it admits no work and arms no recovery, but the open response holds the Durable Object awake for as long as the client keeps it, with a heartbeat (server.cursor) every 15 seconds while nothing happens. A client that wants an idle server to hibernate closes the stream and resumes later from the last cursor it saw.

Servers are created lazily: the first request addressed to a name boots that name's object, and creating the first Session there succeeds. There is no create-server call.

Workspace instructions

Before admitting a prompt, Harness loads the non-empty AGENTS.md in the Session workdir. Ancestors above the workdir are not loaded eagerly. The file is applied as a developer instruction and re-read for every new prompt, so an edit or removal applies to the next admitted Turn. The admitted content is retained with that Turn for recovery and context compaction. Each file must fit the existing 50 KiB workspace text-read limit; unreadable, invalid, or oversized files fail prompt admission instead of being silently dropped.

Successful standard file reads and directory listings below the Session workdir also discover any nested AGENTS.md files between the target and that workdir, nearest-first. Newly discovered files are attached to the read result. For direct reads, they are not attached again while that result remains in model-visible history. A later read can load them again after compaction or revert removes the result. Nested edits therefore take effect after the prior result leaves context, or in a new Session.

Instructions discovered by script child reads are also retained in Session chronology, independently of the script's returned value. They appear as developer messages after the tool round. Known script-discovered files are re-read at new prompt and explicit compaction admission; edits and removals apply there while the currently admitted Turn retains its captured content.

Server lifecycle

DELETE /v1/servers/:server cancels and deletes every Session through the ordinary Session deletion authority (running work is cancelled and settled, overlays are removed, Session credentials are forgotten with revocation), purges the retired Session identities so the emptied server starts with a fresh namespace, forgets the shared host credentials, and drops the adopted catalog pins. The receipt lists the deleted Session identities; repeating the delete is idempotent and reports an empty list.

Kept deliberately: workspace files and provider configuration stay behind on every host. Server delete never removes user files (on Node they live on the caller's disk) and never touches provider connections (explicitly managed through the provider API). Full file teardown is a follow-up requiring explicit product sign-off, not part of this delete.

Idle objects hibernate under Cloudflare's standard Durable Object lifecycle; the alarm self-disarms when nothing is due, so an emptied server costs nothing until its next request. SQLite pages freed by deletion are reused in place; there is no automatic vacuum or pruning in v1, so long-lived servers grow with retained history, credentials, and workspace files.

Concurrency

One Durable Object serves one request at a time through its input gate, but Turns are admitted concurrently across and within Sessions: two Turns on one server may execute model and tool work at the same time. Container and workspace writes are uncoordinated — Harness offers no file locking — so engines that run concurrent Turns must coordinate shared paths themselves. Sequenced multi-step work is fully supported; concurrent writes to one file are last-writer-wins at the container. Session admits, credential mutations, and overlay leases stay atomic through the per-Session mutation gate and the Durable Object's serialized execution.

Hybrid CodeMode

Agents can combine tool calls using CodeMode. It is on by default; set HARNESS_CODEMODE_ENABLED to false to turn it off. Existing tool permissions still apply. Interrupted scripts are not automatically replayed after a restart.

Permission configuration

The Worker applies these variables as the Server permission scope:

Worker variable Value
HARNESS_PERMISSIONS JSON array of ordered { action, resource, effect } rules
HARNESS_PERMISSION_CASE_SENSITIVE true or false; defaults to true
HARNESS_APPROVAL_TIMEOUT_MS Positive milliseconds or unlimited; omitted means unlimited

The last matching rule wins, and no match asks. The effective order is Ze defaults (allow inside the session workdir, ask for external_directory paths and .env reads), then these Worker rules, then Session and Agent rules. Durable Allow always grants belong only to the owning Session and cannot override a configured deny. Operator rules for scoped Sessions match workdir-relative file resources. A relative --workdir given at Session creation resolves against the server's workspace root (/workspace), which remote clients cannot see; an absolute one is used as-is and must name a directory inside that root. Remote Sessions default to the whole workspace.

Singleton provider authority Durable Object

The deployment binds a singleton Durable Object, HarnessProviderAuthorityObject, via HARNESS_PROVIDER_AUTHORITY:

  • Statically bundles the supported models.dev catalog and Ze's native chat, responses, and messages wire adapters. OpenAI, OpenCode Go, and OpenRouter are enabled when HARNESS_PROVIDERS is absent.
  • Stores provider connections in Durable Object SQLite storage: a credential change persists the connection, then publishes a new catalog revision; concurrent connection writes are serialized. Label-only updates persist without publishing a new revision.
  • Backs the gateway's authenticated runtime model and provider management endpoints (/v1/runtime/models, /v1/runtime/providers) through typed Durable Object RPC methods; the gateway decodes each route once and maps the authority's typed failures to protocol errors.
  • Serves stored credentials to server Durable Objects through its providerCredentials() RPC. Each server runtime resolves its inference keys once at initialization — a stored key wins over the deployment environment key, and a stored-but-unreadable entry blocks the environment fallback so inference matches the catalog snapshot — so connection changes apply to servers initialized after the change, exactly like other deployment configuration.

Per-server provider configuration (stored on the server object instead of the singleton) is planned and tracked separately; this deployment keeps the singleton authority above.

Provider transport configuration

HARNESS_PROVIDERS is an optional JSON object with the same provider map documented by @zephyrcloudio/harness-server-node. For example, this Worker variable enables a known catalog provider:

{
  "HARNESS_PROVIDERS": "{\"deepseek\":{}}"
}

The map is an allowlist. "{}" is invalid because it enables no provider. Unknown providers without a models map discover their inventory from the authenticated GET <baseURL>/models endpoint at startup and runtime reload (see @zephyrcloudio/harness-server-node). A models map may still be declared explicitly, with discovery: true combining discovery and explicit overrides. Keep API keys in stored provider connections or Worker secrets named by definition.credentials.environment; do not put secrets in the plain HARNESS_PROVIDERS variable.

Definitions can select chat, responses, or messages, routes, authentication, headers, Session-derived headers, and per-model endpoints and body defaults. options.requestTimeoutMs is a positive integer in milliseconds or null for unlimited. The deadline covers the full HTTP request, not idle time between chunks, and expiration is a typed attempt-limit failure. Shared wire-parser bounds are documented in the LLM package README.

MCP host boundary

The server Durable Object connects remote MCP servers over Streamable HTTP. Local stdio is unavailable because Workers cannot spawn local processes; the Worker does not emulate stdio or silently fall back to HTTP.

Cloudflare applies these platform limits:

  • Each isolate has 128 MB of memory, shared by its concurrent requests.
  • Workers Free allows 50 external subrequests and 1,000 subrequests to Cloudflare services per invocation. Workers Paid defaults to 10,000 subrequests per invocation; limits.subrequests can raise that configured limit up to 10 million.
  • Each request can have six outgoing connections waiting for response headers at once. An MCP SSE response stops using one of those six slots after its response headers arrive.
  • DurableObjectState.waitUntil() does not extend a Durable Object's lifetime. The Durable Object transport passes escaped cleanup to the actual state so it is not a floating promise while the current request, response stream, or runtime-recognized pending I/O keeps the object active. Plain fetch subrequests do not prevent eventual eviction, and Durable Objects provide no shutdown hook.
  • Scope and transport finalizers are best effort if Cloudflare abruptly terminates an isolate.

These are platform ceilings, not hidden Harness defaults. Transport limits remain explicit and configurable, and reaching one is a typed terminal outcome.

MCP OAuth credentials in the Worker

Each server Durable Object composes one server-wide MCP OAuth credential lifecycle on its own SQLite database: one credential store, one lifecycle, startup recovery while the object blocks concurrency during initialization, and the composed oauthTokenResolver on the remote Streamable HTTP transport for first authorization, 401 challenge refresh, and deduplicated concurrent refresh. Host-configured servers share one host credential namespace across every Session on the server; overlay-declared servers keep per-Session credentials that leave with their Session. The Worker's HARNESS_MCP_CALLBACK_BASE_URL HTTPS origin is the public callback address; the gateway route GET /oauth/callback/:server forwards the external browser redirect to the owning server object, the pending OAuth state selects the completing Session, deployment host and origin policy still apply, and callback query values are never logged. An OAuth credential reference without a composed runtime is a typed authorization_required transport failure, never configured functionality.

Rollout runbook

The HarnessServerObject class on the HARNESS_SERVERS binding carries the v3 SQLite migration tag, and v4 formally retires the old HarnessSessionObject class (its orphaned objects are deleted on deploy); the v1/v2 session-object history is inert. Because the pre-release data model changed incompatibly (session-owned to server-owned storage, fresh v13 session schema), deploy into staging expecting wiped Durable Object state:

  1. Deploy the Worker; the v4 migration deletes the old session objects.
  2. Verify addressing: PUT /v1/servers/<name>/sessions/<id> lazily creates the server, GET /v1/servers/<name>/sessions lists it, and a second Session shares the object.
  3. Verify lifecycle: DELETE /v1/servers/<name> empties the server and a re-created Session reuses identities cleanly.
  4. Verify OAuth: one browser flow authorizes a host server for every Session on it.
  5. Promote the same build to production; no per-server migration step exists by design.

Each server object migrates its own SQLite database the first time it wakes on new code (see the harness-storage-sqlite README), and idle objects catch up when next used. The release that introduced storage migrations requires wiping pre-migration Durable Object state once: an object created before it fails with StorageResetRequired. Provider authority objects adopt their existing provider_connections table in place. Before a wrangler rollback, check whether the newer build shipped a breaking migration; objects it migrated refuse to open under the older build with StorageNewerThanBinary.

Embedding in a host Durable Object

@zephyrcloudio/harness-worker-host exports the embeddable WorkerHarnessHost. One DO instance shares one host-provided Workspace (file home plus Computer backend) across N HarnessSessions on one SQLite database, with injected providerDefinitions (embedder-implemented providers, selected per turn), contentResolver (attachment references, resolved only after visibility filtering), and toolDispatch (host custody after Harness permission resolution). Embedding hosts pass credentialMode: 'injected-only' so credentials arrive only through injected ports; ambient discovery stays disabled. The host has no bearer check (in-DO calls are trusted) and no deployment-wide provider singleton. One multiplexed DO alarm serves Harness recovery, Computer sync, and host-owned participants (registerAlarmParticipant + setParticipantDue); direct SessionServices admissions call notifyAdmission() so the drain runs via waitUntil. The host's model (the standalone Worker's HARNESS_MODEL) is optional: without it, prompts that name no model and title generation use the catalog's first available model. The standalone HarnessServerObject is itself composed on this host with env-sourced config, proving the same path embedding hosts take. Pin the exact published version; all workspace packages share one version and pack:smoke enforces it.

host.watchTurn(sessionId, turnId, { afterRevision }) returns the same event stream the /turns/:turnId/events SSE route sends, as an Effect Stream with no HTTP or SSE parsing: durable rows after the cursor, synthesized deltas, heartbeats, and one terminal event after which the stream ends. Omit afterRevision to receive only events after the current revision; pass the last revision you saw to resume without a gap.

import { Effect, Stream } from 'effect';

const events = await host.watchTurn(sessionId, turnId, { afterRevision });
await Effect.runPromise(Stream.runForEach(events, (event) => Effect.sync(() => render(event))));

host.watchSession(sessionId, { afterActivityCursor }) and host.watchServer({ afterServerCursor }) are the in-process counterparts of the /sessions/:sessionId/events and /events SSE routes: the Session's or the whole server's activity after the cursor, with heartbeats, as a Stream that never ends on its own. Omitting the Session cursor replays that Session's stamped history; omitting the server cursor starts live, with a first server.cursor naming the current cursor to bootstrap from. Interrupt the stream to stop watching. Both are observational and arm no recovery. host.fetch also serves the server-level routes (the Session list, the activity stream, server deletion) for an embedder's in-process client.

The host's providers map is an allowlist compiled against the host's catalog (the bundled snapshot, or a newer modelCatalog). By default (unknownProviders: 'fail') an id with no catalog entry and no complete definition fails host construction for every session. Embedders that build the map from external input can pass unknownProviders: 'report': the unknown ids are left out of the plan and the host constructs with the rest. The drop is never silent; (await host.resources()).rejectedProviders lists each { providerId, message }, and providerIds lists what compiled. The diagnostic names only the id, never a credential.

Pass modelCatalog: { document, publishedAt } to compile from a published models.dev-format catalog the embedder fetched instead of the bundled snapshot. It is used only when valid and published (Last-Modified) at or after the bundled snapshot's build time; otherwise the host uses the bundled snapshot and reports why on (await host.resources()).modelCatalog. See the Worker host README.

Runtime plugins

With a worker_loaders binding named LOADER, a server object loads Session plugins at runtime from prebuilt bundles with no redeploy. A Session adds one with a Bundle source, the credential-free https:// URL of a self-contained ES module (see the plugin README):

ze-harness --remote URL --server NAME plugins add --session SESSION_ID \
  --bundle https://plugins.example.com/notes.js --plugin-id example.notes --options '{"folder":"notes"}'
  • Resolution fetches the bundle and pins the SHA-256 of its bytes as the revision's integrity (sha256:<hex>) and, unless the client declared a version, its version (bundle-sha256:<hex>). A declared integrity that does not match the fetched bytes is a typed declared_mismatch; a changed file at the same URL never replaces an activated plugin until plugins refresh.
  • The bundle is cached in the server object's own SQLite by digest, so recovery after eviction reloads the exact activated bytes without the URL. A lost cache refetches and verifies the digest.
  • Each bundle runs in its own Dynamic Worker isolate, shared by every Session in the server object that activates the same bytes, and inherits the Worker's network access. Plugins are trusted code, as on Node. The isolate is stateless across calls; see the plugin README for what that means for setup.
  • Package, File, and Git sources are a typed unsupported capability on the Worker; Bundle sources are a typed unsupported capability on the Node host.
  • Without the binding the Worker stays static-only and Bundle sources fail as a typed unsupported capability. The test environment declares the binding, so test/bundle-plugins.test.ts runs a fixture bundle through miniflare's Worker Loader.

Embedding hosts pass the binding as dynamicPlugins: { loader: env.LOADER } on WorkerHarnessHostOptions.

Embedding hosts can also link Session plugins into their Worker at build time with staticSessionPlugins on WorkerHarnessHostOptions. Each entry names a sourceId, a pluginId, a version, and the plugin definition; a Session adds it with the { _tag: 'Static', id: sourceId } source. Nothing is fetched or loaded at runtime, and static plugins are trusted host code. The deployed Worker links none; its tests link a Session overlay fixture through test/support/worker.ts.

Server plugins (embedding hosts)

An embedding host can also give its Durable Object a list of server plugins: bundles loaded into every Session of the object through the host generation, the embedder's counterpart of the Node config file's plugins. The embedder owns and persists the list; the harness resolves, loads, and reports it.

const host = new WorkerHarnessHost({
  ...options,
  dynamicPlugins: { loader: env.LOADER },
  serverPlugins: [{ url: 'https://plugins.example.com/notes.js', pluginId: 'example.notes' }],
});
await host.setServerPlugins([...entries]); // replaces the list and reloads the host generation
await host.serverPlugins(); // each entry's url, pluginId, pinned integrity, and active/failed state
  • Each entry names a bundle URL and the plugin id it must declare, with an optional integrity: 'sha256:<hex>' pin and options. An entry that cannot load is reported failed and left out of the generation; it never blocks the server.
  • A changed list is a distinct host generation (the revision id carries a digest of the loaded list). New Sessions get it immediately and existing Sessions on their next Turn; Session plugins follow the new host baseline.
  • setServerPlugins is refused with WorkerServerPluginConflict while any Session runs one of the plugin ids as its own Session plugin, so a room plugin never shadows a thread's copy. A new generation that does not activate is WorkerServerPluginRejected; the previous list stays active either way.
  • There are no HTTP routes or CLI commands for server plugins; the standalone Worker has none.

Platform limits, all surfaced as explicit plugin failures rather than hangs:

  • Dynamic Workers require the Workers Paid plan and are billed per unique Dynamic Worker per day (each distinct bundle is one).
  • A Durable Object can have at most 10 distinct Dynamic Workers with requests in flight, across all of its Sessions.
  • Workers RPC messages are limited to 32 MiB, which bounds tool inputs, results, and hook payloads.

Recovery backoff and quarantine

A failing recovery drain records its outcome before surfacing it. The first failure re-arms the pending revision after 60s, then 2m, 4m, and so on up to a 30-minute cap. After 10 consecutive failures the revision moves to the recovery_quarantine diagnostic row (revision, failures, error, quarantined_at) and stops re-arming, so a deterministically failing drain — for example poisoned rows that fail validation on every read — can no longer burn alarm wakes at full rate. The quarantine error and the alarm log carry the error name only, never the message, so a validator that echoes a bad value cannot land in storage or logs.

Both alarm paths (WorkerHarnessHost.alarm() and the standalone server object) run recovery only when that revision is due. An earlier Computer sync or participant wake does not run a backed-off drain. They catch the recorded failure, still run that other work, then succeed: the retry is already scheduled durably through backoff, so the platform never fast-retries a recorded recovery failure. Unexpected failures still propagate and fail the alarm.

Recovery still runs first on every admission via waitUntil; the alarm is the backstop. A new admission after quarantine re-arms recovery with a fresh failure budget, and the quarantine row keeps the last stuck revision for operators. Read it through HostAlarmCoordinator.quarantineState() from inside the owning Durable Object. The alarm log names the error class; it does not repeat the message.

Protocol references

Public TypeScript exports

  • HarnessProviderAuthorityObject
  • HarnessServerObject
  • WorkspaceProxy