# Harness Worker Cloudflare Worker composition for Ze Harness. Each server Durable Object (`HarnessServerObject`, bound via `HARNESS_SERVERS`) hosts any number of Sessions sharing one SQLite database, one SQLite-backed Computer Workspace, one attached Container, and the alarm that coordinates Turn recovery with post-command filesystem synchronization. ## Servers and Sessions A Session is a context boundary: its turns, history, permission grants, and overlay state. A server is the files-and-state home: the SQLite database, the workspace files, the container, the host MCP definitions, and the shared host OAuth credentials. One named server holds N sessions; deleting a Session never disturbs its siblings. Every session route carries both identities: `/v1/servers/:server/sessions/:sessionId/...`. Server-level routes carry only the server: `GET /v1/servers/:server/sessions` lists the server's Sessions oldest-first, `GET /v1/servers/:server/events` streams the server's activity (below), and `DELETE /v1/servers/:server` deletes the server (below). The gateway forwards by server name and sets `x-zephyr-server` (plus `x-zephyr-session-id` on Session routes); the object rejects mismatched identities with 404. ### Server activity stream `GET /v1/servers/:server/events?afterServerCursor=` is the SSE feed the client's `watchServer` consumes: every Session creation, title change, and deletion plus every Turn change on any Session of the server, each stamped with the server-wide cursor, replayed from `afterServerCursor` and then live. The gateway streams the body through unchanged. The route is observational: it admits no work and arms no recovery, but the open response holds the Durable Object awake for as long as the client keeps it, with a heartbeat (`server.cursor`) every 15 seconds while nothing happens. A client that wants an idle server to hibernate closes the stream and resumes later from the last cursor it saw. Servers are created lazily: the first request addressed to a name boots that name's object, and creating the first Session there succeeds. There is no create-server call. ## Workspace instructions Before admitting a prompt, Harness loads the non-empty `AGENTS.md` in the Session workdir. Ancestors above the workdir are not loaded eagerly. The file is applied as a developer instruction and re-read for every new prompt, so an edit or removal applies to the next admitted Turn. The admitted content is retained with that Turn for recovery and context compaction. Each file must fit the existing 50 KiB workspace text-read limit; unreadable, invalid, or oversized files fail prompt admission instead of being silently dropped. Successful standard file reads and directory listings below the Session workdir also discover any nested `AGENTS.md` files between the target and that workdir, nearest-first. Newly discovered files are attached to the read result. For direct reads, they are not attached again while that result remains in model-visible history. A later read can load them again after compaction or revert removes the result. Nested edits therefore take effect after the prior result leaves context, or in a new Session. Instructions discovered by script child reads are also retained in Session chronology, independently of the script's returned value. They appear as developer messages after the tool round. Known script-discovered files are re-read at new prompt and explicit compaction admission; edits and removals apply there while the currently admitted Turn retains its captured content. ## Server lifecycle `DELETE /v1/servers/:server` cancels and deletes every Session through the ordinary Session deletion authority (running work is cancelled and settled, overlays are removed, Session credentials are forgotten with revocation), purges the retired Session identities so the emptied server starts with a fresh namespace, forgets the shared host credentials, and drops the adopted catalog pins. The receipt lists the deleted Session identities; repeating the delete is idempotent and reports an empty list. Kept deliberately: workspace files and provider configuration stay behind on every host. Server delete never removes user files (on Node they live on the caller's disk) and never touches provider connections (explicitly managed through the provider API). Full file teardown is a follow-up requiring explicit product sign-off, not part of this delete. Idle objects hibernate under Cloudflare's standard Durable Object lifecycle; the alarm self-disarms when nothing is due, so an emptied server costs nothing until its next request. SQLite pages freed by deletion are reused in place; there is no automatic vacuum or pruning in v1, so long-lived servers grow with retained history, credentials, and workspace files. ## Concurrency One Durable Object serves one request at a time through its input gate, but Turns are admitted concurrently across and within Sessions: two Turns on one server may execute model and tool work at the same time. Container and workspace writes are uncoordinated — Harness offers no file locking — so engines that run concurrent Turns must coordinate shared paths themselves. Sequenced multi-step work is fully supported; concurrent writes to one file are last-writer-wins at the container. Session admits, credential mutations, and overlay leases stay atomic through the per-Session mutation gate and the Durable Object's serialized execution. ## Hybrid CodeMode Agents can combine tool calls using [CodeMode](/docs/harness.md). It is on by default; set `HARNESS_CODEMODE_ENABLED` to `false` to turn it off. Existing tool permissions still apply. Interrupted scripts are not automatically replayed after a restart. ## Permission configuration The Worker applies these variables as the Server permission scope: | Worker variable | Value | | ----------------------------------- | ------------------------------------------------------------- | | `HARNESS_PERMISSIONS` | JSON array of ordered `{ action, resource, effect }` rules | | `HARNESS_PERMISSION_CASE_SENSITIVE` | `true` or `false`; defaults to `true` | | `HARNESS_APPROVAL_TIMEOUT_MS` | Positive milliseconds or `unlimited`; omitted means unlimited | The last matching rule wins, and no match asks. The effective order is Ze defaults (allow inside the session workdir, ask for `external_directory` paths and `.env` reads), then these Worker rules, then Session and Agent rules. Durable **Allow always** grants belong only to the owning Session and cannot override a configured deny. Operator rules for scoped Sessions match workdir-relative file resources. A relative `--workdir` given at Session creation resolves against the server's workspace root (`/workspace`), which remote clients cannot see; an absolute one is used as-is and must name a directory inside that root. Remote Sessions default to the whole workspace. ## Singleton provider authority Durable Object The deployment binds a singleton Durable Object, `HarnessProviderAuthorityObject`, via `HARNESS_PROVIDER_AUTHORITY`: - Statically bundles the supported models.dev catalog and Ze's native `chat`, `responses`, and `messages` wire adapters. OpenAI, OpenCode Go, and OpenRouter are enabled when `HARNESS_PROVIDERS` is absent. - Stores provider connections in Durable Object SQLite storage: a credential change persists the connection, then publishes a new catalog revision; concurrent connection writes are serialized. Label-only updates persist without publishing a new revision. - Backs the gateway's authenticated runtime model and provider management endpoints (`/v1/runtime/models`, `/v1/runtime/providers`) through typed Durable Object RPC methods; the gateway decodes each route once and maps the authority's typed failures to protocol errors. - Serves stored credentials to server Durable Objects through its `providerCredentials()` RPC. Each server runtime resolves its inference keys once at initialization — a stored key wins over the deployment environment key, and a stored-but-unreadable entry blocks the environment fallback so inference matches the catalog snapshot — so connection changes apply to servers initialized after the change, exactly like other deployment configuration. > Per-server provider configuration (stored on the server object instead of the singleton) > is planned and tracked separately; this deployment keeps the singleton authority above. ## Provider transport configuration `HARNESS_PROVIDERS` is an optional JSON object with the same provider map documented by [`@zephyrcloudio/harness-server-node`](/docs/server-node.md). For example, this Worker variable enables a known catalog provider: ```json { "HARNESS_PROVIDERS": "{\"deepseek\":{}}" } ``` The map is an allowlist. `"{}"` is invalid because it enables no provider. Unknown providers without a `models` map discover their inventory from the authenticated `GET /models` endpoint at startup and runtime reload (see [`@zephyrcloudio/harness-server-node`](/docs/server-node.md)). A `models` map may still be declared explicitly, with `discovery: true` combining discovery and explicit overrides. Keep API keys in stored provider connections or Worker secrets named by `definition.credentials.environment`; do not put secrets in the plain `HARNESS_PROVIDERS` variable. Definitions can select `chat`, `responses`, or `messages`, routes, authentication, headers, Session-derived headers, and per-model endpoints and body defaults. `options.requestTimeoutMs` is a positive integer in milliseconds or `null` for unlimited. The deadline covers the full HTTP request, not idle time between chunks, and expiration is a typed attempt-limit failure. Shared wire-parser bounds are documented in the [LLM package README](/docs/llm.md). ## MCP host boundary The server Durable Object connects remote MCP servers over Streamable HTTP. Local stdio is unavailable because Workers cannot spawn local processes; the Worker does not emulate stdio or silently fall back to HTTP. Cloudflare applies these platform limits: - Each isolate has 128 MB of memory, shared by its concurrent requests. - Workers Free allows 50 external subrequests and 1,000 subrequests to Cloudflare services per invocation. Workers Paid defaults to 10,000 subrequests per invocation; `limits.subrequests` can raise that configured limit up to 10 million. - Each request can have six outgoing connections waiting for response headers at once. An MCP SSE response stops using one of those six slots after its response headers arrive. - `DurableObjectState.waitUntil()` does not extend a Durable Object's lifetime. The Durable Object transport passes escaped cleanup to the actual state so it is not a floating promise while the current request, response stream, or runtime-recognized pending I/O keeps the object active. Plain fetch subrequests do not prevent eventual eviction, and Durable Objects provide no shutdown hook. - Scope and transport finalizers are best effort if Cloudflare abruptly terminates an isolate. These are platform ceilings, not hidden Harness defaults. Transport limits remain explicit and configurable, and reaching one is a typed terminal outcome. ## MCP OAuth credentials in the Worker Each server Durable Object composes one server-wide MCP OAuth credential lifecycle on its own SQLite database: one credential store, one lifecycle, startup recovery while the object blocks concurrency during initialization, and the composed `oauthTokenResolver` on the remote Streamable HTTP transport for first authorization, 401 challenge refresh, and deduplicated concurrent refresh. Host-configured servers share one host credential namespace across every Session on the server; overlay-declared servers keep per-Session credentials that leave with their Session. The Worker's `HARNESS_MCP_CALLBACK_BASE_URL` HTTPS origin is the public callback address; the gateway route `GET /oauth/callback/:server` forwards the external browser redirect to the owning server object, the pending OAuth state selects the completing Session, deployment host and origin policy still apply, and callback query values are never logged. An OAuth credential reference without a composed runtime is a typed `authorization_required` transport failure, never configured functionality. ## Rollout runbook The `HarnessServerObject` class on the `HARNESS_SERVERS` binding carries the `v3` SQLite migration tag, and `v4` formally retires the old `HarnessSessionObject` class (its orphaned objects are deleted on deploy); the `v1`/`v2` session-object history is inert. Because the pre-release data model changed incompatibly (session-owned to server-owned storage, fresh `v13` session schema), deploy into staging expecting wiped Durable Object state: 1. Deploy the Worker; the `v4` migration deletes the old session objects. 2. Verify addressing: `PUT /v1/servers//sessions/` lazily creates the server, `GET /v1/servers//sessions` lists it, and a second Session shares the object. 3. Verify lifecycle: `DELETE /v1/servers/` empties the server and a re-created Session reuses identities cleanly. 4. Verify OAuth: one browser flow authorizes a host server for every Session on it. 5. Promote the same build to production; no per-server migration step exists by design. Each server object migrates its own SQLite database the first time it wakes on new code (see the `harness-storage-sqlite` README), and idle objects catch up when next used. The release that introduced storage migrations requires wiping pre-migration Durable Object state once: an object created before it fails with `StorageResetRequired`. Provider authority objects adopt their existing `provider_connections` table in place. Before a `wrangler rollback`, check whether the newer build shipped a breaking migration; objects it migrated refuse to open under the older build with `StorageNewerThanBinary`. ## Embedding in a host Durable Object `@zephyrcloudio/harness-worker-host` exports the embeddable `WorkerHarnessHost`. One DO instance shares one host-provided `Workspace` (file home plus Computer backend) across N HarnessSessions on one SQLite database, with injected `providerDefinitions` (embedder-implemented providers, selected per turn), `contentResolver` (attachment references, resolved only after visibility filtering), and `toolDispatch` (host custody after Harness permission resolution). Embedding hosts pass `credentialMode: 'injected-only'` so credentials arrive only through injected ports; ambient discovery stays disabled. The host has no bearer check (in-DO calls are trusted) and no deployment-wide provider singleton. One multiplexed DO alarm serves Harness recovery, Computer sync, and host-owned participants (`registerAlarmParticipant` + `setParticipantDue`); direct SessionServices admissions call `notifyAdmission()` so the drain runs via `waitUntil`. The host's `model` (the standalone Worker's `HARNESS_MODEL`) is optional: without it, prompts that name no model and title generation use the catalog's first available model. The standalone `HarnessServerObject` is itself composed on this host with env-sourced config, proving the same path embedding hosts take. Pin the exact published version; all workspace packages share one version and `pack:smoke` enforces it. `host.watchTurn(sessionId, turnId, { afterRevision })` returns the same event stream the `/turns/:turnId/events` SSE route sends, as an Effect `Stream` with no HTTP or SSE parsing: durable rows after the cursor, synthesized deltas, heartbeats, and one terminal event after which the stream ends. Omit `afterRevision` to receive only events after the current revision; pass the last revision you saw to resume without a gap. ```ts import { Effect, Stream } from 'effect'; const events = await host.watchTurn(sessionId, turnId, { afterRevision }); await Effect.runPromise(Stream.runForEach(events, (event) => Effect.sync(() => render(event)))); ``` `host.watchSession(sessionId, { afterActivityCursor })` and `host.watchServer({ afterServerCursor })` are the in-process counterparts of the `/sessions/:sessionId/events` and `/events` SSE routes: the Session's or the whole server's activity after the cursor, with heartbeats, as a `Stream` that never ends on its own. Omitting the Session cursor replays that Session's stamped history; omitting the server cursor starts live, with a first `server.cursor` naming the current cursor to bootstrap from. Interrupt the stream to stop watching. Both are observational and arm no recovery. `host.fetch` also serves the server-level routes (the Session list, the activity stream, server deletion) for an embedder's in-process client. The host's `providers` map is an allowlist compiled against the host's catalog (the bundled snapshot, or a newer `modelCatalog`). By default (`unknownProviders: 'fail'`) an id with no catalog entry and no complete `definition` fails host construction for every session. Embedders that build the map from external input can pass `unknownProviders: 'report'`: the unknown ids are left out of the plan and the host constructs with the rest. The drop is never silent; `(await host.resources()).rejectedProviders` lists each `{ providerId, message }`, and `providerIds` lists what compiled. The diagnostic names only the id, never a credential. Pass `modelCatalog: { document, publishedAt }` to compile from a published models.dev-format catalog the embedder fetched instead of the bundled snapshot. It is used only when valid and published (`Last-Modified`) at or after the bundled snapshot's build time; otherwise the host uses the bundled snapshot and reports why on `(await host.resources()).modelCatalog`. See the [Worker host README](/docs/worker-host.md). ## Runtime plugins With a `worker_loaders` binding named `LOADER`, a server object loads Session plugins at runtime from prebuilt bundles with no redeploy. A Session adds one with a `Bundle` source, the credential-free `https://` URL of a self-contained ES module (see the [plugin README](/docs/plugin.md)): ```sh ze-harness --remote URL --server NAME plugins add --session SESSION_ID \ --bundle https://plugins.example.com/notes.js --plugin-id example.notes --options '{"folder":"notes"}' ``` - Resolution fetches the bundle and pins the SHA-256 of its bytes as the revision's `integrity` (`sha256:`) and, unless the client declared a version, its `version` (`bundle-sha256:`). A declared `integrity` that does not match the fetched bytes is a typed `declared_mismatch`; a changed file at the same URL never replaces an activated plugin until `plugins refresh`. - The bundle is cached in the server object's own SQLite by digest, so recovery after eviction reloads the exact activated bytes without the URL. A lost cache refetches and verifies the digest. - Each bundle runs in its own Dynamic Worker isolate, shared by every Session in the server object that activates the same bytes, and inherits the Worker's network access. Plugins are trusted code, as on Node. The isolate is stateless across calls; see the plugin README for what that means for `setup`. - `Package`, `File`, and `Git` sources are a typed unsupported capability on the Worker; `Bundle` sources are a typed unsupported capability on the Node host. - Without the binding the Worker stays static-only and `Bundle` sources fail as a typed unsupported capability. The `test` environment declares the binding, so `test/bundle-plugins.test.ts` runs a fixture bundle through miniflare's Worker Loader. Embedding hosts pass the binding as `dynamicPlugins: { loader: env.LOADER }` on `WorkerHarnessHostOptions`. Embedding hosts can also link Session plugins into their Worker at build time with `staticSessionPlugins` on `WorkerHarnessHostOptions`. Each entry names a `sourceId`, a `pluginId`, a `version`, and the plugin definition; a Session adds it with the `{ _tag: 'Static', id: sourceId }` source. Nothing is fetched or loaded at runtime, and static plugins are trusted host code. The deployed Worker links none; its tests link a Session overlay fixture through `test/support/worker.ts`. ### Server plugins (embedding hosts) An embedding host can also give its Durable Object a list of **server plugins**: bundles loaded into every Session of the object through the host generation, the embedder's counterpart of the Node config file's `plugins`. The embedder owns and persists the list; the harness resolves, loads, and reports it. ```ts const host = new WorkerHarnessHost({ ...options, dynamicPlugins: { loader: env.LOADER }, serverPlugins: [{ url: 'https://plugins.example.com/notes.js', pluginId: 'example.notes' }], }); await host.setServerPlugins([...entries]); // replaces the list and reloads the host generation await host.serverPlugins(); // each entry's url, pluginId, pinned integrity, and active/failed state ``` - Each entry names a bundle URL and the plugin id it must declare, with an optional `integrity: 'sha256:'` pin and `options`. An entry that cannot load is reported `failed` and left out of the generation; it never blocks the server. - A changed list is a distinct host generation (the revision id carries a digest of the loaded list). New Sessions get it immediately and existing Sessions on their next Turn; Session plugins follow the new host baseline. - `setServerPlugins` is refused with `WorkerServerPluginConflict` while any Session runs one of the plugin ids as its own Session plugin, so a room plugin never shadows a thread's copy. A new generation that does not activate is `WorkerServerPluginRejected`; the previous list stays active either way. - There are no HTTP routes or CLI commands for server plugins; the standalone Worker has none. Platform limits, all surfaced as explicit plugin failures rather than hangs: - Dynamic Workers require the Workers Paid plan and are billed per unique Dynamic Worker per day (each distinct bundle is one). - A Durable Object can have at most 10 distinct Dynamic Workers with requests in flight, across all of its Sessions. - Workers RPC messages are limited to 32 MiB, which bounds tool inputs, results, and hook payloads. ## Recovery backoff and quarantine A failing recovery drain records its outcome before surfacing it. The first failure re-arms the pending revision after 60s, then 2m, 4m, and so on up to a 30-minute cap. After 10 consecutive failures the revision moves to the `recovery_quarantine` diagnostic row (`revision`, `failures`, `error`, `quarantined_at`) and stops re-arming, so a deterministically failing drain — for example poisoned rows that fail validation on every read — can no longer burn alarm wakes at full rate. The quarantine `error` and the alarm log carry the error name only, never the message, so a validator that echoes a bad value cannot land in storage or logs. Both alarm paths (`WorkerHarnessHost.alarm()` and the standalone server object) run recovery only when that revision is due. An earlier Computer sync or participant wake does not run a backed-off drain. They catch the recorded failure, still run that other work, then succeed: the retry is already scheduled durably through backoff, so the platform never fast-retries a recorded recovery failure. Unexpected failures still propagate and fail the alarm. Recovery still runs first on every admission via `waitUntil`; the alarm is the backstop. A new admission after quarantine re-arms recovery with a fresh failure budget, and the quarantine row keeps the last stuck revision for operators. Read it through `HostAlarmCoordinator.quarantineState()` from inside the owning Durable Object. The alarm log names the error class; it does not repeat the message. ## Protocol references - [MCP 2025-06-18 authorization](https://modelcontextprotocol.io/specification/2025-06-18/basic/authorization) - [MCP 2025-06-18 transports](https://modelcontextprotocol.io/specification/2025-06-18/basic/transports) - [MCP 2025-06-18 schema](https://modelcontextprotocol.io/specification/2025-06-18/schema) - [JSON-RPC 2.0](https://www.jsonrpc.org/specification) - [WHATWG server-sent events](https://html.spec.whatwg.org/multipage/server-sent-events.html) - [RFC 9728: OAuth 2.0 Protected Resource Metadata](https://www.rfc-editor.org/rfc/rfc9728.html) - [Cloudflare Workers limits](https://developers.cloudflare.com/workers/platform/limits/) - [Cloudflare Worker context](https://developers.cloudflare.com/workers/runtime-apis/context/) - [Cloudflare Durable Object lifecycle](https://developers.cloudflare.com/durable-objects/concepts/durable-object-lifecycle/) ## Public TypeScript exports - `HarnessProviderAuthorityObject` - `HarnessServerObject` - `WorkspaceProxy` --- Generated from [`packages/worker/README.md`](https://github.com/ZephyrCloudIO/ze-harness/blob/main/packages/worker/README.md).