Provision runners from your own platform

A push pool has no hosts. For each run or check it is to serve, Spotlight calls an HTTPS endpoint of yours, and the endpoint starts a container that does that one piece of work and exits. Use it when your platform starts containers on demand, such as a job scheduler or a serverless container service.

Two things work differently from a host, and they decide whether a push pool suits a line:

• The container has no egress service. A run whose harness profile gives an MCP server a secret never goes to a push pool; it waits for a host.
• The host's sandbox.env does not apply. Harness keys go in the container's environment, where the agent can read them.

You need:

• Terraform 1.11 or later, with the provider set up using a token that has the runners scope. See Manage Spotlight with Terraform.
• An HTTPS endpoint on a host that Spotlight's deployment allows outbound calls to.
• A platform that can start a container which reaches your Spotlight address over HTTPS.
• An API key for each harness your lines use.

Create the poolCopy Anchor Link

Give the pool a provision block with your endpoint's URL and a signing secret, and keep the secret where the endpoint can read it:

data "spotlight_workspace" "acme" {
  slug = "acme"
}

resource "random_password" "provision" {
  length  = 32
  special = false
}

locals {
  provision_secret = "whsec_${base64encode(random_password.provision.result)}"
}

resource "spotlight_runner_pool" "jobs" {
  workspace_id = data.spotlight_workspace.acme.id
  key          = "jobs"
  labels       = ["jobs"]

  provision = {
    url            = "https://provision.acme.example/spotlight"
    secret         = local.provision_secret
    secret_version = 1
  }
}

output "provision_secret" {
  value     = local.provision_secret
  sensitive = true
}

Run terraform apply, then give the endpoint the value of terraform output -raw provision_secret.

Work from any line that does not name runners may come to this pool as well as to your hosts. To send a line's or a station's work here in particular, name one of the pool's labels on it, as in Send a line's work to particular runners. To cap how many containers run at once, add max_runs.

Verify each callCopy Anchor Link

Spotlight signs every call as Standard Webhooks does, so a Standard Webhooks library can verify it. To verify it yourself, compute an HMAC-SHA256 over <webhook-id>.<webhook-timestamp>.<raw body>, keyed with the bytes that the part of the secret after whsec_ decodes to, and compare it with the base64 value after v1, in webhook-signature. In Node:

import { createHmac, timingSafeEqual } from 'node:crypto';

const key = Buffer.from(process.env.PROVISION_SECRET.slice('whsec_'.length), 'base64');

export function verified(headers, rawBody) {
  const id = headers['webhook-id'];
  const timestamp = headers['webhook-timestamp'];
  if (!id || !timestamp || Math.abs(Date.now() / 1000 - Number(timestamp)) > 300) {
    return false;
  }
  const expected = createHmac('sha256', key).update(`${id}.${timestamp}.${rawBody}`).digest();
  return (headers['webhook-signature'] ?? '').split(' ').some((each) => {
    const [version, value = ''] = each.split(',');
    const given = Buffer.from(value, 'base64');
    return version === 'v1' && given.length === expected.length && timingSafeEqual(given, expected);
  });
}

Verify against the body exactly as it arrived, before parsing it. Answer 401 to a call that fails.

Start the runnerCopy Anchor Link

The body is JSON of type work.provision. Read these fields from data:

data.kind is run or check; start the same container for both. Provision call lists every field.

Start the container with those two variables and your harness keys, and leave the image's entrypoint and user as they are. The entrypoint is spotlight runner, which runs as root and hands the agent a user of its own. With Docker, the equivalent is:

docker run --rm --name "spotlight-<runner id>" \
  -e SPOTLIGHT_URL="<spotlightUrl>" \
  -e SPOTLIGHT_TOKEN="<runner token>" \
  -e ANTHROPIC_API_KEY="<your key>" \
  <image>

For other harnesses, pass the variables they read instead, as listed in Harnesses. The agent receives everything in the container's environment except the SPOTLIGHT_ variables, so pass only what the harness needs.

The runner registers, claims its work, does it, and exits. If nothing is waiting for it after 5 minutes, it exits too. Remove the container when it stops.

Answer quicklyCopy Anchor Link

Answer any 2xx within 10 seconds, and start the container after answering or in the background. Spotlight treats a later answer, a redirect or any other status as a failure. It then offers the work to the next push pool that fits, or, after refused, unreachable or lapsed, tries again later, up to 5 attempts in all. The other causes below fail the run at once.

Every call names a new runner, and its id is also the webhook-id header. If your endpoint receives the same runner id twice, start only one container. A name derived from the id, as above, does this on most platforms.

Check that it worksCopy Anchor Link

Start a task from the backlog at a station whose work the pool may take, as in Add a task and move it along a line. On Runners, a row jobs appears that reads Push pool, with the new runner in a row beneath it. The runner reads Starting until it claims the work, then Running.

If the run fails instead, its message begins No runner was provisioned after N attempts. Last failure:, followed by the cause:

RelatedCopy Anchor Link

• spotlight_runner_pool: every pool field.
• Provision call: the call a push pool receives, and its failures.
• Give hosts your harness keys: which key each harness needs.
• Run a line in your own runner image: build the image a push pool is asked to start.
• Ways in: pools, trusts and tokens: when to choose a push pool over hosts.