Improve a station from its runs
Use this when a station's runs keep failing, get sent back or cost more than they should. You need:
- To be on the team, or an owner or admin of the workspace. Only they can see how the station's runs turned out.
- A host serving the workspace with the harness you want to plan with. The chat runs there like any other run. See Install a host.
- The team's configuration in Terraform, to apply the change. See Manage Spotlight with Terraform.
Read how its runs turned out
Open the line's board and select the name of a work station. On the Overview tab, read Latest runs on this config and Recent runs.
To compare configs, select See in Insights. Under Runs by config, compare neighbouring groups, unfold a group to list its runs, and select a run's ending to open it.
Station outcomes says what each part shows and how runs are grouped.
Ask the agent for a change
On the Station page, open the Improve tab. Under Worth improving, select Ask the agent beside the signal you want to fix. The Agent page opens a new chat with the station already in the message, the same way a shared trace appears, and the signal written out before What should change at this station? If no signal is listed, select Review with the agent instead. To start an ordinary chat, delete the station from the message.
In the agent picker in the composer, choose the harness under Harness. For a model other than the harness's default, choose it under Model. If the harness offers an effort, choose it under Effort the same way.
If the team's Terraform is in a GitHub repository the GitHub App can see, select No repositories below the composer and tick it. The agent then writes its proposal against the files as they are.
Add to the message what the station should do better, such as the runs you saw sent back, and press Enter.
The first message names the station and its line. The agent writes
the station's recent runs into its workspace with
spotlight history and
reads them. It proposes one change at a time, as a diff of the station's block
in the line's spotlight_line resource and of its prompt file,
prompts/<team>/<line>/<station>.md, and says which outcome should show that
the change worked.
The agent cannot change the station itself. Reply in the chat to question the proposal or ask for another one.
Try the change on past tasks
To see how the proposed change does on tasks the station has already worked before you apply it, propose it as a replay candidate and replay some of the station's past runs under it. See Replay past tasks against a change to a station. To have an agent propose and replay changes for you, and open a pull request for the one you pick, see Search for a better station config.
Apply the change
Make the proposed change in the team's Terraform: the station's block, its prompt file, or both. See Change a line's stations.
Run terraform apply.
Check whether it helped
Once runs at the station have ended with the new config, open the station again and select See in Insights. Their group is at the top of Runs by config, named by the first of its changes. Compare it with the group below it, or select Impact on it to set the runs before the change against the runs after it.
If the change did not help, say so in the same chat and ask for the next change.