Replays

A replay runs some of a work station's past task runs again under a candidate config, and scores each one. The same data is read and written through the Replays API, candidates through the replay-candidate resource, and both from the spotlight replay command.

Only people on the team, the workspace's owners and admins, and workspace tokens with the read scope can read a team's replay sets, replays, candidates and replay runs. Anyone else is answered 404. Opening a set, renaming or deleting it, adding or removing runs, starting or stopping a replay, and proposing or deleting a candidate also need plan. See Team outcomes.

Station pageCopy Anchor Link

A work station's "Improve" tab, …/stations/<stationId>/improve, has a "Replay sets" card under "Try a change", at #replays. It lists the station's replay sets, one row each:

"Create a replay set" asks for a name, then opens the new set's page. With no sets, the card reads "No replay sets yet" and A replay set holds past runs to replay under a candidate. Create one to start comparing., with "Create a replay set". To a person who may not read the team's replays, the card reads Only team members can see its replays.

Success lineCopy Anchor Link

The success line reads the replay's success with its range, as in 40% (20–60%) succeeded, or No score yet, then:

<counterpart> is the counterpart's label in lower case: baseline, live config, search's starting config or earlier baseline. <share> is the counterpart's success on the tasks both replays scored, with its range when known. Without that success, the label shows alone. Once any task was set side by side, the line ends with · Side by side with <replay>: <n> wins, <n> losses, <n> ties, <replay> being the counterpart as shown, as in the baseline.

Replay set pageCopy Anchor Link

/w/<slug>/team/<teamId>/lines/<lineId>/stations/<stationId>/replay-sets/<setId>

The top bar holds the breadcrumbs, <team> › <line> › <station> › Replay sets › <set>, where "Replay sets" opens the station's replay sets, and "Rename" and "Delete" at its right. Under it, the heading shows the station's mark, the set's name and <n> runs · opened <when>, with this pill:

To someone who may read the team's replays, an unknown set, or one of another station, reads "Replay set not found" and This team has no replay set at that address. Someone may have deleted it. To anyone else, the page reads "Only team members can see its replays".

Then come four panels: "Replays" and "Health" one under the other, then "Runs" and "Checks" side by side. Each is titled with an ⓘ that explains it on hover, and holds its actions at its bottom left. A button that cannot be used is disabled and gives the reason on hover.

The ⓘ of "Checks" reads spotlight replay check replays a pull request's change of this station on this set. One guarding set per station., then Line sets no replay_check. Its pull requests aren't checked. when the line has no replay_check, or else Fails past <n> regressions. Ceiling per check: <ceiling> · Minimum real gain: +<n> pts, from the line's replay_check.

"Rename" opens "Rename the replay set", Its runs and replays stay as they are., with "Name" and "Rename". "Delete" opens Delete <set>?, This also deletes the set's replays and results. Spotlight stops keeping its runs' starting workspaces. You can't undo this., with "Keep it" and "Delete set". Delete is disabled while a replay of the set is queued or running, and while a search of the set is running, with A replay or search of this set is running. Please delete the set after it ends. on hover.

"Add runs" opens Add runs to <set>, The station's latest ended runs. Spotlight keeps a run's starting workspace while it's in the set. It lists the station's ended task runs, the 50 created most recently, newest first, after any runs suggested to even the mix. Runs that are no longer in the workspace are left out after those 50 are read, so the list can hold fewer. Each shows its title, its state, when it ended and, when it cannot be replayed, the refusal, then "Add", "In the set" or "Can't be replayed". "Add" reads "Adding…" while the run is added; when the server refuses it, the dialog shows nothing, and the server's message shows in an error notice. With no ended run, it reads No ended runs to replay yet. Runs appear here once they end. While the suggestions load, the dialog shows a placeholder, and when they cannot be read, Suggested runs could not be loaded. with "Try again". "Done" closes it. A set holds at most 50 runs.

"Start a replay" is disabled with Please add runs to the set first. on an empty set, and with The station is gone from its line. once the station has left the line. It opens Replay <set>, Each of the set's <n> runs restarts from its starting point under the chosen config. Nothing reaches the task, GitHub or any inbox.:

A new baseline is needed when the set has no standing baseline (queued, running or finished) that holds every run now in the set, replays each run at least as many times as asked, and was scored the way the live config scores now. So a baseline is needed when the set has none, when a run was added since the last one (taking a run out does not count), when the last one replayed each run fewer times, when the last one was stopped or went over budget, and when the live config's scoring changed: its fingerprint, the options and ends it has on the line, its check, its diffs or its judge's rubric. With "Live config", the replay itself then becomes the new baseline; otherwise it replays beside the set's baseline. With a proposed candidate, a baseline of the live config starts beside it, with a ceiling of its own equal to this one and the same repeats, so the two may spend up to twice the ceiling. Either baseline may check the set. At 1 or 2 repeats, a checking baseline replays each run 3 times under a larger ceiling; at 3 to 5 repeats, it keeps the repeats and the ceiling asked for. When the baseline beside a candidate checks the set, the two may spend up to the ceiling plus the baseline's: 4 times the ceiling at 1 repeat, 2.5 times at 2 repeats and twice at 3 to 5 repeats, with the baseline's ceiling at most 1,000. The dialog gives what the two may spend only in the two notes below for a candidate with no baseline or more repeats than the baseline, and these say twice the ceiling at any repeats.

Under the ceiling, the dialog reads The most that this replay can spend. Its runs count, and so do the drafting and judging of their changes. Nothing more starts after the replay spends it., then one of these notes:

When the set holds 5 runs or more and its noise is not measured, the first four notes end with The first baseline checks the set. It replays each run 3 times with 3 times the ceiling. It judges and checks each change twice. The note reads the same at any repeats. It holds at 1 repeat. The baseline that checks the set replays each run 3 times, or the repeats asked for when more, under the ceiling times those repeats divided by the repeats asked for: at 2 repeats, 3 times under 1.5 times the ceiling; at 3 to 5 repeats, as asked, under the same ceiling.

"Start replay" opens the new replay's page. The dialog refuses a ceiling of 0 or less with Please enter a ceiling above 0., more than 1,000 with Maximum ceiling: 1,000, a currency that is not three letters with Please enter a three-letter currency code, like USD., and repeats out of bounds with Please enter a whole number from 1 to 5. When the candidates cannot be read, it reads Sorry, the station's candidates couldn't be loaded. You can still replay the live config.

What a set is made ofCopy Anchor Link

The "Runs" panel reads the set's original runs as they stand when the page loads.

OutcomesCopy Anchor Link

How an original run turned out comes from the first thing a person did on its task after it, other than answering an approval, and from how it ended. The first row that applies decides.

When people did two things on the task at the same moment, the first of sending back, closing, answering, parking and starting counts.

OriginsCopy Anchor Link

Where a run's task came from is recorded when the task is opened.

Suggested runsCopy Anchor Link

Once the set holds runs, the "Add runs" dialog lists up to 5 runs first, under "Suggested to even the mix", with Recent runs with outcomes that the set has few of. These runs join the set only when you add them. Each shows its outcome and "Add". None is ever added without it.

The suggestions come from the same 50 ended task runs the dialog lists. From those, Spotlight keeps the runs that can be replayed and are not in the set, then takes the newest run of each task, leaving out tasks with a run in the set already. Runs are then picked one at a time: the outcome among "Accepted", "Sent back", "Asked a person" and "Failed" that the set, with the runs picked so far, holds fewest of, that some run left has, and that is held fewer times than the most common outcome, "Other" included. Ties go to "Accepted", then "Sent back", "Asked a person" and "Failed". The newest run left of that outcome is picked. Picking stops at 5, at the room left in the set below 50, or when no outcome is short. "Other" is never suggested.

An empty set, a full set, a set whose station has left its line, and a set already even get no suggestions, and the part is not shown. Suggested runs are not listed again below it.

Replay pageCopy Anchor Link

/w/<slug>/team/<teamId>/lines/<lineId>/stations/<stationId>/replays/<replayId>

To someone who may read the team's replays, an unknown replay, or one of another station, reads "Replay not found" and This team has no replay at that address. Someone may have deleted it. To anyone else, the page reads "Only team members can see its replays".

The top bar holds the breadcrumbs, <team> › <line> › <station> › Replay sets › <set> › <candidate>. While the replay is queued or running, "Stop" at its right opens "Stop this replay?", Running runs stop and no more start. Finished runs keep their results., with "Keep it running" and "Stop replay".

The heading shows the station's mark and the candidate's name, "Live config", or "A removed candidate", with these pills:

Under it: the set's name, linked to the set; · from its search, linked to the search, for a replay a search started; then · <when>, when it ended, or when it started while it has not ended, with Started <when>, ended <when>, or Started <when>, on hover.

Then come these tiles, each with an ⓘ that explains it on hover:

TasksCopy Anchor Link

The "Tasks" card has one row per task, regressions first, then in the set's order. Its ⓘ reads Each task of the set, replayed and compared with <counterpart>. Regressions come first. Pick a task to see its runs and judging., or Each task of the set, replayed. Pick a task to see its runs and judging. with nothing to compare with. <counterpart> is the counterpart as shown, such as the live config. With a counterpart to compare with, the columns are:

Without a counterpart, the columns are "Task", "Result", "Judge" once any copy was judged, "Closer" once any task was compared, "Cost" and "Time", for this replay alone, the judge cell listing each copy's score, and cost and time averaged over the copies. A value not known reads –.

With no tasks, the card reads "No tasks in this replay" and The set had no runs when this replay started. Please add runs to the set, then replay it again., with "Open the set".

Task detailsCopy Anchor Link

Selecting a task opens a pane beside the card. Its top holds the task's title, "Original run", linked to the original run, and an X that closes it. Then come this replay's copies, under "This replay", or "Result" with nothing to compare with, then the counterpart's, under its label: "Baseline", "Live config", "Search's starting config" or "Earlier baseline". With more than one copy, each is titled with its copy, as in This replay, copy 2 or Live config, copy 2. Each copy shows:

• its result, with "Open the run", linked to the replay run;
• on claims, each claim with "Met" or "Not met" and its reasons, then the pass share, as on the Station page;
• once the copy was asked to be set side by side, Closer to the accepted change: with "This one", "The other change" or "Neither", and "The other run", linked to the counterpart's run, then what the side-by-side judging found;
• under "Why it succeeded", "Why it failed" or "Why it went unscored", each of its reasons, with chips for the steps and hunks the reason cites, as Step <n> and <file>, line <n> or <file>, lines <a>–<b>;
• under "Checks", each check of its change stack: the command, then "Still checking.", Skipped: <reason>, or each step, "The whole change" or a layer, as <step> · passed in <time> or <step> · failed in <time>, with the output of a failed step;
• under "What it changed", the files it changed.

A chip opens the step, or the file's diff at the hunk, in place of the copy's reasons, checks and changes, with "Back" to return.

Replay statesCopy Anchor Link

A replay is running once its first run is queued, and finished once every run has a result. A stopped replay starts no more runs, and its running runs are stopped. An over-budget replay starts no more runs, and its runs already running finish. A run that never started reads Never ran, as does a run a stopped replay left waiting, or Couldn't be replayed or started: <refusal> when it was refused as it came to start, as for dispatch-failed, forgone or candidate-unfit. Through the API, a person can also stop an over-budget replay whose runs are still running: those runs stop, and its state stays over-budget.

ResultsCopy Anchor Link

A result does not decide alone whether a task regressed: a copy that succeeded, or one that is unscored but has a judge's verdict, can regress through a lower judge score.

A run is scored by the candidate's own station block:

With both, a run must pass both. When a person closed the original run's task as the first thing they did on it after the run, the judge is also given the original's change, and scores the replay against it. A replay's judge score is not a run's judge score: it counts toward no agreement, standing, alert or Insights figure of the team's runs. It counts only toward the replay's success, which the Candidates section of the Insights Stations tab shows.

Each result carries reasons, shown as <reason>: <detail> where there is a detail:

When the replay stops, or goes over budget, before a run that finished could be judged, that run is unscored with Judge couldn't judge it: replay stopped before judging. In a replay that checks the set, changes are judged and checked a second time; the second answer never changes a result.

The judge's reasons cite the replay run's steps and the hunks of its change, as a run's judge reasons do. On claims, each claim's mark and reasons are kept with the claims as worded when it was judged.

Run failuresCopy Anchor Link

A failed replay run is classified by its failure code:

A run Spotlight gave up on as abandoned with no code is unscored. A failure code Spotlight does not know shows as the detail as it is. When an attempt is queued again, as after timed-out, its cost is not part of the run's cost: only the run's last attempt, and attempts parked on the way, count toward the replay's spend.

Check skipsCopy Anchor Link

A replay's checks run as a station's checks do. See Checks.

Judging failuresCopy Anchor Link

SuccessCopy Anchor Link

A replay's success is worked out task by task, a task being one run of the set. A task's success is the share of its scored copies that succeeded: 1 or 0 with one copy, and 2/3 when 2 of 3 copies succeeded. The replay's success is the average over the tasks with a scored copy. Unscored and pending runs are left out, so a run that could not be replayed or started, or that its replay stopped, never counts against the candidate.

Beside it is its range: the 95% interval of the average, from 2,000 resamples of the scored tasks, shown as <low>–<high>%. Reading the same results again gives the same range.

GainsCopy Anchor Link

A replay compared with another, its counterpart, reads its gain over it: on each task both scored, its success less the counterpart's, averaged, in points, with the 95% interval of that average from the same resampling of those tasks.

Only the replay page's "Gain" tile writes "Within noise" with a capital. Elsewhere, as in the "Gain" hover on the set's page, it reads within noise.

min_gain is the line's replay_check.min_gain, or 0.05 when the line has no replay_check. The app shows it as +5 pts. A replay reads it from the line as it is now; a replay check keeps the one it started under.

Side by sideCopy Anchor Link

A replay whose candidate has a judge, and that keeps a counterpart in pairedWith, is set beside it, task by task, once the replay has finished. Baselines keep no counterpart, and a replay with the answer keeps none in pairedWith, so neither is ever set side by side. A stopped or over-budget replay never finishes, so it is never set side by side. Each copy of a task is set beside the counterpart's copy with the same number, or else its first copy, as the counterpart's copies conclude, when all of these hold:

• Both runs completed with a change of at least one line, and have their result.
• People accepted the original run's change: the first thing a person did on its task after it was closing the task.
• Both replays are judged under the same rubric version. Otherwise, and when the counterpart has no judge, the copy reads other-rubric.
• The ceiling has room. Each pair is tried at most twice, and before asking, Spotlight sets aside four times what judging the copy's own change cost, two readings for each of two attempts, on top of what the replay has spent and what its pairs still being judged have set aside. When that first judging cost nothing, nothing is set aside. A copy that would pass the ceiling is not asked; once no pair is still being judged, it reads budget.

The judge is given the rubric, the claims or both of the replay's own candidate, the task's title and description, the two changes as hunks labelled A and B, and the accepted change, which is neither. With no candidate or no judge, the copy reads unreadable-rubric. The changes are cut at 60,000 characters, and each hunk at 200 lines. It is not told which change is the replay's. It answers A, B or a tie, with 1 to 5 reasons, each citing 1 to 10 hunks of A or B and never of the accepted change. It reads the pair twice: once in an order drawn from the two runs, and once the other way round. When the two readings name the same change, or both a tie, that is the outcome; otherwise the outcome is a tie. Reasons that cite no hunk, or cite the accepted change, are dropped, and a reading is ungrounded when half its reasons or fewer stand. The replay's change winning is a win, the counterpart's a loss. At most 4 pairs are judged at once, and a pair whose judging fails is tried once more at most; one that failed no-evidence, interrupted or unavailable is not tried again.

A task with copies is a win when more of its compared copies won than lost, a loss when more lost, and a tie otherwise. The replay's wins, losses and ties count tasks with at least one compared copy. They never change a run's result, the replay's success, its gain or its regressions.

Reading a side-by-side judgingCopy Anchor Link

In a copy's details, until it is judged, a copy reads The judge hasn't compared the changes side by side yet. It compares them when the replay finishes. When the two readings disagreed, it reads The judge picked a different change depending on which change it saw first. This counts as a tie. Under "Both readings", With this change shown first: and With the other change shown first: each give their outcome and reasons, each reason with the hunks it cites, as in Cites 2 hunks of this change and 1 hunk of the other.

A copy not compared reads Not judged side by side: <reason>.:

no-evidence means either change, or the accepted change, has no hunks, or the evidence of one of the three runs could not be read.

What side-by-side judging spends counts against the replay's ceiling, but not toward its spend totals or its cost per run. The replay page shows it apart, as "Side by side" in the ⓘ of "Spent".

HealthCopy Anchor Link

The "Health" panel of the set's page says whether the set can tell a better config from a worse one. Its ⓘ reads Shows whether this set can tell a better config from a worse one. The first baseline of the live config replays each run 3 times. Then Spotlight replays the live config with each run's answer. Spotlight doesn't check sets of fewer than 5 runs. It is read for the station's live config as it is now: only replays of the live config at the station's current config fingerprint count.

Each reading is a row: its label with an ⓘ that gives its advice on hover, then what it found, marked with a tick when it found no problem, a warning sign when it found one, an hourglass while it is being measured, and a dashed circle while it cannot be read, and "View replay", which opens the replay it was read from.

NoiseCopy Anchor Link

Until it is measured, the advice reads The first baseline of the live config replays each run 3 times to measure noise. Noise shows how much success moves by chance.

Room to improveCopy Anchor Link

Before any replay with the answer, or when one finished but could not be compared, the reading comes from the live config's success over the whole set:

The last two end with If the set's runs were accepted or sent back, Spotlight measures this with the answer after a replay of the whole set.

Once there is a replay with the answer, the reading comes from it, and "View replay" opens it:

<rates> is Live config: <share> on <n> runs. With the answer: <share>., both read on the runs it replayed. Every advice in this table but "Couldn't measure with the answer" ends with Answer: the change people accepted, or what they said when they sent the run back.

Judge agreeing with itselfCopy Anchor Link

<n> counts the changes judged twice, both times with a verdict. While second judgings are still to come, the advice ends with <n> second judgings are still to come.

Flaky checksCopy Anchor Link

<m> counts the changes checked twice, both times passing or failing. The tasks whose check flipped are listed under it, each linked to the replay run, with Passed, then failed or Failed, then passed. While second checks are still to come, the advice ends with <n> second checks are still to come.

Replay with the answerCopy Anchor Link

After a replay of the live config over every run of the set finishes, other than a search's, Spotlight starts a replay of the same live config handed each run's answer, when:

• the station still runs that config, at the same config fingerprint;
• the station has not left its line;
• at least one of the set's original runs was "Accepted" or "Sent back" (see Outcomes);
• the set has no replay with the answer of that config that is queued, running or finished.

Nobody starts it, and it cannot be started from the app or the API. Its candidate is named With the answer, <date>, and it shows under "Replays" like any other replay. It replays only the set's runs whose original was "Accepted" or "Sent back", among the runs that live replay replayed and that are still in the set, with the repeats and a ceiling of its own equal to those of the live replay whose finishing started it. Set health reads it against that live replay, on the runs it replayed. That live replay is also its counterpart: on the replay page and in the API, its tasks and gain are compared with it, shown as the live config, or as the baseline while it is the set's baseline. When that live replay is gone, it is compared with nothing. It is never set side by side. A stopped one reads A person stopped the replay. No more runs start., also when Spotlight stopped it.

Each replay run's briefing ends with a section headed ## The answer:

Which kind of answer each run gets is fixed when the replay starts; a run whose original change has no hunks, or whose sending back came with no words, is left out. The hunks and people's words are read again when each run is queued, and the answer is cut at 60,000 characters. If a run's answer is gone by then, the run is not replayed and reads answer-gone (see Why a run cannot be replayed). If the station's config changes while it is queued or running, Spotlight stops it.

Checking the setCopy Anchor Link

Nobody starts a check, and it cannot be started from the app or the API. A baseline of the live config checks the set when both hold:

• the set holds 5 runs or more;
• no replay of the live config at its current config fingerprint over the whole set has checked it, or is checking it, while queued, running or finished.

This applies to a "Live config" replay that becomes the baseline, and to a baseline that starts beside a proposed candidate. A search's replay of the live config and the baseline of a replay check never check the set.

The checking baseline replays each run 3 times, or the repeats asked for when more. Its ceiling grows in step: the ceiling asked for, times its repeats, divided by the repeats asked for, and at most 1,000. One repeat at 5.00 becomes 3 repeats at 15.00. Its 3 copies measure the set's noise.

Once it finishes, every change whose check passed or failed is checked a second time, and every change the judge gave a verdict on is judged a second time. Each change is weighed on its own, in order: it is judged again when what its first judging cost still fits under the ceiling, on top of what is spent and set aside so far. A change that does not fit is not judged again, a later, cheaper one still can be, and a change whose first judging cost nothing always fits. A second judging or check never changes a run's result. Its finishing then starts the replay with the answer, as any replay of the live config over the whole set does.

CounterpartsCopy Anchor Link

A replay's counterpart is the replay of the set it is compared with. Its task-by-task rows and its gain are measured against it, and the API names it in comparedWith. These replays keep a counterpart, chosen when the replay starts:

The first three also hold it in pairedWith, the replay their changes are set side by side with. A replay whose kept counterpart is gone has none, and is compared with nothing.

A replay that keeps none, such as a baseline, has as its counterpart the set's baseline as it stands when it is read: the newest replay of the live config the set started as a baseline, while it is queued, running or finished. The baseline itself has none. A stopped or over-budget baseline is not compared with, and the next replay of the set starts a new one.

The counterpart is shown, and labelled, as:

The shown form names it in running text, as in the "Tasks" ⓘ. The "Closer" column and the judge hover capitalise it, as in "The baseline". The label heads its column under "Tasks" and starts its success on the set page's "Gain" hover. In lower case, it follows the verdict on the "Gain" tile, and on the success line it follows vs after the gain, or · without one.

RegressionsCopy Anchor Link

A task regressed when the counterpart's success on it is higher than this replay's by more than 50 points (0.5), or when this replay's judge score, averaged over its copies, is more than 0.01 below the counterpart's. With one copy each, that is the counterpart succeeding and this replay failing. Either is enough: a task whose copies all succeeded still regresses when its judge score is lower. A task can regress only once every copy of it, on both sides, has settled and has its result. A copy that has neither a scored result nor a judge's verdict cannot make a task regress.

Score comparisonsCopy Anchor Link

Judge scores are compared only when both runs were judged, under the same rubric version, either both or neither against the original's accepted change, and on the same scale. Scores on different scales are not compared, and the cell reads –. Otherwise the "Judge" column says why:

Why a run cannot be replayedCopy Anchor Link

A run that cannot be replayed is refused when it is added to a set, with 409. A run in a set that can no longer be replayed carries a red warning sign on the set's page, with its refusal on hover, and the "Add runs" dialog shows the refusal beside the run. The sign shows only unknown-run, not-a-task-run, task-gone, still-running, no-starting-point, no-briefing and archive-gone. The dialog shows these for the runs it lists. It leaves out runs that are no longer in the workspace, so it never shows unknown-run. The other refusals of an added run show only as the 409. A run refused when its replay comes to start it is unscored, with Couldn't be replayed or started: <refusal>, the refusal in lower case.

attachments-gone, repository-gone and not-at-the-station can be refused both when a run is added and when its replay comes to start it. answer-gone, candidate-unfit, cannot-start, forgone and dispatch-failed arise only when a replay comes to start the run. candidate-unfit means the candidate was removed, or its block no longer fits the line as it is now. cannot-start means the candidate's station runs no harness a runner can start. dispatch-failed follows 10 failed attempts to queue the run. unchecked means checking the run met an unexpected error when it was added, with the reason It could not be checked just now. Please try again later.; adding it again checks it again.

A workspace archive is kept for at least 30 days after the latest run that started from it, and while a run that started from it is in a replay set. Once the run's task is deleted, the replay set no longer keeps it. See Workspace archives.

Spend and ceilingCopy Anchor Link

A replay's spend is the cost of its runs, every copy included, of drafting their change stacks and of judging them, counted in the ceiling's currency. A drafting or judging attempt that failed counts what it was billed. A cost in another currency is summed apart and does not count toward the ceiling, nor does a cost that was never reported, for a replay started from the app or the API. Replays a search or a replay check starts, and their baselines, count such a run at a fixed amount per run against the ceiling; a replay with the answer does the same when the live replay that started it does. Side-by-side judging counts against the ceiling, but not toward the spend; see Side by side. Each replay has its own ceiling. A baseline started beside a replay has a ceiling of its own, equal to the replay's, or larger when it checks the set. The second judgings of a baseline that checks the set count against its ceiling, but not toward its success or what its runs cost.

Once a replay's spend reaches its ceiling, it ends as over-budget: nothing more of it starts, its runs not yet started are withdrawn, and its runs already running finish. What they spend still counts, so a replay can end above its ceiling. The replay page then reads Over the ceiling by <amount>, and spotlight replay status adds , over the ceiling by <amount>, spent by runs already running when it was reached.

Replay runsCopy Anchor Link

A replay run's Run page opens at /w/<slug>/team/<teamId>/runs/<runId>, like any run, from "Open the run" in a task's details. Its breadcrumb ends with the original run's task title and Replay at <station>, or Replay at a removed station. Above the run's briefing and steps, a banner shows:

• A replay of <task title>, linked to the original run;
• Under <candidate>, with (the baseline) for a baseline; for a search's replay, · candidate <name> in its search, or · the live config it started from in its search, linked to the search; then in a replay of <set>, linked to the replay page.

When the replay cannot be read, the second line reads Sorry, the replay couldn't be loaded. Please try again later. The run's result, claims and side-by-side judging show in the task's details on the replay page.

The message box is disabled, with The agent runs autonomously. It can't receive messages. A replay run can be read, with its config and its event stream, only by those who may read the team it replayed for, recorded when it was queued, even after its station is removed. Anyone else is answered 404, and the page reads This team has no such run.

A replay run differs from a task run:

See Inside a sandbox for the phases every run goes through.