Driving Hass Bench from your own code

Everything the web app's four lanes do is available over HTTP. Send a task, your Home Assistant automation as a line-numbered document, and the read your side made of it, and get back one JSON object. The parts the browser does for free — the YAML reader that resolves scalars the way Home Assistant's own PyYAML-derived loader does, the deterministic rules that run over the parsed tree, and the dry run against a world state you set — are not re-run server-side. If you drive the API directly you build automation_digest, you build facts, and you build dry_run, because facts.browser_findings is the only thing the model is held accountable to.

The task field comes first

This is a multi-lane app with one system prompt and one model. Which lane you get is decided entirely by task. Send it on every call, including /estimate — hold_credits differs per lane because the prompts and output caps differ, and pricing one lane while running another is the most common arithmetic mistake against this API.

If task is missing or unrecognised the model picks the closest lane, follows that lane's contract exactly, names the lane it chose in the task field of its reply, and says in the first sentence of summary why. It never blends two lanes: a review does not quietly hand you a rewrite, and a harden does not stop to narrate.

Base URL and headers

The response envelope

Every response, success or failure, is the same shape. Read ok before you touch data.

{"ok": true,  "data":  { ... }}
{"ok": false, "error": {"code": "insufficient_credits", "message": "...", "details": { ... }}}

What the browser does before it calls the API

The web app is not a text box in front of a model. By the time it posts anything, four things have happened locally, and all four travel in the request as data the API takes at face value.

1. The YAML is parsed — not by a general-purpose JavaScript library, but by a reader written for the subset Home Assistant configuration uses, for two reasons. Every node keeps its line number, because a finding that cannot say "line 6" is a finding nobody acts on. And scalars resolve the way PyYAML resolves them, which is YAML 1.1: the plain scalar on is the boolean true, so to: on does not test for the string a switch reports, and the automation silently never fires. A YAML 1.2 library reads that as "on" and reports no problem at all.
2. Deterministic rules run over the parsed tree — arithmetic and pattern matching over nodes and lines, not judgement: a YAML 1.1 boolean where a state string was meant, a sexagesimal integer where a time was meant, mode: single in front of a delay, an action that writes to an entity that also triggers the automation. Each one that fires becomes an entry in facts.browser_findings with an id, a severity, a line and the refs it points at.
3. The document is redacted and line-numbered — credential-shaped values replaced before anything leaves the browser, every line prefixed with its real line number so a finding can name one and a fabricated one is visible. That is automation_digest. The structure the rules found travels beside it in facts: the ref vocabulary, the entity ids that exist, and the flags.
4. Optionally, a dry run — you set entity states and a clock, the app walks the automation the way Home Assistant would, and the transcript becomes dry_run. No Home Assistant instance is contacted and none is required.

None of that happens server-side. The API is one model call with your digest, your facts and your transcript in the prompt. Two consequences:

• Send the document as raw_yaml when it parsed fine, and you throw away the line numbers, the ref vocabulary and every flag. The model then reads the text as a YAML 1.2 reader would — which is to say it reads to: on as a string and tells you the automation is fine. Number it and send facts.
• Send facts.browser_findings: [] and reconciliation comes back empty, which makes the reply unfalsifiable. Every distinct browser_findings[].flag_id you send comes back as exactly one reconciliation entry with a status of confirmed, cleared or not-assessed and a one-sentence note. That is the only mechanical check there is on whether the model read your automation or wrote around it.

The model is held to facts afterwards: the prompt forbids contradicting a count you sent, and every flag must be answered by name. If the model thinks a flag is wrong it argues in the note and sets cleared. What it may not do is ignore it.

The input object

Send automation_digest or the raw_yaml/parse_error pair, never both — sending both is a validation_error. With raw_yaml there are no refs and no structural reading, so the prompt puts the parse failure first in findings as HB-001 at critical severity, quotes the smallest fragment that shows the problem, gives the corrected fragment in yaml_fix, and drops the verdict to that lane's most cautious value — do-not-reload, unpredictable, rewritten-with-assumptions or not-enough-entities. reconciliation still carries one entry per flag_id, and most of them being not-assessed is the correct answer. That path exists so a broken paste still gets a useful answer — usually "there is a tab character on line 12 and Home Assistant will not load this at all" — not so you can skip building a digest.

You build the digest. Nothing rebuilds it for you.

An automation is small — twenty to eighty lines — so unlike a scanner log it is sent whole rather than sampled. That makes automation_digest simpler than the name suggests: it is the document itself, as text, redacted, with every line prefixed by its real line number, behind a short header saying what it is. It is not a JSON object, and it is not a structural summary. The structure travels separately, in facts.

Here is the automation used in every example below — the most common shape there is, and it has never fired once — shown as exactly what goes in automation_digest: header, blank line, then the numbered document. Strip the NN | prefixes and you have the user's file, unchanged.

HOME ASSISTANT AUTOMATION DOCUMENT, read in the user's browser.
Lines: 18. Blocks found: 1.
Every line below is prefixed with its real line number in the user's file. Cite those numbers exactly; never invent one.

 1 | alias: Hallway motion light
 2 | description: Light the hallway when someone walks through after dark
 3 | trigger:
 4 |   - platform: state
 5 |     entity_id: binary_sensor.hallway_motion
 6 |     to: on
 7 | condition:
 8 |   - condition: numeric_state
 9 |     entity_id: sun.sun
10 |     attribute: elevation
11 |     below: 4
12 | action:
13 |   - service: light.turn_on
14 |     entity_id: light.hallway
15 |   - delay: 00:02:00
16 |   - service: light.turn_off
17 |     entity_id: light.hallway
18 | mode: single

The numbers are the entire reason the document is reformatted at all: they let a finding say "line 6", and they make a fabricated line visible as one — a reply citing line 24 of an eighteen-line document can be rejected without reading a word. Numbering always starts at 1, so the line in the digest is the line in the user's file with no offset to undo.

Redaction happens before the wire, not after

The browser redacts the document on the way into the digest, line by line, and it is deliberately blunt:

• A value under a key whose name says credential — token, password, secret, api_key, bearer, client_secret, access_key, and also latitude/longitude — is replaced wholesale with [value removed by Hass Bench before sending]. A value that is already !secret something is left alone, because that is the fix, not the problem.
• A credential-shaped run of characters anywhere — 28 or more characters of unbroken identifier, a eyJ JWT, a ghp_-style token, a Slack xox token — is replaced in place with [N-character value removed by Hass Bench]. Anything containing a dot is left alone, which is how entity ids and service names survive.
• Slack, Discord and Telegram webhook URLs are replaced by name.

Over-redacting a long harmless identifier costs the review a little context. Under-redacting sends someone's token to a language model. Those are not comparable, so the rule errs one way on purpose. When anything was redacted the header gains a REDACTED: line telling the model how many lines were touched, not to ask for them, and not to treat the placeholder as the real value — and the prompt's answer to an inline credential is always the same: rotate it, then reference it from secrets.yaml. If you build the digest yourself, redact first. Nothing downstream will do it for you.

Truncation cuts on block boundaries

The cap is 26,000 characters. Someone will paste their entire automations.yaml, and a blind slice(0, MAX) of a YAML document cuts an automation in half and produces a confident reply about a fragment. So the cut is on whole-automation boundaries: as many complete blocks as fit, never a partial one. The header then gains a TRUNCATED: line naming how many of how many blocks are below, the line range they cover, and an instruction not to comment on blocks it cannot see. Do the same if you are assembling this yourself — the failure mode you are avoiding is not a missing answer, it is a plausible answer about half an automation.

One more thing goes into the digest: when a dry run was produced, its transcript is appended under its own header, introduced as the browser's own dry run with the warning that anything marked NOT EVALUATED is a gap in the dry run rather than a fact about the automation. The dry_run field carries the same transcript; the copy inside the digest is what keeps it adjacent to the lines it refers to.

The facts object

facts travels as a real object, not a string. This is the structural read — what the browser knows, as opposed to the document, which is what the user typed:

{
  "readable": true,
  "is_automation": true,
  "document_kind": "automation",
  "line_count": 18,
  "automation_count": 1,
  "reader_warnings": [],
  "secret_tags_used": 0,
  "counts": {"blocker": 1, "warn": 2, "note": 1, "total": 4},
  "confidence": {"score": 44, "band": "do not reload yet"},
  "automations": [
    {"index": 0, "line": 1, "alias": "Hallway motion light", "id": "",
     "script_id": "", "is_script": false,
     "mode": "single", "mode_given": true, "max": null,
     "trigger_key": "trigger", "condition_key": "condition", "action_key": "action",
     "triggers": ["T1:state"],
     "conditions": ["C1:numeric_state"],
     "action_steps": ["A1:light.turn_on", "A2:delay", "A3:light.turn_off"]}
  ],
  "entities": [
    {"entity_id": "binary_sensor.hallway_motion", "domain": "binary_sensor",
     "seen_in": "trigger", "lines": [5]},
    {"entity_id": "sun.sun", "domain": "sun", "seen_in": "condition", "lines": [9]},
    {"entity_id": "light.hallway", "domain": "light", "seen_in": "action",
     "lines": [14, 17]}
  ],
  "browser_findings": [
    {"flag_id": "HL-01", "code": "bool-where-string", "severity": "blocker", "line": 6,
     "title": "`to: on` is the boolean true, so this trigger can never match"},
    {"flag_id": "HL-02", "code": "legacy-service-key", "severity": "warn", "line": 13,
     "title": "`service:` is the pre-2024.8 spelling of a service call"},
    {"flag_id": "HL-03", "code": "long-delay-single-mode", "severity": "warn", "line": 18,
     "title": "mode: single in front of a 2-minute delay drops re-triggers silently"},
    {"flag_id": "HL-04", "code": "legacy-singular-key", "severity": "note", "line": 3,
     "title": "Singular trigger/condition/action keys; 2024.10+ spells them plural"}
  ],
  "dry_run": {"started_at": "21:40", "chosen_trigger": "", "fires": false,
              "blocked_by": "T1", "why": "no trigger can match", "trace_steps": 0,
              "stopped_early": false, "not_evaluated": 0}
}

facts.automations[].triggers, .conditions and .action_steps are deliberately flat strings of the form ref:type. They are not a structural dump — the document is right there in the digest, line-numbered — they exist to fix the ref vocabulary so a reply cannot invent a trigger. facts.entities is the closed set of entity ids that exist for the card lane. facts.dry_run is a summary of the transcript, present only when a dry run ran.

browser_findings, and why the ids move

Every distinct flag_id you send comes back as exactly one reconciliation entry, with a status of confirmed, cleared or not-assessed and a one-sentence note. That is the single assertion about the reply the browser can check entirely on its own, and it is the whole reason facts exists. Send browser_findings: [] and reconciliation comes back empty, which makes the reply unfalsifiable.

Two fields identify a finding, and the difference matters:

• flag_id is positional. The flags are sorted by severity (blocker, then warn, then note), then by a rank that floats a disclosed credential above every other blocker regardless of where in the file it sits, then by line — and only then numbered HL-01, HL-02, and so on. So HL-01 is always the most serious finding in this document, which is what makes it readable out loud, and it means HL-01 is not a stable name for a rule. Do not persist it, do not compare it across documents, and do not build a suppression list out of it.
• code is the stable rule identity: bool-where-string, sexagesimal-time, self-trigger-loop, long-delay-single-mode, inline-secret, legacy-service-key, and about sixty more. That is the field to key your own logic on.

Four codes change the reply's behaviour outright, so compute them or accept that you lose the guarantee:

facts.confidence is a single blunt number: 100, minus 22 per blocker, 7 per warn and 2 per note, and then capped at 44 if there is any blocker at all. The bands are ready to reload (85 and up), reload after a read (60–84), needs changes (45–59), do not reload yet (below 45), plus unreadable when nothing parsed and not an automation when it parsed but held no automation or script. Note what the cap does: 44 is one below the needs changes boundary, so any blocker puts the document in the worst band by construction, whatever the rest of the arithmetic says. That is deliberate — a blocker means it does not work, and a score above the reload line would be a lie. Copy the formula or use your own, but send something: every lane reads the band, and review reads it before it writes a verdict.

Refs and lines are how you point at a step

Every trigger, condition and top-level action step has a ref, and the refs are short: T1, T2 for triggers, C1 for conditions, A1, A2, A3 for action steps, numbered from 1 in document order. They are fixed by facts.automations[] — that is what the arrays of "T1:state" strings are for — and body.trigger_review[].ref, body.condition_review[].ref and body.action_review[].ref quote them back. The prompt forbids inventing a ref, renumbering, or naming a step the document does not have.

Nested action steps extend the parent ref with a letter for the branch and a number for the position: s for sequence, t for then, e for else, d for default, p for parallel, and c1, c2 for the options of a choose. So the second step inside the then branch of action 3 is A3t2, and the first step of the second choose option of action 4 is A4c2s1. Only top-level steps get an action_review entry; nested ones are named in a why when they matter.

findings[] uses a line and a key instead of a ref, because a finding is often about a key that is absent — no for: on a motion trigger, no mode: at all, no id — and an absent key has no ref. key is the dotted path, such as trigger[0].to or triggers[1].to depending on which spelling the document uses. Where a finding genuinely cannot be placed on a line, line is 0 — that is the contract, and it is there so that nothing is ever tempted to guess a plausible number.

The app audits this. Every ref in the reply is checked against the document that was parsed at the moment of the run — so a later edit to the paste box cannot make a good reply look fabricated — and every line is checked against facts.line_count. Anything that does not resolve is rendered with an explicit marker rather than drawn as if it were real. Reproduce both checks if you drive the API directly: keep facts.automations and line_count on your side, and treat an unresolvable ref as a failed reply rather than as a finding. The cheapest way for a model to sound precise is to invent a line number, and a line-numbered digest is what makes that cheap trick catchable.

The dry run is a transcript, not a simulation the API runs

dry_run is plain text you produce. The API neither executes it nor checks it — it goes into the prompt as evidence, and the prompt's rule is that where the dry run actually evaluated something, its result is treated as fact. That is a strong instruction, so be careful what you put there.

The format the web app emits, and the shape the prompt is written against: a header line with the clock, the sun state and the run mode, then a TRIGGERS block with one line per trigger, a CONDITIONS block, a one-line VERDICT, then an ACTION TRACE if it got that far, and finally the two blocks that keep it honest.

DRY RUN of Hallway motion light at 21:40, sun below horizon, mode single

TRIGGERS
  T1 state: does not fire - `to` loaded as the boolean true (line 6); the entity
     reports the string "on", and "on" != true

CONDITIONS
  C1 numeric_state: PASS - sun.sun elevation -12.4 is below 4

VERDICT: it does not run - no trigger can match the state this entity reports

NOT EVALUATED BY THE DRY RUN (0)

A run that does fire carries the trace as well, one line per step, each prefixed with the simulated clock so a delay is visible as time passing rather than as a step that did nothing:

VERDICT: it runs - T1 fired and C1 passed

ACTION TRACE
  21:40 A1 [service] light.turn_on -> light.hallway = "on"
  21:40 A2 [delay] wait 00:02:00 (clock advances to 21:42)
  21:42 A3 [service] light.turn_off -> light.hallway = "off"

NOT EVALUATED is load-bearing. The browser's simulator does not implement Jinja templates, device triggers or zones, and when it meets one it says so rather than guessing. The prompt is told explicitly that such a line is a gap in the dry run and not a defect in the automation — without that, "the dry run did not evaluate this template" comes back as a finding about the template. If you write your own transcript, keep that block and keep the wording: an unknown you declare is worth more than a result you invented.

Send "" when you did not simulate. An empty dry_run makes review say so in assumptions, which is exactly the disclosure you want. A transcript you got by asking a model to imagine one is the failure case: the lane will treat its results as fact and reason from them, and you will have laundered a guess into evidence.

The output contract

Every lane returns one JSON object — no prose around it, no code fence — with the same envelope; only body differs.

{
  "task": "review",
  "title": "one line naming the automation and the job, under 90 chars",
  "verdict": "one of the lane's allowed verdicts",
  "summary": "two to five sentences",
  "assumptions": ["..."],
  "open_questions": ["..."],
  "findings": [{"id":"HB-001","severity":"critical|high|medium|low",
                "line":6,"key":"trigger[0].to",   // a line the digest shows, or 0
                "issue":"","why":"","fix":"","yaml_fix":""}],
  "reconciliation": [{"flag_id":"HL-01","status":"confirmed|cleared|not-assessed",
                      "note":""}],
  "next_lane": "review|explain|harden|card|",
  "body": { }
}

Every array is present even when it is empty, and an empty array is a real answer rather than a placeholder. findings is for the automation, not for Home Assistant and not for your YAML style. A trigger that cannot match, a condition always false at the hour the trigger fires, a mode that silently drops events, a template with no default, an action ordered after the delay that gates it — those are findings. "This would be tidier as a script" is not, and the prompt says so.

The closed verdict sets

Each lane has exactly three allowed verdicts. They are closed sets, and the app colour-codes on them.

An off-contract verdict is displayed, not colour-coded. If a reply comes back with verdict: "probably-fine", the app renders the string exactly as sent and drops the badge styling rather than guessing which of the three it meant. Do the same: switch on the closed set, fall through to a neutral rendering, and never map an unknown verdict onto the nearest known one. A model that invented a verdict has told you something about that reply, and flattening it hides that.

Three house rules shape every reply. What the loader produces wins over what was typed: wherever raw and value differ, the text says the value first. Entity existence is never assumed — the digest proves an id was referenced, never that it exists, so replies say "the automation targets light.hallway", not "your hallway light". And nothing is recommended that needs a destructive reload to test: verify_steps prefers Developer Tools and manual triggering over "reload and see".

One worked example per lane

Every example below runs against the hallway automation from the digest section, so you can read the four replies against each other. automation_digest is that digest stringified and facts is the object next to it; both are elided for width. Only review is shown with the full envelope — the envelope is identical for every lane, so the other three show the fields that differ.

task: "review" — Will it fire, and is it safe to reload

Request

{"task": "review",
 "automation_digest": "HOME ASSISTANT AUTOMATION DOCUMENT, read in the user's browser.\nLines: 18. Blocks found: 1.\nEvery line below is prefixed with its real line number in the user's file. Cite those numbers exactly; never invent one.\n\n 1 | alias: Hallway motion light\n 2 | description: Light the hallway when someone walks through after dark\n 3 | trigger:\n 4 |   - platform: state\n 5 |     entity_id: binary_sensor.hallway_motion\n 6 |     to: on\n...\n18 | mode: single",
 "context": "Aqara P1 motion sensor, hallway. It has never turned the light on once.",
 "ha_version": "2025.8",
 "dry_run": "DRY RUN of Hallway motion light at 21:40, sun below horizon, mode single\n\nTRIGGERS\n  T1 state: does not fire - ...\n\nVERDICT: it does not run - no trigger can match\n",
 "facts": { ... }}

Reply

{
  "task": "review",
  "title": "Hallway motion light - the state trigger compares against a boolean, so it never fires",
  "verdict": "reload-with-changes",
  "summary": "This automation cannot fire as written. On line 6, `to: on` is loaded as the boolean true, not the string \"on\", and binary_sensor.hallway_motion reports the string - so the trigger never matches and no action has ever run. Your dry run confirms it. Quote the value and it works; two smaller issues follow, in mode and in the service-call syntax.",
  "assumptions": [
    "binary_sensor.hallway_motion reports the strings \"on\" and \"off\"",
    "light.hallway exists - the digest proves it is referenced, not that it is real"
  ],
  "open_questions": [
    "Is the two-minute delay the whole hold time, or a minimum before a second motion event can re-arm?"
  ],
  "findings": [
    {"id":"HB-001","severity":"critical","line":6,"key":"trigger[0].to",
     "issue":"`to: on` is the boolean true, not the state string \"on\"",
     "why":"Home Assistant loads YAML through a PyYAML-derived SafeLoader, which implements YAML 1.1 resolution: plain `on`, `off`, `yes` and `no` are booleans. The trigger compares a state string against True, which never matches.",
     "fix":"Quote the value, and every other state string in the file.",
     "yaml_fix":"    to: \"on\""},
    {"id":"HB-002","severity":"medium","line":18,"key":"mode",
     "issue":"`mode: single` in front of a two-minute delay drops re-triggers",
     "why":"Motion in a hallway repeats. While the run sits in the delay a second trigger is dropped with a log warning, not queued - so the light goes off two minutes after the FIRST movement.",
     "fix":"Use `mode: restart` if the delay is a hold time.",
     "yaml_fix":"mode: restart"}
  ],
  "reconciliation": [
    {"flag_id":"HL-01","status":"confirmed","note":"Line 6 carries raw `on` resolving to the boolean true, and the dry run shows the non-match."},
    {"flag_id":"HL-02","status":"confirmed","note":"Both light service calls use the legacy top-level entity_id (lines 14, 17)."},
    {"flag_id":"HL-03","status":"confirmed","note":"mode: single with a 00:02:00 delay and a repeating trigger - re-triggers are dropped."},
    {"flag_id":"HL-04","status":"cleared","note":"Singular keys still load on 2025.8, so cosmetic rather than a defect; harden will fix it."}
  ],
  "next_lane": "harden",
  "body": {
    "reload_safety": "reload-with-changes",
    "will_it_fire": "No. Not once, in any world state. The trigger waits for binary_sensor.hallway_motion to become the boolean true, and a binary_sensor reports \"on\" or \"off\" as strings. Reloading is harmless because nothing runs; it is also pointless until line 6 is quoted.",
    "trigger_review": [
      {"ref":"T1","trigger_type":"state","verdict":"never-fires",
       "why":"`to` loaded as the boolean true (line 6), so the comparison can never succeed. There is also no `for:`, so once quoted this fires on every flicker of the sensor."}
    ],
    "condition_review": [
      {"ref":"C1","verdict":"passes",
       "why":"Elevation below 4 is the right shape for 'after dark' and is true at the hour the trigger would fire. Four rather than zero reads like a deliberate choice."}
    ],
    "action_review": [
      {"ref":"A1","verdict":"ok","why":"light.turn_on with a single entity target. No brightness, so the light comes up at whatever it was last set to."},
      {"ref":"A2","verdict":"risky","why":"A blocking 00:02:00 delay is what makes mode: single a problem, and a restart during it leaves the light on with nothing left to turn it off."},
      {"ref":"A3","verdict":"ok","why":"light.turn_off on the same entity - the step skipped if the run is interrupted."}
    ],
    "mode_note": "single. The YAML reads as 'hold the light for two minutes, extending on new motion', and that intent is `restart`. With single, a second motion event inside the window is dropped and logged as a warning while the light still goes off on the original schedule.",
    "loop_risk": "none",
    "loop_note": "No action writes to binary_sensor.hallway_motion or to any entity that triggers this, and nothing calls automation.trigger or homeassistant.reload. light.hallway is a target only (A1, A3) and appears in no trigger."
  }
}

One trigger_review entry per trigger in the digest, one condition_review per
condition, one action_review per action, each in document order, none skipped and
none added. `will_it_fire` answers with a word before it explains. `loop_risk` is
`likely` whenever self-trigger-loop is confirmed and the cycle is then traced by ref in
`loop_note`; `none` requires having actually looked, which is why the note names
what it checked. A reachability claim must cite a line.

review.body = {
  reload_safety:    "safe-to-reload" | "reload-with-changes" | "do-not-reload",
                    // repeats `verdict`, so a consumer that reads only `body` still gets the gate
  will_it_fire:     string,   // opens with a word a human can act on: "No.", "Yes.", "Only when..."
  trigger_review:   [{ref, trigger_type,
                      verdict: "fires" | "never-fires" | "fires-too-often" | "unclear", why}],
  condition_review: [{ref, verdict: "passes" | "blocks" | "unclear", why}],
                    // `blocks` = always false when the trigger fires, not merely false right now
  action_review:    [{ref, verdict: "ok" | "risky" | "broken", why}],
                    // `broken` will raise or do nothing; `risky` works and you may not want it to
  mode_note:        string,   // what the configured mode does to THIS automation, and which mode
                              // the apparent intent implies. Never a definition of the four modes
  loop_risk:        "none" | "possible" | "likely",   // forced to `likely` when self-trigger-loop fired
  loop_note:        string    // the cycle traced by ref, or what was checked when there is none
}

One entry in each of the three review arrays per corresponding digest entry, in
document order: none skipped, none added.

task: "explain" — Narrate it for someone who did not write it

Request:  {"task": "explain", "automation_digest": "<the numbered document>", "facts": { ... },
           "context": "My partner keeps asking why the hallway light does what it
                       does. Explain it to them, not to me.",
           "ha_version": "2025.8", "dry_run": ""}

Reply (the fields that differ):
{
  "task": "explain",
  "verdict": "has-surprises",
  "assumptions": ["No dry run was supplied, so the trigger analysis is static"],
  "body": {
    "one_liner": "It is meant to light the hallway for two minutes when someone walks through after dark - but a typo means it currently never runs.",
    "plain_english": "There are three parts to this, and they run in order.\n\nThe trigger wakes it up: the hallway motion sensor changing to 'motion detected'.\n\nThen one check before anything runs: the sun has to be more than a few degrees below the horizon. That is stricter than 'after sunset' - there is a stretch of dusk where motion will not turn the light on.\n\nIf the check passes: the light turns on, the automation waits exactly two minutes, then the light turns off. The waiting happens inside the automation, which matters below.\n\nAnd the part that is not on paper. On line 6 the value the trigger waits for was written without quotation marks, and Home Assistant reads an unquoted 'on' as a true/false value rather than as the word 'on'. The sensor sends the word. So the trigger waits for something that never arrives. Two quotation marks fix it and everything above becomes true.",
    "when_it_runs": [
      "Intended: whenever the motion sensor goes from clear to detecting, if the sun is more than 4 degrees below the horizon",
      "Actually, today: never - the trigger cannot match (line 6)",
      "Not on a schedule, and not when the light is switched on by hand"
    ],
    "what_it_does": [
      "Turns light.hallway on, at whatever brightness it was last left at",
      "Waits two minutes with the run held open",
      "Turns light.hallway off again"
    ],
    "surprises": [
      {"situation":"Somebody walks through the hallway after dark, today",
       "behaviour":"Nothing happens. This is the line-6 problem, and it is the whole of why the automation looks broken."},
      {"situation":"Once fixed: somebody is still there when the two minutes are nearly up, and moves again",
       "behaviour":"The light still goes off on the original schedule. The second motion event does not extend anything - the mode is `single`, so it is discarded while the first run is waiting."},
      {"situation":"Once fixed: Home Assistant restarts during the two-minute wait",
       "behaviour":"The light stays on indefinitely. The step that turns it off lived inside the interrupted run."},
      {"situation":"Someone turns the light on at the wall switch, then walks past the sensor",
       "behaviour":"The light is turned on again (no visible change) and then switched off underneath them two minutes later. It does not check who turned the light on."}
    ],
    "housemate_note": "The hallway light is supposed to come on by itself when you walk through after dark and go off two minutes later. Right now it does not work at all - there is a one-character fix pending. Once fixed: if you stand in the hallway longer than two minutes it will go dark on you, and if you switch the light on by hand and then walk past the sensor it will switch itself off a couple of minutes later. Neither is a fault."
  }
}

`plain_english` is prose for a person who does not edit YAML: no key names, no
refs, no line numbers except where naming one is the fix. `one_liner` is under 140
characters. `surprises` is the point of this lane - each entry pairs a concrete
situation with the behaviour that will actually happen, ordered by how likely
somebody is to hit it, and a mode or restart surprise is always included when the
automation has a blocking wait. `housemate_note` is pasteable into a message to
somebody who lives in the house: no jargon, no editor instructions.

explain.body = {
  one_liner:      string,     // under 140 chars: the purpose, plus the headline defect if any
  plain_english:  string,     // multi-paragraph, newline-separated. Trigger, then conditions,
                              // then actions, then anything that contradicts that reading.
                              // No YAML keys, no refs
  when_it_runs:   [string],   // every path in. Where intent and behaviour differ, both, labelled
  what_it_does:   [string],   // one per observable effect, in execution order. Waits count
  surprises:      [{situation, behaviour}],
                              // concrete situations, not categories. Where mode semantics,
                              // restart survival and manual-override interaction land
  housemate_note: string      // pasteable into a chat with someone who does not administer the
                              // system: no entity ids, no line numbers, no advice
}

task: "harden" — Rewrite it properly, and list what changed

This is the lane where ha_version earns its place in the input. The same automation gets three different rewrites depending on what will load it, and none of the three is wrong YAML.

Request:  {"task": "harden", "automation_digest": "<the numbered document>", "facts": { ... },
           "context": "Aqara P1 motion sensor. I want the light to stay on while
                       there is still movement.",
           "ha_version": "2025.8", "dry_run": "world:\n  ...\nresult: did not fire\n"}

Reply (the fields that differ):
{
  "task": "harden",
  "verdict": "rewritten-with-assumptions",
  "assumptions": [
    "The two-minute delay was a hold time, not a rate limit - your context says the light should stay on while there is movement, so mode: restart is the right reading",
    "The instance is on 2025.8, as ha_version said: the plural keys below will not load on anything before 2024.10"
  ],
  "next_lane": "card",
  "body": {
    "posture": "needs-syntax-update",
    "changes": [
      {"order":1,"change":"Quote the trigger's `to` value: `to: \"on\"`","line":6,"kind":"semantics",
       "reason":"Unquoted `on` loads as the boolean true under PyYAML's YAML 1.1 resolution. This is the change that makes the automation work at all; the others improve something that already ran."},
      {"order":2,"change":"Rename the section keys to their plural forms, and `platform:` to `trigger:`","line":3,"kind":"syntax",
       "reason":"2024.10 introduced the plural keys and they are what the UI editor writes. The singular forms still load, so this is not urgent - but a file that mixes both is a file where nobody can tell which convention it follows."},
      {"order":3,"change":"Replace `service:` with `action:` and move entity_id under `target:`","line":13,"kind":"syntax",
       "reason":"`action:` is the 2024.8 spelling of a service call and `target:` is the documented way to aim one. Both old forms still load, and both are what the editor rewrites away the moment you open the automation in the UI."},
      {"order":4,"change":"Change mode from `single` to `restart`, and add `for: \"00:00:02\"` to the trigger","line":18,"kind":"semantics",
       "reason":"With `single`, a second motion event during the delay is dropped and the light goes off on the first run's schedule; with `restart` each new event starts a fresh two-minute hold. The two-second settle stops a flickering PIR from restarting the run several times a second. This is a behaviour change, and it is the one to be sure about."}
    ],
    "rewritten_yaml": "alias: Hallway motion light\ndescription: Light the hallway when someone walks through after dark\ntriggers:\n  - trigger: state\n    entity_id: binary_sensor.hallway_motion\n    to: \"on\"\n    for: \"00:00:02\"\nconditions:\n  - condition: numeric_state\n    entity_id: sun.sun\n    attribute: elevation\n    below: 4\nactions:\n  - action: light.turn_on\n    target:\n      entity_id: light.hallway\n  - delay: \"00:02:00\"\n  - action: light.turn_off\n    target:\n      entity_id: light.hallway\nmode: restart\n",
    "kept_deliberately": [
      "The sun condition stays at `below: 4` rather than becoming `condition: sun, after: sunset`. Elevation 4 is a stricter, more predictable test and reads like a choice somebody made on purpose.",
      "The blocking `delay:` stays a delay. A `wait_for_trigger` on the sensor going \"off\" would be more elegant and would also leave the light on forever if the sensor drops off the network.",
      "No brightness was added to light.turn_on: that would change what the light looks like, which is not a hardening decision to make on your behalf.",
      "Durations are quoted. `00:02:00` happens to load as a string because of the leading zero, but `0:02:00` would not, and quoting every duration removes the question."
    ],
    "verify_steps": [
      "Developer Tools -> YAML -> check configuration, before reloading anything. The plural keys are a load-time error on anything before 2024.10, and this is where you find that out.",
      "Developer Tools -> States: confirm binary_sensor.hallway_motion reads on (a string) rather than True. That is the assumption change 1 rests on.",
      "Reload automations, then walk past the sensor after dark. The light should come on within two seconds and go off two minutes after the LAST movement.",
      "Automation -> Traces: a second walk-through inside the window should produce a new trace rather than a dropped-run warning in the log."
    ]
  }
}

`rewritten_yaml` is the WHOLE automation, always - including on `already-modern`,
where it comes back unchanged so that pasting it can never be a downgrade. It is a
JSON string with newlines escaped, it targets the `ha_version` that was sent, and
it contains no placeholder or ellipsis: a rewrite you cannot paste is not a
rewrite. Every entry in `changes` maps to something visibly different in that
YAML, `order` is the order to apply them in, and a `semantics` change - one that
alters behaviour rather than spelling - always says so in `reason` and gets a
matching entry in `assumptions`. `kept_deliberately` exists to stop the lane from
tidying: it is where the things a rewrite could have changed and should not go.

harden.body = {
  posture:           "already-modern" | "needs-syntax-update" | "needs-rework",
                     // orthogonal to `verdict`: posture describes the input, verdict the rewrite
  changes:           [{order, change, reason, line,
                       kind: "syntax" | "semantics" | "robustness" | "style"}],
                     // `order` is the application order, behaviour-changing edits last, so a
                     // partial apply cannot leave a half-changed automation
  rewritten_yaml:    string,   // the whole automation, pasteable, no ellipses, targeting
                               // ha_version. Never empty - unchanged on `already-modern`
  kept_deliberately: [string], // what was left alone and why; guards against silent restyling
  verify_steps:      [string]  // things that could actually fail, in order, starting with a
                               // check-configuration before any reload
}

task: "card" — Build the dashboard card for it

Request:  {"task": "card", "automation_digest": "<the numbered document>", "facts": { ... },
           "context": "I want to see on the dashboard whether this thing is armed
                       and when it last ran.",
           "ha_version": "2025.8", "dry_run": ""}

Reply (the fields that differ):
{
  "task": "card",
  "verdict": "card-with-placeholders",
  "summary": "Three entities came out of the automation itself - the motion sensor, the hallway light and sun.sun - and those are real. The automation's own entity id is not in the digest, because it comes from a unique_id and this file has none, so automation.hallway_motion_light is a PLACEHOLDER: check it in Developer Tools before pasting.",
  "findings": [
    {"id":"HB-001","severity":"medium","line":1,"key":"unique_id",
     "issue":"The automation has no unique_id, so its entity id is generated from the alias",
     "why":"Without unique_id, renaming the alias changes the entity id, and every card and script pointing at the old id breaks silently. It also means this card cannot be certain of the id it needs.",
     "fix":"Add a unique_id. Any stable string will do; it is never shown.",
     "yaml_fix":"unique_id: hallway_motion_light_v1"}
  ],
  "next_lane": "",
  "body": {
    "card_kind": "entities",
    "cards": [
      {"title":"Hallway motion light",
       "purpose":"The one card to keep: arm/disarm, last run, live sensor, manual trigger.",
       "yaml":"type: entities\ntitle: Hallway motion light\nshow_header_toggle: false\nentities:\n  - entity: automation.hallway_motion_light   # PLACEHOLDER - confirm this id\n    name: Automation armed\n  - type: attribute\n    entity: automation.hallway_motion_light   # PLACEHOLDER - confirm this id\n    attribute: last_triggered\n    name: Last ran\n  - entity: binary_sensor.hallway_motion\n    name: Motion now\n  - entity: light.hallway\n    name: Hallway light\n  - type: call-service\n    name: Run it now\n    action_name: Trigger\n    service: automation.trigger\n    service_data:\n      entity_id: automation.hallway_motion_light\n"},
      {"title":"Why it is not firing",
       "purpose":"A diagnostic card, worth keeping only until the trigger is fixed. The elevation row is the condition that gates the automation, so a dark hallway with a positive elevation is answered here rather than in the logs.",
       "yaml":"type: entities\ntitle: Hallway automation - diagnostics\nentities:\n  - type: attribute\n    entity: sun.sun\n    attribute: elevation\n    name: Sun elevation (needs to be below 4)\n  - entity: binary_sensor.hallway_motion\n    name: Motion sensor state\n    secondary_info: last-changed\n  - entity: light.hallway\n    name: Light state\n    secondary_info: last-changed\n"},
      {"title":"Compact tile version",
       "purpose":"One tile for a grid instead of a list. Same arm toggle, no last-run row.",
       "yaml":"type: tile\nentity: automation.hallway_motion_light   # PLACEHOLDER\nname: Hallway motion\nstate_content:\n  - state\n  - last_triggered\nfeatures:\n  - type: toggle\n"}
    ],
    "view_yaml": "title: Hallway\npath: hallway\ntype: sections\nsections:\n  - type: grid\n    cards:\n      - type: heading\n        heading: Hallway motion light\n      - type: entities\n        show_header_toggle: false\n        entities:\n          - entity: automation.hallway_motion_light   # PLACEHOLDER\n            name: Automation armed\n          - entity: binary_sensor.hallway_motion\n            name: Motion now\n          - entity: light.hallway\n            name: Hallway light\n",
    "entities_used": ["binary_sensor.hallway_motion", "light.hallway", "sun.sun"],
    "missing_entities": [
      "automation.hallway_motion_light - the automation's own entity. Not in the digest: an automation's entity id comes from its unique_id, and this automation has none, so the id is generated from the alias slug at first load. Confirm it under Developer Tools -> States before pasting, or add a unique_id and set the id yourself."
    ],
    "placement_note": "The entities card is a list, so it wants a full column rather than a grid slot; in a sections view give it its own grid section with a heading above it. Use the tile version if you already have a row of tiles. Keep the diagnostics card on a separate admin view - it is a card you will want twice a year, and it is the kind that ends up load-bearing if it lives somewhere prominent."
  }
}

Every entity id in a card must come from `facts.entities` or be listed in
`missing_entities` with a reason - a plausible-looking guess like light.hallway_2
is the one thing this lane must never do. Placeholders appear in the YAML with an
inline PLACEHOLDER comment so a paste that was not checked is visible in the
dashboard editor. `cards` is ordered most-useful-first and each entry says in
`purpose` what question it answers. `view_yaml` is a complete view containing the
first card, for somebody starting a dashboard from nothing. Card types are limited
to what the target `ha_version` ships - no custom cards from HACS, ever, because a
card that needs an install is not a card that pastes.

card.body = {
  card_kind:        string,    // the primary card type, naming cards[0]: "entities" | "tile" |
                              // "vertical-stack" | "conditional" | "history-graph" | "custom:..."
                              // A custom card must name its HACS repo in placement_note
                              // and be called a dependency
  cards:            [{title, purpose, yaml}],
                    // most useful first; `yaml` is a complete card, pasteable into the raw editor
  view_yaml:        string,    // a whole view wrapping cards[0], for a dashboard that does not
                               // exist yet. "" when a view would add nothing
  entities_used:    [string],  // only ids in facts.entities, plus the automation's own
                              // automation.<object_id> when alias or id makes it derivable.
                              // The app audits this
  missing_entities: [string],  // every id in the YAML not in facts.entities, each with why it
                               // was invented and how to confirm it. Non-empty forces the
                               // verdict card-with-placeholders. Fewer than two entities in
                               // facts.entities forces not-enough-entities instead
  placement_note:   string     // where on a dashboard these belong, and in what kind of view
}

Step by step

# Every call below uses these two. The token comes from step 2.
BASE="https://api.skillsafe.ai/v1/app-api"
TOKEN="YOUR_TOKEN"

import json, os, urllib.request

BASE  = "https://api.skillsafe.ai/v1/app-api"
TOKEN = os.environ.get("HASS_BENCH_TOKEN", "YOUR_TOKEN")

def call(method, path, body=None, headers=None):
    data = json.dumps(body).encode() if body is not None else None
    req = urllib.request.Request(BASE + path, data=data, method=method)
    req.add_header("Authorization", "Bearer " + TOKEN)
    if data:
        req.add_header("Content-Type", "application/json")
    for k, v in (headers or {}).items():
        req.add_header(k, v)
    with urllib.request.urlopen(req) as r:
        payload = json.loads(r.read())
    if not payload.get("ok"):
        raise RuntimeError(payload["error"]["code"] + ": " + payload["error"]["message"])
    return payload["data"]

const BASE = "https://api.skillsafe.ai/v1/app-api";
const TOKEN = "YOUR_TOKEN";

async function call(method, path, body, extraHeaders) {
  const res = await fetch(BASE + path, {
    method,
    headers: {
      "Authorization": `Bearer ${TOKEN}`,
      ...(body ? { "Content-Type": "application/json" } : {}),
      ...(extraHeaders || {})
    },
    body: body ? JSON.stringify(body) : undefined
  });
  const payload = await res.json();
  if (!payload.ok) throw new Error(`${payload.error.code}: ${payload.error.message}`);
  return payload.data;
}

package main

import (
    "bytes"
    "encoding/json"
    "errors"
    "io"
    "net/http"
    "os"
)

const base = "https://api.skillsafe.ai/v1/app-api"

// Falls back to the placeholder so the snippet runs as written.
func token() string {
    if t := os.Getenv("HASS_BENCH_TOKEN"); t != "" {
        return t
    }
    return "YOUR_TOKEN"
}

type envelope struct {
    OK    bool            `json:"ok"`
    Data  json.RawMessage `json:"data"`
    Error *struct {
        Code    string `json:"code"`
        Message string `json:"message"`
    } `json:"error"`
}

func call(method, path string, body any, extra map[string]string) (json.RawMessage, error) {
    var rdr io.Reader
    if body != nil {
        b, _ := json.Marshal(body)
        rdr = bytes.NewReader(b)
    }
    req, _ := http.NewRequest(method, base+path, rdr)
    req.Header.Set("Authorization", "Bearer "+token())
    if body != nil {
        req.Header.Set("Content-Type", "application/json")
    }
    for k, v := range extra {
        req.Header.Set(k, v)
    }
    res, err := http.DefaultClient.Do(req)
    if err != nil {
        return nil, err
    }
    defer res.Body.Close()
    var env envelope
    if err := json.NewDecoder(res.Body).Decode(&env); err != nil {
        return nil, err
    }
    if !env.OK {
        return nil, errors.New(env.Error.Code + ": " + env.Error.Message)
    }
    return env.Data, nil
}

import java.net.URI;
import java.net.http.*;
import java.util.Map;

public class HassBench {
    static final String BASE = "https://api.skillsafe.ai/v1/app-api";
    static final String TOKEN =
        System.getenv("HASS_BENCH_TOKEN") != null ? System.getenv("HASS_BENCH_TOKEN")
                                                  : "YOUR_TOKEN";
    static final HttpClient CLIENT = HttpClient.newHttpClient();

    static String call(String method, String path, String jsonBody,
                       Map<String, String> extra) throws Exception {
        HttpRequest.Builder b = HttpRequest.newBuilder()
            .uri(URI.create(BASE + path))
            .header("Authorization", "Bearer " + TOKEN);
        if (jsonBody == null) {
            b.method(method, HttpRequest.BodyPublishers.noBody());
        } else {
            b.header("Content-Type", "application/json")
             .method(method, HttpRequest.BodyPublishers.ofString(jsonBody));
        }
        if (extra != null) extra.forEach(b::header);
        HttpResponse<String> res = CLIENT.send(b.build(), HttpResponse.BodyHandlers.ofString());
        return res.body(); // parse with your JSON library; check "ok" before "data"
    }
}

require "json"
require "net/http"
require "uri"

BASE  = "https://api.skillsafe.ai/v1/app-api"
TOKEN = ENV.fetch("HASS_BENCH_TOKEN", "YOUR_TOKEN")

def call(method, path, body = nil, extra = {})
  uri = URI(BASE + path)
  req = (method == "GET" ? Net::HTTP::Get : Net::HTTP::Post).new(uri)
  req["Authorization"] = "Bearer #{TOKEN}"
  if body
    req["Content-Type"] = "application/json"
    req.body = JSON.generate(body)
  end
  extra.each { |k, v| req[k] = v }
  res = Net::HTTP.start(uri.hostname, uri.port, use_ssl: true) { |h| h.request(req) }
  payload = JSON.parse(res.body)
  raise "#{payload["error"]["code"]}: #{payload["error"]["message"]}" unless payload["ok"]
  payload["data"]
end

<?php
const BASE = "https://api.skillsafe.ai/v1/app-api";
define("TOKEN", getenv("HASS_BENCH_TOKEN") ?: "YOUR_TOKEN");

function call(string $method, string $path, ?array $body = null, array $extra = []): array {
    $headers = ["Authorization: Bearer " . TOKEN];
    $opts = ["http" => ["method" => $method, "ignore_errors" => true]];
    if ($body !== null) {
        $headers[] = "Content-Type: application/json";
        $opts["http"]["content"] = json_encode($body);
    }
    foreach ($extra as $k => $v) { $headers[] = "$k: $v"; }
    $opts["http"]["header"] = implode("\r\n", $headers);
    $raw = file_get_contents(BASE . $path, false, stream_context_create($opts));
    $payload = json_decode($raw, true);
    if (empty($payload["ok"])) {
        throw new RuntimeException($payload["error"]["code"] . ": " . $payload["error"]["message"]);
    }
    return $payload["data"];
}

using System.Net.Http;
using System.Net.Http.Headers;
using System.Text;
using System.Text.Json;

static class HassBench {
    const string Base = "https://api.skillsafe.ai/v1/app-api";
    static readonly string Token =
        Environment.GetEnvironmentVariable("HASS_BENCH_TOKEN") ?? "YOUR_TOKEN";
    static readonly HttpClient Client = new HttpClient();

    public static async Task<JsonElement> Call(HttpMethod method, string path,
            object? body = null, Dictionary<string, string>? extra = null) {
        var req = new HttpRequestMessage(method, Base + path);
        req.Headers.Authorization = new AuthenticationHeaderValue("Bearer", Token);
        if (body != null) {
            req.Content = new StringContent(JsonSerializer.Serialize(body),
                                            Encoding.UTF8, "application/json");
        }
        if (extra != null) foreach (var kv in extra) req.Headers.Add(kv.Key, kv.Value);
        var res = await Client.SendAsync(req);
        using var doc = JsonDocument.Parse(await res.Content.ReadAsStringAsync());
        var root = doc.RootElement;
        if (!root.GetProperty("ok").GetBoolean()) {
            var err = root.GetProperty("error");
            throw new Exception(err.GetProperty("code").GetString() + ": "
                              + err.GetProperty("message").GetString());
        }
        return root.GetProperty("data").Clone();
    }
}

curl -s -X POST "$BASE/guest" \
  -H "Content-Type: application/json" \
  -d '{"slug":"hass-bench"}'
# -> {"ok":true,"data":{"token":"aut_...","guest_id":"gst_..."}}
# This token can call /me and /estimate. /run returns 403 forbidden.

req = urllib.request.Request(
    "https://api.skillsafe.ai/v1/app-api/guest",
    data=json.dumps({"slug": "hass-bench"}).encode(),
    headers={"Content-Type": "application/json"},
    method="POST")
with urllib.request.urlopen(req) as r:
    TOKEN = json.loads(r.read())["data"]["token"]   # /me and /estimate only

const res = await fetch("https://api.skillsafe.ai/v1/app-api/guest", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ slug: "hass-bench" })
});
const TOKEN = (await res.json()).data.token;   // /run needs a personal token

b, _ := json.Marshal(map[string]string{"slug": "hass-bench"})
res, _ := http.Post(base+"/guest", "application/json", bytes.NewReader(b))
defer res.Body.Close()
// decode into envelope, then env.Data -> {"token": "...", "guest_id": "..."}

String json = call("POST", "/guest", "{\"slug\":\"hass-bench\"}", null);
// json.data.token is your guest token

uri = URI("https://api.skillsafe.ai/v1/app-api/guest")
res = Net::HTTP.post(uri, JSON.generate({ "slug" => "hass-bench" }),
                     "Content-Type" => "application/json")
guest_token = JSON.parse(res.body)["data"]["token"]

<?php
$token = call("POST", "/guest", ["slug" => "hass-bench"])["token"];

var guest = await Call(HttpMethod.Post, "/guest",
                       new Dictionary<string, string> { ["slug"] = "hass-bench" });
var token = guest.GetProperty("token").GetString();

curl -s "$BASE/me" -H "Authorization: Bearer $TOKEN"
# -> {"ok":true,"data":{"subject_type":"user","credits":48210,...}}
# subject_type "guest" here means /run will 403 no matter what you send.

me = call("GET", "/me")
print(me["subject_type"], me["credits"])
if me["subject_type"] != "user":
    print("guest token: /me and /estimate only, no runs")

const me = await call("GET", "/me");
console.log(me.subject_type, me.credits);
if (me.subject_type !== "user") console.warn("guest token: no runs");

data, err := call("GET", "/me", nil, nil)
if err != nil {
    panic(err)
}
fmt.Println(string(data))

System.out.println(call("GET", "/me", null, null));

me = call("GET", "/me")
puts "#{me["subject_type"]} #{me["credits"]}"

<?php
$me = call("GET", "/me");
echo $me["subject_type"], " ", $me["credits"], "\n";

var me = await Call(HttpMethod.Get, "/me");
Console.WriteLine(me.GetProperty("subject_type").GetString());

# input.json is the whole input object: task, automation_digest (the
# line-numbered document, a STRING), facts (an OBJECT), context, ha_version,
# dry_run. No {"input": ...} wrapper.
curl -s -X POST "$BASE/estimate" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d @input.json
# -> {"ok":true,"data":{"model":"gpt-5.6-terra","model_alias":"gpt-terra",
#      "markup_bps":1000,"hold_credits":4180,"min_credits":260,"sponsor_enabled":false}}

digest = build_digest(parsed)   # redacted, line-numbered document text
facts  = build_facts(parsed)    # counts, confidence, automations[], entities[]

INPUT = {
    "task": "review",
    "automation_digest": digest,                # the numbered document text
    "facts": facts,                            # an OBJECT, not a string
    "context": "Aqara P1 motion sensor. It has never turned the light on.",
    "ha_version": "2025.8",
    "dry_run": transcript,                     # or "" if you did not simulate
}

est = call("POST", "/estimate", INPUT)
print(est["model_alias"], est["hold_credits"], est["min_credits"])

# Free, and it creates no job. Re-estimate per lane: hold_credits differs.
for lane in ("review", "explain", "harden", "card"):
    print(lane, call("POST", "/estimate", dict(INPUT, task=lane))["hold_credits"])

const INPUT = {
  task: "review",
  automation_digest: digest,                   // the numbered document text
  facts,                                       // an OBJECT, not a string
  context: "Aqara P1 motion sensor. It has never turned the light on.",
  ha_version: "2025.8",
  dry_run: transcript                          // or "" if you did not simulate
};

const est = await call("POST", "/estimate", INPUT);
console.log(est.model_alias, est.hold_credits, est.min_credits);
// hold_credits is a reservation. You pay charged_credits, from the run.

input := map[string]any{
    "task":              "review",
    "automation_digest": digest, // the numbered document text
    "facts":             facts, // an object
    "context":           "Aqara P1 motion sensor. It has never turned the light on.",
    "ha_version":        "2025.8",
    "dry_run":           transcript,
}

data, err := call("POST", "/estimate", input, nil)
if err != nil {
    panic(err)
}
fmt.Println(string(data))

String inputJson = mapper.writeValueAsString(Map.of(
    "task", "review",
    "automation_digest", digest,
    "facts", facts,
    "context", "Aqara P1 motion sensor. It has never turned the light on.",
    "ha_version", "2025.8",
    "dry_run", transcript));

System.out.println(call("POST", "/estimate", inputJson, null));

input = {
  "task"              => "review",
  "automation_digest" => digest,   # the numbered document text
  "facts"             => facts,                   # a Hash, not a String
  "context"           => "Aqara P1 motion sensor. It has never turned the light on.",
  "ha_version"        => "2025.8",
  "dry_run"           => transcript
}

est = call("POST", "/estimate", input)
puts "#{est["model_alias"]} #{est["hold_credits"]} #{est["min_credits"]}"

<?php
$input = [
    "task"              => "review",
    "automation_digest" => $digest,   // the numbered document text
    "facts"             => $facts,                 // an array -> JSON object
    "context"           => "Aqara P1 motion sensor. It has never turned the light on.",
    "ha_version"        => "2025.8",
    "dry_run"           => $transcript,
];

$est = call("POST", "/estimate", $input);
echo $est["hold_credits"], " ", $est["min_credits"], "\n";

var input = new Dictionary<string, object> {
    ["task"]              = "review",
    ["automation_digest"] = digest,   // the numbered document text
    ["facts"]             = facts,                              // an object
    ["context"]           = "Aqara P1 motion sensor.",
    ["ha_version"]        = "2025.8",
    ["dry_run"]           = transcript
};

var est = await Call(HttpMethod.Post, "/estimate", input);
Console.WriteLine(est.GetProperty("hold_credits").GetInt32());

# 1. submit. The lane is in the key, so running the next lane over the same
#    input.json cannot replay this job's answer.
JOB=$(curl -s -X POST "$BASE/run" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -H "Idempotency-Key: hass-bench:review:$(shasum -a 256 input.json | cut -c1-16):a1" \
  -d @input.json | python3 -c 'import json,sys;print(json.load(sys.stdin)["data"]["job_id"])')

# 2. poll to terminal
until curl -s "$BASE/jobs/$JOB" -H "Authorization: Bearer $TOKEN" \
  | tee /dev/stderr | grep -q '"status":"succeeded"'; do sleep 2; done

import hashlib, time

# The lane MUST be in the key: two lanes over one automation are two runs.
h = hashlib.sha256(json.dumps({k: INPUT[k] for k in
    ("automation_digest", "context", "ha_version", "dry_run") if k in INPUT},
    sort_keys=True).encode()).hexdigest()[:16]
key = "hass-bench:" + INPUT["task"] + ":" + h + ":a1"

job_id = call("POST", "/run", INPUT, {"Idempotency-Key": key})["job_id"]

while True:
    job = call("GET", "/jobs/" + job_id)
    if job["status"] in ("succeeded", "failed", "cancelled"):
        break
    time.sleep(2)

result = json.loads(job["output"]["output"])   # the envelope from the contract
print(job["status"], job.get("charged_credits"), result["verdict"])

# The three checks the web app makes on every reply, and you should too.
assert result["task"] == INPUT["task"], ("wrong lane - input wrapper?", result["task"])

sent = {f["flag_id"] for f in INPUT["facts"]["browser_findings"]}
got  = [r["flag_id"] for r in result["reconciliation"]]
assert sent == set(got) and len(got) == len(sent), "unreconciled flags"

known = {s.split(":")[0] for a in facts["automations"]
         for s in a["triggers"] + a["conditions"] + a["action_steps"]}
quoted = {e.get("ref") for k in ("trigger_review", "condition_review", "action_review")
          for e in result["body"].get(k, [])}
print("fabricated refs:", sorted(r for r in quoted if r and r not in known))
print("bad lines:", [f["line"] for f in result["findings"]
                     if f.get("line", 0) > facts["line_count"]])

const key = `hass-bench:${INPUT.task}:${hash(JSON.stringify({
  automation_digest: INPUT.automation_digest, context: INPUT.context,
  ha_version: INPUT.ha_version, dry_run: INPUT.dry_run
}))}:a1`;

const { job_id } = await call("POST", "/run", INPUT, { "Idempotency-Key": key });

let job;
do {
  await new Promise(r => setTimeout(r, 2000));
  job = await call("GET", `/jobs/${job_id}`);
} while (!["succeeded", "failed", "cancelled"].includes(job.status));

const result = JSON.parse(job.output.output);
console.log(job.status, job.charged_credits, result.verdict);

// Audit before you render anything.
if (result.task !== INPUT.task) throw new Error("wrong lane - input wrapper?");

const sent = new Set(INPUT.facts.browser_findings.map(f => f.flag_id));
const got  = result.reconciliation.map(r => r.flag_id);
if (got.length !== sent.size || got.some(id => !sent.has(id))) {
  console.warn("reconciliation does not match the flags sent", got);
}

const known = new Set(INPUT.facts.automations.flatMap(a =>
  [...a.triggers, ...a.conditions, ...a.action_steps].map(s => s.split(":")[0])));
const quoted = ["trigger_review", "condition_review", "action_review"]
  .flatMap(k => (result.body[k] || []).map(e => e.ref));
console.log("fabricated refs:", quoted.filter(r => r && !known.has(r)));

b, _ := json.Marshal(input)
extra := map[string]string{
    // "review" here is input["task"]. Never a constant that outlives the lane.
    "Idempotency-Key": "hass-bench:review:" + hash(b) + ":a1",
}
data, err := call("POST", "/run", input, extra)
if err != nil {
    panic(err)
}
// read data.job_id, then GET /jobs/{id} every 2s until status is terminal,
// then json.Unmarshal(data.output.output) and check: result.task == the lane
// you sent, one reconciliation per flag, every ref resolves, every line
// <= facts.line_count.

String key = "hass-bench:review:" + hash(inputJson) + ":a1";
String submitted = call("POST", "/run", inputJson,
                        Map.of("Idempotency-Key", key));
// read data.job_id, poll GET /jobs/{id} until terminal, then parse
// data.output.output and verify result.task is the lane you asked for.

key = "hass-bench:#{input["task"]}:#{hash(input)}:a1"
job_id = call("POST", "/run", input, { "Idempotency-Key" => key })["job_id"]

job = nil
loop do
  job = call("GET", "/jobs/#{job_id}")
  break if %w[succeeded failed cancelled].include?(job["status"])
  sleep 2
end

result = JSON.parse(job["output"]["output"])
raise "wrong lane: #{result["task"]}" unless result["task"] == input["task"]
puts "#{result["verdict"]} charged=#{job["charged_credits"]}"

<?php
$key = "hass-bench:" . $input["task"] . ":"
     . substr(hash("sha256", json_encode($input)), 0, 16) . ":a1";

$jobId = call("POST", "/run", $input, ["Idempotency-Key" => $key])["job_id"];
do {
    sleep(2);
    $job = call("GET", "/jobs/" . $jobId);
} while (!in_array($job["status"], ["succeeded", "failed", "cancelled"], true));

$result = json_decode($job["output"]["output"], true);
if ($result["task"] !== $input["task"]) {
    throw new RuntimeException("wrong lane - check for an input wrapper");
}
echo $result["verdict"], "\n";

var key = $"hass-bench:{input["task"]}:{Hash(input)}:a1";
var run = await Call(HttpMethod.Post, "/run", input,
                     new Dictionary<string, string> { ["Idempotency-Key"] = key });
var jobId = run.GetProperty("job_id").GetString();

JsonElement job;
do {
    await Task.Delay(2000);
    job = await Call(HttpMethod.Get, "/jobs/" + jobId);
} while (job.GetProperty("status").GetString() is not ("succeeded" or "failed" or "cancelled"));

using var parsed = JsonDocument.Parse(
    job.GetProperty("output").GetProperty("output").GetString()!);
Console.WriteLine(parsed.RootElement.GetProperty("verdict").GetString());
// Then: result.task == the lane you sent, one reconciliation per flag, refs resolve.

curl -N -s -X POST "$BASE/run-stream" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -H "Idempotency-Key: hass-bench:review:abc123def4567890:a1" \
  -d @input.json
# event: job     data: {"job_id":"job_..."}
# event: delta   data: {"text":"{\"task\":\"review\","}
# event: done    data: {"job_id":"job_...","status":"succeeded",
#                       "output":{"output":"..."},"charged_credits":1140}

req = urllib.request.Request(BASE + "/run-stream",
                             data=json.dumps(INPUT).encode(), method="POST")
req.add_header("Authorization", "Bearer " + TOKEN)
req.add_header("Content-Type", "application/json")
req.add_header("Idempotency-Key", key)

buf, final, event = "", None, None
with urllib.request.urlopen(req) as r:
    for raw in r:
        line = raw.decode().rstrip("\n")
        if line.startswith("event:"):
            event = line[6:].strip()
        elif line.startswith("data:"):
            payload = json.loads(line[5:].strip())
            if event == "delta":
                buf += payload.get("text", "")
            elif event in ("done", "pending"):
                final = payload
            elif event == "error":
                raise RuntimeError(payload["code"] + ": " + payload["message"])

# Prefer the done payload: a replayed run sends no deltas at all.
text = final["output"]["output"] if final and final.get("output") else buf
result = json.loads(text)
print(result["verdict"], final.get("charged_credits"), final.get("truncated"))

const res = await fetch(`${BASE}/run-stream`, {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${TOKEN}`,
    "Content-Type": "application/json",
    "Idempotency-Key": key
  },
  body: JSON.stringify(INPUT)
});

const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = "", text = "", final = null;

for (;;) {
  const { done, value } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });
  let idx;
  while ((idx = buffer.indexOf("\n\n")) >= 0) {
    const frame = buffer.slice(0, idx);
    buffer = buffer.slice(idx + 2);
    let event = "message", dataStr = "";
    for (const line of frame.split("\n")) {
      if (line.startsWith("event:")) event = line.slice(6).trim();
      else if (line.startsWith("data:")) dataStr += line.slice(5).trim();
    }
    if (!dataStr) continue;
    const payload = JSON.parse(dataStr);
    if (event === "delta") {
      text += payload.text || "";
      // Cheap, honest progress: the keys arrive in contract order.
      if (text.includes('"reconciliation"')) setStage("reconciling");
      else if (text.includes('"findings"')) setStage("findings");
      else if (text.includes('"verdict"')) setStage("verdict");
    } else if (event === "done" || event === "pending") {
      final = payload;
    } else if (event === "error") {
      throw new Error(`${payload.code}: ${payload.message}`);
    }
  }
}

const result = JSON.parse(final?.output?.output ?? text);
console.log(result.verdict, final?.charged_credits, final?.truncated);

req, _ := http.NewRequest("POST", base+"/run-stream", bytes.NewReader(b))
req.Header.Set("Authorization", "Bearer "+token())
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Idempotency-Key", "hass-bench:review:"+hash(b)+":a1")
res, _ := http.DefaultClient.Do(req)
defer res.Body.Close()

sc := bufio.NewScanner(res.Body)
sc.Buffer(make([]byte, 0, 64*1024), 1024*1024) // deltas can be long
for sc.Scan() {
    line := sc.Text()
    // "event: delta" then "data: {...}" - accumulate payload.text.
    // On "event: done", payload.output.output is authoritative.
    _ = line
}

HttpResponse<java.util.stream.Stream<String>> res =
    CLIENT.send(streamRequest, HttpResponse.BodyHandlers.ofLines());
StringBuilder text = new StringBuilder();
res.body().forEach(line -> {
    // "event: delta" then "data: {...}" - append the "text" field.
    // On "done", prefer data.output.output over the accumulated text.
});

Net::HTTP.start(uri.hostname, uri.port, use_ssl: true) do |http|
  http.request(stream_req) do |res|
    res.read_body do |chunk|
      # split on "\n\n", read "event:" then "data:" lines,
      # accumulate delta text, keep the done payload
    end
  end
end

<?php
$stream = fopen(BASE . "/run-stream", "r", false, stream_context_create($opts));
while (($line = fgets($stream)) !== false) {
    // "event: delta" then "data: {...}"; on "done" take output.output
}
fclose($stream);

using var res = await Client.SendAsync(streamReq, HttpCompletionOption.ResponseHeadersRead);
using var reader = new StreamReader(await res.Content.ReadAsStreamAsync());
string? line;
while ((line = await reader.ReadLineAsync()) != null) {
    // "event: delta" then "data: {...}"; on "done" take output.output
}

Five things worth knowing

Estimate is free and creates no job. Compare hold_credits against the balance from /me before you submit; a 402 after submitting is a bug in your client, not in the user's wallet. And remember which number to show a human: hold_credits is the reservation, charged_credits on the finished job is the bill, and on a small automation the second is a fraction of the first.

A run between min_credits and hold_credits still executes, with a reduced output cap, and the job comes back with "truncated": true. Each lane fails differently under truncation: review returns fewer action_review entries than the digest has actions, card loses its later cards[] entries, and harden is the dangerous one, because rewritten_yaml is last and a truncated rewrite is a YAML file that ends mid-block. Check truncated before you offer anything as pasteable, and on harden confirm rewritten_yaml parses on your side before you show a copy button.

A reply that does not parse is worth exactly one retry. The web app re-sends the identical input with a fresh Idempotency-Key ending :a2 and stops there. Two retries on one input is a loop, and it bills twice. The same applies to a reply that parses but comes back on the wrong lane: fix the request rather than retrying it, because the second attempt will do the same thing.

The API never touches a Home Assistant instance. No URL is called, no long-lived access token is involved, no entity is read, no service is invoked, nothing is reloaded. Every reply is a reading of the digest, the facts and the transcript you sent — which is why a stale digest produces a confident answer about a file you already changed, and why verify_steps talks about Developer Tools rather than promising something has been checked. Everything you pass in is the whole world the model has: the entity ids in your digest are the entity ids in the reply, and nothing will ever discover the one you forgot.

The YAML 1.1 resolution rules are the thing to internalise. More than half of every reply this app produces traces back to one of them, and they are not intuitive. Plain on, off, yes, no, true, false and their capitalised forms are booleans — but bare y and n are not, whatever the YAML 1.1 spec says, because PyYAML does not implement that part. 7:30:00 is the integer 27000 by sexagesimal resolution; 07:30:00 is a string, because the leading zero disqualifies it; 0755 is the octal integer 493. An empty value, ~ and null are all null, so a key with nothing after it is not an empty string. Build a digest with a YAML 1.2 library and you get none of this, and the model will politely tell you your broken automation looks fine.