Perception
Capture pages over CDP, distill them to compact interactive trees, assign stable element IDs, and send diffs instead of full page dumps.
@rote/browser · @rote/perception
docs/02 · architecture
The context window is RAM. Observations are pages, dropping them is eviction, diffing is delta encoding, the prompt cache is L2, compaction is GC, a playbook is a cached compiled program, and site memory is the persistent store. Every one of those has a manager in an OS — Rote gives them one in a browser agent.
the memory spine
Each tier caches what the one below keeps forgetting. None of it re-enters the window unverified.
within one run
the action ledger; evicts stale observations, diffs the live page
across runs of a task
verified playbooks, replayed deterministically at zero LLM tokens
across tasks on a site
how the portal behaves — briefs, quirks, transition models
trust gate — assertion on the way back in
Memory that might be wrong is worse than no memory. Success is page state, never the absence of an exception.
where the code lives
Capture pages over CDP, distill them to compact interactive trees, assign stable element IDs, and send diffs instead of full page dumps.
@rote/browser · @rote/perception
Own the context layout, tag every model call by source, and skip model calls entirely when memory or replay can safely act.
@rote/agent · @rote/llm
Typed browser actions with settledness detection, self-healing element resolution, per-step assertions, and a scoped repair ladder.
@rote/action · @rote/executor
Record every run losslessly, promote verified runs into replay candidates, and measure everything behind launch gates.
@rote/recorder · @rote/bench
the control loop
The loop below is the actual execution order — a hard environment gate first, memory next, the model last. Deterministic replay steps cost zero LLM tokens; every exit path ends at the verify gate. Compression shrinks a step; Rote declines to run it.
solid dot — built · hollow dot — designed (P2)
the perception pipeline
Every number below is measured on the real WordPress admin fixture — 120 seeded posts, 100 rows per page — with zero variance across 15 repetitions.
capture
CDP snapshot of the live page
6,784 elements
wp-admin, 120 posts
distill
interactive tree, noise dropped
797 nodes
787 actionable
stable IDs
role + name + ancestry hash
survives re-renders
zero variance ×15 runs
render
budgeted compact markup
~22,279 tok
full page, once
diff
vs last grounded bootstrap
median 24 chars
−99.6% vs re-send
When a first or replacement observation blows past the ordinary budget, the renderer records one grounded bootstrap under a 100,000-character emergency ceiling, then returns to diffs — and fails before planning above the ceiling rather than emitting a selector-free summary.
the context assembler
The ContextAssembler is the only module allowed to order messages. Tests fail if any volatile token — a timestamp, a run id — lands above the stable line, and the runtime re-asserts prefix immutability on every call. That discipline is what makes the prefix cache-eligible: the 50 lines of cache plumbing are trivial; the discipline is not.
With eviction on, a step adds ~37 tokens instead of ~172 — the quadratic term belongs to observations, and observations don't accumulate.
System prompt, task, tool schemas. Nothing above this line may ever mutate — not a timestamp, not a run id. A SHA-256 of the prefix routes through prompt_cache_key.
Tier-2 knowledge of the portal, held stable across the session so it stays cache-eligible. Planned — P2.
One compact JSON action per prior step. This is what survives eviction: keep what you did, not what you saw.
The current page only — full on bootstrap, then diffs against the last grounded observation. Prior observations are evicted.
the action plane
Before anything dispatches, the page must be settled and the target must resolve safely; after it lands, page state has to prove it.
An action dispatches only when the page is quiet: zero pending network requests plus an unchanged DOM for a 250ms window, polled every 50ms, capped at 5s.
waitForSettled
Element targets self-heal in order — stable ID, then exact role + name, then text proximity. If no safe target resolves, the step fails closed; it never guesses.
resolveElementTarget
Assertions come from a closed set of ten primitives over page state. Judgment steps classify into a closed enum — an out-of-enum answer is a hard error, not a shrug.
evaluateExpect
failure is a first-class path
Transient failures — a settle-and-retry with no model involvement.
One model call over a narrow context; emits a versioned patch, never an in-place edit.
The plain agent takes over, the run is recorded, the failure is re-learned. The fallback logs why it fired.
Errors are typed, carry the failing step and run id, and are never swallowed into a boolean. A scoped repair costs about one step; a blind restart costs the whole task — the ladder exists so it rarely gets that far.
the learning plane
The recorder is a stdio proxy that tees the tool channel unmodified — observationally invisible to both sides — while writing an append-only trajectory. Nothing is edited in place, ever; a verified run is promoted, and an exact fingerprint match replays it without a model in the loop.
.rote/runs/<runId>/
trajectory.jsonl · manifest.json · blobs/
the monorepo
Ten packages in strict layers: everything may depend on core; core depends on nothing; CI forbids cycles and deep imports.
the type spine
Schemas are the single source of truth — every type is z.infer'd, never hand-duplicated. Five of them carry the design.
| StableNodeId | role + accessible name + ancestry content hash — the same control keeps the same id across re-renders |
| Observation | full | diff | summary — a page is sent whole exactly once, then as deltas |
| Action | navigate | click | fill | select | extract | done | fail — a small closed verb set, not 50 overlapping tools |
| StepClass | replay | speculated | grounded-routine | frontier | recovery — every step knows why it ran |
| Expect | ten closed assertion primitives over page state — the trust gate's vocabulary |