Atlas recap

2026-10-20

Admin assistant: saved threads and rich blocks

Fimo Admin staff now keep saved assistant threads that run on the shared chat engine, get answers drawn as rich blocks, and change data only through a single-use confirmation card. Customers never see these threads, and Fimo pays for the turns. Four live QA passes show what the agent does well and what it still misses.

Scope PR 3214 · 10 commits · FIMO-2955 Base 1d733e6a → 07c9870a Depth Extensive Evidence unit + integration, live E2E, 4 live QA passes Draft PR · not merged

Draft PR

PR 3214 · Admin assistant threads and blocks

Moves the Admin assistant from a stateless route to saved chat-engine threads, adds History, 12 block types, single-use plans and the isolation and cost guards.

Outcome

Staff can now keep their Admin assistant conversations. Each conversation is a saved thread. A History view lists the threads, with search, open, new and delete. A reload during an answer joins the running answer again. Stop works, and a second message sent during an answer waits in a queue. The assistant now answers with rich blocks (record, cards, table, timeline, stats, chart, excerpt, media, choices, actions, form, handoff), as in the mockups. A change to data still happens only after a person confirms a plan card.

The threads use the chat engine that Studio and Slack already use. An admin thread is an ordinary agent_threads row with surface = 'admin', no project, the staff member's personal organization and created_by = that staff member. A different agent (the ops assistant) runs the turn with four tools: ops, ops_search, ui and open_page. The engine gives persistence, stream resume, Stop, titles, usage and traces for free. Fimo pays for the turns: charge: false with a $5 ceiling per turn, so nothing is charged to the personal organization.

The boundary is strict. Customer readers keep surface = 'app', so no admin thread reaches Studio, the CLI or MCP, not even for the staff member or other members of the personal organization. Only the owner writes to a thread. A platform admin can read any admin thread, and that read is written to the audit log. Live QA ran 4 passes and about 60 conversations. The final live E2E run passed 6 of 6 tests in 3 of 3 runs. The model still has gaps: it sometimes says a source was empty without reading it, and it seldom picks some block types by itself.

What staff see

The assistant answers in the docked panel: text first, then one or two blocks

The panel opens with Ctrl/⌘ J. It docks beside the page at 1680 px and wider, and it covers the page as an overlay below that. The header has three buttons: new thread, History and close. There are no tabs.

In the hero shot below, staff asked "What needs attention right now?". The assistant read health.overview, wrote a short list, and drew a table block. The "1 source" line folds the reads it made.

"What needs attention right now?"Docked · 1680 · dark · Dev Remote · QA pass 4
Empty, docked1680 · dark · pass 4
Empty, overlay1280 · dark · pass 4

Architecture

An admin thread is a normal chat-engine thread with its own surface and agent

Before this PR, the assistant route was stateless: the browser kept the only copy of the conversation. Now the route calls the same engine function as org chat (runChatTurnForRoute) with project: null, surface: 'admin' and a different agent.

The server sets the surface, the organization and the creator. The client cannot choose them. A staff member with no personal organization gets 409 personal_org_missing. The route never falls back to another organization.

flowchart TB
  Staff[Staff in<br/>Admin panel] --> Route["/api/ops/assistant"<br/>Ops principal only]
  Route --> Engine[Chat engine<br/>runChatTurn]
  Engine --> Agent[Ops assistant<br/>Sonnet · 12 steps]
  Agent --> Tools[ops · ops_search<br/>ui · open_page]
  Tools --> Ops[Ops registry<br/>reads run · writes plan]
  Engine --> Store[(agent_threads<br/>surface = admin)]
  Store -. never listed .-> Readers[Studio · CLI · MCP<br/>surface = app only]
Source: assistant.routes.ts, ops-assistant.ts, threads.ts. The dotted line is a guard, not a call.
agent_threads (admin row)New use
surface
'admin' · set by the server
kind
'chat'
project_id
NULL
organization_id
staff member's personal org
created_by
staff user id · only the owner writes
Turn costSponsored
charge
false · no hold, no settlement
maxCostUsd
5 per turn
queued turn
keeps the same sponsor
migrations
none · surface is a free varchar
›POST/api/ops/assistant/chatChangedSend a message; creates the thread

Body: { threadId?, message: { id, text }, formResponse? }. Response: the same NDJSON stream as Studio's chat (head line with the thread id, UI message chunks, then data-turn-status). A send during an answer returns 202 queued. Attachments and unknown fields are refused. Text and form answers are bounded (20,000 characters in total).

AssistantChatTransportassistant.routes.tsrunChatTurnForRoute
›GET/threads · /threads/:id · /live · /streamAddedList own threads, read, rejoin

The list shows only your own threads, newest first, with title search. Read, live state and stream accept the owner, or a platform admin for audit. A cross-owner read writes an admin_resource_read audit row.

›POST/PATCH/DELETE/threads/:id/abort · /threads/:idAddedStop, rename, delete (owner only)

A platform admin who is not the owner cannot stop, rename or delete a thread.

›GET/api/ops/operations/applied?planHash=…AddedWhich plans are already applied (max 50)

A saved plan card asks this when it renders, so an applied card stays applied after a reload. See the single-use plans topic.

›CLIpnpm ops assistant threads list | get --thread <id>AddedCoding agents read a thread

Two Ops registry commands (assistant.threads.list, assistant.threads.get), read-only and human-only. The handoff block puts the get command in its prompt.

Decision

Reuse the chat engine instead of new tables

Two independent reviews (Fable and Astra, under FIMO-2937) compared two designs before any code. Both chose the engine. The engine already runs org-level chat turns with no sandbox and no charge. New tables would give persistence only, and stream resume, Stop, titles, usage and traces would have to be built again.

Those reviews proposed a dedicated internal Fimo organization. The final spec puts threads in each staff member's personal organization instead. A later security audit of the personal-org design said it is safe with six guards, and that a dedicated org would "reduce billing and deletion exposure". The sources do not record why the personal org won.

A

Chosen: engine + surface='admin' + personal org. 0 migrations. Free features: resume, Stop, queue, titles, usage, traces, compaction through the existing context window. Cost: one agent class, one transport port to Admin, and the six guards below.

B

Rejected: new ops thread tables + simple route. About 800 lines and one migration, but persistence only. The reviewers expected a second agent system that drifts from the engine. Astra also noted that the admin generic-resource reader exposes every schema table, so new tables would not be private by default.

A′

Not taken: a dedicated internal org. Needs a config id and org provisioning. It keeps staff threads out of personal orgs. Account deletion still deletes a personal org and its threads (cascade); with the chosen design, deleting a staff account deletes that person's assistant history.

Agent and tools

Four tools: run a command, look up its inputs, draw a block, open a page

ops runs one Ops command with the staff member's own permissions. Reads run at once. Writes only return a plan. ops_search is progressive disclosure: the prompt lists each command on one line with its required keys, and ops_search returns the full input details. ui draws one block and validates it against hard limits; a refused block comes back with the field and the limit. open_page now runs on the server and navigates on a live turn, not on a reload.

The prompt changed over three fix rounds as QA found model mistakes. Each rule below fixes a bug from QA (ids in brackets).

packages/agents/src/agents/ops-assistant/prompt.tsRules added from QA findings (excerpt)
551'- The command list shows the keys each command requires, e.g. `project.get(project)` … Never guess an input key.',
56'- If `ops` answers `invalid_input`, fix the call from its `expectedInput` and retry once.',
582'- Write no text before or between your reads ("Let me check…"). When the reads are done, write the answer once.',
603'- … Call it "the confirmation card", never "above" or "below". … never write "Confirmed", "Done" …',
614'- Say a source (logs, Sentry, Braintrust, Pylon, …) was checked, or had nothing, only if an Ops read of it ran …',
62'- Quote times and amounts from the read … Never convert credits to money unless a read gives the money.',
715'- When `ops` returned a plan, its confirmation card is the only way to confirm it: never add an `op` action …',
72'- A chart over time has one point per day … for every day of the window, 0 for a day without data …',
  1. 1

    B3, guessed input keys. Pass 1: 13 failed reads in 10 conversations. Pass 4: 2 in about 47, each fixed in one retry.

  2. 2

    B5, narration. Fixed for good in pass 4 (0 notes in 15 turns).

  3. 3

    N6 fixed; position words not. The model still writes "the handoff above" for a block that renders below.

  4. 4

    N4, unread sources. Still broken in about 1 of 15 replies.

  5. 5

    B2, two confirm paths. Fixed (2 of 2 write proposals had one card).

Block gallery · data

Data blocks: record, table, stats and chart come naturally; cards and timeline only on request

Each shot is the assistant panel, cut from a real 1680 px capture on the final code (QA pass 4). "Natural" means the model chose the block without being asked for it.

This matters for the product: a block the model never picks is a block staff never see unless they know to ask. Here the model defaults to record and table. Cards and timeline render well, but only appeared when the question named them.

recordGood · natural
tableGood · its favourite
statsGood · natural
chartGood · on "chart it"
cardsOK · only "as cards"
timelineOK · only "as a timeline"
rec

record shows status, failure reason, attempt id, deadline. The text above it says Sentry evidence came from project.get: a borderline N4 claim.

tbl

table fits the 339 px column now (N1 fixed). Narrow columns still break words mid-word ("in_prog/ress", "Severi/ty") and dates truncate.

sta

stats quotes platform.metrics. Its numbers (4 users, 5 orgs) differ from the dashboard cards beside it (6, 7). Not investigated: it can be a time window or counting difference.

cht

chart has one bar per day for the full 7 days, zeros included (B12 and N8 fixed). The text says the 10,334 total twice.

crd

cards renders well and flags the duplicate member row (a seed mistake). Asked "who are the members", the model draws a table instead.

tml

timeline uses the attempt time, not the deadline (N5 fixed). Long times such as 21:12:09 touch the status dot: a small layout bug seen in the shot.

Block gallery · evidence

Excerpts quote what the customer wrote or what failed; untrusted images wait for a click

The excerpt block has two forms: one customer message, or a few log lines with one line highlighted. The media block shows images. After the security review, an image from a non-Fimo host never loads by itself: the block shows the host name and a "Load image" button.

excerpt · messageGood · natural
excerpt · log linesGood · natural
mediaSafe · only with a URL
msg

excerpt (message) quotes the last customer message with time and source. Below it, a log question gets an honest "no log lines" answer: Loki is empty on Dev Remote.

log

excerpt (log lines) shows a real AbortError stack from health.providers (Braintrust spans). It names the two span signatures and the trace ids.

img

media was drawn only when staff pasted an image URL: no Ops command returns images today. The gate held: no request to the image host and no <img> until the click.

Block gallery · interaction

Choices and actions ask; only the confirmation card changes data

An op action does not run a write. It asks the server for a plan and shows the plan card: the change, the risk, a required reason, then Confirm. After Confirm the card turns into an "applied" line and the spent buttons are disabled. The assistant then checks the result with a read.

1 · actionsonly "as buttons"
2 · confirmation cardHigh risk · reason
3 · appliedspent buttons disabled
choicesOK · on "give me the choices"
record + impersonateGood · natural
Impersonate confirmName and email, not a raw id (B10 fixed) · 1680 · pass 4
act

actions work end to end: plan → card → reason → Confirm → applied → a check read. The model offers options in prose unless asked for buttons.

cho

choices renders well, but its text claims "logs/support evidence turned up empty" with no log read in the turn. This is the clearest remaining N4 case.

imp

impersonate opens a confirm with name and email. Impersonation itself was not completed in QA (Cancel only).

Block gallery · forms

A form pins above the composer like Studio, and its answer becomes the next message

A form ends the model's turn. The thread shows "Waiting for your answer…" and the form pins above the input. A multi-step form shows one question at a time and keeps earlier answers as short summaries. A one-step form uses its question as its title. When staff submit, the answers appear as an answered card and the model continues.

Multi-step · step 1natural
Multi-step · step 3answers summarised
Answered → plan cardreason not prefilled
One-stepnatural · unknown customer

Block gallery · handoff

Handoff continues a thread in Claude Code, Codex or Cursor

The handoff block copies Studio's "Open with" area: Copy prompt, three agent logos, and the command that loads the thread (pnpm ops assistant threads get --thread … --profile local). The profile is now the real backend profile, not a <profile> placeholder (B9 half fixed).

handoffGood · natural
hnd

Works: 12 reads, a record with the diagnosis, then the handoff. The block renders below the text.

gap

Misses: the text says "the handoff above" although it is below, and the reply repeats itself ("Continue the investigation…" then "I checked, there is no additional evidence…"). It runs 900 to 1,500 characters.

Failure path · writes

A plan hash is single-use, so an applied card cannot apply again after a reload

QA pass 1 found the most serious bug (B1, High). Saved threads replay the stored plan output. After a reload, an applied takedown card came back as "Apply this change?". Once the project was restored, the old plan hash was valid again, and Confirm took the project down a second time.

The fix is on the server, so the CLI is covered too. An apply audit row records the plan hash. A second apply of the same hash returns 409 plan_already_applied. The same change can be planned again later (take down, restore, take down): each apply moves the next plan to a new hash "generation". The card asks GET /operations/applied when it renders.

Before · pass 1applied card asks again
After · pass 2"Applied: …", no Confirm
apps/api/src/domain/ops/operations/execute.tsapplyOperation refuses a spent plan
2881if (request.planHash && (await findAppliedPlanHashes([request.planHash], [action])).has(request.planHash)) {
289  await deny(OPS_PLAN_ALREADY_APPLIED, 'This plan was already applied; plan the change again to make it again', 409);
290}
291
295  plan = await definition.plan(input, ctx);
2962  planHash = await currentPlanHash(id, input, plan, action);
  1. 1

    Spent means audited. The lookup reads success rows with applied: true for this operation's own audit actions, so it stays on the action/time index. The refusal itself is audited as denied.

  2. 2

    Generations. Without them, a restore would make an old takedown hash valid again, which is exactly the B1 repro.

Threads and history

Stop, queued messages and History work on desktop and on a phone

The Admin panel uses a port of Studio's chat transport. A head line carries the thread id. A reload in the middle of an answer rejoins the stream. Stop shows "Stopped." at once (B7 fixed). While an answer runs, Send becomes "Queue message" and a chip says "Queued. It runs when the current reply ends." (B8 fixed). A recovery loop reloads the saved thread when a stream stalls or a queued reply exists. This fixed N2 (first send hung) and N3 (reply to a queued Confirm never showed).

History lists only your own threads, grouped Today / Yesterday / Earlier, with title search. On a phone the open thread's row always shows its trash button, and coarse pointers always show it (B6).

Stop1280 · light
Queued1280 · light
Queued ran1280 · light
History search1280 · light
History390 · dark
Delete on touch390 · dark
Table390 · dark
Chart390 · dark
History, overlay1280 · light · pass 4

Mockup vs shipped

The shipped panel follows the mockups; the gaps are data, not layout

The mockups use invented data at a large scale (18,402 users). The shipped panel uses the same header, spacing, text-first order, timeline, log excerpt, pinned form and history list. The differences come from what Ops can read today.

Mockup · answerassistant-blocks-chat · invented data
Shipped · timeline1680 · light · pass 4
Mockup · pinned forminvented data
Mockup · historyinvented data
Mockup · block catalog (data)assistant-blocks.html · thumbnails blurred
=

Same: header (new, history, close), text then blocks, timeline dots, log excerpt with a highlighted line, pinned multi-step form with a red step number, history rows with time on the right.

≠

Not built (marked "later" or "new" in the mockup): diff, compare, PostHog replay and live-site preview blocks. Replay needs a recordings read that Ops does not have.

≠

Data gap: the mockup answers "what happened" with release → merge → 500 → Pylon. On Dev Remote there are no release logs or Pylon data, so real answers are thinner.

≠

Small: no "Recommended" badge was seen on a shipped form option; the shipped composer shows the page-context chip as an eye icon only.

Security · guards

Six guards keep admin threads away from users and free of charge

"Personal" does not mean private: a personal org can have other members, and the staff member is also a normal Studio user. Astra's audit of the design found two real gaps in the shared engine before any admin code existed. First, the scope check skipped every projectless thread. Second, a message sent during a turn ran as a "bundled" follow-up with a hard-coded credit cost, so it would charge the personal org. Both were fixed here, and each test was seen failing before the fix.

  • 1 · scope checkEvery turn compares org, project (also null) and surface. A projectless thread with another surface is refused.
  • 2 · queued costA bundled follow-up keeps the sender's sponsorship: charge: false, $5 ceiling.
  • 3 · server-set identityThe route sets surface, org and creator. No personal org → 409. A non-admin or another person's thread → 404.
  • 4 · customer readersStudio lists, messages, live/stream, history search and recall keep surface = 'app'. Not widened.
  • 5 · no attachmentsUploads are public URLs, so file parts and unknown fields are refused before anything is saved.
  • 6 · ops principalRead, stream, abort and delete authorize the Ops principal, never org membership.
apps/api/src/domain/agents/chat/chat.tsGuard 1: the projectless bypass is gone

Before

319async function assertConversationScope(deps, send, threadId) {
320  if (!send.project) return;
321  const thread = await deps.conversation.getThread(threadId);
322  if (!thread || !threadMatchesSendScope(thread, send)) throw new ThreadScopeMismatchError();

After

320async function assertConversationScope(deps, send, threadId) {

321  const thread = await deps.conversation.getThread(threadId);
322  if (!thread || !threadMatchesSendScope(thread, send)) throw new ThreadScopeMismatchError();
apps/api/src/domain/agents/chat/chat.tsGuard 2 and finding M2: a bundled turn keeps its sponsor and hears Stop
1004      const bundleAbort = new AbortController();
10071      const tearDownAbort = await subscribeCancellation(deps, send, threadId, bundleAbort);
1008      const turn = deps.agents.stream(send.agent, {
10252        cost: chatTurnCost(send.sponsorship),
1026        …
10463        void turn.receipt.catch(() => undefined).then(tearDownAbort);
  1. 1

    Stop reaches the successor. Before, Stop and delete could not stop a bundled turn; it kept reading and spending. This was an existing engine limit that the assistant exposed.

  2. 2

    Same sponsor. Before, this line was a fixed credit cost, which would put a hold and a settlement on the personal org.

  3. 3

    Cleanup. The subscription ends when the receipt settles, or at once when admission fails.

Security · review

An independent diff review found four issues; all four are fixed with tests

Astra reviewed the first full diff (f5cad86300) by source inspection and in-memory probes. Verdict: "ship after fixes". It found no billing or customer-reader regression. The PR body says a re-review of the fixes was in progress; this recap has no result for it.

H

Image URLs could leak thread data. A model, steered by customer content, could draw an image whose URL carries private thread text. The browser would fetch it with no click, and again on every replay. Fix: blocks load images by themselves only from Fimo asset hosts; any other host shows a placeholder and "Load image". Assistant markdown renders no images. Proven in QA: zero network requests to the untrusted host.

M

Unaudited cross-owner live reads. A platform admin could read another person's live state or stream with no audit row; only the saved-thread read was audited. Fix: all three read routes write admin_resource_read.

M

Stop missed bundled turns. See guard 2 above. Fix: subscribe before admission; integration test "stops a follow-up that was queued behind a turn".

M

Unbounded form answers. A probe sent a 1,000,000-character form answer and the server built a 1,000,122-character model message. Fix: per-answer bounds, a 20,000-character limit on the composed message, and a body limit. Customer Studio forms are unchanged.

apps/admin/src/features/assistant/blocks/images.tsxThe trusted-host gate
101export const TRUSTED_IMAGE_HOSTS: ReadonlySet<string> = new Set([
12  'assets.fimo.ai', // marketing and docs CDN
14  'uploads.fimo.team', 'uploads.dev.fimo.team', // uploads bucket
17  'chat-uploads.fimo.ai',
18]);
322  return parsed.protocol === 'https:' && TRUSTED_IMAGE_HOSTS.has(parsed.hostname);
  1. 1

    Allowlist, not a proxy. The review warned that an open image proxy would still leak the URL.

  2. 2

    Exact host and HTTPS. The click choice lasts while the block is on screen; a reload asks again.

Security · proof

Integration tests on the real schema prove isolation and zero cost

These run against the real database schema (PGlite harness), not mocks. They are API paths with no screen, so each row names a trigger and a result.

They are the acceptance criteria from FIMO-2955. If one of them broke, a customer could read a staff conversation or be charged for it, so they carry more weight than the screenshots.

01
Another member of the personal org gets nothing
Starts with
A personal org with two members. Staff A writes a secret in an admin thread. Prepared by setup.
Trigger
Member B, and A as a normal user, call every customer reader: Studio lists, messages, live/stream/abort by id, /chat/org, history search and recall, and the admin thread list.
Result
No reader returns the thread or the secret.
Check
"never return an admin thread to a member of the personal organization, or to its owner as a customer"; "gives another staff member nothing".
Not proven
Readers the audit marked unverified: Tinybird pipes, exports and webhooks, and staff-only admin readers (invocation detail, raw resources, traces).
02
Overlapping sends charge nothing
Starts with
A running admin turn.
Trigger
A second message arrives during the turn.
Result
202 queued; it runs after the first turn.
Check
Zero holds, zero settlements, no credit change. Live QA agrees: 0 holds and 0 settlements on the staff personal org after 24+ threads.
Not proven
Hyperline itself was not checked; Hyperline reads settlements only, and there are none.
03
Platform admin reads on the record, but cannot write
Starts with
Staff A owns a thread; B is a platform admin.
Trigger
B reads, streams, then tries to send, stop or delete.
Result
Reads succeed and are audited; writes are refused.
Check
"lets a platform admin read any thread on the record, but not write to it".
Not proven
What happens to a staff member's threads after the person loses Ops access (the rows stay).
04
Refusals before anything is saved
Starts with
A staff session.
Trigger
Service token; attachments or unknown fields; no personal org; oversized form answer, composed message or body.
Result
403 for a service token; 400 for attachments, unknown fields and oversized answers; 409 personal_org_missing; 413 for an oversized body.
Check
Status codes, plus no new message or thread row for attachments, the missing org and oversized answers.
Not proven
The service-token and body-size cases check only the status code.

QA journey

Four live QA passes found 22 issues; 2 remain open, plus 3 small wording and layout gaps

Each pass ran on a fresh Dev Remote box with the live model (Sonnet 5 through the dev gateway), about 15 conversations each, at 390, 1252/1280 and 1680 px, dark and light. After each pass, one server lane and one client lane fixed the list, and the next pass re-tested it.

B1

High · applied card re-applies after reload. Found pass 1 → fixed pass 2 (single-use plans). Verified in E2E and CLI.

P1

High · the answer folded away into "N sources". Created by the pass-3 fix for narration → fixed pass 4. Verified: 15 of 15 answers fully visible.

B2

Medium · two confirm paths for one write. Pass 1 → fixed pass 2.

B3

Medium · guessed input keys, commands called as tools. Pass 1 → improved pass 2 → fixed pass 3 (required keys in the prompt index).

B4

Medium · ui refusals cost a step. Pass 1 → mostly fixed pass 2 (1 refusal in 19).

B5

Medium · narration and repetition. Pass 1 → fixed pass 4. Long, repetitive answers remain (Low).

B6

Medium · delete unreachable on touch. Pass 1 → fixed pass 2 (partly verified).

N1

Medium · table overflowed the panel. Pass 2 → fixed pass 3.

N2

Medium–High · first send hung for 180 s. Pass 2 → not seen since (33 fresh threads in pass 4).

N3

Medium · reply to a queued Confirm never showed. Pass 2 → fixed pass 4 (2 of 2 QA, 3 of 3 E2E).

N4

Medium · claims about sources it did not read. Pass 2 → still open: about 1 in 15 replies.

B13

Low · reason not prefilled in the plan card. Pass 1 → still open.

B7–B12, N5–N8

Low. Stopped line, queue from composer, handoff placeholder, impersonate dialog, column truncation, per-event bars, wrong inferences, "Confirmed" on a plan, blocks above text, chart window: all fixed. "Above"-wording (B9/N7 wording) still open.

Before · pass 1 · B2"pending" record + 2 cards
Before · pass 1 · B11key column cut off
Before · pass 3 · P1answer in the fold
After · pass 4answer shown, table fits

E2E

Six live journeys pass 6 of 6 in 3 of 3 runs on the final code

The file hosting/admin-assistant.ui.live.test.ts uses the live provider on Dev Remote. Earlier passes failed on real bugs (N2 in pass 2 run A, N3 in pass 2 run B and pass 3 run 3, B3 in pass 2 run C). Pass 4 is the first fully green set.

01
Read a project, propose a takedown, apply only from the card
Starts with
A staff session and a project. Prepared by setup.
You do
Ask to take the project down, give a reason, Confirm. Restore it. Reload.
You see
A plan card, then "Applied"; after reload the card stays applied with no Confirm.
Check
An Ops read happened first; no failed read (" — "); an audit row with the reason; replay of the old plan hash → 409 plan_already_applied.
Not proven
Other write commands through the card (only takedown/restore in E2E; merge.fail and top-up seen in QA only).
02
Panel overlays a laptop, docks on a wide screen, covers a phone
Starts with
A staff session.
You do
Open the assistant at three widths.
You see
Overlay, docked and full-screen layouts.
Check
Panel geometry per width.
Not proven
Real touch input.
03
A user without Ops access is refused
Starts with
A signed-in normal user.
You do
Call the assistant routes.
You see
Refusal.
Check
Status codes on the assistant routes.
Not proven
Nothing beyond the check.
04
A thread survives reload, lists in History, switches back, deletes
Starts with
A staff session.
You do
Send, reload, open History, start a new thread, switch back, delete.
You see
The same messages after reload; the row in History; the row gone after delete.
Check
Saved text after reload; History rows; API 404 after delete.
Not proven
Rejoin during a running answer is QA-only (pass 1, question 9).
05
A project status answer draws a block from Ops
Starts with
A project.
You do
Ask for the project's status.
You see
A block with the project.
Check
The block exists and the sources list a project or find read.
Not proven
The other 11 block types are QA screenshots and unit renders only.
06
An admin thread is invisible to every Studio chat reader
Starts with
An admin thread with a known secret.
Trigger
The same person calls Studio chat readers as a customer.
Result
No thread, no secret.
Check
Every reader response lacks the thread id and the secret.
Not proven
CLI and MCP surfaces directly (they use the same routes).
Verifiedops-takedown.cli

Single-use plan, stale plan (plan_stale), replay after restore → plan_already_applied. Passed in pass 4.

Verifiedops-agent-chats.cli, Studio chat/threads.ui

Passed in pass 2: the shared engine changes did not break Studio chat.

What the agent can do

Accurate on what it reads; weak on what it did not read and on block variety

Marc asked to see what the agent can do and what it misses. This is the honest balance after pass 4, on thin Dev Remote data.

The pattern: when the agent reads, it is right. Its mistakes are claims about things it did not read, block types it does not think of, and data that Ops cannot give it yet.

✓

Real values. Ids, names, statuses, times and amounts come from Ops reads and name their source. Failed reads: 2 in about 47, each fixed in one retry.

✓

Fitting blocks. record for status, table for lists, stats for numbers, a per-day chart for time, excerpt for traces and messages, form and choices for decisions, handoff to continue. Text first; at most two blocks.

✓

Safe writes. One plan card per change, reason required, single-use, stale plans refused, audit linked to the thread.

✓

Honest limits. Asked to audit every org, it said "There's no command to enumerate every organization" instead of guessing.

✗

Unread sources (N4). About 1 in 15 replies says logs or Sentry were empty with no read of them.

✗

Block choice. It never chose cards, timeline, actions, media or choices on its own; it uses them when asked. Media has no image source at all.

✗

Wording. It says "above" for blocks that render below, although the prompt forbids position words. Handoff and choices replies run 900–1,500 characters and repeat their opening.

✗

Missing Ops reads. No org list (it searched find with "a"), no publish/build history, no daily usage series, no Hyperline invoices, no session recordings. "What needs attention" skips platform.pool.status, which the dashboard's own Needs attention shows.

Follow-ups

Most remaining work is new Ops reads and prompt adherence, not plumbing

The threads, transport, guards and write path are done and proven. What is left changes what the assistant can know and how well it follows its rules. None of these items needs a schema change. Items 1 and 2 matter most, because they decide whether staff can trust a "nothing found" answer.

1

N4: enforce "checked only if read" outside the prompt, for example by listing the thread's actual reads to the model, or flag a reply that names an unread source.

2

Ops reads: org.list, publish/deploy history, daily usage, Hyperline invoices, PostHog recordings (for a replay block).

3

B13: pass the reason from the form or chat into the plan card.

4

Layout: no mid-word breaks in narrow table columns, full dates, timeline time column width.

5

Block choice: intent hints for timeline (events in order), cards (people/projects), actions (next steps).

6

Ship gates: security re-review result, PR out of draft so CI suites run, real touch-device check.

Proof and footprint

Evidence by layer, every changed file, and what was not examined

Counts come from the PR body and the QA report. This recap did not rerun any test; it read the diff, the QA report, the reviews and the screenshots.

Each layer below is separate. Green unit tests do not prove the live E2E, and the live E2E does not prove CI or production. The tree lists all 84 changed files; click a row to read the file at the pinned revisions.

VerifiedUnit and integration

admin 337; agents ops-assistant 22 (package 1042); API ops 829 unit + 311 integration; contracts 1687. Guards 1 and 2 seen red before the fix (PR body).

VerifiedLive E2E (Dev Remote, live provider)

6/6 in 3 of 3 runs on 07c9870a52; ops-takedown.cli green.

VerifiedLive browser QA

4 passes, about 60 conversations, 390/1252/1280/1680, dark and light. 0 holds and 0 settlements on the staff personal org.

PartialSecurity review

4 findings fixed and tested; the re-review of the fixes had no result in the sources.

PartialTouch

Open-row delete proven at 390; coarse-pointer rule not seen on a device.

UnknownCI

PR is a draft; full suites run only when it is marked ready.

UnknownStaging and production

Not merged, not deployed.

Full footprint84 files · +10,180 / −775
⌄▱.agents/skills/fimo-support/references
⌄▱apps/admin/src/assets/logos
⌄▱apps/admin/src/components/ui/shadcn-io/ai
⌄▱apps/admin/src/features/assistant
⌄▱apps/admin/src/features/assistant/blocks
⌄▱apps/admin/src/features/assistant/transport
⌄▱apps/admin/src
⌄▱apps/api/src/domain/agents/chat
⌄▱apps/api/src/domain/ops/assistant
⌄▱apps/api/src/domain/ops/operations/admin-writes
⌄▱apps/api/src/domain/ops/operations
⌄▱apps/api/src/domain/ops/operations/ops-writes
⌄▱apps/api/src/http/routes/chat
⌄▱apps/api/src/http/routes/ops
⌄▱apps/api
⌄▱e2e/poms/admin
⌄▱e2e/tests/hosting
⌄▱packages/agents/src/agents
⌄▱packages/agents/src/agents/ops-assistant
⌄▱packages/agents/src/engine
⌄▱packages/agents/src
⌄▱packages/api-contracts/src
⌄▱packages/api-contracts/src/ops/commands
⌄▱packages/api-contracts/src/ops
⌄▱packages/api-contracts/src/types

Source peek

Open at head
Path copied