One display problem. Studio shows a sentence written next to a tool that draws no row (internal linking close). Studio also repeats the question of a one-step form with no title; none of the failures below depends on it.
PR 3801 · QA failures, drawn
What a person reads in the chat for each failed QA check, and whether the problem is the words or the drawing.
Outcome
This page shows each QA failure on draft PR 3801 the way a person sees it in the Studio chat. The person does not see the routine's report. They see the agent's message, its form, and a short "Question closed" line when the agent closes a form. Each card draws that view with the stored words, word for word, and then says what is wrong, why, and what could fix it.
Most failures are content problems. The agent wrote too much, said the same thing twice, used a word the team does not use, or put no sentence before a form. Studio draws those words correctly. One failure is also a display problem. Studio shows every sentence that sits next to a tool that draws no row, such as "close a form". So the note "Close the waiting form; no message needed…" shows as a message.
The live judge reads almost what Studio draws, but not exactly. It reads every stored sentence, joined with no space. It reads each form step's header, which an open form never shows. It does not read the form title or a connect card, which Studio shows. A Studio test that ran outside the repo confirms these drawing rules. The pictures are mockups built from the stored words. They are not screenshots of a running Studio.
Summary
Nine of the eleven failures are about what the agent wrote, not about what Studio draws
"Display" means Studio draws something that the person should not see. "Content" means Studio draws the agent's words correctly, but the words are the problem. "Caused by this PR" uses the verdicts in the PR body. "Not run on develop" means nobody ran that check on develop, so we cannot compare.
| Failure | Display or content | Caused by this PR? | Proposed fix (one line) |
|---|---|---|---|
| Internal linking, close: a note to itself shows (2 of 4) | Content Display | Not run on develop | Studio: hide a sentence that sits before a silent tool when more words follow it. |
| Internal linking, ask: "only reach it by typing its address" | Content | Not run on develop; 1 of 3 trials | No code fix. The agent must keep the routine's fact: the sitemap lists the page. |
| The close Marc approved fails the judge 3 of 3 | Content Judge | Judge disagrees with Marc | Marc decides: change "has links to it" to "is linked from another page", or accept it and tune the judge. |
| Technical SEO first run: a "Connect Google Search Console" card, no words | Content | Yes: 5 of 9 here, 0 of 5 on develop | Say a missing connection once per site, not on every first run; Marc decides if the card is noise. |
| Brand compliance: the same point three times | Content | Not run on develop | Prompt: give the reason once, in the message or in the form, not in both. |
| block-lifecycle-qa 15: a question over 120 characters that carries the reason | Content | Yes: 1 of 6 here, 5 of 6 on develop | Move the reason to the sentence before the form, or let Marc raise the 120-character limit. |
| block-lifecycle-qa 17: a form with no sentence before it | Content | Worse: 0 of 6 here, 3 of 6 on develop | Prompt: always write one sentence before a form that asks about many changes. |
| Content refresh, a price: "analytics shows no visits yet" | Content | Yes: the "say what you could not check" guidance | Pass on "not checked" only when it changes the decision the person makes. |
| Content lifecycle: an option that ends "You still publish." | Content | Not run on develop | Write the step in full words, for example "It goes live when you publish." |
| AI citation: an option over 240 characters, so the first form was refused | Neither: not shown | No: a known limit, not the judge | None in this PR. The retry put the long quote in the question, and the person saw a good form. |
| chat-ui-tool 06 and 26: a before/after card, and a form where no question was expected | Content | Mostly on develop too | 06: say in words that the edit cannot happen here. 26: Marc decides if asking which story is right. |
Judge read is not what is drawn in six cards: internal linking close (texts glued; maybe only one drawn), internal linking ask, content refresh and content lifecycle (step header read but hidden, form title shown but not read), the approved close (judged under the full form; the person sees "Question closed"), and Technical SEO (a card shown but never judged). Also, the AI citation test failed on a form that the person never saw.
Sources. Verdicts from the PR body (round 4). Words from the stored chat rows, the eval captures and the run logs. Drawing rules from the Studio code at the PR head and a throwaway test.
In each mockup, a dashed outline or a marked phrase shows the problem. Studio does not draw these marks.
How Studio draws a turn
Studio hides the report and the work, shows the words and the form, and folds a closed form to one line
A routine's report is a hidden message: Studio draws no row for it (Messages.tsx:136). The agent's turn then shows its words and its form. A tool that has no entry in Studio's tool map draws no row. block_close and block_list are such tools (groupMessageTasks.ts:1230).
Words before a tool that draws a row fold away once the turn answers. Words next to a silent tool do not fold. So in the closing turn "words → block_close → words", nothing separates the two sentences, and Studio draws both, as two bubbles. One case differs: if the turn starts straight with words, with no thinking step, Studio keeps only the last sentence (MessageParts.tsx:335, lastWordsOnly). The stored capture kept no thinking parts, so we cannot tell which case happened. Opus 5.5 streams a thinking summary, and Studio's turn sends it to the chat (sendReasoning), so "both drawn" is the likely case.
A form shows its title in the top row. A form with no title and one step shows its question there instead, and then again as the big question (chat-block-state.ts:45, QuestionForm.tsx:236). A step's header shows only after an answer. A closed form becomes one muted line, "Question closed" (BlockLifecycle.tsx:23). A form the tool refused draws nothing.
The judge reads all text parts joined with no space (field-notes.fixture.ts:818, seo-agent.api.live.test.ts:170). For each shown form it reads every step's words, the header included, but not the form title (field-notes.fixture.ts:834).
/tmp/fimo-3801-display/display.test.tsxRenders the real MessageParts on stored parts · vitest 5.1.0 · 8 passedA thinking, note, block_close, close
drawn: the note AND the close (2 bubbles)
B same, with no thinking step
drawn: the close only
C same, not marked as a reply to a report
drawn: the note AND the close
D control: a tool WITH a row in between
drawn: the close only (the note folds)
E a turn that only closes a form
drawn: nothing at all
F open form, one step, no title
drawn: the question twice (top row and
big title); header "Link to Press page"
not drawn
G the same form after the close
drawn: "Question closed"
H AI citation: refused form, then retry
drawn: 1 form, "About page: quotable intro"
Test Files 1 passed (1)
Tests 8 passed (8)- The test file sits outside the repo. It runs the worktree's vitest with the worktree's sources (api-contracts read from
src, no build). Nothing in the worktree changed. - The stored forms had
fallbackandinternalNoteremoved by the capture script. The test puts back a placeholderfallback, which Studio does not draw.
⌄apps/ui/src/features/chat/components/Messages
⌄e2e/tests/automation/chat
Internal linking · close
The agent's note to itself shows as a message, above the real closing message
The routine said that the Press page is gone, so the open question is not needed. The agent closed the form. Before the close, it wrote a note to itself. After the close, it wrote the message for the team. Studio draws both. This happened in 2 of the 4 closings that the trials reached.
Internal linking: the agent closes the Press question
- What is wrong
- A sentence the agent wrote for itself ("Close the waiting form; no message needed to the team beyond that.") shows to the person.
- Shown but should not be?
- Yes, and the words are also wrong. The agent should not write the note. But Studio also shows it because the tool next to it (close a form) draws no row, so nothing folds the note away. Behind a tool that draws a row, Studio would have hidden it (test D).
- Judge read vs drawn
- The judge read both sentences glued together ("…beyond that.You can ignore…"). Studio draws them as two bubbles if the turn began with a thinking step (likely). If the turn began straight with words, Studio drew only the second sentence, and the judge failed text that the person never saw. The capture has no thinking parts, so this is not captured.
- Cause and verdict
- Seen on this PR in 2 of 4 closings. The other leak read: "The form is still waiting even though the page is gone, so close it. No message is needed after that." followed by "I closed the question about linking the Press page, since the page has been taken down. A new check found nothing else to fix in your site's internal links." Its part order is not captured. This check did not run on develop, so we cannot say if this PR caused it.
- Proposed fix
- Studio: treat a sentence before a silent tool (close or list forms) as work, and fold it when more words follow. Prompt, as a second guard: every sentence you write is shown to the team.
- Not proven
- Which case Studio drew: the thinking parts of this turn were not captured.
Internal linking · ask
The agent turned "only the sitemap lists it" into "only reach it by typing its address"
The routine wrote: "Only the sitemap lists it, so visitors and Google can only reach it directly." The agent shortened this to "visitors and Google can only reach it by typing its address". That is not true for Google, which reads the sitemap. The judge failed it for that reason.
Internal linking: the agent asks where to link the Press page
- What is wrong
- "Google can only reach it by typing its address" is wrong. Google finds the page in the sitemap.
- Shown but should not be?
- No. Studio draws the words correctly. The words are the problem.
- Judge read vs drawn
- The judge read the step header "Press page link", which an open form does not show. It did not read the form title "Link the Press page", which Studio shows in the top row. This does not change the verdict.
- Cause and verdict
- Judge failure in 1 of 3 trials (trial 2). The seed is the routine's own "can only reach it directly". Not run on develop.
- Proposed fix
- No code fix now. Keep watching. If it repeats, ask the routine to name how a page can still be found (the sitemap).
- Not proven
- How develop words the same ask: this check did not run on develop, and only one trial failed.
Internal linking · the approved close
The judge reads "every one of them has links to it" as links to the removed page
Marc approved the closing message on the left. The judge fails it 3 of 3, and two other judge models agree. The judge reads "it" as the Press page: the page is gone, but every page links to it? The message on the right passed. It says why the question is closed and does not count links.
The same moment, two wordings
- What is wrong
- "every one of them has links to it" can mean "links to the Press page". The page is gone, so that reads as broken links.
- Shown but should not be?
- No. This is about the words.
- Judge read vs drawn
- The judge reads the close under the full earlier question, form included. The person sees only "Question closed" there, because Studio folds a closed form to one line. The asking message of round 3 is not captured in full, so the mockups show only the closed line and the close.
- Cause and verdict
- Judge problem, or a real ambiguity: the PR left it to Marc. Not tuned away.
- Proposed fix
- Marc decides. Option 1: keep the wording and accept this judge fail. Option 2: say "each of the 5 pages left is linked from another page".
- Not proven
- Round 3's asking message, which sits above the close: not captured in full.
Technical SEO · first run
A first run that found nothing now shows a "Connect Google Search Console" card, with no words around it
The routine found no problems. It then said what it could not check: Search Console is not connected. That sentence is the guidance this PR added. The agent passed it on as one card and wrote no text. On develop, the same run was quiet, and the person saw nothing.
Technical SEO: the baseline run reports a missing connection
- What is wrong
- A run with nothing to fix now puts a card in the chat. The test expects a quiet run, so it fails before the agent replies.
- Shown but should not be?
- No, for Studio. Studio draws the card the agent chose. Whether the card is noise is a product call. The second trial showed the same card with: "Connect it so I can check which pages Google has indexed and catch old addresses Google still shows. It also lets me work on the searches each page almost ranks for."
- Judge read vs drawn
- The judge never read this card. The test stopped at "the run must be quiet". Also, the judge reads forms only, not cards. The person sees a card the judge does not check.
- Cause and verdict
- Caused by this PR. The run reported 5 of 9 times here, 0 of 5 on develop.
- Proposed fix
- Say a missing connection once per site, not on each first run. Marc decides if this card is noise.
- Not proven
- Whether a person finds the card useful or noisy: nobody asked a person.
Brand compliance
The message, the question and one option all say "this breaks your brand rules"
The routine found the line "Field Notes doubles your readers in a month." The agent asks what to do with it. The reason appears three times: in the message, in the question, and in the "Keep it as is" option. The person reads the same point three times before they choose.
Brand compliance: a promised result on the home page
- What is wrong
- The same reason shows three times. The judge: it "repeats the same issue".
- Shown but should not be?
- No. Studio draws each part once. The agent wrote the repetition.
- Judge read vs drawn
- Match. The judge read the step header "Home page claim". Studio shows the same words as the form title.
- Cause and verdict
- Judge failure on this PR. Not run on develop. It fits this PR's "say why" guidance, but that link is a guess.
- Proposed fix
- Prompt: give the reason once, in the message or in the form question, not in both.
- Not proven
- That this PR caused the repetition: develop was not run on this check.
block-lifecycle-qa 15
The question carries the reason and the risk, so it passes 120 characters
The agent asks about three meta tag changes. Each question also says why, and what could go wrong. That is what this PR asks for. But the eval fails any form question over 120 characters. The first question here has 210 characters, the third has 226.
Meta tags: three changes, three long questions
- What is wrong
- Questions of 210, 111 and 226 characters. The eval limit is 120.
- Shown but should not be?
- No. The person sees one step at a time, with a short message above. The words are long, not hidden or wrong.
- Judge read vs drawn
- Not the judge. The eval counts characters.
- Cause and verdict
- Caused by this PR. 1 of 6 pass here, 5 of 6 on develop. Develop's questions run 76 to 103 characters.
- Proposed fix
- Move the reason into the sentence before the form or into the option text, or let Marc raise the 120-character limit.
- Not proven
- That a shorter question reads better to a person: the limit belongs to the eval.
block-lifecycle-qa 17
The agent opens a ten-change form with no sentence before it
The agent found ten changes on two pages. It shows a three-step form at once. There is no sentence that says what was checked or why these changes matter. The person must start from the form.
Many changes: the form comes alone
- What is wrong
- No sentence before the form. In trial 6, a question also reached 143 characters: "Which home page content changes should I make? I recommend the two alt texts and the link. The H1 and intro sentence change text visitors read."
- Shown but should not be?
- No. Something is missing, not shown by mistake.
- Judge read vs drawn
- Not the judge. The eval checks for text before the form and the question length.
- Cause and verdict
- Worse on this PR, already on develop. No sentence 5 of 6 here, 3 of 6 on develop. Long questions 3 of 6 here, 0 of 6 on develop.
- Proposed fix
- Prompt: before a form about many changes, always write one sentence that says what you found.
- Not proven
- Why the agent skips the sentence: no prompt change was tried for it.
Content refresh · a price
A sentence about what was not checked sits between the problem and the questions
Two pages disagree on what members get. The agent says so, then adds: "I couldn't look for pages losing visitors because analytics shows no visits yet." That sentence has nothing to do with either question. It comes from this PR's guidance to say what a run could not check.
Content refresh: free trial and early access
- What is wrong
- The last sentence is noise for this decision. The judge: it "is unrelated to either decision and adds noise".
- Shown but should not be?
- No. The agent chose to write it. Studio draws it correctly.
- Judge read vs drawn
- The judge read both form steps at once, with their headers "Free trial length" and "Early access". The person sees step 1, then step 2, and no headers. It did not read the title "Membership terms". This does not change the verdict.
- Cause and verdict
- Caused by this PR ("say what the run could not check").
- Proposed fix
- Pass on "not checked" only when it changes the decision the person makes.
- Not proven
- A develop run of this check: the cause comes from the words of the new guidance.
Content lifecycle
The recommended option ends with "You still publish.", which a person cannot place
A story is a word-for-word copy of another one. The agent asks what to do with the copy. The "Retire it" option ends with "You still publish." The routine wrote "A person still publishes." The agent cut it down until it lost its meaning. This turn also has no sentence before the form.
Content lifecycle: retire a duplicate story
- What is wrong
- "You still publish." is a fragment. The judge: "an unexplained fragment".
- Shown but should not be?
- No. Studio shows the option text as written.
- Judge read vs drawn
- The judge read the header "Retire the copy?", which is not shown. It did not read the title "Duplicate story", which is shown. This does not change the verdict.
- Cause and verdict
- Judge failure on this PR. Not run on develop.
- Proposed fix
- Write the step in full words, for example "It goes live when you publish."
- Not proven
- That this PR caused the fragment: develop was not run on this check.
AI citation
The first form was refused for a long option, but the person only sees the good retry
The agent put the whole proposed intro (over 240 characters) in an option description. The form tool refused it, and Studio drew nothing for it. The agent then put the intro in the question, and the form showed. The test fails because it counts the refusal, not because of what the person read.
AI citation: add a quotable intro to the About page
- What is wrong
- The first form broke the 240-character limit for an option description. Nothing wrong reached the person.
- Shown but should not be?
- No. The refused form is not shown (test H). This turn also has no sentence before the form.
- Judge read vs drawn
- The judge did not decide this one. The test failed on a form the person never saw.
- Cause and verdict
- Not this PR. A known form limit, tracked on its own.
- Proposed fix
- None in this PR. If the limit stays, the form tool's description can say where long wording goes (the question).
- Not proven
- How the judge reads the shown form: the test stopped at the refusal first.
chat-ui-tool 06 and 26
A plain request gets a preview card, and a "pick for me" request gets a question
In 06, the person asks for a new headline. The eval workspace cannot be edited. The agent shows a before/after card for an edit it did not make, then one sentence. In 26, the person asks "Pick the best cover for me." The agent asks which story the cover is for. The eval expects no question; a person grades it.
chat-ui-tool: cards and forms where the eval wants words or an action
- What is wrong
- 06: a preview card for a change that did not happen. 26: a question where the eval expects the agent to choose.
- Shown but should not be?
- No. Studio draws what the agent sent. Both are the agent's choice.
- Judge read vs drawn
- Not this judge. 06 uses an automatic scorer; 26 is graded by a person.
- Cause and verdict
- 06: mostly on develop too (passes 2 of 6 there, 0 of 6 here; the card shows 4 of 6 there, 6 of 6 here). 26: the agent asks 6 of 6 here, 4 of 6 on develop.
- Proposed fix
- 06: when an edit cannot happen, say so in words, with no preview card. 26: Marc decides if asking "which story?" is right; if yes, change the eval.
- Not proven
- The order of the card and the sentence in 06: not captured.
Proof
The drawing rules are tested; the pictures are mockups of stored words
Each layer below says what was checked and what was not. The drawing rules come from running Studio's own code on the stored parts. The words come from the captures. The pictures are drawn by hand from both. No live run, Dev Remote box or paid model call was made for this page.
A throwaway vitest renders the real MessageParts on the stored parts: 8 passed. It confirms the note shows next to a silent tool, folds behind a tool with a row, a closed form becomes "Question closed", the step header is hidden, and a refused form draws nothing.
Copied from the stored chat rows, the eval captures and the run logs on the review boxes, word for word.
Mockups at 400 px, built from Studio's classes and tokens. They are not screenshots of a running Studio. Agent names (SEO, GEO, Content, Compliance) come from the agent templates; eval chats show "Fimo".
Not captured. They decide whether Studio drew the note (likely) or only the last sentence.
Round 3's asking message for the approved close (only "so visitors can't reach it and Google treats it as unimportant." is known); the order of card and text in chat-ui-tool 06; the second leak's part order.