perf: session cache mirror writes full message arrays on every SSE token #460

New issue

Open

opened 2026-07-25 23:35:26 +02:00 by dries · 0 comments

dries commented

2026-07-25 23:35:26 +02:00

Owner

Code-quality audit (P2). frontend/src/pages/session-detail/useSession.ts:361-374.

useEffect(() => {
  if (!sessionId || !view.session) return;
  if (view.sessionId !== sessionId) return;
  const { defaultAgent, defaultModel, ...sessionForCache } = view.session;
  updateCachedSession(sessionId, (prev) => ({
    ...prev,
    session: sessionForCache,
    messages: view.messages,
    parts: view.parts,
    totalMessages: Math.max(prev.totalMessages ?? 0, totalMessages),
  }));
}, [sessionId, view.sessionId, view.session, view.messages, view.parts, totalMessages, updateCachedSession]);

Why it matters

The effect depends on view.messages / view.parts, whose identity changes on every streaming token delta. Each run clones the entire sessionCache Map (apiStore.ts:209) and writes a new SessionDetail, which notifies every useApiStore subscriber. At token rate, with up to 200 retained messages and their parts, that is a Map clone plus a store-wide notification per token — the dominant cost of watching a streaming session.

The cache exists only to make a revisit warm. It does not need to be current mid-stream.

Suggested fix

Debounce the mirror (~500 ms), or better: flush only on the session.idle edge and on unmount. Both give the same warm-revisit behaviour at a tiny fraction of the writes.

Acceptance criteria

Streaming a session produces O(1) cache writes per turn, not O(tokens).
The cache is still correct after navigating away mid-stream (unmount flush).
Test asserts the write count over a burst of simulated deltas.

Effort: S.

Code-quality audit (P2). `frontend/src/pages/session-detail/useSession.ts:361-374`. ```js useEffect(() => { if (!sessionId || !view.session) return; if (view.sessionId !== sessionId) return; const { defaultAgent, defaultModel, ...sessionForCache } = view.session; updateCachedSession(sessionId, (prev) => ({ ...prev, session: sessionForCache, messages: view.messages, parts: view.parts, totalMessages: Math.max(prev.totalMessages ?? 0, totalMessages), })); }, [sessionId, view.sessionId, view.session, view.messages, view.parts, totalMessages, updateCachedSession]); ``` ## Why it matters The effect depends on `view.messages` / `view.parts`, whose identity changes on **every streaming token delta**. Each run clones the entire `sessionCache` Map (`apiStore.ts:209`) and writes a new `SessionDetail`, which notifies every `useApiStore` subscriber. At token rate, with up to 200 retained messages and their parts, that is a Map clone plus a store-wide notification per token — the dominant cost of watching a streaming session. The cache exists only to make a *revisit* warm. It does not need to be current mid-stream. ## Suggested fix Debounce the mirror (~500 ms), or better: flush only on the `session.idle` edge and on unmount. Both give the same warm-revisit behaviour at a tiny fraction of the writes. ## Acceptance criteria - [ ] Streaming a session produces O(1) cache writes per turn, not O(tokens). - [ ] The cache is still correct after navigating away mid-stream (unmount flush). - [ ] Test asserts the write count over a burst of simulated deltas. Effort: S.