Skip to content

feat(billing): drop trace table, make 'analyse my spend' clickable#2331

Open
pauldambra wants to merge 3 commits into
mainfrom
posthog-code/spend-banner-drop-traces-add-analyse-cta
Open

feat(billing): drop trace table, make 'analyse my spend' clickable#2331
pauldambra wants to merge 3 commits into
mainfrom
posthog-code/spend-banner-drop-traces-add-analyse-cta

Conversation

@pauldambra
Copy link
Copy Markdown
Member

Problem

Two issues with the PostHog Code token spend banner:

  1. The Top traces table is noise. Trace IDs are opaque strings (UUIDs or JSON-shaped device blobs like {"device":"...","session":"..."}), the list is long, and there's no actionable next step from seeing it. Users get more information than they can use.
  2. The "drop the exploring-llm-costs skill into your agent" line is a dead link. It links to the SKILL.md on GitHub, but that doesn't actually move the user toward the analysis they want. They have to manually start a new task, paste the cost data in, ask for advice, and remember to load the skill — friction nobody overcomes for a "maybe save some money later" task.

Changes

  1. Drop the TraceTable component and the trace-related branch in generateSuggestions. Backend PR #59796 deprecates the top_traces API field (returns empty); removing the rendering here means we don't show an empty section.
  2. Replace the static exploring-llm-costs GitHub link with a Sparkle-iconed button: "Ask an agent to analyse this and suggest reductions". Clicking it calls navigateToTaskInput({ initialPrompt }) with a markdown report of the user's spend (summary, by_product, by_tool top 10, by_model) plus a prompt instructing the new agent to load the skill from the PostHog skill store via mcp__posthog__execllma-skill-get and rank reduction advice by impact.

The prefill saves the new task an API round-trip and gives the agent the full breakdown in its initial context, so it can answer "what should I do to spend less" immediately without fetching data first.

How did you test this?

I'm an agent — Claude Code via PostHog Code.

  • pnpm exec biome check apps/code/src/renderer/features/billing/components/TokenSpendAnalysisBanner.tsx — clean.
  • pnpm -F @posthog/code exec tsc --noEmit — only the existing unrelated auth/service.test.ts errors, none in the modified file.
  • No manual UI run — I haven't booted the app. Worth a human eyeball on (a) the button styling/placement, (b) the prefilled prompt arriving correctly in the new task input, (c) the new task actually loading the skill on its first turn.

Publish to changelog?

no


Created with PostHog Code

Two improvements to the PostHog Code token spend banner:

1. Drop the Top traces table. Trace IDs are opaque strings (UUIDs or
   JSON-shaped device blobs) that aren't actionable, and the list is
   too long to scan. The corresponding API field is being deprecated
   on the backend in PR #59796 (returns empty there); removing the
   rendering means we don't show an empty section.

2. Replace the static GitHub link to `exploring-llm-costs` with a
   button that opens a new task prefilled with a markdown report of
   the user's spend (summary, by_product, by_tool top 10, by_model)
   plus a prompt asking the agent to load the skill from the PostHog
   skill store (`mcp__posthog__exec` -> `llma-skill-get`) and rank
   reduction advice by impact.

The prefill saves the new task an API round-trip and gives the agent
the full breakdown in its initial context, so it can answer the
'what should I do to spend less' question immediately without
fetching data first.

Generated-By: PostHog Code
Task-Id: f9d5d152-49c6-46cf-8fde-079105ba2e67
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 23, 2026

Comments Outside Diff (1)

  1. apps/code/src/renderer/features/billing/components/TokenSpendAnalysisBanner.tsx, line 422-425 (link)

    P2 The initial callout (shown before data loads) still advertises "by trace" — but the trace table was removed in this PR, so the description will be stale. Consider dropping "by trace" from the copy to match the current feature set.

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: apps/code/src/renderer/features/billing/components/TokenSpendAnalysisBanner.tsx
    Line: 422-425
    
    Comment:
    The initial callout (shown before data loads) still advertises "by trace" — but the trace table was removed in this PR, so the description will be stale. Consider dropping "by trace" from the copy to match the current feature set.
    
    How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
apps/code/src/renderer/features/billing/components/TokenSpendAnalysisBanner.tsx:422-425
The initial callout (shown before data loads) still advertises "by trace" — but the trace table was removed in this PR, so the description will be stale. Consider dropping "by trace" from the copy to match the current feature set.

### Issue 2 of 2
apps/code/src/renderer/features/billing/components/TokenSpendAnalysisBanner.tsx:242-261
Markdown table injection from unescaped data values — if a `product`, `tool`, or `model` name contains a pipe character `|`, the generated markdown table will have misaligned cells, causing the agent to misread row boundaries. For example a tool name like `bash | grep` would silently split into extra columns. Consider replacing `|` with `\|` (or a similar escape) in all cell values before interpolation.

Reviews (1): Last reviewed commit: "feat(billing): drop trace table, make 'a..." | Re-trigger Greptile

…task

Paul-reviewer flagged the previous "Ask an agent to analyse this and
suggest reductions" copy as reading like it kicks off something inline,
when it actually navigates to a new task input with a prefilled prompt.
"Open a task to analyse this with an agent" makes the destination
explicit.

Generated-By: PostHog Code
Task-Id: f9d5d152-49c6-46cf-8fde-079105ba2e67
@pauldambra
Copy link
Copy Markdown
Member Author

Note

🤖 Automated comment by QA Swarm — not written by a human

Single-reviewer pass: paul-reviewer (focused review chosen for this small frontend-only change).

Verdict: 💬 APPROVE WITH NITS → 1 fix pushed, rest deferred

The diff is doing the right things — drop the unactionable trace table, swap the static GitHub link for a "send the spend data into a fresh agent task" affordance that mirrors the inbox `ReportDetailPane` pattern. No blockers.

Fixed in `a457d50e`

  • 🟢 (NIT) Button copy — "Ask an agent to analyse this and suggest reductions" read like it kicked off something inline. Changed to "Open a task to analyse this with an agent" to make the destination explicit.

Deferred (non-blocking)

  • 🟡 (MEDIUM) MCP soft dependency — the prompt asks the new agent to load `exploring-llm-costs` via `mcp__posthog__exec` → `llma-skill-get`. If the receiving agent doesn't have the posthog MCP wired up (or uses a different server name), the instruction silently no-ops. Mitigation: the prompt already includes the full data in the initial context, so even without the skill the agent has the numbers and can give generic advice. The skill is the playbook; the data is the substance.
  • 🟢 (LOW) Missing analytics event — no `track(ANALYTICS_EVENTS.X)` call in the button handler. This repo's analytics layer would need a new event added to `ANALYTICS_EVENTS` + `EventPropertyMap`, which is a bigger change than the finding warrants. Worth a follow-up so we can measure: did anyone click the button → did it open a task → did the task produce a useful answer?
  • 🟢 (LOW) Format-string duplication — `buildAnalysisPrompt` re-derives the same `formatUsd` / `formatTokens` / "(no tool)" / "(unknown)" fallback strings the React tables use. If someone tweaks the tables, the prompt drifts silently. Acceptable for now; consider extracting shared format helpers if a second prompt-builder appears.
  • (NIT) Skill name string coupling — `SKILL_NAME = "exploring-llm-costs"` is a magic-string contract with the PostHog skill store. If the skill gets renamed, this silently asks for one that doesn't exist. Future fragility note, not a fix today.

Reviewer summary

Reviewer Assessment
👤 paul Ship as you see fit. Diff is the right shape; the main concerns are about resilience of the MCP-skill-load handshake and a missing analytics event so we can measure adoption.

Automated by QA Swarm — not a human review

Greptile flagged that tool/product/model names containing `|` would
split markdown-table cells mid-row, causing the receiving agent to
misread the row boundaries. For example a tool name like `bash | grep`
would silently extend the row by an extra column.

Add a small `escapeTableCell` helper and apply it to every cell value
that comes from the data (product, tool, model names). Numbers and
$-prefixed formatted strings are pipe-free.

Generated-By: PostHog Code
Task-Id: f9d5d152-49c6-46cf-8fde-079105ba2e67
* agent_mode string). Unescaped pipes would split the cell mid-row and the receiving agent
* would misread row boundaries. */
function escapeTableCell(value: string): string {
return value.replace(/\|/g, "\\|");
@pauldambra pauldambra added the Stamphog This will request an autostamp by stamphog on small changes label May 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Stamphog This will request an autostamp by stamphog on small changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants