Read analytics & improve an agent

Advanced · ~10 min · Prerequisites: a published agent with usage

What you'll do

Open the analytics panel for a published agent, read the usage signals, translate what you find into concrete edits, and publish an improved version.

Open analytics

In the builder at /admin/agents/<slug>/edit, navigate to the Analytics tab (or open it from the agent's admin entry at /admin/agents). The panel shows activity since the agent was first published. You need at least a few real conversations before the data is meaningful — this guide assumes those exist.

For a complete reference to every metric and chart, see Analytics.

Read the signals

Focus on three areas:

Usage volume and trends. How many conversations has the agent had? Is usage growing, flat, or declining? Declining usage after an initial spike often means members tried it and found it unhelpful — worth investigating.

Common tasks and topics. What are members actually asking for? If the top topics don't match the role you designed, either the agent is being used differently than intended or the hub description is attracting the wrong requests. Both are fixable.

Failures and drop-offs. Look for conversations where the agent gave an unhelpful response, a tool error appeared, or the member sent multiple follow-up corrections. These are the clearest signal of where the instructions or tools fall short.

Turn findings into edits

Map each signal to a specific change:

• Wrong answers on a recurring topic → open the Instructions section and add explicit guidance for that topic, or add an example of the expected output.
• Tool errors or unexpected tool calls → open the Tools section, review which tools are enabled, and tighten the instructions around when each tool should (and shouldn't) be called.
• Members asking for things outside the agent's scope → update the agent's description in Identity & model so the hub card sets clearer expectations, and add a boundary statement to the instructions.
• Correct answers but poor formatting → add a formatting example to the instructions (show the agent what a good response looks like).

Make targeted edits — change one thing at a time so you can measure the effect of each change.

Publish and compare

After editing:

1. Test the updated draft in the test panel, running the exact prompts that produced failures in analytics.
2. When the test results look right, click Publish to create a new version.
3. Return to analytics after a few days of real usage and compare the failure rate and topic distribution against the prior period.

Repeat the loop — read, edit, publish, compare — until the agent's behavior matches its role. See Versions & publishing if you need to roll back a change that made things worse.

Try this next