Monitor performance and gain insights into your app’s usage.
Overview
The analytics dashboard provides real-time visibility into how your agentic app performs—tracking users, sessions, messages, and resource consumption across agents, tools, and models.
Dashboard Components
Key Metrics
┌─────────────────────────────────────────────────────────────────┐
│ Usage Overview │
├───────────────┬───────────────┬───────────────┬─────────────────┤
│ Users │ Sessions │ Messages │ Tokens │
│ 1,234 │ 3,456 │ 45,678 │ 2.3M │
│ ↑ 12% │ ↑ 8% │ ↑ 15% │ ↑ 18% │
└───────────────┴───────────────┴───────────────┴─────────────────┘
| Metric | Description |
|---|
| Users | Unique active users in the period |
| Sessions | Total conversation sessions |
| Messages | Messages exchanged (user + agent) |
| Tokens | Total token consumption |
Trends
Compare current period to previous: Daily/hourly breakdowns, Week-over-week comparisons, or Growth trends.
Run Analytics
Track execution across your app’s components.
Agent Runs Example
| Agent | Runs | Avg Response | Tokens | Success |
|---|
| Support Agent | 1,234 | 2.3s | 450K | 98.5% |
| Billing Agent | 567 | 1.8s | 180K | 99.1% |
| Order Agent | 890 | 3.1s | 320K | 97.8% |
Tool Runs Example
| Tool Type | Tool | Runs | Avg Time | Success |
|---|
| Workflow | get_order | 890 | 450ms | 99.2% |
| Code | validate_input | 1,200 | 120ms | 99.8% |
| MCP | crm_lookup | 456 | 890ms | 96.5% |
| Knowledge | faq_search | 2,100 | 340ms | 99.9% |
Model Runs Example
| Model | Invocations | Avg Latency | Tokens | Cost |
|---|
| gpt-4o | 3,400 | 1.2s | 1.8M | $45.20 |
| gpt-3.5 | 1,200 | 0.4s | 320K | $0.64 |
Traces
Traces provide detailed visibility into individual request lifecycles.
What’s in a Trace
Trace: req_abc123
├── Start: 2024-01-15 14:30:22.123
├── End: 2024-01-15 14:30:25.456
├── Duration: 3.333s
│
├── Events
│ ├── [14:30:22.123] Request received
│ ├── [14:30:22.145] Agent selected: Support Agent
│ ├── [14:30:22.200] Tool invoked: get_order_status
│ ├── [14:30:22.650] Tool response received
│ ├── [14:30:22.700] LLM generation started
│ ├── [14:30:25.400] LLM generation completed
│ └── [14:30:25.456] Response sent
│
├── Spans
│ ├── Agent Processing: 3.2s
│ ├── Tool Execution: 450ms
│ └── LLM Generation: 2.7s
│
└── Generations
└── Support Agent response
├── Model: gpt-4o
├── Input tokens: 1,234
├── Output tokens: 256
└── Latency: 2.7s
Trace Benefits
- Debug request flow issues.
- Identify bottlenecks.
- Understand agent behavior.
- Optimize performance.
Sessions
Sessions track continuous user interactions.
Session View
Session: sess_xyz789
├── User: user_456
├── Started: 2024-01-15 14:25:00
├── Duration: 12 minutes
├── Traces: 5
│
├── Trace 1: "What's my order status?"
│ └── Agent: Support Agent, Duration: 3.3s
│
├── Trace 2: "When will it arrive?"
│ └── Agent: Support Agent, Duration: 2.1s
│
├── Trace 3: "Can I change the address?"
│ └── Agent: Order Agent, Duration: 4.5s
│
├── Trace 4: "What's the cost?"
│ └── Agent: Billing Agent, Duration: 1.8s
│
└── Trace 5: "Thanks, that's all"
└── Agent: Support Agent, Duration: 0.8s
Total Cost: $0.12
Generations
Track individual LLM outputs within traces.
Generation Details
| Field | Value |
|---|
| Model | gpt-4o |
| Input tokens | 1,234 |
| Output tokens | 256 |
| Latency | 2.7s |
| Cost | $0.032 |
| Temperature | 0.7 |
Quality Assessment
- Review response quality.
- Identify hallucinations.
- Track instruction following.
Filtering
Customize your analytics view:
| Filter | Options |
|---|
| Time Range | Last hour, Last 24 hours, Last 7 days, Last 30 days, or Custom range |
| Environment | Draft (development), Staging, or Production |
| Dimensions | By agent, tool, model, or user segment |
Exporting Data
Download analytics for external analysis:
Available Exports
- CSV: Spreadsheet-compatible
- JSON: Programmatic analysis
- PDF: Shareable reports
Export Options
export:
format: csv
date_range: last_30_days
include:
- sessions
- traces
- generations
- tool_runs
filters:
environment: production
agent: Support Agent
Alerts
Configure notifications for important events:
Alert Types
alerts:
- name: High error rate
condition: error_rate > 5%
window: 1 hour
action: email
- name: Slow responses
condition: avg_latency > 5s
window: 15 minutes
action: slack
- name: Cost spike
condition: daily_cost > $100
window: 1 day
action: email
Audit Logs
Track all changes made across your account.
What’s Logged
- User actions (create, update, delete)
- Configuration changes
- Deployments
- Access events
Log Entry
Event: Tool Updated
User: alice@company.com
Time: 2024-01-15 14:30:00
Details:
Tool: get_order_status
Changes:
- timeout: 30s → 60s
- description: Updated
Compliance Uses
- Track who changed what.
- Maintain audit trail.
- Support security reviews.
Best Practices
Monitor Key Metrics
Focus on metrics that matter:
- Success rate: Are requests completing successfully?
- Latency: Is performance acceptable?
- Cost: Is spending within budget?
- User satisfaction: Are users getting help?
Set Baselines
Establish normal ranges to detect anomalies:
baselines:
success_rate: 95-99%
avg_latency: 1-3s
daily_cost: $20-50
Review Regularly
- Daily: Quick health check
- Weekly: Trend analysis
- Monthly: Deep dive and optimization
Act on Insights
Use analytics to drive improvements:
- Slow agent? Optimize tools or prompts.
- High error rate? Review configurations.
- Cost spike? Check token usage patterns.
