Otter — AI Chat Copilot

Meet Otter, your AI copilot for querying data, managing resources, and creating dashboards

Meet Otter 🦦 — your AI-powered observability copilot built into OpenLIT. Otter lets you interact with your observability platform using natural language. It can query your telemetry data, create and manage platform resources, and generate complete dashboards — all through conversation.

Meet Otter

Otter is always just a click away via the floating chat button (bottom-right corner of every page) or from the Chat entry in the sidebar. Otter paddles through your traces, metrics, and logs so you don’t have to.

Key Features

• Natural Language Queries: Ask questions about your traces, metrics, costs, and tokens — the AI converts them to ClickHouse SQL and executes them
• Trace and Span Analysis: Run structured AI analysis on a full trace hierarchy or an individual span, with findings grouped by performance, reliability, cost, token efficiency, and execution path
• Prompt Improvement: Ask Otter to review an existing Prompt Hub prompt and suggest improvements without automatically saving changes
• Resource Management: Create and manage rules, contexts, prompts, vault secrets, and custom models through conversation
• Dashboard Generation: Describe the dashboard you want and get a complete importable layout with working queries
• SQL Execution: Generated queries run automatically with results displayed inline as tables and charts
• Save as Widget: Save any query result as a reusable dashboard widget

Get Started

1. Configure AI Provider

   Navigate to Chat → Settings (gear icon in chat sidebar) and configure:

   1. Provider — Select from 14 supported providers (OpenAI, Anthropic, Google, etc.)
   2. Model — Choose a model from the provider’s list (includes custom models from Manage Models)
   3. API Key — Select a vault secret containing your provider’s API key

   Store your API key in the Vault first, then select it in Chat Settings.

2. Start a Conversation

   Click New Chat in the sidebar or use the floating chat button available on any page.

   Try example prompts like:

   • “What are my top 5 most expensive models this week?”
   • “Show me error rate by provider over the last 24 hours”
   • “Create a rule that triggers when request duration exceeds 5 seconds”
3. View Results

   • SQL queries appear in a collapsible accordion with syntax highlighting
   • Query results display automatically as tables, bar charts, line charts, or pie charts
   • Entity cards link to created resources (rules, contexts, prompts)
   • Dashboard cards show an import button for generated dashboards

Data Queries

Ask questions about your OpenTelemetry data and the AI generates ClickHouse SQL with dynamic time filters.

Supported Tables

Table	Description
otel_traces	Traces and spans (LLM calls, API requests, etc.)
otel_metrics_gauge	Gauge metrics (GPU temperature, memory, etc.)
otel_metrics_sum	Counter/sum metrics (token counts, etc.)
otel_metrics_histogram	Histogram metrics (request durations, etc.)

Example Queries

"How many requests did I get today?"
"What is my total cost breakdown by provider?"
"Show me the slowest requests in the last hour"
"Which models have the highest error rate?"

Save as Widget

Click Save as Widget on any query result to:

1. Choose a widget type (Stat Card, Bar Chart, Line Chart, Pie Chart, Area Chart, Table)
2. Select a dashboard to add it to
3. The query is automatically converted to use dynamic time filters ({{filter.timeLimit.start/end}})

AI Analysis for Traces and Spans

Otter can analyze telemetry directly from chat or from the trace detail view. Use this when you want a structured explanation of what happened in a trace, why a span is slow or expensive, or which part of an agent path should be improved.

What Otter Reviews

Trace and span analysis evaluates:

• Performance: slow spans, blocking operations, duration hotspots, and inefficient execution paths.
• Reliability: failed spans, exception signals, retry patterns, and unstable downstream calls.
• Cost: model usage, high-cost spans, token usage, and estimated spend.
• Token efficiency: input/output token balance, cache usage, repeated context, and response verbosity.
• Prompt and model behavior: prompt structure, model selection, and output quality signals.
• Telemetry quality: missing span attributes, weak resource metadata, and incomplete trace context.
• Actionability: each analysis section is reviewed by a grader pass to make recommendations clearer and more useful.

How to Run It

You can ask Otter questions like:

"Analyze this trace and explain the cost drivers"
"Review span <span_id> and suggest improvements"
"Find token waste in traces for session.id dc7b1ba1-..."
"Analyze the slowest trace from the last hour"

When a trace or span is referenced from a telemetry view, Otter opens the same analysis experience with the relevant context already selected. Existing analysis runs are reused unless you explicitly ask for a fresh run.

Output

The response streams progress while each analysis dimension runs, then collapses into a completed analysis summary. Findings include the affected spans, severity, supporting evidence, and suggested fixes.

Prompt Improvement from Chat

Otter can review Prompt Hub prompts by name or ID. For review-only requests, it loads the prompt and suggests changes without mutating the saved prompt.

Example prompts:

"Can you help me improve prompt with prompt name music_recommend?"
"Review prompt customer_summary and make it more concise"
"Suggest safer output-format instructions for prompt_id <uuid>"

Otter only updates a Prompt Hub version when you explicitly ask it to save, apply, update, publish, or create a new version.

Resource Management

The chat can create and manage platform resources through natural language. All operations execute immediately.

Supported Operations

Resource	Operations
Rules	Create, update, delete, list, get details, link/unlink entities
Contexts	Create, update, delete, list
Prompts	Create, read, improve, update version, delete, list
Vault Secrets	Create, update, delete, list
Custom Models	Create, update, delete, list

Examples

"Create a vault secret called OPENAI_KEY with value sk-..."
"Create a rule named high-cost-alert that triggers when cost exceeds 0.1"
"Create a prompt template for customer feedback summarization"
"Link the customer-context to the high-cost-alert rule"
"List all my vault secrets"

Vault key names are automatically normalized to UPPER_SNAKE_CASE (e.g., “my openai key” becomes “MY_OPENAI_KEY”).

Dashboard Generation

Describe the dashboard you want and the AI generates a complete importable layout.

How it Works

1. Ask: “Create a GPU monitoring dashboard with utilization, memory, and temperature widgets”
2. The AI generates a dashboard JSON with:
   • Widget definitions (type, title, SQL query, display properties)
   • Grid layout positions
   • Dynamic time filters ({{filter.timeLimit.start/end}})
3. A dashboard card appears with:
   • Dashboard title and widget count
   • Widget preview chips
   • Import Dashboard button
   • Download JSON button
4. Click Import Dashboard to create it instantly

Dashboard Queries

All generated queries use the Mustache template pattern for dynamic time filtering:

WITH
  parseDateTimeBestEffort('{{filter.timeLimit.start}}') AS start_time,
  parseDateTimeBestEffort('{{filter.timeLimit.end}}') AS end_time
SELECT ...
FROM otel_traces
WHERE Timestamp >= start_time AND Timestamp <= end_time

This ensures dashboards respond to the time filter controls in the UI.

Cost & Token Tracking

Each conversation tracks:

• Per-message: Input tokens, output tokens, estimated cost
• Per-conversation: Accumulated totals displayed in the sidebar

The cost formula:

cost = (input_tokens / 1M) × input_price + (output_tokens / 1M) × output_price

Hover over the info icon next to the chat input to see your current provider, model, and pricing details.

Conversation Management

• URL-based navigation: Each conversation has a unique URL (/chat?id=<uuid>)
• Auto-generated titles: Titles are generated after the first message exchange
• Persistent history: Conversations and messages are stored in ClickHouse
• Search: Filter conversations by title in the sidebar
• Delete: Hover over a conversation and click the trash icon

Configuration

Chat Settings

Access via the gear icon in the chat sidebar or navigate to /chat/settings.

Setting	Description
Provider	LLM provider (OpenAI, Anthropic, Google, Mistral, Cohere, Groq, etc.)
Model	Model from the provider’s list (includes custom models)
API Key	Vault secret reference for the provider’s API key

Supported Providers

OpenAI, Anthropic, Google, Mistral, Cohere, Groq, Perplexity, Azure OpenAI, Together AI, Fireworks AI, DeepSeek, xAI, Hugging Face, Replicate

Security

• All SQL queries execute with readonly mode — no write operations possible
• Queries are validated against an allowlist of tables (otel_traces, otel_metrics_*)
• DDL/DML statements (INSERT, UPDATE, DELETE, DROP, etc.) are rejected
• API keys are stored in the Vault and never sent to the client
• Query results have a LIMIT cap of 10,000 rows