Monitor TypeSafe Jev using OpenTelemetry

Monitor TypeSafe Jev System One calls with OpenTelemetry auto-instrumentation for decision spans, token usage, and cost tracking.

OpenLIT uses OpenTelemetry Auto-Instrumentation to monitor applications that call TypeSafe Jev through the official SDKs. This includes tracking latency, token usage, cost, and the questions sent to System One.

Auto-instrumentation means you don’t have to set up monitoring manually. Add OpenLIT to the application and TypeSafe system_one / systemOne calls emit decision spans.

The integration is compatible with:

• TypeSafe Python SDK typesafe-sdk >= 0.6.0
• TypeSafe TypeScript/JavaScript SDK @typesafe-ai/sdk >= 0.6.0

OpenLIT wraps the native System One API (TypeSafeClient.system_one in Python and TypeSafeClient.systemOne in TypeScript). Vercel AI Gateway experimental_evaluate is not instrumented in this release; that path can be added when AI SDK evaluate is stable.

Get started

1. Install OpenLIT

   Open your command line or terminal and run:

   Python

   pip install openlit
   

   TypeScript

   npm install openlit
   
2. Initialize OpenLIT in your Application

   Python

   Zero Code Instrumentation

   Perfect for existing applications - no code modifications needed:

   Via CLI Arguments

   # Configure via CLI arguments
   openlit-instrument \
     --service-name my-ai-app \
     --environment production \
     --otlp-endpoint YOUR_OTEL_ENDPOINT \
     python your_app.py
   

   Via Environment Variables

   # Configure via environment variables
   export OTEL_SERVICE_NAME=my-ai-app
   export OTEL_DEPLOYMENT_ENVIRONMENT=production
   export OTEL_EXPORTER_OTLP_ENDPOINT=YOUR_OTEL_ENDPOINT
   
   # Run with zero code changes
   openlit-instrument python your_app.py
   

   Perfect for: Legacy applications, production systems where code changes need approval, quick testing, or when you want to add observability without touching existing code.

   One-Line Instrumentation

   Via Function Parameters

   import openlit
   
   openlit.init(otlp_endpoint="YOUR_OTEL_ENDPOINT")
   

   Via Environment Variables

   Add the following two lines to your application code:

   import openlit
   
   openlit.init()
   

   Then, configure the your OTLP endpoint using environment variable:

   export OTEL_EXPORTER_OTLP_ENDPOINT=YOUR_OTEL_ENDPOINT
   

   Typescript

   One-Line Instrumentation (SDK)

   Via Function Parameters

   import openlit from "openlit"
   
   openlit.init({ otlpEndpoint: "YOUR_OTEL_ENDPOINT" })
   

   Via Environment Variables

   Add the following two lines to your application code:

   import openlit from "openlit"
   
   openlit.init()
   

   Then, configure the your OTLP endpoint using environment variable:

   export OTEL_EXPORTER_OTLP_ENDPOINT=YOUR_OTEL_ENDPOINT
   

   Replace: YOUR_OTEL_ENDPOINT with the URL of your OpenTelemetry backend, such as http://127.0.0.1:4318 if you are using OpenLIT and a local OTel Collector.

   To send metrics and traces to other Observability tools, refer to the supported destinations.

   For more advanced configurations and application use cases, visit the OpenLIT Python repository or OpenLIT Typescript repository.

What is captured

Each System One call creates a CLIENT span named decision {model} (for example decision jev-1.13.0). That follows the OpenTelemetry GenAI span-name format {gen_ai.operation.name} {gen_ai.request.model}. The operation is decision rather than chat so Jev traces are not auto-evaluated as chat completions. Provider is gen_ai.provider.name=typesafe. Shared inference attributes match OpenAI chat spans where they apply. Sampling knobs such as temperature are omitted because System One has no such request fields.

Instrumentor aliases: typesafe, jev, typesafe-ai, typesafe_sdk.

Set TYPESAFE_API_KEY in the environment. Filter traces on span name decision jev-1.13.0 (or decision jev-latest).

Data	Attribute
Operation	gen_ai.operation.name = decision
Provider	gen_ai.provider.name = typesafe
Model	gen_ai.request.model
Stream	gen_ai.request.stream (always false)
Finish reasons	gen_ai.response.finish_reasons
Token usage / cost	gen_ai.usage.*, gen_ai.client.token.usage
Question ids and types	typesafe.question_ids, typesafe.question_types

Example

Call openlit.init() once. Keep using the official TypeSafe client. No extra OpenTelemetry setup in application code.

Python

import openlit
from typesafe_sdk import TypeSafeClient

openlit.init(otlp_endpoint="http://127.0.0.1:4318")

client = TypeSafeClient()
result = client.system_one(
    state={
        "prompt": "What is the refund window?",
        "response": "Enterprise customers get 60 days.",
        "ground_truth_context": "Enterprise refunds: 30 days from purchase.",
    },
    questions={
        "hallucination": {
            "type": "noul",
            "instructions": "Flag invented or contradictory claims versus the ground-truth context.",
        }
    },
    model="jev-1.13.0",
)
print(result.answers["hallucination"].noul)

TypeScript

import openlit from "openlit"
import { TypeSafeClient } from "@typesafe-ai/sdk"

openlit.init({ otlpEndpoint: "http://127.0.0.1:4318" })

const client = new TypeSafeClient()
const result = await client.systemOne({
  state: {
    prompt: "What is the refund window?",
    response: "Enterprise customers get 60 days.",
    ground_truth_context: "Enterprise refunds: 30 days from purchase.",
  },
  questions: {
    hallucination: {
      type: "noul",
      instructions:
        "Flag invented or contradictory claims versus the ground-truth context.",
    },
  },
  model: "jev-1.13.0",
})
console.log(result.answers.hallucination.noul)

AsyncTypeSafeClient.system_one is instrumented the same way in Python.

Configuration

By default OpenLIT records metadata, usage, and cost but not the content of your System One calls. Enable capture_message_content (captureMessageContent in TypeScript) to include the state and questions on the span, for example while debugging. Leave it off if your inputs contain sensitive data.

Python

openlit.init(
    otlp_endpoint="http://127.0.0.1:4318",
    capture_message_content=True,
)

TypeScript

openlit.init({
  otlpEndpoint: "http://127.0.0.1:4318",
  captureMessageContent: true,
})

You can also set OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true. See SDK configuration for every option.

Use Jev as an OpenLIT judge

TypeSafe Jev is also an evaluation provider in the OpenLIT UI. Select TypeSafe and a Jev model under Evaluations → Configuration, store the API key in Vault, and keep your existing evaluation types. Issue types map to noul; quality types map to score. See Evaluators. Jev is not available in Chat or OpenGround.

Pin jev-1.13.0 when a verdict threshold depends on a model version. jev-latest moves when TypeSafe ships a release.

---

Quickstart: LLM Observability

Production-ready AI monitoring setup in 2 simple steps with zero code changes

Configuration

Configure the OpenLIT SDK according to your requirements.

Evaluators

Use TypeSafe Jev as the in-product evaluation judge

Destinations

Send telemetry to Datadog, Grafana, New Relic, and other observability stacks