GPU Performance Monitoring

Monitor NVIDIA and AMD GPUs with key metrics like usage, temperature, and power using OpenTelemetry for AI workloads

OpenLIT uses OpenTelemetry to help you monitor NVIDIA and AMD GPUs for AI applications. Track GPU metrics like utilization, temperature, memory usage, and power consumption during AI training and inference workloads.

Choose your method

GPU monitoring can be implemented in two ways depending on your setup and requirements:

OpenLIT SDK

It is useful if you already have an AI application running on GPU that’s instrumented with OpenLIT.

It extends your existing observability to include GPU metrics alongside your LLM traces.

OpenTelemetry GPU Collector

It is useful for remote GPUs with only LLM models hosted, containerized deployments.

This approach allows you to get GPU metrics without modifying application code.

Supported Parameters

SDK Configuration Options

Parameter	Type	Default	Description
collect_system_metrics	boolean	False	Enable GPU and system metrics collection
otlp_endpoint	string	None	OpenTelemetry OTLP endpoint URL
otlp_headers	string	None	Authentication headers for OTLP endpoint
service_name	string	"unknown_service"	Name of your AI application
environment	string	None	Deployment environment (dev, staging, prod)

Environment Variables

Variable	Description	Example
OPENLIT_COLLECT_SYSTEM_METRICS	Enable GPU monitoring	true
OTEL_EXPORTER_OTLP_ENDPOINT	OTLP endpoint URL	http://127.0.0.1:4318
OTEL_SERVICE_NAME	Service name for telemetry	my-gpu-app
OTEL_DEPLOYMENT_ENVIRONMENT	Deployment environment	production

---

Deploy OpenLIT

Deployment options for scalable LLM monitoring infrastructure

Integrations

60+ AI integrations with automatic instrumentation and performance tracking

Destinations

Send telemetry to Datadog, Grafana, New Relic, and other observability stacks