Langfuse gives you tracing and evaluations without lock-in, and it can be self-hosted, which is the deciding factor for anyone whose prompts contain customer data. Some governance features are commercial rather than open. The entries below say which tools are open source, which are hosted only, and which charge by trace volume.
8 tools reviewedLast reviewed Ranked by us, not by votes
What counts as LLM observability and evaluation
LLM observability and evaluation tools capture what an LLM or agent application actually did in production and let teams judge whether it did it well. They record traces (prompts, model calls, tool calls, retrievals, outputs, cost and latency) and they provide scoring tooling: code assertions, LLM-as-judge evaluators and human review. The evaluation half is what separates them from general application monitoring, because it turns production traffic into datasets and regression tests so quality changes are caught before users hit them. A tool belongs here if it does both the recording and the scoring; plain API gateways, generic APM and standalone prompt playgrounds do not.
How we judged them
Entries were judged on four things: tracing depth (whether the tool models agent steps, tool calls and multi-turn sessions rather than flat request logs), evaluation tooling (offline experiments, LLM-as-judge and code scorers, building datasets from production traces, regression detection), self-hostability (whether the full platform can run in your own infrastructure, and under what license or contract), and cost behavior at volume (which unit is metered and how predictable spend stays as trace volume grows). Every claim below was checked against the vendor's own site during this research pass. Pricing is described by shape only, because published figures go stale quickly; confirm current numbers on the vendor page before budgeting.
All 8 LLM observability and evaluation in this guide, in ranked order.
langfuse.com · Free managed tier plus paid cloud tiers metered in consumption units (traces, observations and scores) with included allowances and graduated overage; self-hosting the open source build is free.
Best for Teams that want full-featured tracing and evals without vendor lock-in, especially those who need the whole stack running inside their own infrastructure.
An open source LLM engineering platform covering tracing, evaluation, prompt management and datasets, available as managed cloud or self-hosted.
Strengths
Core platform is MIT licensed and the self-hosted build carries no usage limits, so cost at high trace volume is largely your own infrastructure spend.
Covers the full loop: tracing, scores and LLM-as-judge evals, datasets, experiments, annotation, prompt management and a playground.
Framework agnostic, with OpenTelemetry support and SDKs and integrations across common Python and JavaScript stacks.
Identical architecture across cloud, open source self-hosted and enterprise self-hosted, so moving between them does not require reinstrumenting your app.
Where it falls short
Some enterprise features (SCIM provisioning, extended audit logging, data retention policies) require a commercial license key when self-hosting.
A production self-hosted deployment means operating ClickHouse, Postgres, Redis and object storage yourself, which is real ongoing ops work.
The managed free tier limits how far back you can access data and how many users you get.
langchain.com · Free single-seat developer tier; paid tier is per seat plus usage-based trace charges, with extended retention priced separately; enterprise is custom quoted.
Best for Teams that want deep agent trace inspection plus mature online and offline eval tooling, and are content on managed cloud unless they can buy an enterprise plan.
LangChain's hosted platform for tracing, evaluating and monitoring LLM and agent applications.
Strengths
Step by step agent tracing with message threading for multi-turn conversations, and SDKs for Python, TypeScript, Go and Java.
Framework agnostic despite the LangChain name: it ingests OpenTelemetry and works with the OpenAI SDK, Anthropic SDK, Vercel AI SDK, LlamaIndex and custom code.
Evaluation covers both offline experiments and online LLM-as-judge and code evaluators running against live production traffic.
Automatic clustering of traces to surface recurring failure modes and usage patterns.
Operational alerting through webhooks and PagerDuty.
Where it falls short
Self-hosted and bring-your-own-cloud deployment are restricted to the Enterprise plan.
Managed cloud data is stored in a single Google Cloud region by default, which can conflict with data residency requirements.
Billing stacks per-seat charges on top of per-trace usage, and longer retention is charged separately, so spend grows along more than one axis at volume.
braintrust.dev · Free starter tier with capped allowances; paid tier is a flat monthly fee with included allowances and consumption overages across processed data, scores and model credits; enterprise is custom quoted.
Best for Teams whose central problem is proving quality and catching regressions, and who want production failures to feed directly into regression suites.
A commercial platform built around evaluation and experimentation for AI applications, with production tracing and dataset management attached.
Strengths
Evaluation is the core of the product: experiments against datasets, with scoring by LLM judge, code or human review.
Production traces convert into eval datasets in one step, so regression tests get built from real failures rather than synthetic examples.
Automatic topic clustering over production logs to surface recurring issues by task, issue and sentiment.
Playground for comparing prompts and models side by side against the same data.
Enterprise customers can run the data plane in their own infrastructure.
Where it falls short
Not open source, and on-prem or hybrid deployment is available only on the Enterprise tier.
Enterprise pricing is not published.
Metering runs across several separate dimensions at once (processed data volume, number of scores, model credits, optional extended retention), which makes spend at volume harder to forecast than a single-unit model.
arize.com · Phoenix is free to self-host; the managed Arize AX product has a free tier and a paid tier metered by span volume and storage, with enterprise custom quoted.
Best for Developers who want capable tracing and eval tooling running on their own laptop or cluster with no account and no per-trace bill.
A local-first, source-available platform for tracing, evaluating and experimenting on LLM and agent applications, built on OpenTelemetry.
Strengths
Runs locally, in Docker, on Kubernetes or in the cloud, and starts on a laptop in minutes.
Native OpenTelemetry support using the OpenInference specification, which reduces lock-in to any single vendor's instrumentation format.
Includes evals, dataset creation from traces, experiments and a prompt IDE, not just a trace viewer.
Free to self-host, so cost at volume is your own storage and compute rather than a metered bill.
Where it falls short
Licensed under Elastic License v2, a source-available license that restricts offering the software as a competing managed service; it is not a permissive license like MIT or Apache 2.0.
Scaling to managed hosting, long retention and team governance generally means moving onto the paid Arize AX product.
Self-hosted Arize AX is Enterprise tier only, and Enterprise pricing is not published.
wandb.ai · Free tier with capped ingestion and storage; paid tiers combine per-seat charges with usage-based Weave data ingestion and storage, with volume discounts tied to annual commitments; enterprise is custom quoted.
helicone.ai · Free tier with capped monthly requests and a single seat; paid tiers charge a fixed monthly fee plus usage-based overage on requests and storage; enterprise is custom quoted; self-hosting the open source build is free.
Best for Teams that want request-level logging, cost tracking and caching working within a day, with a self-host option available.
An open source LLM observability layer that logs requests through a gateway or asynchronously and reports usage, cost and latency analytics.
Strengths
Gateway integration means changing a base URL rather than instrumenting application code, which is the fastest onboarding in this list.
Apache 2.0 licensed with a documented docker compose self-host path.
Async logging via OpenLLMetry is available for teams that do not want requests routed through a proxy.
The gateway adds operational features beyond observability: caching, rate limiting, fallbacks and routing across many models.
Detailed cost and usage analytics broken down per user and per request.
Where it falls short
Evaluation and experiment tooling is thinner than in the eval-first platforms in this category.
The proxy path puts a third party in the live request path, adding a network hop and a failure mode; the async path avoids that but gives up gateway features.
The production Helm chart and on-prem deployment are gated behind contacting the enterprise team.
Cloud billing combines a fixed tier fee with metered requests and storage, and log ingestion rate limits differ by tier.
Best for Enterprises with data residency or network constraints that need traces to stay inside their own cloud while still buying a managed product.
A commercial agent observability and evaluation platform aimed at enterprise teams, offered as SaaS, hybrid or fully self-hosted.
Strengths
Three deployment models, including a hybrid mode where traces stay in your cloud and the vendor runs only the control plane, plus a Kubernetes-based fully self-hosted option.
Tracing built for long-running agents, with full trajectory visibility and replay across prompts, tool calls, outputs and handoffs.
Evaluation via LLM judges, code evaluators and human feedback, comparing agent versions on identical datasets.
Prompt management with CI/CD integration that runs eval suites on each release.
States SOC 2 Type II, GDPR and HIPAA compliance on its own site.
Where it falls short
No public pricing: the site routes buyers to a demo request or signup rather than listing tiers.
Not open source, so even the self-hosted deployment depends on a commercial agreement.
The self-hosted option is Kubernetes-based, so there is no lightweight single-container path for a small team to trial it in its own environment.
Fewer third-party integrations and less community-written material than the longest-established tools in this category.