| Quick answer |
|---|
| The best AI tools for engineering performance tracking in 2026 are Langfuse, free and unlimited when self-hosted, LangSmith at $39 per seat for LangChain teams, and Braintrust at $249 a month when evaluation is the bottleneck. Arize Phoenix is the other free self-hosted option. |
| How we compare |
|---|
| Tools are ranked on what the free tier really includes, which unit the vendor meters — traces, spans, units or scores — and whether the tool runs inside your own infrastructure. Every price and overage rate was read off the vendor’s own pricing page on 12 September 2026 and links straight to it. Every price was re-checked against the vendor’s own pricing page in September 2026, and figures available only from third-party trackers are marked as reported. No vendor paid for placement. We ran no head-to-head lab test and publish no benchmark of our own; the only numbers we generated are the divisions shown below. |
Table of Contents
What engineering performance tracking means for AI systems
Tracking an AI feature is not tracking a service. Latency and error rate still matter, but the failure mode that costs you users is a request that returns 200 OK with a wrong answer. Standard APM never sees it.
So the LLM observability category splits into two jobs that vendors deliberately blur. LLM tracing records what happened on a single request: the prompt, retrieved context, tool calls, tokens and latency. AI evaluation asks whether a change made the system better across a fixed dataset. Most teams buy tracing, discover six weeks later that they cannot answer the second question, and buy again. Quality checks on model output are covered in our guide to AI evaluation tools for production.
Best AI tools for engineering performance tracking compared
| Tool | Entry price | Billing unit | Self-host |
|---|---|---|---|
| Langfuse | Free self-hosted; cloud $29/mo | Units (100,000 included on Core) | Yes, MIT licence, no unit limits |
| Arize Phoenix | Free self-hosted; AX Pro $50/mo | Spans and GB | Yes, source-available |
| LangSmith | Free for 1 seat; Plus $39/seat | Traces, plus per-seat | Enterprise only |
| Braintrust | Free Starter; Pro $249/mo | Spans, GB and scores | Enterprise only |
| Weights and Biases Weave | Free tier; Pro from $60/mo | Seats plus usage | Enterprise arrangement |
The billing unit matters more than the price
Nothing in this category is comparable on sticker price, because no two vendors meter the same thing. A trace contains many spans. A Langfuse unit is not a LangSmith trace. A Braintrust score is a separate meter again.
Here is one number nobody in this category publishes: what a million events a month costs on each vendor’s own overage rate. Two vendors publish a per-unit price you can extend, so the arithmetic below is theirs, not ours. Figures read from the pricing pages linked above in September 2026.
| Plan | Base price | Included | Published overage rate | Arithmetic for 1,000,000 a month | Total |
|---|---|---|---|---|---|
| Langfuse self-hosted | $0 licence | Unlimited units | None — you pay infrastructure | no licence meter | $0 in licence |
| Langfuse Cloud Core | $29 | 100k units | $8 per 100k units | 29 + (900,000 ÷ 100,000 × 8) | $101 |
| Langfuse Cloud Pro | $199 | 100k units | $8 per 100k units | 199 + (900,000 ÷ 100,000 × 8) | $271 |
| Arize AX Pro | $50 | 50k spans | $0.0008 per span | 50 + (950,000 × 0.0008) | $810 |
| Braintrust Pro | $249 | Unlimited spans, 5 GB data | $3 per extra GB | no span meter to extend | $249 if under 5 GB |
| LangSmith Plus | $39 / seat | 10k base traces | Metered in LCU and LSU | no published traces-per-unit ratio | Not derivable |
A Langfuse unit, an Arize span and a Braintrust span are not the same object, so this is a shape comparison, not a like-for-like price. The shape is still worth seeing: at a million events a month the two metered plans land at $101 and $810, an eight-fold gap between products whose entry prices are $29 and $50. Braintrust removes the event meter entirely on Pro and charges for data volume instead, so payload size rather than request count is the variable to watch there. And self-hosting Langfuse converts the entire line into infrastructure you already know how to budget.
The tools in detail
Langfuse
Open source under an MIT licence and self-hostable free with no unit limits, which makes it the default for teams that must keep prompts and outputs inside their own infrastructure — the same constraint that pushes teams toward running models offline in the first place. Cloud Hobby is free with 50,000 units a month; Core is $29 including 100,000 units, Pro $199, and Enterprise $2,499, with additional units at $8 per 100,000 on every paid cloud tier.
LangSmith
Developer is free for one seat with up to 5,000 base traces a month, then pay-as-you-go. Plus is $39 per seat per month with up to 10,000 base traces and unlimited seats. Overage moved to a two-unit model in 2026: LangChain Compute Units at $1.50 for work done and LangChain Storage Units at $1.00 for traces, with consumption varying by what your agents actually do. Tightest integration with LangChain and LangGraph, which is both its advantage and its lock-in.
Braintrust
Evaluation-first rather than tracing-first. Starter is free with $10 of model credits, 1 GB of processed data at $4 per extra GB, 10,000 scores and 14-day retention, plus unlimited users, projects, datasets and experiments. Pro is $249 a month with $100 of credits, 5 GB of processed data at $3 per extra GB, 50,000 scores and 30-day retention, with longer retention billed at $0.50 per GB per month. Note what is not metered on Pro: spans.
Arize Phoenix
Source-available observability for LLM applications, RAG pipelines and agent workflows, free to self-host and framework-agnostic. The hosted Arize AX free tier gives 25,000 spans and 1 GB a month; AX Pro is $50 a month for 50,000 spans and 10 GB, with additional spans at $0.0008 each and extra ingestion at $3 per GB. That published per-span rate is what makes the million-event comparison above possible at all — most vendors in this category do not offer one.
Weights and Biases Weave
Weights and Biases Weave traces model and function calls, capturing inputs, outputs, metadata, latency and token usage, with evaluations, scorers, production monitoring and LLM-as-a-judge built in. Weave is on the free Weights and Biases plan and Pro starts at $60 a month, but no per-unit Weave rate is published, so it cannot appear in the arithmetic above — scope volume with the vendor. For the operations side, our guide to AI ops tools for operational performance covers monitoring in production.
How to choose in one afternoon
- If prompts or outputs cannot leave your network, the shortlist is Langfuse or Arize Phoenix. Both self-host free; everything else is an enterprise conversation.
- If you are on LangChain or LangGraph, LangSmith saves days of instrumentation. Accept that this is the lock-in you are buying.
- If you already have traces and still cannot ship confidently, the gap is evaluation, and Braintrust is built for exactly that gap.
- If you already run Weights and Biases for training, Weave keeps experiments and production in one place.
- Instrument against the OpenTelemetry GenAI semantic conventions where the SDK allows it. Emitting the standard attribute names means switching vendor later is a config change rather than a rewrite, and it is the one decision on this page that costs nothing and saves the most.
On the failure modes these tools are meant to catch, our explainer on AI hallucinations covers what to score for, fine-tuning versus RAG covers the architecture choice upstream of it, and multi-agent AI systems covers why traces get hard to read once agents call agents. For the tooling your team writes the code with, see our AI coding tools comparison. For checking the code before it ships, see our guide to AI code review tools.
Frequently Asked Questions
What are the best AI tools for engineering performance tracking?
Langfuse is the best default because the MIT-licensed core self-hosts free with no unit limits, and Braintrust at $249 a month is the strongest if evaluation rather than tracing is the bottleneck. LangSmith at $39 per seat suits LangChain teams, and Arize Phoenix is the other free self-hosted option.
What does evaluation tooling cost at production volume for engineering managers?
Far more than the entry price, because every vendor meters a different unit. On published September 2026 rates, one million Langfuse cloud units on Core is $101. The same volume on Arize AX Pro is $810. Braintrust Pro is $249 with no span meter at all, so its bill moves with data volume rather than request count. LangSmith now meters in LangChain Compute and Storage Units at $1.50 and $1.00, with no published traces-per-unit ratio, so only its own calculator can answer for your workload. Those units are not equivalent, so treat the comparison as shape rather than price.
Do you need a dedicated eval tool for engineering managers if you already log traces?
Only when you need to answer “did this change make it better?” Tracing tells you what happened on one request. Evaluation tells you whether release B beats release A across a fixed dataset, which needs versioned datasets, scorers and a comparison view that trace logs do not provide. If you are shipping prompt or model changes weekly, you need both. If you ship quarterly, tracing plus manual review is usually enough.
How do you evaluate LLM output without ground-truth labels for engineering managers?
Use reference-free scorers and comparisons. LLM-as-a-judge grades an output against a rubric rather than a gold answer. Groundedness and faithfulness scorers check the output against the retrieved context, which you already have in RAG. Pairwise comparison asks a judge which of two outputs is better, avoiding absolute scoring altogether. Then sample a few dozen judged cases for human review to check the judge itself, because an uncalibrated judge produces confident nonsense.
How much do AI engineering performance tracking tools cost?
List prices read off each vendor’s pricing page on 12 September 2026: Langfuse self-hosted is free and unlimited, cloud Hobby free for 50,000 units, Core $29 for 100,000, Pro $199 and Enterprise $2,499. LangSmith Developer is free for one seat with 5,000 base traces, Plus $39 per seat with 10,000. Braintrust Starter is free, Pro $249. Arize Phoenix self-hosts free; hosted Arize AX Free covers 25,000 spans and AX Pro is $50 a month for 50,000. Weights and Biases includes Weave on its free plan, with Pro from $60 a month.
Conclusion
Start with Langfuse self-hosted. It is free, MIT-licensed, imposes no unit limits, and gives you real LLM tracing without a procurement cycle. If instrumentation time matters more than licence cost and you are already on LangChain, LangSmith Plus at $39 per seat is the faster route.
Add Braintrust when the question stops being “what happened on that request” and becomes “is this release better than the last one”. That is the moment tracing stops being enough — and the moment most teams discover they have been measuring throughput while their users were measuring correctness. One clarification: everything on this page monitors the AI system you ship. Monitoring how AI systems describe your company is a separate category with separate tools, covered in our guide to AI search monitoring platforms.
