Seven Open-Source LLM Ops Platforms, One Table: Pick by the Row You Can’t Ship Without

Nobody wins this table. Seven self-hostable LLM ops platforms, eleven rows, and every column has at least two cells it would rather you didn’t read.An LLM ops platform is the layer between your application and the model provider, or beside it, that records every call, versions the prompts, runs…

Nobody wins this table. Seven self-hostable LLM ops platforms, eleven rows, and every column has at least two cells it would rather you didn’t read.An LLM ops platform is the layer between your application and the model provider, or beside it, that records every call, versions the prompts, runs evaluations, and tracks spend. You install one when a production app has more prompts, models and bills than you can follow by reading logs.A table with no overall winner is still useful, because each row has a clear one. If you need a spend cap or a cached response, three of the seven can do it and four can’t, by architecture. If you need a human labeling queue, the list flips: three have one, four don’t. Pick the row you can’t ship without, read that row across, and the shortlist is usually two or three names long before you look at anything else.Compared the LLM Ops platformWe self-hosted all seven and built the same prompt on each: a support triage agent that changes its instructions for VIP customers and lists their open tickets. Same customer message, same downstream model.The rows are the questions that came up while doing that, plus every capability at least three competitors have and AcruxCore doesn’t.Compare 7 LLM Ops platformsThe rest of this post walks the rows in the order they tend to decide a choice: where the platform sits, what your prompt can do, who runs your tools, what “open source” means for each, what each one leaves for you to build, and what sitting in the request path costs in milliseconds.Row 3 decides more than any other: record or decision pointTwo different shapes hide under one category name.Beside your request path. Your code calls the provider directly. An SDK wraps that call and ships a trace afterwards. Langfuse, Laminar, Opik and Phoenix work this way.In your request path. Your code calls the platform, and the platform calls the provider. Helicone, MLflow’s AI Gateway and AcruxCore work this way.A trace ingested afterwards is a record. A request passing through a gateway is a decision point. Only a decision point can swap in a cheaper model, return a cached response instead of paying for a new one, refuse a key that has passed its monthly budget, or hand your team a virtual key that never exposes the real provider key.You can’t add those four later from beside the path. By the time an ingest-only tool sees the call, the money is spent, and the response exists. That’s architecture, not roadmap, and it’s why one cell in row 3 predicts most of what a platform can and can’t do about cost.The trade runs the other way too. A tool beside your path works with any provider, any SDK, and any framework, with nothing new in front of production traffic that can fail. Laminar commits fully to that shape and auto-instruments fifteen-plus agent frameworks from one line of setup. If you already enforce budgets in your own code, or you run a separate proxy, the gateway row may not matter to you at all.Row 5: two platforms let a prompt branch, five make you do it in codePrompt templating looked like a formality when we built the grid. It wasn’t.Our test prompt needs a branch and a loop. VIP customers get one instruction, everyone else gets another, and the ticket list has unknown length. On MLflow and AcruxCore, that logic lives in the stored template, and the server renders it:You are a support triage agent for {{ company }}.{% if is_vip %}This customer is VIP. Prioritize them and skip standard hold times.{% else %}Standard support flow applies.{% endif %}{% for ticket in tickets %}- #{{ ticket.id }}: {{ ticket.title }}{% endfor %}The other five do flat {{variable}} substitution, so the branch can't live in the stored prompt. It moves into your code. We wrote that flattening five times, once per platform. This is the shape it takes:# Same prompt, on a platform that only substitutes variables.# The stored template can't branch or loop, so this runs before every call.def build_system_prompt(variables): lines = [f"You are a support triage agent for {variables['company']}."] if variables["is_vip"]: lines.append("This customer is VIP. Prioritize them and skip standard hold times.") else: lines.append("Standard support flow applies.") for ticket in variables["tickets"]: lines.append(f"- #{ticket['id']}: {ticket['title']}") return "\n".join(lines)Both produce identical text. What changes is where the prompt’s behaviour is versioned. In the first, changing how VIPs are treated is a new prompt version, reviewable and revertable by whoever owns the prompt. In the second, it’s a code deploy, and the platform’s version history shows a string that never changed.The partial cells need a word each. Langfuse documents storing a Jinja2 template and rendering it yourself with an external library, which keeps the version history but takes the playground out of the loop. Opik has a Jinja2 prompt type in its SDK, and rendering it in the playground is an open feature request.Laminar has no prompt registry at all: its playground is one mutable row of messages, which is deliberate, because the product is about agent runs rather than the calls inside them.MLflow’s prompt registry is the one competitor that matches AcruxCore here, with full Jinja2 plus a version diff and @production style aliases.Row 4: six store the tool’s schema, one runs itEvery tool you give a model has two halves. The definition is the JSON schema the model reads to decide whether to call it. The implementation is the code that runs when it does. Each platform has to decide who owns which.Six of the seven own only the first half, or neither. Langfuse and Laminar keep a schema in a field beside a prompt, and nothing executes it. MLflow catalogs whole MCP servers, where MCP is the Model Context Protocol, a standard way to expose tools to a model.The unit there is a server, not a tool, so there’s no per-tool version history. Helicone, Opik, and Phoenix have nothing in this row, which fits their shape: your tool call arrives as one more span in the trace.AcruxCore is the only one where a tool is a versioned record the gateway itself calls and measures. That gives the tool the same version history and rollback your prompt gets. It also has a cost worth naming: the tool’s endpoint and its transform then live on the platform instead of in your repo. For the other six, the implementation stays in your code, which is where plenty of teams want it.Rows 1 and 6: three different things called open sourceFive of the seven are plain Apache 2.0: AcruxCore, Helicone, Laminar, MLflow, and Opik. Install it, change it, run it commercially.Langfuse is MIT with an exception. Its LICENSE covers the repo except for a ee/ folder under a separate commercial license, unlocked with an enterprise license key. The audit log lives in that folder. That's why the free self-hosted install doesn't have one, and why row 6 has exactly one ✅ across seven platforms.Phoenix is Elastic License 2.0. That’s source-available: you can read it and self-host it, but it isn’t OSI-approved, isn’t permissive, and you can’t offer it as a managed service. If your legal review has a “must be OSI-approved” line, Phoenix fails it and the other six pass.Two more of the seven are open-source front ends to paid products, and that’s the easiest thing in this table to get wrong. MLflow’s human labeling sessions need Databricks’ hosted MLflow. Phoenix’s alerting lives in the paid Arize AX product. A feature in the docs isn’t always a feature in your install. Check which host the page you’re reading describes.Rows 7 to 10: what each platform leaves for you to buildThese four rows exist because a comparison drawn only from our own feature list would show AcruxCore winning nearly everything, and that list is our feature set. So the grid also carries every capability at least three of the six competitors have, and we don’t. AcruxCore loses all four.Guardrails. Opik ships topic and PII guardrails configurable per project. Helicone runs Prompt Guard and Llama Guard behind a request header. MLflow attaches PII and safety guardrails per gateway endpoint. Laminar has a project-level PII redaction toggle. Langfuse offers a masking hook in the SDK, with server-side masking behind Enterprise. Phoenix traces a third-party setup rather than shipping its own. AcruxCore inspects nothing in a call’s content.Alerts. Helicone wires a threshold to a Slack channel or an email address. Langfuse’s Monitors alert on cost, quality, or latency to Slack, webhooks, or GitHub Actions. Opik has Slack, PagerDuty, and webhook destinations. Laminar’s Signals can page Slack. MLflow’s budget webhooks cover spend only. AcruxCore’s only channel is email.Human labeling queue. Langfuse, Opik, and Laminar each have a queue with reviewer assignment and a score schema. Phoenix has annotations but no queue in the open-source build. MLflow’s sessions need the paid host. AcruxCore and Helicone have nothing here.Hierarchy. Langfuse has a real organization-above-project structure with a role at each level. Laminar has a workspace above project with three roles. Helicone has an organization tier with no projects beneath it. AcruxCore, MLflow, Opik, and Phoenix are each one flat team, and in self-hosted MLflow, Opik, and Phoenix, there is no login screen at all.Put together, alphabetically:Deficiencies in different LLM platforms which you have to build yourselfWhat sitting in the request path costsWe timed each platform against its own direct-to-provider baseline, in its own session, on different days. So read down a row, not across. These are seven separate measurements, not one benchmark.Latency comparison between different LLM ops platforms.Two things fall out.The four that sit beside the path — Langfuse, Laminar, Opik and Phoenix — add nothing you can measure, which is what you would expect. They send the trace in the background, after the response has already gone back to the caller, so nobody waits for it. A few of their numbers come out slightly negative. That is measurement noise. Wrapping your client in an SDK does not make the provider faster.The three that sit in the path — AcruxCore, Helicone and MLflow — are far apart from each other. AcruxCore added tens of milliseconds. MLflow added hundreds, and was the clear outlier: a tenth of a second or more in every single run. Helicone’s number is not a fair comparison, because that leg only forwarded the call and logged nothing.So the request path does cost you something, but the size of that cost comes from how a platform is built, not from where it sits. Measure it on your own network before trusting any published number, including mine.Who owns what you’re installingWorth ten minutes before you commit a year of tooling to one of these.Helicone joined Mintlify on 3 March 2026 and is in maintenance mode: security fixes, new models, and bug fixes, with no stated commitment to open source or self-hosting going forward.Langfuse was acquired by ClickHouse on 16 January 2026, with an explicit commitment in the announcement that it stays fully open source.MLflow is Databricks’, Phoenix is Arize’s, and Opik is Comet’s. Three open-source projects attached to commercial products, which is exactly what puts some features on a paid host.Laminar is a Y Combinator S24 company and the smallest of the six with a public star count.AcruxCore is the newest and has no community to show yet. That’s a real column in the table.How to pickThere’s no overall winner because these rows aren’t the same size for any two teams. Choosing from scratch:You need routing, caching, spend caps, or virtual keys. You need a gateway, so the shortlist is Helicone, MLflow, or AcruxCore before you look at anything else. Weigh Helicone’s maintenance mode and MLflow’s measured overhead against that.Your prompt’s behaviour has to be versioned outside your code. MLflow or AcruxCore, the only two that render conditionals and loops server-side.Legal needs an OSI-approved license. Drop Phoenix. If you also need a free audit log, drop Langfuse.You’re instrumenting agents built on existing frameworks. Laminar’s one-line integrations are the shortest path, and having no prompt registry may not bother you.You need guardrails or Slack alerts in the box. Opik, Helicone or Langfuse cover both. AcruxCore covers neither today.You need human labeling to build eval datasets. Langfuse, Opik, or Laminar. But if you want your end user to build dataset, choose AcruxCore.Each platform has its own write-up, built by running that same prompt on both sides: Langfuse, Phoenix, Opik, Helicone, MLflow, and Laminar. The matrix with a source link in every cell is at acruxcore.com/compare.LangSmith isn’t in the table because it’s closed source. It does offer self-hosting, so that’s the reason, not “hosted only”.The row that’s hardest to call is the third one. Is a gateway in your request path worth one more thing that can fail in front of production traffic, or are you better off beside the path and giving up routing and spend caps entirely?Both answers hold up, and the latency table doesn’t settle it. If you’ve run any of these seven in production, which row turned out to matter more than you expected when you chose?This story is published under the Generative AI publication. Connect with us on LinkedIn and follow Zeniteq to stay in the loop with the latest AI stories. Let’s shape the future of AI together!Seven Open-Source LLM Ops Platforms, One Table: Pick by the Row You Can’t Ship Without was originally published in Generative AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Source: Generative AI Pub — Published September 9, 2026 at 01:31 UTC — Category: Image AI