AI Observability Leaderboard

That the vendor's own documentation shows the feature at the strictness the criterion asks for. It does not mean the feature was installed, exercised or found to work. Nothing on this board was run. A half means the feature exists with a gating condition or at reduced scope: an enterprise plan, a hybrid where the vendor keeps the control plane, scrubbing that happens after ingestion rather than before. A cross means a page was checked and the feature is documented as absent.

Because it is not a failure. An instrumentation library has no prompt registry and no hosted compliance posture, and scoring those as zero would rank libraries last for being libraries. A not-applicable cell leaves both the numerator and the denominator, which is what lets a tool and a platform share one table. There are 10 of them.

Weighted points over the weighted maximum, scaled to 100. A documented feature earns two points, a partial one earns one, a documented absence earns nothing, and a not-applicable criterion is not in the sum at all. The six weightings above the table change which criteria carry more: balanced counts all ten the same, and the other five weight for autonomous agents, regulated enterprise, high-throughput SaaS, continuous quality gating and full-stack APM.

Because coverage no longer separates anything here. Every applicable cell on this board carries a status and a link, so a row cannot look better by having been checked more. What still varies is how good those links are: 10 cells rest on vendor marketing and 5 on a documentation root rather than a page about the criterion. The evidence figure says so beside the score, and never inside it, because a board that scored its own sourcing would be marking its own work.

Two. CI/CD PR gating has three products clearing the bar out of 18, and the first pass scored an identical testing plugin as documented for one product and partial for four others, which means it was measuring the auditor as much as the product. SOC 2 Type II and HIPAA is the weakest sourced column on the board: certification evidence usually lives behind a third-party trust portal, off the vendor's own domain, which the sourcing rule does not accept. Both are written up under Cautions.

4 of the 22 audited carry a row and no rank, each with its reason on the page: a vendor winding down, a semantic-convention specification that is not a product anyone deploys, a platform whose hosted tier has been withdrawn, and a project with no commit in three months whose cells rest on a link dump. They were read against the same criteria and carry the same links. Ranking them would compare things that are not alternatives to each other.

A cell that moved after the first read. A down arrow means an adversarial pass attacked the claim and the evidence did not hold, so the status here is the lower one; 19 cells carry a mark of either kind. An up arrow means a review pass checked that attack, found it wrong and put the cell back. Only the documented claims were attacked, so a partial cell has been examined less than a tick, which is the opposite of what you might expect.

Never. They are shown for the 15 open-source rows because attention is worth knowing, they break a tie, and they do nothing else. The dates beside them are the useful part. Two products on this board read as entirely current in their documentation and had stopped shipping: one had not been pushed to in twenty months, and one commits daily with no tagged release in thirteen. No vendor's own site says so.

Because documentation is written about products, not about ownership. Six of these products changed hands or changed shape inside the audit window and not one documentation site says so: Langfuse is owned by ClickHouse, Galileo by Cisco, Weights & Biases by CoreWeave, and Datadog renamed its product while leaving the URL alone. Those were found by reading company pages and repository activity, which is why every open-source row carries a repository fact beside it.

There is no fixed schedule. The date at the top is the last time the data changed, not the last time the page was built. A weekly job checks every source link still resolves and opens an issue when one does not; it never edits a status.