NewHow we catch a seller listing the wrong model, and refund you if one does.

How MartLLM (formerly UberLLM) routing picks the cheapest healthy provider

For each request, MartLLM looks up every seller offering the model you named, ranks them by live price, and sends the request to the cheapest healthy one that is fast enough for your prompt size. If that seller fails before any output has streamed, the request is retried on the next cheapest healthy seller — so you get the lowest available price without losing reliability, and you are never billed for a failed attempt.

The one-sentence version

Cheapest healthy seller, decided per request. Both words carry weight: cheapest, so you pay the lowest live price; healthy, so you do not get routed to a seller that is erroring or slow.

Step by step

  1. Resolve the model. The model id you send maps to one logical model (old ids keep resolving to their current replacement, so links and code do not rot).
  2. Gather the sellers. MartLLM collects every active offer for that model — the gateway’s own curated route plus any seller-pool offers listed on the marketplace.
  3. Rank by live price. Offers are ordered by their current price. The price you are quoted has no MartLLM fee added, so ranking and billing use the same number.
  4. Filter to healthy. Sellers that are failing, rate-limiting, or probing unhealthy are skipped. Health is tracked continuously from real traffic and background probes.
  5. Send to the cheapest one that is fast enough. Of the healthy sellers, those within 2x of the fastest measured time to first token at your prompt size are kept, and the cheapest of them gets the request. Long prompts also go to a second fast seller if the first is slow to start. The details are in smart routing.
  6. Fail over before output. If that seller errors before any tokens have streamed to you, the request is retried on the next cheapest healthy seller. A request that ultimately returns no result costs $0.

Why "healthy" is not optional

A pure cheapest-wins router sends you to whichever seller underprices everyone — including a seller that is cheap because it is broken. MartLLM’s health filter is what makes "cheapest" safe: the live board surfaces each provider’s reliability, and you can inspect it yourself on the uptime page. Routing uses the same signal.

What you can see

  • The live price per model, no fee added — pricing board.
  • Which models get the most traffic — rankings.
  • Provider reliability over time — uptime.

For sellers

The same mechanism is the seller side of the marketplace: list spare capacity for a model, stay healthy and competitively priced, and requests route to you. See how selling works and the sellers doc.

Keep reading

FAQ

How does MartLLM decide which provider serves my request?

It filters out unhealthy sellers of the model you named, keeps those fast enough for your prompt size (within 2x of the fastest measured), and sends the request to the cheapest of them. If that seller fails before output starts, the request is retried on another seller.

What does "healthy" mean in routing?

A seller that is currently serving successfully — not erroring, rate-limiting, or probing unhealthy. Health is tracked from real traffic and background probes, and unhealthy sellers are skipped so "cheapest" never means "broken".

Am I charged if the chosen provider fails?

No. A failure before any output has streamed is retried on the next healthy seller, and a request that returns no result costs $0.