NewHow we catch a seller listing the wrong model, and refund you if one does.

Models & routing

MartLLM sends each request to an available third-party provider for the model you asked for. If the provider fails before it starts answering, the request is retried, and a request that still fails is not billed. Each request is billed what the provider charged, with no MartLLM fee added, and is listed with its cost in Dashboard → Activity. You choose a model, not a provider.

Model ids

Use the model id from the catalog, e.g. deepseek/deepseek-v4-flash, anthropic/claude-sonnet-5 or z-ai/glm-5.3. List what you can call right now:

  • GET https://api.martllm.com/v1/models: the chat models a request can be sent to right now, so a tool that fills its model menu from this list (SillyTavern, LibreChat, n8n, Cline) shows only models it can chat with. Add ?type=video (or image, music, speech, transcription, audio for both of those, chat, several joined with commas) for other kinds, or ?type=all for every model, chat first, then image, video, music, speech and transcription. Each entry has type (chat, image, video, music, speech, transcription), endpoints (the paths that serve it) and pricing.
  • Each chat model also says what it can do: context_length, max_output_tokens, input_modalities (image = it reads pictures), supports_tools, supports_reasoning, supports_vision and supported_parameters (the request fields it honours). The same facts are under OpenRouter's names too (architecture.input_modalities, top_provider.context_length, top_provider.max_completion_tokens, and pricing.prompt / pricing.completion in dollars per token), so tools built for OpenRouter's list read ours as is.
  • GET https://api.martllm.com/v1/models/<model id>: one model, with its price, its context (chat) and, for a media model, parameters: a JSON Schema of the request body (see Make a video or image).
  • Every entry, in both answers, also has:
    • available: true when a request for it can be served right now. false means nobody is serving it at the moment, so a request would be refused with no_available_provider. Pick another model or try again later.
    • input_modalities: what the model takes. A model that needs a picture (image to video, reference to video, picture editing) lists "image" next to "text"; one that takes a video (upscaling) lists "video".
    • nsfw: true for a model made for adult or unfiltered content (Lustify, the "uncensored" models and similar). Skip these if your product must stay safe for work.
  • Public, no key: GET https://api.martllm.com/api/v1/public/models (with prices, and the same available, input_modalities and nsfw fields)
  • Web: martllm.com/#models, and a page per model at martllm.com/models/<model id>

What happens on every request

  1. MartLLM checks the model exists, that your prompt fits its context window, and that it supports tools if your request uses them.
  2. It reserves the estimated cost from your balance and forwards the request.
  3. The request is sent to an available provider for that model. With the default Balanced routing the provider is chosen again for each call: the cheapest one that starts answering fast enough for your prompt size (within 6x of the fastest measured, or under 1 second). That provider can be priced above the listed price, and the call then costs what it charged. A long streamed prompt also goes to a second fast provider when the first is slow to start, and a call that has not started after 5 seconds can move to a faster provider; you get and pay for one reply. Your key's routing setting (below) or sort in the request changes this. How it works: smart routing.
  4. When the reply finishes, MartLLM settles at the price the marketplace reports for that call and releases the rest of the reservation.

Automatic retries

If a call fails before any content reaches you, MartLLM retries it for you. The marketplace re-matches a seller on each try, and it charges nothing for a call that failed before it was routed, so a retry costs you only time.

  • Retried: no seller for the model right now (no_sellers_for_model, no_healthy_sellers), the marketplace ran out of time to route (route_deadline_exceeded), a 429 asking to wait 5 seconds or less, any 5xx, a connection error, a seller whose own provider key or account failed, or no first byte in time.
  • Not retried: the model doesn't exist, the request is invalid, or a billing or auth problem. Those come back at once.
  • Schedule: up to 4 attempts, waiting 0.3 s, 0.9 s, then 2.7 s between them (or the Retry-After the marketplace sent, if it is 5 seconds or less). When nobody is selling the model, the answer comes back within a few seconds.
  • Waiting for the first byte: a stream gets 10 seconds (more for a long prompt, and more for a model that is usually slow); a reply without "stream": true gets 85 seconds, because nothing arrives until it is complete. An attempt that runs out the full 85 seconds is not waited on a second time. No retry starts after 150 seconds.

Once streaming has started, nothing is retried: a failure mid-stream ends the stream and you are billed only for what was produced.

If every attempt fails you get an error instead of a reply, and the reservation is fully released. The common ones:

HTTP code When
503 no_available_provider Nobody could serve that model right now, even after retries: no seller, every seller failing or too slow to start, or your provider options left nothing to route to. The message names the model. Try again shortly or pick another model.
503 service_unavailable A temporary problem on MartLLM's side. Retry in a few minutes.
404 model_not_found The model id isn't available.

See Errors for the full list.

Fallback models

Send models with a list of other model ids to try, in order, when the first cannot be served:

{ "model": "deepseek/deepseek-v4-flash",
  "models": ["z-ai/glm-5.3", "qwen/qwen3.7-plus"],
  "messages": [{ "role": "user", "content": "hi" }] }

The next model is tried only when nobody can serve the one before it right now (no_available_provider), or nobody sells it within your provider.max_price. A request error (a bad body, an unknown model id in either field) and a money refusal (balance, key limits) are answered at once and never passed down the list. At most 5 models in model and models together. The reply's model field and the Activity row name the model that answered. models works on /v1/chat/completions and on image, video, music and speech requests; /v1/messages and /v1/audio/transcriptions do not read it.

Model aliases

Instead of a model id you can name what you need, and MartLLM picks the model on every request:

  • best-price:<job>: the available model with the lowest price for one ordinary request. A chat model is compared by its price per million tokens with output counted three times; an image model by one image (one 1024×1024 image when it is priced per megapixel); a video model by its shortest clip; a music model by one track; a speech model by 1K characters; a transcription model by one minute.
  • fastest:<job>: the model with the lowest median time over its successful requests on MartLLM in the last 7 days (time to the first token for chat, time to the finished file for media), counting only models with at least 5 such requests. When no model has enough, the request is refused with 503 no_timing_data, and best-price: still works.

Both pick only among models that are available right now: the live market shows a healthy seller for it, and MartLLM's last request to it in the past 10 minutes did not find every seller busy or down (the same rule as available on /v1/models). On a media route, when the model an alias picked answers that nobody can take the request after all, the same request moves on to the next model the alias stands for (up to 3 models), and the reply's model names the one that answered. When no model for the job is available, the answer is 503 no_available_provider and nothing is charged.

<job> is one of chat, image, image-edit, text-to-video, image-to-video, music, speech, transcription. Send a chat alias to /v1/chat/completions or /v1/messages, and a media one to its media route. The reply's model field names the model it resolved to. GET /v1/models/aliases lists every alias that resolves right now and its model; GET /v1/models/best-price:chat describes the model it stands for, with alias set.

Your key's routing setting

Each API key has a routing setting, on Dashboard → API keys (create a key or edit one):

Setting What it does
Balanced (default) Fast and cheap: the cheapest provider that starts fast enough, as described above. A faster provider priced above the listed price may answer, and the call costs what it charged.
Lowest price Always the lowest-priced provider for the request. No faster, pricier provider is picked, no second provider is raced, and a slow call is not moved. Can be slower.
Fastest The provider that starts answering first, whatever it costs (the same as "sort": "latency").
Never charge more than the listed price On or off, with any of the three. A request is sent only to providers at or under the model's listed price when the request came in, and is never billed above what its tokens cost at that price. If no provider sells at that price right now, the request is refused with 400 max_price_exceeded and costs nothing.
Minimum discount 0–99%, with any of the above (empty or 0 = no minimum). A chat request is sent only to providers selling at least that much below the model's official price, input and output each, and is never billed above that % off what its tokens cost at the official price (the same official price Activity's "% off" uses). If no provider is that cheap right now, the request is refused with 400 max_price_exceeded and costs nothing. A model with no official price is refused the same way, because the discount cannot be measured.

Images and video follow the key's setting too (since 2026-11-19). Many image and video models have more than one seller, and one seller can have more than one way to make a model (a cheaper, slower one and a dearer, faster one). We time how long each takes and count how often each fails. The seller market is ranked in the same list as the direct sellers, at its own price and speed.

  • The default (auto, every key that has not picked another): the cheapest seller that makes exactly what you asked. It only steps over a seller that keeps failing, or one that is more than twice as slow as the fastest we have timed. So a key that never changed its setting does not pay more for a small gain in speed.
  • Balanced (balanced): the cheapest seller that is at most 20% slower than the fastest one we have timed. A seller we have not timed yet counts as fast enough. Pick this when a faster result is worth a higher price.
  • Lowest price (cheapest): always the cheapest seller that makes exactly what you asked, however slow.
  • Fastest (fastest): the seller we have timed as quickest. It may cost more than the cheapest seller, but never more than the model's listed price for your request.

A seller is timed once it has finished at least 3 jobs for the model in the last 7 days; while none is timed, every setting picks the cheapest. A seller that made fewer than 7 in 10 of its recent jobs (10 or more tries) goes after the healthy ones on every setting, and the time we compare is the time a job is expected to take: a seller that loses one job in five counts as 25% slower. The price in a quote ("validate": true) and a job's estimated_cost_usd are the price of the seller your key's setting picks, and that is what the job is billed when the seller charges what it lists. If that seller cannot take the job, the next one in the same order does.

The setting applies when a request does not say. A request's own provider.sort, :floor / :nitro, or provider.max_price wins over it. provider.only: ["uberllm"] is not a sort, so the key's setting still applies. The minimum discount is the exception: it always applies, and together with a request's own provider.max_price the stricter price holds.

Through the dashboard API: routing (auto, balanced, cheapest or fastest; for text, balanced is the same as auto) and price_cap_listed (true/false) and min_discount_pct (whole number 0–99) on POST /api/v1/keys and PATCH /api/v1/keys/{id}; all are returned by GET /api/v1/keys.

Routing options

The provider object is read on /v1/chat/completions. An option MartLLM cannot apply is refused with 400 unsupported_routing_option and nothing is charged, so a request never runs with a setting silently dropped.

Option What it does
only / allow Only these routes or providers. uberllm (MartLLM's marketplace route) and seller (sellers' own offers) choose between our routes. Any other name, such as openai or api.venice.ai, is a provider on the marketplace: the request then goes through the uberllm route, which uses only offers from those providers.
ignore / deny Never these. Route names as above. Provider names (at most 32) are refused on the marketplace, and the request then goes through the uberllm route.
order Preferred order. Route names order our routes; provider names set the marketplace's attempt order. It never makes a provider eligible or ineligible.
sort price (the default), latency (fastest observed first, within a price band) or uptime (most reliable first, within the same band). throughput orders our own routes only.
max_price The most you pay per million tokens, in US dollars: { "prompt": 1.0, "completion": 3.0 }. Routes priced above it are skipped; the marketplace route is held to sellers at least as far below the model's list price as your ceiling needs. If nothing is left, the request is refused with 400 max_price_exceeded and nothing is charged. When you set both prompt and completion, a request is never billed above them for the tokens it used.
allow_fallbacks false turns off the automatic retries above; the first failure is returned to you.
data_collection Only "allow" (the default) is accepted.
zdr, require_parameters Only false is accepted.

The :floor and :nitro model suffixes are accepted and stripped from the model id; :floor sorts by price and :nitro by throughput.

For image, video, music and speech, each request is sent to an available third-party provider for the model. provider.max_price there is the most one unit (image, second, megapixel, minute, 1K characters or request) may cost, as a number: { "max_price": 0.05 }; it also caps the bill. Every other provider option is refused for media.

Tags

Send metadata, an object of up to 16 text values (keys up to 64 characters without a colon, values up to 512), on any request:

{ "model": "deepseek/deepseek-v4-flash", "messages": [...],
  "metadata": { "project": "spring-ads", "run": "42" } }

The tags are stored with the request and never sent to a provider. They show on the request in Dashboard → Activity, on a video or music job's status, and in the CSV export (metadata column). Filter Activity by a tag with GET /api/v1/usage/activity?tag=project:spring-ads (repeat tag to require several); the CSV link takes the same filter. On /v1/messages Anthropic's own metadata ({"user_id": "…"}) is stored the same way.