Models & routing
MartLLM sends each request to an available third-party provider for the model you asked for. If the provider fails before it starts answering, the request is retried, and a request that still fails is not billed. Each request is billed what the provider charged, with no MartLLM fee added, and is listed with its cost in Dashboard → Activity. You choose a model, not a provider.
Model ids
Use the model id from the catalog, e.g. deepseek/deepseek-v4-flash,
anthropic/claude-sonnet-5 or z-ai/glm-5.3. List what you can call right now:
GET https://api.martllm.com/v1/models: the chat models a request can be sent to right now, so a tool that fills its model menu from this list (SillyTavern, LibreChat, n8n, Cline) shows only models it can chat with. Add?type=video(orimage,music,speech,transcription,audiofor both of those,chat, several joined with commas) for other kinds, or?type=allfor every model, chat first, then image, video, music, speech and transcription. Each entry hastype(chat,image,video,music,speech,transcription),endpoints(the paths that serve it) andpricing.- Each chat model also says what it can do:
context_length,max_output_tokens,input_modalities(image= it reads pictures),supports_tools,supports_reasoning,supports_visionandsupported_parameters(the request fields it honours). The same facts are under OpenRouter's names too (architecture.input_modalities,top_provider.context_length,top_provider.max_completion_tokens, andpricing.prompt/pricing.completionin dollars per token), so tools built for OpenRouter's list read ours as is. GET https://api.martllm.com/v1/models/<model id>: one model, with its price, its context (chat) and, for a media model,parameters: a JSON Schema of the request body (see Make a video or image).- Every entry, in both answers, also has:
available:truewhen a request for it can be served right now.falsemeans nobody is serving it at the moment, so a request would be refused withno_available_provider. Pick another model or try again later.input_modalities: what the model takes. A model that needs a picture (image to video, reference to video, picture editing) lists"image"next to"text"; one that takes a video (upscaling) lists"video".nsfw:truefor a model made for adult or unfiltered content (Lustify, the "uncensored" models and similar). Skip these if your product must stay safe for work.
- Public, no key:
GET https://api.martllm.com/api/v1/public/models(with prices, and the sameavailable,input_modalitiesandnsfwfields) - Web: martllm.com/#models, and a page per model at
martllm.com/models/<model id>
What happens on every request
- MartLLM checks the model exists, that your prompt fits its context window, and that it supports tools if your request uses them.
- It reserves the estimated cost from your balance and forwards the request.
- The request is sent to an available provider for that model. With the
default Balanced routing the provider is chosen again for each call: the
cheapest one that starts answering fast enough for your prompt size (within
6x of the fastest measured, or under 1 second). That provider can be priced
above the listed price, and the call then costs what it charged. A long
streamed prompt also goes to a second fast provider when the first is slow
to start, and a call that has not started after 5 seconds can move to a
faster provider; you get and pay for one reply. Your key's routing setting
(below) or
sortin the request changes this. How it works: smart routing. - When the reply finishes, MartLLM settles at the price the marketplace reports for that call and releases the rest of the reservation.
Automatic retries
If a call fails before any content reaches you, MartLLM retries it for you. The marketplace re-matches a seller on each try, and it charges nothing for a call that failed before it was routed, so a retry costs you only time.
- Retried: no seller for the model right now (
no_sellers_for_model,no_healthy_sellers), the marketplace ran out of time to route (route_deadline_exceeded), a429asking to wait 5 seconds or less, any5xx, a connection error, a seller whose own provider key or account failed, or no first byte in time. - Not retried: the model doesn't exist, the request is invalid, or a billing or auth problem. Those come back at once.
- Schedule: up to 4 attempts, waiting 0.3 s, 0.9 s, then 2.7 s between
them (or the
Retry-Afterthe marketplace sent, if it is 5 seconds or less). When nobody is selling the model, the answer comes back within a few seconds. - Waiting for the first byte: a stream gets 10 seconds (more for a long
prompt, and more for a model that is usually slow); a reply without
"stream": truegets 85 seconds, because nothing arrives until it is complete. An attempt that runs out the full 85 seconds is not waited on a second time. No retry starts after 150 seconds.
Once streaming has started, nothing is retried: a failure mid-stream ends the stream and you are billed only for what was produced.
If every attempt fails you get an error instead of a reply, and the reservation is fully released. The common ones:
| HTTP | code |
When |
|---|---|---|
| 503 | no_available_provider |
Nobody could serve that model right now, even after retries: no seller, every seller failing or too slow to start, or your provider options left nothing to route to. The message names the model. Try again shortly or pick another model. |
| 503 | service_unavailable |
A temporary problem on MartLLM's side. Retry in a few minutes. |
| 404 | model_not_found |
The model id isn't available. |
See Errors for the full list.
Fallback models
Send models with a list of other model ids to try, in order, when the first
cannot be served:
{ "model": "deepseek/deepseek-v4-flash",
"models": ["z-ai/glm-5.3", "qwen/qwen3.7-plus"],
"messages": [{ "role": "user", "content": "hi" }] }
The next model is tried only when nobody can serve the one before it right now
(no_available_provider), or nobody sells it within your provider.max_price.
A request error (a bad body, an unknown model id in either field) and a money
refusal (balance, key limits) are answered at once and never passed down the
list. At most 5 models in model and models together. The reply's model
field and the Activity row name the model that answered. models works on
/v1/chat/completions and on image, video, music and speech requests;
/v1/messages and /v1/audio/transcriptions do not read it.
Model aliases
Instead of a model id you can name what you need, and MartLLM picks the model on every request:
best-price:<job>: the available model with the lowest price for one ordinary request. A chat model is compared by its price per million tokens with output counted three times; an image model by one image (one 1024×1024 image when it is priced per megapixel); a video model by its shortest clip; a music model by one track; a speech model by 1K characters; a transcription model by one minute.fastest:<job>: the model with the lowest median time over its successful requests on MartLLM in the last 7 days (time to the first token for chat, time to the finished file for media), counting only models with at least 5 such requests. When no model has enough, the request is refused with503 no_timing_data, andbest-price:still works.
Both pick only among models that are available right now: the live market shows
a healthy seller for it, and MartLLM's last request to it in the past 10 minutes
did not find every seller busy or down (the same rule as available on
/v1/models). On a media route, when the model an alias picked answers that
nobody can take the request after all, the same request moves on to the next
model the alias stands for (up to 3 models), and the reply's model names the one
that answered. When no model for the job is available, the answer is
503 no_available_provider and nothing is charged.
<job> is one of chat, image, image-edit, text-to-video,
image-to-video, music, speech, transcription. Send a chat alias to /v1/chat/completions or
/v1/messages, and a media one to its media route. The reply's model field
names the model it resolved to. GET /v1/models/aliases lists every alias that
resolves right now and its model; GET /v1/models/best-price:chat describes the
model it stands for, with alias set.
Your key's routing setting
Each API key has a routing setting, on Dashboard → API keys (create a key or edit one):
| Setting | What it does |
|---|---|
| Balanced (default) | Fast and cheap: the cheapest provider that starts fast enough, as described above. A faster provider priced above the listed price may answer, and the call costs what it charged. |
| Lowest price | Always the lowest-priced provider for the request. No faster, pricier provider is picked, no second provider is raced, and a slow call is not moved. Can be slower. |
| Fastest | The provider that starts answering first, whatever it costs (the same as "sort": "latency"). |
| Never charge more than the listed price | On or off, with any of the three. A request is sent only to providers at or under the model's listed price when the request came in, and is never billed above what its tokens cost at that price. If no provider sells at that price right now, the request is refused with 400 max_price_exceeded and costs nothing. |
| Minimum discount | 0–99%, with any of the above (empty or 0 = no minimum). A chat request is sent only to providers selling at least that much below the model's official price, input and output each, and is never billed above that % off what its tokens cost at the official price (the same official price Activity's "% off" uses). If no provider is that cheap right now, the request is refused with 400 max_price_exceeded and costs nothing. A model with no official price is refused the same way, because the discount cannot be measured. |
Images and video follow the key's setting too (since 2026-11-19). Many image and video models have more than one seller, and one seller can have more than one way to make a model (a cheaper, slower one and a dearer, faster one). We time how long each takes and count how often each fails. The seller market is ranked in the same list as the direct sellers, at its own price and speed.
- The default (
auto, every key that has not picked another): the cheapest seller that makes exactly what you asked. It only steps over a seller that keeps failing, or one that is more than twice as slow as the fastest we have timed. So a key that never changed its setting does not pay more for a small gain in speed. - Balanced (
balanced): the cheapest seller that is at most 20% slower than the fastest one we have timed. A seller we have not timed yet counts as fast enough. Pick this when a faster result is worth a higher price. - Lowest price (
cheapest): always the cheapest seller that makes exactly what you asked, however slow. - Fastest (
fastest): the seller we have timed as quickest. It may cost more than the cheapest seller, but never more than the model's listed price for your request.
A seller is timed once it has finished at least 3 jobs for the model in the
last 7 days; while none is timed, every setting picks the cheapest. A seller
that made fewer than 7 in 10 of its recent jobs (10 or more tries) goes after
the healthy ones on every setting, and the time we compare is the time a job
is expected to take: a seller that loses one job in five counts as 25% slower.
The price in a quote ("validate": true) and a job's estimated_cost_usd are
the price of the seller your key's setting picks, and that is what the job is
billed when the seller charges what it lists. If that seller cannot take the
job, the next one in the same order does.
The setting applies when a request does not say. A request's own
provider.sort, :floor / :nitro, or provider.max_price wins over it.
provider.only: ["uberllm"] is not a sort, so the key's setting still applies.
The minimum discount is the exception: it always applies, and together with a
request's own provider.max_price the stricter price holds.
Through the dashboard API: routing (auto, balanced, cheapest or
fastest; for text, balanced is the same as auto) and
price_cap_listed (true/false) and min_discount_pct (whole number 0–99) on
POST /api/v1/keys and PATCH /api/v1/keys/{id}; all are returned by
GET /api/v1/keys.
Routing options
The provider object is read on /v1/chat/completions. An option MartLLM
cannot apply is refused with 400 unsupported_routing_option and nothing is
charged, so a request never runs with a setting silently dropped.
| Option | What it does |
|---|---|
only / allow |
Only these routes or providers. uberllm (MartLLM's marketplace route) and seller (sellers' own offers) choose between our routes. Any other name, such as openai or api.venice.ai, is a provider on the marketplace: the request then goes through the uberllm route, which uses only offers from those providers. |
ignore / deny |
Never these. Route names as above. Provider names (at most 32) are refused on the marketplace, and the request then goes through the uberllm route. |
order |
Preferred order. Route names order our routes; provider names set the marketplace's attempt order. It never makes a provider eligible or ineligible. |
sort |
price (the default), latency (fastest observed first, within a price band) or uptime (most reliable first, within the same band). throughput orders our own routes only. |
max_price |
The most you pay per million tokens, in US dollars: { "prompt": 1.0, "completion": 3.0 }. Routes priced above it are skipped; the marketplace route is held to sellers at least as far below the model's list price as your ceiling needs. If nothing is left, the request is refused with 400 max_price_exceeded and nothing is charged. When you set both prompt and completion, a request is never billed above them for the tokens it used. |
allow_fallbacks |
false turns off the automatic retries above; the first failure is returned to you. |
data_collection |
Only "allow" (the default) is accepted. |
zdr, require_parameters |
Only false is accepted. |
The :floor and :nitro model suffixes are accepted and stripped from the
model id; :floor sorts by price and :nitro by throughput.
For image, video, music and speech, each request is sent to an available
third-party provider for the model. provider.max_price there is the most one unit (image, second,
megapixel, minute, 1K characters or request) may cost, as a number: { "max_price": 0.05 }; it also caps the bill.
Every other provider option is refused for media.
Tags
Send metadata, an object of up to 16 text values (keys up to 64 characters
without a colon, values up to 512), on any request:
{ "model": "deepseek/deepseek-v4-flash", "messages": [...],
"metadata": { "project": "spring-ads", "run": "42" } }
The tags are stored with the request and never sent to a provider. They show on
the request in Dashboard → Activity, on a video or music job's status, and in
the CSV export (metadata column). Filter Activity by a tag with
GET /api/v1/usage/activity?tag=project:spring-ads (repeat tag to require
several); the CSV link takes the same filter. On /v1/messages Anthropic's own
metadata ({"user_id": "…"}) is stored the same way.