Call a Model
A Model answers like any OpenAI chat model: set the base URL to your serving URL and the model to the Model’s name. Any OpenAI SDK, or anything built on one, works unchanged.
Your serving URL and key
- Serving URL: in the Code view of any Model’s Overview. Store it as
FACET_BASE_URL. - API key: make one under Settings, Keys. Keys start with
fct_. Store it asFACET_API_KEY, in your app’s secrets, never in code.
A call
import os
from openai import OpenAI
client = OpenAI(base_url=os.environ["FACET_BASE_URL"], api_key=os.environ["FACET_API_KEY"])
reply = client.chat.completions.create(
model="support-triage",
messages=[{"role": "user", "content": "My card was charged twice"}],
)
print(reply.choices[0].message.content)
import OpenAI from "openai";
const client = new OpenAI({ baseURL: process.env.FACET_BASE_URL, apiKey: process.env.FACET_API_KEY });
const reply = await client.chat.completions.create({
model: "support-triage",
messages: [{ role: "user", content: "My card was charged twice" }],
});
console.log(reply.choices[0].message.content);
curl "$FACET_BASE_URL/chat/completions" \
-H "Authorization: Bearer $FACET_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "support-triage", "messages": [{"role": "user", "content": "My card was charged twice"}]}'
Send only the input. Facet adds the Model’s Instructions to every call and keeps them current with each change, so don’t add your own system prompt.
Pick a version
model: "support-triage"follows the live version, also called Latest: when a change merges, your next call gets it.model: "support-triage@v4"pins v4, and keeps answering from it after newer versions go live.
Until a Model’s first version is live, calls answer 409 model_not_ready.
Parameters
| Parameter | What it does |
|---|---|
model |
The Model’s name, or name@vN to pin a version. Required |
messages |
The conversation, as in OpenAI’s API; roles system, user, assistant and tool. Required |
temperature, top_p |
How varied answers are, 0 to 2 and 0 to 1; lower is more repeatable. Default: the Model’s own setting |
max_tokens or max_completion_tokens |
The longest answer, in tokens (a token is about three quarters of a word). Default and ceiling: the Model’s own |
stream |
true streams the answer as server-sent events; stream_options.include_usage adds token counts at the end |
stop, seed |
As in OpenAI’s API |
response_format |
Only for Models that return text or JSON of any shape; see Return structured output |
tools, tool_choice, parallel_tool_calls |
Passed to the model as in OpenAI’s API; most Models don’t use them |
n |
Leave unset: a Model returns one answer per call |
Long inputs
Some Models read a long input in windows: the ones whose answer is a list of values copied from the input, when the draft’s rows showed it works. You send the whole document and get one whole answer. Facet splits an input longer than one window at line breaks (4,000 characters, or the 8,000 or 16,000 the Model was made with), the Model answers each window, and the answers are merged into one list in the document’s order.
- The
x-facet-windowsresponse header says how many windows the call took. A short input is answered in one piece, with no header. max_tokensapplies to each window.finish_reasonislengthwhen any window’s answer was cut.usagecounts every window’s tokens, and windows share a tenth of their length, so a long input costs a little more than its length.streamis refused for an input read in windows; a short input streams as usual.- One call reads at most 64 windows. A longer input is refused with 413
input_too_long.
Streaming
stream = client.chat.completions.create(
model="support-triage",
messages=[{"role": "user", "content": "My card was charged twice"}],
stream=True,
)
for chunk in stream:
print(chunk.choices[0].delta.content or "", end="")
const stream = await client.chat.completions.create({
model: "support-triage",
messages: [{ role: "user", content: "My card was charged twice" }],
stream: true,
});
for await (const chunk of stream) process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
Response headers
| Header | What it says |
|---|---|
x-facet-version |
The version that answered, such as support-triage@v4 |
x-facet-request-id |
The call’s ID. Quote it when you write to us about a call |
Keys and privacy
A key can be limited to one Model, and a viewer key only reads. Facet keeps each call’s metadata (version, key, token counts, latency), never the messages or the answer. See Privacy and data.
Next steps
- Errors: what each error means and what to do
- Limits: calls and tokens a minute
- Base models: speed and price per call
- Frameworks: Vercel AI SDK, LangChain, LlamaIndex, LiteLLM, Pydantic AI, or moving from the Anthropic SDK