Docs
Sign inJoin the waitlist

Call a Model

A Model answers like any OpenAI chat model: set the base URL to your serving URL and the model to the Model’s name. Any OpenAI SDK, or anything built on one, works unchanged.

Your serving URL and key

  • Serving URL: in the Code view of any Model’s Overview. Store it as FACET_BASE_URL.
  • API key: make one under Settings, Keys. Keys start with fct_. Store it as FACET_API_KEY, in your app’s secrets, never in code.

A call

import os
from openai import OpenAI

client = OpenAI(base_url=os.environ["FACET_BASE_URL"], api_key=os.environ["FACET_API_KEY"])
reply = client.chat.completions.create(
    model="support-triage",
    messages=[{"role": "user", "content": "My card was charged twice"}],
)
print(reply.choices[0].message.content)
import OpenAI from "openai";

const client = new OpenAI({ baseURL: process.env.FACET_BASE_URL, apiKey: process.env.FACET_API_KEY });
const reply = await client.chat.completions.create({
  model: "support-triage",
  messages: [{ role: "user", content: "My card was charged twice" }],
});
console.log(reply.choices[0].message.content);
curl "$FACET_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $FACET_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "support-triage", "messages": [{"role": "user", "content": "My card was charged twice"}]}'

Send only the input. Facet adds the Model’s Instructions to every call and keeps them current with each change, so don’t add your own system prompt.

Pick a version

  • model: "support-triage" follows the live version, also called Latest: when a change merges, your next call gets it.
  • model: "support-triage@v4" pins v4, and keeps answering from it after newer versions go live.

Until a Model’s first version is live, calls answer 409 model_not_ready.

Parameters

Parameter What it does
model The Model’s name, or name@vN to pin a version. Required
messages The conversation, as in OpenAI’s API; roles system, user, assistant and tool. Required
temperature, top_p How varied answers are, 0 to 2 and 0 to 1; lower is more repeatable. Default: the Model’s own setting
max_tokens or max_completion_tokens The longest answer, in tokens (a token is about three quarters of a word). Default and ceiling: the Model’s own
stream true streams the answer as server-sent events; stream_options.include_usage adds token counts at the end
stop, seed As in OpenAI’s API
response_format Only for Models that return text or JSON of any shape; see Return structured output
tools, tool_choice, parallel_tool_calls Passed to the model as in OpenAI’s API; most Models don’t use them
n Leave unset: a Model returns one answer per call

Long inputs

Some Models read a long input in windows: the ones whose answer is a list of values copied from the input, when the draft’s rows showed it works. You send the whole document and get one whole answer. Facet splits an input longer than one window at line breaks (4,000 characters, or the 8,000 or 16,000 the Model was made with), the Model answers each window, and the answers are merged into one list in the document’s order.

  • The x-facet-windows response header says how many windows the call took. A short input is answered in one piece, with no header.
  • max_tokens applies to each window. finish_reason is length when any window’s answer was cut.
  • usage counts every window’s tokens, and windows share a tenth of their length, so a long input costs a little more than its length.
  • stream is refused for an input read in windows; a short input streams as usual.
  • One call reads at most 64 windows. A longer input is refused with 413 input_too_long.

Streaming

stream = client.chat.completions.create(
    model="support-triage",
    messages=[{"role": "user", "content": "My card was charged twice"}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")
const stream = await client.chat.completions.create({
  model: "support-triage",
  messages: [{ role: "user", content: "My card was charged twice" }],
  stream: true,
});
for await (const chunk of stream) process.stdout.write(chunk.choices[0]?.delta?.content ?? "");

Response headers

Header What it says
x-facet-version The version that answered, such as support-triage@v4
x-facet-request-id The call’s ID. Quote it when you write to us about a call

Keys and privacy

A key can be limited to one Model, and a viewer key only reads. Facet keeps each call’s metadata (version, key, token counts, latency), never the messages or the answer. See Privacy and data.

Next steps

Esc