Docs
Sign inJoin the waitlist

Use a Model with LiteLLM

LiteLLM calls any OpenAI-compatible endpoint with the openai/ prefix. Set FACET_BASE_URL and FACET_API_KEY as in Call a Model.

Python SDK

import os
import litellm

reply = litellm.completion(
    model="openai/support-triage",
    api_base=os.environ["FACET_BASE_URL"],
    api_key=os.environ["FACET_API_KEY"],
    messages=[{"role": "user", "content": "My card was charged twice"}],
)
print(reply.choices[0].message.content)

Proxy

Add the Model to your proxy’s config.yaml, and your apps call it by the name you give it:

model_list:
  - model_name: support-triage
    litellm_params:
      model: openai/support-triage
      api_base: os.environ/FACET_BASE_URL
      api_key: os.environ/FACET_API_KEY

Send only the input. Facet adds the Model’s Instructions to every call, so leave out your own system prompt.

Next steps

Esc