Use a Model with LiteLLM
LiteLLM calls any OpenAI-compatible endpoint with the openai/ prefix. Set FACET_BASE_URL and FACET_API_KEY as in Call a Model.
Python SDK
import os
import litellm
reply = litellm.completion(
model="openai/support-triage",
api_base=os.environ["FACET_BASE_URL"],
api_key=os.environ["FACET_API_KEY"],
messages=[{"role": "user", "content": "My card was charged twice"}],
)
print(reply.choices[0].message.content)
Proxy
Add the Model to your proxy’s config.yaml, and your apps call it by the name you give it:
model_list:
- model_name: support-triage
litellm_params:
model: openai/support-triage
api_base: os.environ/FACET_BASE_URL
api_key: os.environ/FACET_API_KEY
Send only the input. Facet adds the Model’s Instructions to every call, so leave out your own system prompt.
Next steps
- Call a Model: every parameter and header a call takes
- Return structured output: get answers as fields or JSON