Prepare for the PgBouncer and IPv4 deprecations on 26th January 2024

Learn more
Back
LiteLLM

LiteLLM

Overview

Simplify LLM API Calls across Anthropic, OpenAI, HuggingFace, Replicate, etc.

Use Supabase to log requests and see total spend across all LLM Providers (OpenAI, Azure, Anthropic, Cohere, Replicate, PaLM)

liteLLM provides success_callbacks and failure_callbacks, making it easy for you to send data to a particular provider depending on the status of your responses.

In this case, we want to log requests to Supabase in both scenarios - when it succeeds and fails.

Create a supabase table

Go to your Supabase project > go to the Supabase SQL Editor and create a new table with this configuration.

Note: You can change the table name. Just don't change the column names.


_14
create table
_14
public.request_logs (
_14
id bigint generated by default as identity,
_14
created_at timestamp with time zone null default now(),
_14
model text null default ''::text,
_14
messages json null default '{}'::json,
_14
response json null default '{}'::json,
_14
end_user text null default ''::text,
_14
error json null default '{}'::json,
_14
response_time real null default '0'::real,
_14
total_cost real null,
_14
additional_details json null default '{}'::json,
_14
constraint request_logs_pkey primary key (id)
_14
) tablespace pg_default;

Use Callbacks

Use just 2 lines of code, to instantly see costs and log your responses across all providers with Supabase:


_10
litellm.success_callback=["supabase"]
_10
litellm.failure_callback=["supabase"]

Complete code


_16
from litellm import completion
_16
_16
## set env variables
_16
os.environ["SUPABASE_URL"] = "your-supabase-url"
_16
os.environ["SUPABASE_key"] = "your-supabase-key"
_16
os.environ["OPENAI_API_KEY"] = ""
_16
_16
# set callbacks
_16
litellm.success_callback=["supabase"]
_16
litellm.failure_callback=["supabase"]
_16
_16
#openai call
_16
response = completion(model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hi 👋 - i'm openai"}])
_16
_16
#bad call
_16
response = completion(model="chatgpt-test", messages=[{"role": "user", "content": "Hi 👋 - i'm a bad call to test error logging"}])

Additional Controls

Different Table name

If you modified your table name, here's how to pass the new name.


_10
litellm.modify_integration("supabase",{"table_name": "litellm_logs"})

Identify end-user

Here's how to map your llm call to an end-user


_10
litellm.identify({"end_user": "krrish@berri.ai"})

Details

DeveloperBerriAI
CategoryDevTools
Websitelitellm.ai
DocumentationLearn