LiteLLM
Overview
Simplify LLM API Calls across Anthropic, OpenAI, HuggingFace, Replicate, etc.
Use Supabase to log requests and see total spend across all LLM Providers (OpenAI, Azure, Anthropic, Cohere, Replicate, PaLM)
liteLLM provides success_callbacks and failure_callbacks, making it easy for you to send data to a particular provider depending on the status of your responses.
In this case, we want to log requests to Supabase in both scenarios - when it succeeds and fails.
Create a supabase table
Go to your Supabase project > go to the Supabase SQL Editor and create a new table with this configuration.
Note: You can change the table name. Just don't change the column names.
_14create table_14 public.request_logs (_14 id bigint generated by default as identity,_14 created_at timestamp with time zone null default now(),_14 model text null default ''::text,_14 messages json null default '{}'::json,_14 response json null default '{}'::json,_14 end_user text null default ''::text,_14 error json null default '{}'::json,_14 response_time real null default '0'::real,_14 total_cost real null,_14 additional_details json null default '{}'::json,_14 constraint request_logs_pkey primary key (id)_14 ) tablespace pg_default;
Use Callbacks
Use just 2 lines of code, to instantly see costs and log your responses across all providers with Supabase:
_10litellm.success_callback=["supabase"]_10litellm.failure_callback=["supabase"]
Complete code
_16from litellm import completion_16_16## set env variables_16os.environ["SUPABASE_URL"] = "your-supabase-url" _16os.environ["SUPABASE_key"] = "your-supabase-key" _16os.environ["OPENAI_API_KEY"] = ""_16_16# set callbacks_16litellm.success_callback=["supabase"]_16litellm.failure_callback=["supabase"]_16_16#openai call_16response = completion(model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hi 👋 - i'm openai"}]) _16_16#bad call_16response = completion(model="chatgpt-test", messages=[{"role": "user", "content": "Hi 👋 - i'm a bad call to test error logging"}])
Additional Controls
Different Table name
If you modified your table name, here's how to pass the new name.
_10litellm.modify_integration("supabase",{"table_name": "litellm_logs"})
Identify end-user
Here's how to map your llm call to an end-user
_10litellm.identify({"end_user": "[email protected]"})
Details
Third-party integrations and docs are managed by Supabase partners.