160 Commits

Author SHA1 Message Date
yuneng-jiang
cb18af542e Merge pull request #17498 from BerriAI/litellm_customer_usage_backend
[Feature] Customer (end user) Usage
2025-12-05 15:31:08 -08:00
Ishaan Jaff
8f592088ed [Feat] Agent Access Control - Enforce Allowed agents by key, team + add agent access groups on backend (#17502)
* init schema.prisma

* init LiteLLM_ObjectPermissionTable with agents and agent_access_groups

* TestAgentRequestHandler

* refatctor agent list

* add AgentRequestHandler

* fix agent access controls by key/team

* feat - new migration for LiteLLM_AgentsTable

* fix add LiteLLM_ObjectPermissionBase with agent and agent groups

* add agent routes to llm api routes

* add agent routes as llm route
2025-12-04 16:31:00 -08:00
yuneng-jiang
562afb208d v0 customer usage, pending tests + extras version bump 2025-12-04 12:30:08 -08:00
yuneng-jiang
dd72ff3abf Add organization_id to spend logs table 2025-11-22 12:54:49 -08:00
yuneng-jiang
6029a24baa Merge remote-tracking branch 'origin' into litellm_org_usage 2025-11-20 20:19:16 -08:00
Ishaan Jaff
c7cf18cf67 [Feat] Prompt Management - Allow storing prompt version in DB (#16848)
* test_dotprompt_auto_detection_with_model_only

* fix _auto_detect_prompt_management_logger

* test_dotprompt_with_prompt_version

* add v1, v2 tests

* add _compile_prompt_helper

* fix _compile_prompt_helper

* test_dotprompt_with_prompt_version

* test_dotprompt_with_prompt_version, test_get_prompt_with_version

* add version in schema

* feat add _get_prompt_spec_for_db_prompt

* add _get_prompt_spec_for_db_prompt

* feat add _get_prompt_spec_for_db_prompt

* update prompt table

* add version in prompt DB

* test_get_prompt_spec_for_db_prompt_with_versions
2025-11-19 13:19:56 -08:00
yuneng-jiang
f36cbccd56 Merge remote-tracking branch 'origin' into litellm_org_usage 2025-11-15 16:28:12 -08:00
Krish Dholakia
8097fafc05 Agents - support agent registration + discovery (A2A spec) (#16615)
* fix: initial commit adding types

* refactor: refactor to include agent registry

* feat(agents/): endpoints.py

working endpoint for agent discovery

* feat(agent_endpoints/endpoints.py): add permission management logic to agents endpoint

* feat: public endpoint for showing publicly discoverable agents

* feat: make /public/agent_hub discoverable

* feat(agent_endpoints/endpoints.py): working create agent endpoint

adds dynamic agent registration to the proxy

* feat: working crud endpoints

* feat: working multi-instance create/delete agents

* feat(migration.sql): add migration for agents table
2025-11-14 18:23:30 -08:00
yuneng-jiang
94da5c076c Add Daily Org Spend Table, read path, and write path 2025-11-12 18:22:26 -08:00
yuneng-jiang
67478a9074 [Fix] Litellm tags usage add request_id (#16111)
* Add request_id into tag spend

* Linting
2025-11-11 18:53:48 -08:00
YutaSaito
6eb74bd62a Feat/persist mcp credentials in db (#16308)
* feat: persist mcp credentials in db

* feat: remove Auth Value field from MCP Tool Testing Playground

* fix: test
2025-11-07 19:22:49 -08:00
YutaSaito
8e27b6c0b4 [MCP] configure static mcp header (#16179)
* feat: configure extra mcp headers in ui

* doc: static header

* build: add new migration file

* chore: add missing image file

* fix: test
2025-11-03 21:06:36 -08:00
Krrish Dholakia
6637beb247 fix: minor fixes + dep updates 2025-11-01 13:45:42 -07:00
Krish Dholakia
43aacf2dc0 (feat) Azure AI Vector Stores - support "virtual" indexes + create vector store on passthrough API (#16160)
* feat(vector_store_endpoints/endpoints.py): add new index_create endpoint

allows admin to create a virtual index, to do permission management for

* feat(key_management_endpoints.py): enable setting allowed_vector_store_indexes on keys

proxy admin can enable dev to create an index on a vector stor

* feat: initial commit adding vector store index passthrough logic to litellm

* feat: add vector store table

* fix(azure_ai/transformation.py): fix headers

* feat: track read/write endpoints by vector store integration

enables permissions by index to work

* fix: azure_ai/vector_stores/search

document the vector store endpoints correctly

 ensures permission management works as expected

* fix(proxy/utils.py): improve error message

* docs(azure_ai_vector_stores_passthrough.md): document azure ai passthrough vector store support

* docs(create.md): document azure ai support via passthrough for vector store create

* fix: fix code qa errors

* fix: document new allowed_vector_store_indexes endpoint
2025-11-01 12:01:32 -07:00
Ishaan Jaff
c59a0c9681 [Feat] UI - Allow setting cache settings on UI (#16143)
* add LiteLLM_CacheConfig

* add CacheSettingsField

* add UI cache saver

* feat add cache_settings_router

* fix schema

* fix ssl_check_hostname

* refactor into utils

* add groups for field names

* add test_connection in base cache

* add test_connection inredis and redis cluster

* feat _decrypt_db_variables

* add cache settings endpoints

* test_test_cache_connection_calls_cache_test_connection_with_params

* fix: add switch_on_llm_response_caching

* feat use CacheSettingsManager

* feat use CacheSettingsManager

* TestCacheSettingsManager

* fix update_config

* Cache Field test
2025-10-31 17:43:59 -07:00
yuneng-jiang
720ba865fb [Infra] Litellm Backend SSO Changes (#16029)
* SSO Backend changes

* Encrypt and Decrypt, load into os env

* Linting and addressing comments
2025-10-30 14:32:08 -07:00
Ishaan Jaff
d8ea1665c7 [Feat] - [Backend] Search APIs - Allow storing configured Search APIs in DB (#15862)
* add LiteLLM_SearchToolsTable

* init SearchToolRegistry

* fix add SearchToolRegistry

* fix add SearchToolRegistry

* fix handling search tool management

* fix search imports

* fix registry

* init search tools in memory

* fix init tools in mem

* fix TypedDict def

* add new SCHEMA

* bump proxy extras

* add LiteLLM_SearchToolsTable_search_tool_name_key

* bump extras with migration

* fix working CRUD Ops

* fix: _init_search_tools_in_db
2025-10-23 17:57:49 -07:00
Ishaan Jaff
527c8f59fa [Feat] Tag Management - Add support for setting tag based budgets (#15433)
* feat: add LiteLLM_TagTable

* fix: use new table for tag management

* fix - allow setting budgets for tags

* working tag creation

* fix schema.prisma

* add tag info

* ui fixes

* ui fix tag info

* TAG_CACHE_IN_MEMORY_TTL_SECONDS

* add Litellm_EntityType

* fix get_aggregated_db_spend_update_transactions

* fix: _update_entity_spend_in_db

* fix _tag_max_budget_check

* add tag budget check

* add tag_list_transactions

* test_get_tag_objects_batch

* test_update_tag_db_without_prisma_client

* fix get_tags_from_request_body

* get_tags_from_request_body

* fix get_tags_from_request_body

* fix spend tracking utils

* get_tags_from_request_body

* test_get_tags_from_request_body_with_metadata_tags

* feat: add _update_tag_cache spend tracking

* fix _PROXY_track_cost_callback

* test_tag_cache_update_multiple_tags

* fix tag info

* docs fix

* docs tag budgets

* doc fix

* docs fix

* fix tag budget

* docs tag budgets

* docs fix

* ruff fix
2025-10-10 19:24:50 -07:00
Ishaan Jaff
930f88078e [MCP Gateway] Backend - Allow storing allowed tools by team/key (#15243)
* add mcp_tool_permissions column to LiteLLM_ObjectPermissionTable

* add mcp_tool_permissions

* add LiteLLM_ObjectPermissionTable

* test key/team mgmt

* fix code QA check

* fix check_tool_permission_for_key_team

* test_key_tool_permission_allows_permitted_tool

* test_list_tools_filters_by_key_team_permissions

* fix _get_tools_from_mcp_servers

* fix logger.debug

* fix check_tool_permission_for_key_team

* fix: filter_tools_by_key_team_permissions

* fix _get_team_object_permission

* fix _set_object_permission

* fix handle_update_object_permission_common

* fix add allowed tools to keys

* test add key/team level tool permissions for MCPs
2025-10-06 18:49:32 -07:00
Krrish Dholakia
39cac8e58a build(ui/): show updated schema.prisma 2025-10-03 19:07:50 -07:00
rishiganesh2002
d36c8d6bbe [Feat] MCP Gateway Fine-grained Tools Addition (#15153)
* feat: UI to add specific tools under creating MCP connection

* chore: pydantic + prisma changes

* feat: adding specific MCP tools now works

* fix: allowed tools filtering

* chore: filtered list to mcp server cost config

* chore: update Readme

* chore: refactor the filtering

* test: Added tests

When the allowed_tests is null, empty list or populated

* chore: resolve the proxy issue

* feat: updating MCP tool filtering
2025-10-03 10:16:29 -07:00
Ishaan Jaff
628cd13755 [Feat] UI - Allow scheduling key rotations when creating virtual keys (#14960)
* fix design of key reset interval

* fix: LiteLLM_VerificationToken

* fix key manager

* add key_rotation_at

* ui fix

* fix ui view

* set key_rotation_at  on creation

* add key_rotation_at

* fix info

* fix: _set_key_rotation_fields

* fix KeyRotationManager

* fix key edit view

* test_update_key_fn_auto_rotate_enable

* fix KeyRotationManager
2025-09-26 16:24:40 -07:00
Ishaan Jaff
ea8d0bb7d5 fix: re-add scheduled rotations 2025-09-26 11:40:46 -07:00
Ishaan Jaffer
b1e56c55ad merge 2025-09-24 22:11:37 -07:00
Ishaan Jaff
ded823a3a3 [Feat] Initial support for scheduled key rotations (#14877)
* add KeyRotationSettings

* add KeyRotationManager

* add key rotation manager

* add KeyRotationManager

* fix _initialize_spend_tracking_background_jobs

* fix add_key_rotation_fields

* fixes

* new fields in schema

* update schema

* fix fixes for init key rotation manager

* fixes for init key rotation manager

* fix KeyRotationManager

* tests for key rotation

* fix key_rotation_settings

* fix KeyRotationManager

* fix LITELLM_KEY_ROTATION_CHECK_INTERVAL_SECONDS

* fix fixes for init key rotation manager

* ruff check fix

* fix LITELLM_KEY_ROTATION_CHECK_INTERVAL_SECONDS

* fix _initialize_spend_tracking_background_jobs

* docs fix

* docs auto rotate

* schema fix

* bump proxy extras

* fix config.yml
2025-09-24 21:37:56 -07:00
Yuta Saito
654f1d3290 fix: stop including spec_version in MCP server registration inserts 2025-09-19 07:06:15 +09:00
Krish Dholakia
3119064e94 Prompt Management - add prompts on UI (#13240)
* fix(create_key_button.tsx): add prompts on UI

* feat(key_management_endpoints.py): support adding prompt to key via `/key/update`

* fix(key_info_view.tsx): show existing prompts on key in key_info_view.tsx

* fix(key_edit_view.tsx): UX - disable premium feature for non-premium users

prevent accidental clicking

* fix(create_key_button.tsx): disable premium features behind flag, prevent errors

* feat(prompts.tsx): add new ui component to view created prompts

enables viewing prompts created on config

* feat(prompt_info.tsx): add component for viewing the prompt information

* feat(prompt_endpoints.py): support converting dotprompt to json structure + accept json structure in promptmanager

allows prompt manager to work with api endpoints

* test(test_prompt_manager.py): add unit tests for json data input

* feat(dotprompt/__init__.py): add prompt data to dotpromptmanager

* fix(prompt_endpoints.py): working crud endpoints for prompt management

* feat(prompts/): support `prompt_file` for dotprompt

allows to precisely point to the prompt file a prompt should use

* feat(proxy/utils.py): resolve prompt id correctly

resolves user sent prompt id with internal prompt id

* feat(schema.prisma): initial pr with db schema for prompt management table

allows post endpoints to work with backend

* feat(prompt_endpoints.py): use db in patch_prompt endpoint

* feat(prompt_endpoints.py): use db for update_prompt endpoint

* feat(prompt_endpoints.py): use db on prompt delete endpoint

* build(schema.prisma): add prompt tale to schema.prisma in litellm-proxy-extras

* build(migration.sql): add new sql migration file

* fix(init_prompts.py): fix init

* feat(prompt_info_view.tsx): show the raw prompt template on ui

allows developer to know the prompt template they'll be calling

* feat(add_prompt_form.tsx): working ui add prompt flow

allows user to add prompts to litellm via ui

* build(ui/): styling fixes

* build(ui/): prompts.tsx

styling improvements

* fix(add_prompt_form.tsx): styling improvements

* build(prompts.tsx): styling improvements

* build(ui/): styling improvements

* build(ui/): fix ui error

* fix: fix ruff check

* docs: document new api params

* test: update tests
2025-08-02 22:33:37 -07:00
Jugal D. Bhatt
e324f76859 [MCP Gateway] add health check endpoints for MCP (#13106)
* add health check endpoints for MCP

* add import

* Clean up endpopints

* fix ruff
2025-07-30 20:40:44 +05:30
Jugal D. Bhatt
10595ea077 [MCP Gateway] add Litellm mcp alias for prefixing (#12994)
* change alias-> server_name

* add server alias uses

* add tests

* schema

* ruff fix

* fix alias for config

* fix tests

* add alias

* fix tests

* fix tests

* add a common util

* ruff fix

* fix migration
2025-07-25 17:57:52 -07:00
Ishaan Jaff
3051a9c68a [Bug Fix] QA - Use PG Vector Vector Store with LiteLLM (#12716)
* store in generic litellm params

* handle storing litellm_params_json

* add litellm_params

* update file loc

* fix startup issue

* ui fix litellm_params

* fix typing

* fix adding PG Vector

* use litellm params

* fix transform_search_vector_store_request

* fix transform

* fix URL

* test fix

* test_pg_vector_search_request_construction
2025-07-18 08:41:18 -07:00
Ishaan Jaff
bccbe921e4 build - sync all schema.prisma files 2025-07-11 20:25:26 -07:00
Jugal D. Bhatt
e3437795bb Litellm mcp access group on UI (#12470)
* added mcp tools on internal user and divide it by teams

* add support for server api call

* Added frontend for test key

* added tools used output

* fix ui for servers

* All servers to personal

* change columns format

* revert ui logic

* Added vertical align

* fix mapped tests

* fix lint

* fix lint

* remove extra file

* fix ui test

* comments fixes

* change query type

* change query type

* mcp acces group init

* add ability to change server display on ui through access groups

* Mcp access group names UI (#12486)

* Added ui changes to reflect mcp_access_groups

* fix edit mcp page

* change to string array (#12491)

* change to string array

* Remove print

* add ability to change server display on ui through access groups

* Litellm mcp access groups accesses (#12498)

* added mcp access groups for keys and teams

* added access groups above servers

* fixed ruff

* fixed mypy

* revert couple changes
2025-07-11 10:29:25 +05:30
Krish Dholakia
c4af2eb5e2 MCP - usage tracking (#12397)
* fix(common_daily_activity.py): initial commit with working mock BE endpoint for mcp usage

* feat(ui/): show mcp server activity on UI

allows admin to know which mcp's are being used

* feat(common_daily_activity.py): return activity by key

* feat(ui/): show top api keys for a given model / mcp server

allow user to know which key is driving spend

* fix(common_daily_activity.py): use known mcp server names

* feat(server.py): log the namespaced tool name (includes server prefix)

allow accurate cost tracking

* feat(db_spend_update_writer.py): log by mcp_namespaced_tool_name

store aggregate daily activity by mcp_namespaced_tool_name

Enables cost / usage tracking by mcp tool name

* fix(server.py): add key/user metadata to mcp calls

* refactor(common_daily_activity.py): update to return mcp activity in API

* fix(common_daily_activity.py): handle empty key

* fix(common_daily_activity.py): track when api key is empty

* test(test_spend_management_endpoints.py): update tests

* fix: fix ui linting error

* fix: fix linting errors

* test: add missing key

* build(schema.prisma): add mcp tool tracking

* fix(migration.sql): add schema migration file

* feat(server.py): add request logging for mcp calls

enables storing the mcp calls

* fix(new_usage.tsx): fix linting errors

* fix: fix code qa errors

* fix(activity_metrics.tsx): fix ui linting errors post-merge

* fix(types/utils.py): fix linting error

* fix(server.py): always have name
2025-07-08 22:08:16 -07:00
Ishaan Jaff
5cad0dd94b [Feat] Add MCP Cost Tracking (#12385)
* fix MCP_TOOL_NAME_PREFIX

* test_mcp_cost_tracking

* init MCPCostCalculator

* add call_mcp_tool

* calculate_mcp_tool_call_cost

* add MCPServerCostInfo

* add mcp_server_cost_info

* add mcp_server_cost_info

* pass through litellm_logging_obj

* logged_standard_logging_payload

* fix logging MCP tool call

* test_mcp_cost_tracking_per_tool

* fix NewMCPServerRequest

* fix add_update_server

* add MCP info to schema.prisma

* fix create_mcp_server

* working custom cost per call

* fix mcp server cost

* fix MCPCostCalculator

* TestMCPCostCalculator
2025-07-07 21:08:10 -07:00
Krrish Dholakia
c3857e60f2 Store batch output file id in DB + Store batch file status in DB + (experimental) BATCH API COST TRACKING 2025-06-25 22:41:22 -07:00
Cole McIntosh
c39b8f2178 feat: add LiteLLM_HealthCheckTable model to schema for health monitoring (#11677) 2025-06-18 08:37:40 -07:00
Krish Dholakia
eb0d0c2f72 Revert "feat(schema): add additional indexes to LiteLLM_SpendLogs for improve…" (#11683)
This reverts commit 2a7f113fde.
2025-06-12 18:18:44 -07:00
Cole McIntosh
2a7f113fde feat(schema): add additional indexes to LiteLLM_SpendLogs for improved query performance (#11675) 2025-06-12 17:55:34 -07:00
hagan
0f449bf038 [Bug Fix] Create/Update team member api 500 errror (#10479)
* Fixes issue with team_endpoints on member budget update

* refactored location of budget membership fix

* added test for _upsert_budget_membership func
2025-06-03 15:57:33 -07:00
Ishaan Jaff
4e6c4beef8 [Feat] Permission management vector stores on LiteLLM Key, Team, Orgs (#11213)
* fix: init commit for object permissions

* fix: init commit for object permissions

* fix: add vector_store_id to permissions

* fix vector store selector

* feat:add vector store permission mgmt

* feat: ui add allowed vector stores dropdown

* feat: add new vector store object permissions

* testing: key mgmt

* fix: stor vector store permissions on team

* ui select vector store for teams

* ui add vector store settings for orgs

* feat: allow setting org vector store permissions

* test: adding team permissions for vector stores
2025-05-28 16:58:53 -07:00
Krrish Dholakia
0017d5f1db build(ui/): Allow empty values in daily agg table + reintroduce 'unassigned' teams in spend tracking 2025-05-26 22:03:34 -07:00
Krrish Dholakia
8da4b3c4f1 style: add comments on schema reflecting org id field on schema
Closes https://github.com/BerriAI/litellm/issues/11092
2025-05-24 12:27:46 -07:00
Krish Dholakia
e9b7059af4 Litellm add file validation (#11081)
* fix: cleanup print statement

* feat(managed_files.py): add auth check on managed files

Implemented for file retrieve + delete calls

* feat(files_endpoints.py): support returning files by model name

enables managed file support

* feat(managed_files/): filter list of files by the ones created by user

prevents user from seeing another file

* test: update test

* fix(files_endpoints.py): list_files - always default to provider based routing

* build: add new table to prisma schema
2025-05-22 23:05:45 -07:00
Ishaan Jaff
ee1557afcd [Feat] Add endpoints for adding, deleting, editing guardrails in DB (#10833)
* feat: add DB Add, Edit, Delete for Guardrails

* feat: endpoints for guardrail management

* add guardrail info helpers

* add presidio logo

* add createGuardrailCall
2025-05-14 14:19:51 -07:00
Ishaan Jaff
3d8c3d7b96 [DB] Add index for session_id on LiteLLM_SpendLogs (#10727)
* build: add index session_id

* build: add db migration
2025-05-10 14:28:20 -07:00
Muhammad Husnain
0675dc109e [FIX] Update token fields in schema.prisma to use BigInt for improved… (#10697)
* [FIX] Update token fields in schema.prisma to use BigInt for improved handling of large values across LiteLLM models.

* [FIX] revert SpendLogs back to Int
2025-05-09 14:14:39 -07:00
Ishaan Jaff
d9f2415f65 [Refactor - Filtering Spend Logs] Add status to root of SpendLogs table (#10661)
* add status to spendlogs

* add status to spendlogs

* add status to spendlogs

* add status to spendlogs

* fix tests with spendlogs
2025-05-08 11:29:25 -07:00
Ishaan Jaff
f524eb4bd5 Contributor PR - MCP Server DB Schema (#10634) (#10641)
* MCP Server DB Schema (#10634)

* feat: initial mcp db schema

* user -> team table for mcp_servers

* remove mcp_servers from team and verificationtoken tables

---------

Co-authored-by: wagnerjt <wagnerjt@github.com>

* add mcp table

* Team and Key Object Permission table for MCP Servers (#10643)

* feat: initial mcp db schema

* user -> team table for mcp_servers

* remove mcp_servers from team and verificationtoken tables

* adding ObjectPermissionTable relationship for mcpservers

* fix: object_permission_id and added relationship to User and Organization tables

---------

Co-authored-by: wagnerjt <wagnerjt@github.com>

* migration file - add_mcp_key_team_permission_mgmt

---------

Co-authored-by: Tyler Wagner <wagnerjt@users.noreply.github.com>
Co-authored-by: wagnerjt <wagnerjt@github.com>
2025-05-07 21:09:51 -07:00
Ishaan Jaff
be885e4a17 [Feat] Vector Stores/KnowledgeBases - Allow defining Vector Store Configs (#10448)
* init vector store configs

* working kb init

* add vector store endpoints

* use litellm_credential_name

* working CRUD vector stores litellm

* working creds with vector DB

* ui cleanup

* clean up vector store id

* fix delete button

* refactored vector store component

* working selector for KBs

* ui vector stores

* add vector store tool calls usage on chat ui

* fixes for vector stores litellm

* test fix

* docs Knowledge Bases

* fixes for vector stores litellm

* fix linting

* add managed vectorstores

* fix orjson ci/cd test

* fix linting

* add types.tsx file
2025-04-30 21:49:59 -07:00
Ishaan Jaff
331e784db4 [Feat] Responses API - Add session management support for non-openai models (#10321)
* add session id in spendLogs

* fix log proxy server request as independant field

* use trace id for SpendLogs

* add _ENTERPRISE_ResponsesSessionHandler

* use _ENTERPRISE_ResponsesSessionHandler

* working session_ids

* working session management

* working session_ids

* test_async_gcs_pub_sub_v1

* test_spend_logs_payload_e2e

* working session_ids

* test_get_standard_logging_payload_trace_id

* test_get_standard_logging_payload_trace_id

* test_gcs_pub_sub.py

* fix all linting errors

* test_spend_logs_payload_with_prompts_enabled

* _ENTERPRISE_ResponsesSessionHandler

* _ENTERPRISE_ResponsesSessionHandler

* expose session id on ui

* get spend logs by session

* add sessionSpendLogsCall

* add session handling

* session logs

* ui session details

* fix on rowExpandDetails

* ui working sessions
2025-04-25 23:24:24 -07:00