28312 Commits

Author SHA1 Message Date
Sameer Kankute
43914796d6 fix failing vertex tests 2025-12-06 00:04:04 +05:30
Krish Dholakia
85d73403f4 Refactor: Skip PublicAI tests if API key is not set (#17540)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-12-05 10:22:07 -08:00
Sameer Kankute
3907667892 fix tests 2025-12-05 23:25:18 +05:30
Alexsander Hamir
c0d149e0a9 Fix: Lack of None value checks & update publicai_chat_transformation tests (#17539)
* fix: handle none content

* fix: defensive check on none value

* Fix test failures: Azure OCR skip, None content handling, PublicAI JSON config

- Skip aocr/ocr call types in Azure test (they don't use Azure SDK client)
- Handle None content in Responses API transformation (skip message creation)
- Update PublicAI tests to use JSON-based configuration system
- Add None check in PublicAI test fixture to fix type error
2025-12-05 09:43:52 -08:00
Krrish Dholakia
c272741d7f docs: fix strings 2025-12-05 09:37:22 -08:00
Krrish Dholakia
c1cbe6ed56 docs: document tool calls spec 2025-12-05 09:37:22 -08:00
Sameer Kankute
a21f1ce21f Merge pull request #17528 from BerriAI/litellm_save_background_checks
Add background health checks to db
2025-12-05 22:25:03 +05:30
Sameer Kankute
558c8f92d1 Merge pull request #17519 from BerriAI/litellm_cursor_integration
Add support for cursor BYOK with its own configuration
2025-12-05 22:23:45 +05:30
Sameer Kankute
49a344ebd9 Merge pull request #17525 from BerriAI/litellm_fix_in_memory_vector_store
Fix vector store configuration synchronization failure
2025-12-05 22:23:17 +05:30
Sameer Kankute
b6867184a8 Merge pull request #17534 from colinlin-stripe/colinlin/opus-budget-thinking
[fix] parse <budget:thinking> blocks for opus 4.5
2025-12-05 22:21:05 +05:30
Sameer Kankute
5f23d94b7e Fixed media resoltion for gemini 3 2025-12-05 22:16:36 +05:30
Alexsander Hamir
96122a8b5a Fix Presidio guardrail test TypeError and license base64 decoding error (#17538)
Fixed two issues:

1. Presidio guardrail test TypeError:
   - Issue: test_presidio_apply_guardrail() was calling apply_guardrail() with
     incorrect arguments (text=, language=) instead of the correct signature
     (inputs=, request_data=, input_type=)
   - Fix: Updated test to use correct method signature:
     - Changed from: apply_guardrail(text=..., language=...)
     - Changed to: apply_guardrail(inputs={'texts': [...]}, request_data={}, input_type='request')
   - Also updated assertions to extract text from response['texts'][0]

2. License verification base64 decoding error:
   - Issue: verify_license_without_api_request() was failing with
     'Invalid base64-encoded string: number of data characters (185) cannot be
     1 more than a multiple of 4' when license keys lacked proper base64 padding
   - Root cause: Base64 strings must be a multiple of 4 characters. Some license
     keys were missing padding characters (=) needed for proper decoding
   - Fix: Added automatic padding before base64 decoding:
     - Calculate padding needed: len(license_key) % 4
     - Add '=' characters to make length a multiple of 4
     - This makes license verification robust to keys with or without padding

Both fixes ensure the code handles edge cases properly and tests use correct APIs.
2025-12-05 08:45:02 -08:00
Alexsander Hamir
0c017f376c fix: code quality issues from ruff linter (#17536)
* fix: resolve code quality issues from ruff linter

- Fix duplicate imports in anthropic guardrail handler
  - Remove duplicate AllAnthropicToolsValues import
  - Remove duplicate ChatCompletionToolParam import

- Remove unused variable 'tools' in guardrail handler

- Replace print statement with proper logging in json_loader
  - Use verbose_logger.warning() instead of print()

- Remove unused imports
  - Remove _update_metadata_field from team_endpoints
  - Remove unused ChatCompletionToolCallChunk imports from transformation

- Refactor update_team function to reduce complexity (PLR0915)
  - Extract budget_duration handling into _set_budget_reset_at() helper
  - Minimal refactoring to reduce function from 51 to 50 statements

All ruff linter errors resolved. Fixes F811, F841, T201, F401, and PLR0915 errors.

* docs: add missing environment variables to documentation

Add 8 missing environment variables to the environment variables reference section:
- AIOHTTP_CONNECTOR_LIMIT_PER_HOST: Connection limit per host for aiohttp connector
- AUDIO_SPEECH_CHUNK_SIZE: Chunk size for audio speech processing
- CYBERARK_SSL_VERIFY: Flag to enable/disable SSL certificate verification for CyberArk
- LITELLM_DD_AGENT_HOST: Hostname or IP of DataDog agent for LiteLLM-specific logging
- LITELLM_DD_AGENT_PORT: Port of DataDog agent for LiteLLM-specific log intake
- WANDB_API_KEY: API key for Weights & Biases (W&B) logging integration
- WANDB_HOST: Host URL for Weights & Biases (W&B) service
- WANDB_PROJECT_ID: Project ID for Weights & Biases (W&B) logging integration

Fixes test_env_keys.py test that was failing due to undocumented environment variables.
2025-12-05 08:40:49 -08:00
Colin Lin
0bd144103d [stripe] simplify opus test 2025-12-05 10:56:09 -05:00
Colin Lin
3046b9f163 [stripe] opus budget thinking 2025-12-05 10:56:02 -05:00
Sameer Kankute
3d6b7f0d3d Add background health checks to db 2025-12-05 14:27:37 +05:30
yuneng-jiang
b9b5d638c8 Merge pull request #17524 from BerriAI/litellm_team_user_settings_fix
[Fix] Select in Edit Membership Modal
2025-12-04 23:13:13 -08:00
yuneng-jiang
37bfe65bdd Adding screenshot to debug 2025-12-04 23:05:00 -08:00
Sameer Kankute
c8fbcc7f1c add tutorial as well 2025-12-05 12:32:23 +05:30
yuneng-jiang
50283a00a3 e2e fix 2025-12-04 22:51:52 -08:00
Sameer Kankute
acc0b5fe27 Merge pull request #17362 from BerriAI/litellm_vertex-bge-cherrypick
[Feat] VertexAI - Add BGE Embeddings support
2025-12-05 11:53:42 +05:30
Sameer Kankute
99fd96687f Fix vector store configuration synchronization failure 2025-12-05 11:46:14 +05:30
Krish Dholakia
b3a3081e8e Guardrails API - new structured_messages param (#17518)
* fix(generic_guardrail_api.py): add 'structured_messages' support

allows guardrail provider to know if text is from system or user

* fix(generic_guardrail_api.md): document 'structured_messages' parameter

give api provider a way to distinguish between user and system messages

* feat(anthropic/): return openai chat completion format structured messages when calls made via `/v1/messages` on Anthropic

* feat(responses/guardrail_translation): support 'structured_messages' param for guardrails

structured openai chat completion spec messages, for guardrail checks when using /v1/responses api

allows guardrail checks to work consistently across APIs
2025-12-04 22:08:00 -08:00
Krish Dholakia
51cc102c30 fix(unified_guardrail.py): support during_call event type for unified guardrails (#17514)
* fix(unified_guardrail.py): support during_call event type for unified guardrails

allows guardrails overriding apply_guardrails to work 'during_call'

* feat(generic_guardrail_api.py): support new 'tool_calls' field for generic guardrail api

returns the tool calls emitted by the LLM API to the user

* fix(generic_guardrail_api.py): working anthropic /v1/messages tool call response

send llm tool calls to guardrail api when called via `/v1/messages` API

* fix(responses/): run generic_guardrail_api on responses api tool call responses

* fix: fix tests

* test: fix tests

* fix: fix tests
2025-12-04 22:06:13 -08:00
Cesar Garcia
316f7671a9 fix(gemini): handle partial JSON chunks after first valid chunk (#17496)
* fix(gemini): allow JSON accumulation on any chunk, not just first

* test(gemini): add tests for partial JSON chunk handling
2025-12-04 22:01:59 -08:00
yuneng-jiang
8da265b901 Fix select in edit membership 2025-12-04 21:59:43 -08:00
Kristian Brünn
63fae79493 fix(sql): Optimize SpendLogs queries to use timestamp filtering for index usage (#17504)
* fix: optimize SpendLogs queries to use timestamp filtering (#17487)

* use timestamptz & enhance test
2025-12-04 21:52:57 -08:00
Krish Dholakia
8776336c3c Enable detailed debugging for reference (#17508)
* Deprecate set_verbose in favor of LITELLM_LOG

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

* Update debugging documentation links

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-12-04 21:51:56 -08:00
Devaj Mody
48b5100c18 fix(guardrails): mask all matching keywords in content filter (#17521)
Fixes #17517

  - Fixed bug where only the first matching blocked keyword was masked
  - Now iterates through ALL blocked keywords and masks each one
  - Added 3 regression tests for multiple keyword masking
2025-12-04 21:50:15 -08:00
Sameer Kankute
392e5059b0 Add steps to add litellm proxy in cursor 2025-12-05 10:02:42 +05:30
Sameer Kankute
01ee46b493 Add steps to add litellm proxy in cursor 2025-12-05 10:01:48 +05:30
Sameer Kankute
4d83a48b59 Add steps to add litellm proxy in cursor 2025-12-05 09:39:58 +05:30
Sameer Kankute
a6006e698c Add support for cursor BYOK with its own configuration 2025-12-05 09:34:49 +05:30
Dominic Feliton
6af693de36 (fix): empty response + vllm streaming (#17516)
* Fix empty response + vllm streaming

* Add unit test
2025-12-04 19:24:28 -08:00
YutaSaito
7106509ef0 chore: add stack trace to mcp error message (#17269) 2025-12-04 19:19:00 -08:00
Ishaan Jaffer
4f3b843efe docs openai 2025-12-04 18:32:23 -08:00
Ishaan Jaff
b2e8d3fd42 [Feat] Allow adding OpenAI compatible chat providers using .json + add public ai provider (#17448)
* feat: Add JSON config for OpenAI-compatible providers

Co-authored-by: ishaan <ishaan@berri.ai>

* feat: Add simple JSON config for OpenAI-compatible providers

Co-authored-by: ishaan <ishaan@berri.ai>

* feat: Implement JSON-based provider config and migrate PublicAI

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* Checkpoint before follow-up message

Co-authored-by: ishaan <ishaan@berri.ai>

* docs fix

* undo change

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
2025-12-04 17:59:25 -08:00
Ishaan Jaff
fadfbb13d3 [Docs] A2a - Permission management (#17515)
* docs add a2a gateway + mcp gateway

* docs a2a permissions

* docs a2a permission

* docs

* docs a2a

* docs a2a

* add new img

* docs agent permissions
2025-12-04 17:29:47 -08:00
Ishaan Jaffer
dfe3124335 UI new build 2025-12-04 16:34:59 -08:00
Ishaan Jaffer
5cc1a5031b fix invoke_agent_a2a 2025-12-04 16:32:15 -08:00
Ishaan Jaff
575e769bff [Feat] UI - Agent Gateway - set allowed agents by key, team (#17511)
* init schema.prisma

* init LiteLLM_ObjectPermissionTable with agents and agent_access_groups

* TestAgentRequestHandler

* refatctor agent list

* add AgentRequestHandler

* fix agent access controls by key/team

* feat - new migration for LiteLLM_AgentsTable

* fix add LiteLLM_ObjectPermissionBase with agent and agent groups

* add agent routes to llm api routes

* add agent routes as llm route

* add AgentPermissionsProps

* add agents on team/key create

* add agent selector on team/key

* add agent selector on key edit /info

* add AgentPermissions

* docs list + invoke agents
2025-12-04 16:31:17 -08:00
Ishaan Jaff
8f592088ed [Feat] Agent Access Control - Enforce Allowed agents by key, team + add agent access groups on backend (#17502)
* init schema.prisma

* init LiteLLM_ObjectPermissionTable with agents and agent_access_groups

* TestAgentRequestHandler

* refatctor agent list

* add AgentRequestHandler

* fix agent access controls by key/team

* feat - new migration for LiteLLM_AgentsTable

* fix add LiteLLM_ObjectPermissionBase with agent and agent groups

* add agent routes to llm api routes

* add agent routes as llm route
2025-12-04 16:31:00 -08:00
yuneng-jiang
5a2e89a49e Customer Usage UI 2025-12-04 15:13:17 -08:00
Ishaan Jaff
e3116da653 feat: Add /global/spend/tags to admin viewer routes (#17501)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
2025-12-04 14:43:33 -08:00
Ishaan Jaffer
c6e26a20b1 refactor invoke 2025-12-04 14:21:01 -08:00
Raghav Jhavar
72eb4c3a1c 🆕 feat: support routing to only websearch supported deployments (#17500)
* support routing to only websearch supported deployments

* add docs
2025-12-04 14:18:20 -08:00
Anas AbdelR
2abcc77944 fix: resolve ruff lint errors (#17490)
Fixed 20 of 22 lint errors:

- batches/batch_utils.py: Added missing model_name parameter to functions
- integrations/custom_guardrail.py: Removed unused Tuple import
- llms/custom_httpx/http_handler.py: Removed unused AIOHTTP_NEEDS_CLEANUP_CLOSED import
- llms/anthropic/chat/guardrail_translation/handler.py: Prefixed unused variable with underscore
- llms/openai/responses/guardrail_translation/handler.py: Removed duplicate BaseModel import, prefixed unused variable
- llms/pass_through/guardrail_translation/handler.py: Prefixed unused variables with underscore
- proxy/guardrails/guardrail_hooks/generic_guardrail_api/generic_guardrail_api.py: Removed unused List and Tuple imports
- proxy/hooks/parallel_request_limiter_v3.py: Removed unused import

Remaining 2 errors are PLR0915 (too many statements) which require refactoring.
2025-12-04 14:12:57 -08:00
Yannay Hammer
2627f0d519 Fix aim security guardrail tests (#17499) 2025-12-04 13:29:20 -08:00
yuneng-jiang
b12ccb1a7a Publish proxy extras 2025-12-04 13:11:20 -08:00
yuneng-jiang
1834377950 Adding migration 2025-12-04 12:57:06 -08:00