mirror of
https://github.com/BerriAI/litellm.git
synced 2025-12-06 11:33:26 +08:00
d612d71ef4
* add PassThroughGuardrailsConfig * init JsonPathExtractor * feat PassthroughGuardrailHandler * feat pt guardrails * pt guardrails * add Pass-Through Endpoint Guardrail Translation * add PassThroughEndpointHandler * execute simple guardrail config and dict settings * TestPassthroughGuardrailHandlerNormalizeConfig * add passthrough_guardrails_config on litellm logging obj * add LiteLLMLoggingObj to base trasaltino * cleaner _get_guardrail_settings * update guardrails settings * docs pt guardrail * docs Guardrails on Pass-Through Endpoints * fix typing * fix typing * test_no_fields_set_sends_full_body * fix typing * Potential fix for code scanning alert no. 3834: Clear-text logging of sensitive information Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> --------- Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
1.7 KiB
1.7 KiB
Pass-Through Endpoint Guardrail Translation
Why This Exists Here
This module is located under litellm/llms/ (instead of with the main passthrough code) because:
- Auto-discovery: The
load_guardrail_translation_mappings()function inlitellm/llms/__init__.pyscans forguardrail_translation/directories underlitellm/llms/ - Consistency: All other guardrail translation handlers follow this pattern (e.g.,
openai/chat/guardrail_translation/,anthropic/chat/guardrail_translation/)
Main Passthrough Implementation
The main passthrough endpoint implementation is in:
litellm/proxy/pass_through_endpoints/
├── pass_through_endpoints.py # Core passthrough routing logic
├── passthrough_guardrails.py # Guardrail collection and field targeting
├── jsonpath_extractor.py # JSONPath field extraction utility
└── ...
What This Handler Does
The PassThroughEndpointHandler enables guardrails to run on passthrough endpoint requests by:
- Field Targeting: Extracts specific fields from the request/response using JSONPath expressions configured in
request_fields/response_fields - Full Payload Fallback: If no field targeting is configured, processes the entire payload
- Config Access: Uses
get_passthrough_guardrails_config()/set_passthrough_guardrails_config()helpers to access the passthrough guardrails configuration stored in request metadata
Example Config
passthrough_endpoints:
- path: "/v1/rerank"
target: "https://api.cohere.com/v1/rerank"
guardrails:
bedrock-pre-guard:
request_fields: ["query", "documents[*].text"]
response_fields: ["results[*].text"]