[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-portkey-migrate-litellm-to-portkey":3,"mdc--xksdkm-key":36,"related-org-portkey-migrate-litellm-to-portkey":3906,"related-repo-portkey-migrate-litellm-to-portkey":3945},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":12,"stars":26,"repoUrl":27,"updatedAt":28,"license":29,"forks":26,"topics":30,"repo":31,"sourceUrl":34,"mdContent":35},"migrate-litellm-to-portkey","migrate Python apps to Portkey","Migrate Python applications from LiteLLM to Portkey AI Gateway. Covers litellm.completion, Router, Proxy, fallbacks, caching, retries, callbacks, embeddings, and image generation. Use when the user wants to replace LiteLLM with Portkey or switch from LiteLLM to Portkey.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},"portkey","Portkey","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fportkey.png","Portkey-AI",[13,17,20,23],{"name":14,"slug":15,"type":16},"LLM","llm","tag",{"name":18,"slug":19,"type":16},"Migration","migration",{"name":21,"slug":22,"type":16},"Python","python",{"name":24,"slug":25,"type":16},"AI Infrastructure","ai-infrastructure",1,"https:\u002F\u002Fgithub.com\u002FPortkey-AI\u002Fskills","2026-07-13T06:00:08.528744","Apache-2.0",[],{"repoUrl":27,"stars":26,"forks":26,"topics":32,"description":33},[],null,"https:\u002F\u002Fgithub.com\u002FPortkey-AI\u002Fskills\u002Ftree\u002FHEAD\u002Fskills\u002Fmigrate-litellm-to-portkey","---\nname: migrate-litellm-to-portkey\ndescription: Migrate Python applications from LiteLLM to Portkey AI Gateway. Covers litellm.completion, Router, Proxy, fallbacks, caching, retries, callbacks, embeddings, and image generation. Use when the user wants to replace LiteLLM with Portkey or switch from LiteLLM to Portkey.\nlicense: Apache-2.0\ncompatibility: Requires Python 3.8+. Replaces litellm with portkey-ai package.\nmetadata:\n author: portkey-ai\n version: \"1.0.0\"\n docs: https:\u002F\u002Fdocs.portkey.ai\n---\n\n# Migrate from LiteLLM to Portkey\n\nStep-by-step guide for migrating Python applications from LiteLLM to the Portkey AI Gateway SDK. Portkey uses an OpenAI-compatible client interface, so most code changes are structural (client instantiation and config) rather than call-signature changes.\n\n**Additional References:**\n- [Detailed Migration Patterns](references\u002FMIGRATION_PATTERNS.md) - Every LiteLLM pattern with before\u002Fafter code\n- [Proxy & Config Migration](references\u002FLITELLM_PROXY_MIGRATION.md) - LiteLLM Proxy → Portkey gateway migration\n\n---\n\n## Pre-Migration Checklist\n\n1. **Get a Portkey API key** at [app.portkey.ai\u002Fapi-keys](https:\u002F\u002Fapp.portkey.ai\u002Fapi-keys)\n2. **Set up AI Providers** in the [Model Catalog](https:\u002F\u002Fapp.portkey.ai) — connect your LLM providers (OpenAI, Anthropic, Azure, Bedrock, etc.) and note each provider's slug (e.g., `openai-prod`, `anthropic-main`)\n3. **Swap the package**:\n ```bash\n pip uninstall litellm\n pip install portkey-ai\n ```\n4. **Set environment variables**:\n ```bash\n export PORTKEY_API_KEY=\"your-portkey-api-key\"\n # Provider keys are now stored securely in the Model Catalog\n # You no longer need OPENAI_API_KEY, ANTHROPIC_API_KEY, etc. in your env\n ```\n5. **(Optional) Verify connectivity** with the Portkey CLI:\n ```bash\n npx portkey verify\n ```\n\n---\n\n## Quick Migration Map\n\n| LiteLLM | Portkey |\n|---------|---------|\n| `from litellm import completion` | `from portkey_ai import Portkey` |\n| `completion(model=\"openai\u002Fgpt-4o\", ...)` | `client.chat.completions.create(model=\"@openai-prod\u002Fgpt-4o\", ...)` |\n| `litellm.acompletion(...)` | `await async_client.chat.completions.create(...)` |\n| `litellm.embedding(...)` | `client.embeddings.create(...)` |\n| `litellm.image_generation(...)` | `client.images.generate(...)` |\n| `Router(model_list=[...])` | `Portkey(config={\"strategy\": ..., \"targets\": [...]})` |\n| `completion(..., fallbacks=[...])` | `Portkey(config={\"strategy\": {\"mode\": \"fallback\"}, ...})` |\n| `completion(..., num_retries=3)` | `Portkey(config={\"retry\": {\"attempts\": 3, ...}})` |\n| `litellm.cache = Cache(...)` | `Portkey(config={\"cache\": {\"mode\": \"semantic\", ...}})` |\n| `litellm.success_callback = [...]` | Built-in dashboard + `trace_id` \u002F `metadata` |\n| `completion(..., timeout=30)` | `Portkey(..., request_timeout=30)` |\n| `completion(..., metadata={...})` | `Portkey(..., metadata={...})` |\n| `\"openai\u002Fgpt-4o\"` (provider\u002Fmodel) | `\"@openai-prod\u002Fgpt-4o\"` (@provider-slug\u002Fmodel) |\n\n---\n\n## Core Migrations\n\n### 1. Basic Completion\n\n**LiteLLM:**\n```python\nfrom litellm import completion\nimport os\n\nos.environ[\"OPENAI_API_KEY\"] = \"sk-...\"\n\nresponse = completion(\n model=\"openai\u002Fgpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\nprint(response.choices[0].message.content)\n```\n\n**Portkey:**\n```python\nimport os\nfrom portkey_ai import Portkey\n\nclient = Portkey(\n api_key=os.environ[\"PORTKEY_API_KEY\"]\n)\n\nresponse = client.chat.completions.create(\n model=\"@openai-prod\u002Fgpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\nprint(response.choices[0].message.content)\n```\n\nKey differences:\n- Stateless function → stateful client object\n- Model format changes from `\"openai\u002Fgpt-4o\"` to `\"@provider-slug\u002Fgpt-4o\"` where the provider slug comes from your Model Catalog\n- Provider API keys no longer needed in env — credentials are stored in the Model Catalog\n\n### 2. Streaming\n\n**LiteLLM:**\n```python\nfor chunk in completion(\n model=\"openai\u002Fgpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Write a poem\"}],\n stream=True,\n):\n print(chunk.choices[0].delta.content or \"\", end=\"\")\n```\n\n**Portkey:**\n```python\nstream = client.chat.completions.create(\n model=\"@openai-prod\u002Fgpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Write a poem\"}],\n stream=True\n)\nfor chunk in stream:\n if chunk.choices[0].delta.content:\n print(chunk.choices[0].delta.content, end=\"\", flush=True)\n```\n\n### 3. Async\n\n**LiteLLM:**\n```python\nimport litellm\nresponse = await litellm.acompletion(\n model=\"openai\u002Fgpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\n```\n\n**Portkey:**\n```python\nfrom portkey_ai import AsyncPortkey\n\nclient = AsyncPortkey(\n api_key=os.environ[\"PORTKEY_API_KEY\"]\n)\nresponse = await client.chat.completions.create(\n model=\"@openai-prod\u002Fgpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\n```\n\n### 4. Fallbacks\n\n**LiteLLM:**\n```python\nresponse = completion(\n model=\"openai\u002Fgpt-4o\",\n messages=messages,\n fallbacks=[\"anthropic\u002Fclaude-3-5-sonnet-20241022\"]\n)\n```\n\n**Portkey:**\n```python\nclient = Portkey(\n api_key=os.environ[\"PORTKEY_API_KEY\"],\n config={\n \"strategy\": {\"mode\": \"fallback\"},\n \"targets\": [\n {\n \"override_params\": {\"model\": \"@openai-prod\u002Fgpt-4o\"}\n },\n {\n \"override_params\": {\"model\": \"@anthropic-main\u002Fclaude-3-5-sonnet-20241022\"}\n }\n ]\n }\n)\nresponse = client.chat.completions.create(messages=messages)\n```\n\n### 5. Load Balancing (Router)\n\n**LiteLLM:**\n```python\nfrom litellm import Router\n\nmodel_list = [\n {\n \"model_name\": \"gpt-3.5-turbo\",\n \"litellm_params\": {\n \"model\": \"azure\u002Fchatgpt-v-2\",\n \"api_key\": os.getenv(\"AZURE_API_KEY\"),\n \"api_base\": os.getenv(\"AZURE_API_BASE\"),\n \"rpm\": 900\n }\n },\n {\n \"model_name\": \"gpt-3.5-turbo\",\n \"litellm_params\": {\n \"model\": \"openai\u002Fgpt-3.5-turbo\",\n \"api_key\": os.getenv(\"OPENAI_API_KEY\"),\n \"rpm\": 100\n }\n }\n]\n\nrouter = Router(model_list=model_list, routing_strategy=\"simple-shuffle\")\nresponse = await router.acompletion(\n model=\"gpt-3.5-turbo\",\n messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\n```\n\n**Portkey:**\n```python\nclient = Portkey(\n api_key=os.environ[\"PORTKEY_API_KEY\"],\n config={\n \"strategy\": {\"mode\": \"loadbalance\"},\n \"targets\": [\n {\n \"override_params\": {\"model\": \"@azure-prod\u002Fgpt-3.5-turbo\"},\n \"weight\": 0.9\n },\n {\n \"override_params\": {\"model\": \"@openai-prod\u002Fgpt-3.5-turbo\"},\n \"weight\": 0.1\n }\n ]\n }\n)\nresponse = client.chat.completions.create(\n messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\n```\n\n### 6. Retries\n\n**LiteLLM:**\n```python\nresponse = completion(\n model=\"openai\u002Fgpt-4o\",\n messages=messages,\n num_retries=3\n)\n```\n\n**Portkey:**\n```python\nclient = Portkey(\n api_key=os.environ[\"PORTKEY_API_KEY\"],\n config={\n \"retry\": {\"attempts\": 3, \"on_status_codes\": [429, 500, 502, 503, 504]}\n }\n)\nresponse = client.chat.completions.create(\n model=\"@openai-prod\u002Fgpt-4o\",\n messages=messages\n)\n```\n\n### 7. Caching\n\n**LiteLLM:**\n```python\nimport litellm\nfrom litellm.caching.caching import Cache\n\nlitellm.cache = Cache(type=\"redis\", host=\"localhost\", port=6379)\n\nresponse = completion(model=\"openai\u002Fgpt-4o\", messages=messages, caching=True)\n```\n\n**Portkey:** (no infrastructure to manage — caching is built-in)\n```python\nclient = Portkey(\n api_key=os.environ[\"PORTKEY_API_KEY\"],\n config={\n \"cache\": {\"mode\": \"semantic\", \"max_age\": 3600}\n }\n)\nresponse = client.chat.completions.create(\n model=\"@openai-prod\u002Fgpt-4o\",\n messages=messages\n)\n```\n\n### 8. Observability\n\n**LiteLLM:**\n```python\nimport litellm\nlitellm.success_callback = [\"langfuse\"]\nlitellm.failure_callback = [\"langfuse\"]\n```\n\n**Portkey:** (built-in — every request is logged automatically)\n```python\nclient = Portkey(\n api_key=os.environ[\"PORTKEY_API_KEY\"],\n trace_id=\"session-123\",\n metadata={\n \"user_id\": \"user-456\",\n \"environment\": \"production\"\n }\n)\n```\n\nView logs, traces, costs, and latency at [app.portkey.ai](https:\u002F\u002Fapp.portkey.ai). No third-party callback integration needed.\n\n---\n\n## Model Naming\n\nLiteLLM uses `provider\u002Fmodel` format. Portkey uses `@provider-slug\u002Fmodel` where the provider slug comes from your Model Catalog:\n\n| LiteLLM Model String | Portkey Model String |\n|----------------------|---------------------|\n| `\"openai\u002Fgpt-4o\"` | `\"@openai-prod\u002Fgpt-4o\"` |\n| `\"anthropic\u002Fclaude-3-5-sonnet-20241022\"` | `\"@anthropic-main\u002Fclaude-3-5-sonnet-20241022\"` |\n| `\"azure\u002Fgpt-4\"` | `\"@azure-us-east\u002Fgpt-4\"` |\n| `\"bedrock\u002Fanthropic.claude-3-sonnet...\"` | `\"@bedrock-main\u002Fanthropic.claude-3-sonnet...\"` |\n| `\"vertex_ai\u002Fgemini-1.5-pro\"` | `\"@vertex-prod\u002Fgemini-1.5-pro\"` |\n\nThe `@slug` prefix maps to an AI Provider in your Model Catalog. You control which models are available under each provider.\n\n---\n\n## Error Handling\n\n**LiteLLM:**\n```python\nimport litellm\ntry:\n response = completion(model=\"openai\u002Fgpt-4o\", messages=messages)\nexcept litellm.AuthenticationError:\n print(\"Bad API key\")\nexcept litellm.RateLimitError:\n print(\"Rate limited\")\n```\n\n**Portkey:**\n```python\ntry:\n response = client.chat.completions.create(\n model=\"@openai-prod\u002Fgpt-4o\", messages=messages\n )\nexcept Exception as e:\n status = getattr(e, 'status_code', None)\n if status == 401:\n print(\"Bad API key\")\n elif status == 429:\n print(\"Rate limited\")\n```\n\n---\n\n## AI Coding Agent Migrations\n\n### Claude Code\n\nIf Claude Code is routed through a LiteLLM Proxy (via `ANTHROPIC_BASE_URL`), the fastest migration is:\n\n```bash\nnpx portkey setup\n```\n\nThis replaces the LiteLLM Proxy env vars (`ANTHROPIC_BASE_URL`, `ANTHROPIC_API_KEY`) with Portkey equivalents. See [Proxy Migration — Claude Code section](references\u002FLITELLM_PROXY_MIGRATION.md#claude-code-litellm-proxy--portkey) for manual setup, Bedrock\u002FVertex model mappings, and Config-based routing.\n\n### OpenAI Codex CLI\n\nIf Codex CLI is routed through a LiteLLM Proxy (via `openai_base_url` or `OPENAI_BASE_URL`), update `~\u002F.codex\u002Fconfig.json` to point at Portkey with a `portkey` provider and `model: \"@provider-slug\u002Fmodel\"` format. See [Proxy Migration — Codex section](references\u002FLITELLM_PROXY_MIGRATION.md#openai-codex-cli-litellm-proxy--portkey) for step-by-step setup.\n\n---\n\n## Zero-Change Path: Keep LiteLLM, Route Through Portkey\n\nIf you want Portkey's observability and reliability without rewriting any code, you can keep using `litellm.completion()` and just point it at Portkey's gateway:\n\n```python\nfrom litellm import completion\n\nresponse = completion(\n model=\"gpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Hello!\"}],\n api_base=\"https:\u002F\u002Fapi.portkey.ai\u002Fv1\",\n api_key=os.environ[\"PORTKEY_API_KEY\"],\n extra_headers={\"x-portkey-provider\": \"@openai-prod\"}\n)\n```\n\nThis gives you Portkey logging, caching, and guardrails immediately — then you can migrate call-by-call to the Portkey SDK at your own pace.\n\n---\n\n## OpenAI Client Approach (Minimal Changes)\n\nIf your codebase uses the OpenAI SDK to talk to LiteLLM's proxy, you can switch to Portkey by only changing `base_url` and adding headers — no import changes required:\n\n```python\nimport openai\nfrom portkey_ai import PORTKEY_GATEWAY_URL, createHeaders\n\nclient = openai.OpenAI(\n api_key=\"not-used\",\n base_url=PORTKEY_GATEWAY_URL, # https:\u002F\u002Fapi.portkey.ai\u002Fv1\n default_headers=createHeaders(\n api_key=\"your-portkey-api-key\"\n )\n)\n\nresponse = client.chat.completions.create(\n model=\"@openai-prod\u002Fgpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\n```\n\nThis also works in **TypeScript\u002FJavaScript**:\n\n```typescript\nimport OpenAI from \"openai\";\n\nconst client = new OpenAI({\n baseURL: \"https:\u002F\u002Fapi.portkey.ai\u002Fv1\",\n apiKey: process.env.PORTKEY_API_KEY,\n defaultHeaders: { \"x-portkey-provider\": \"@openai-prod\" }\n});\n\nconst response = await client.chat.completions.create({\n model: \"gpt-4o\",\n messages: [{ role: \"user\", content: \"Hello!\" }]\n});\n```\n\n### Quick Verification with cURL\n\nBefore changing any code, test the gateway connection:\n\n```bash\ncurl -s https:\u002F\u002Fapi.portkey.ai\u002Fv1\u002Fchat\u002Fcompletions \\\n -H \"Content-Type: application\u002Fjson\" \\\n -H \"x-portkey-api-key: $PORTKEY_API_KEY\" \\\n -H \"x-portkey-provider: @openai-prod\" \\\n -d '{\n \"model\": \"gpt-4o\",\n \"max_tokens\": 5,\n \"messages\": [{\"role\": \"user\", \"content\": \"Say ok\"}]\n }'\n```\n\n### Routing via Headers\n\nPortkey supports two routing modes, set via headers:\n\n| Header | Use Case |\n|--------|----------|\n| `x-portkey-provider: @slug` | Route to a specific AI Provider from your Model Catalog |\n| `x-portkey-config: pc-config-xxx` | Use a Config (with fallbacks, caching, etc.) from the dashboard |\n\nWith the `@provider-slug\u002Fmodel` format in the `model` parameter, you typically don't need to set these headers explicitly.\n\n---\n\n## Migration Workflow\n\n1. **Verify** — Test connectivity before touching code:\n ```bash\n npx portkey verify\n ```\n2. **Audit** — Grep your codebase for LiteLLM usage:\n ```bash\n grep -rn \"from litellm\\|import litellm\\|litellm\\.\" --include=\"*.py\"\n ```\n3. **Quick win** — Point existing LiteLLM calls at Portkey with `api_base` (see [Zero-Change Path](#zero-change-path-keep-litellm-route-through-portkey))\n4. **Map** — For each usage, find the Portkey equivalent using the table above or [MIGRATION_PATTERNS.md](references\u002FMIGRATION_PATTERNS.md)\n5. **Replace** — Swap imports, instantiate a Portkey client, and update call sites\n6. **Configure** — Move routing\u002Ffallback\u002Fcache logic from code into Portkey `config` dicts (or dashboard [Config IDs](https:\u002F\u002Fapp.portkey.ai\u002Fconfigs))\n7. **Test** — Run your test suite; response formats are identical (OpenAI-compatible)\n8. **Clean up** — Remove `litellm` from requirements, remove provider API key env vars\n\n---\n\n## Resources\n\n- **Portkey Docs**: [docs.portkey.ai](https:\u002F\u002Fdocs.portkey.ai)\n- **Dashboard**: [app.portkey.ai](https:\u002F\u002Fapp.portkey.ai)\n- **Python SDK**: [github.com\u002Fportkey-ai\u002Fportkey-python-sdk](https:\u002F\u002Fgithub.com\u002Fportkey-ai\u002Fportkey-python-sdk)\n- **Discord**: [portkey.ai\u002Fdiscord](https:\u002F\u002Fportkey.ai\u002Fdiscord)\n",{"data":37,"body":43},{"name":4,"description":6,"license":29,"compatibility":38,"metadata":39},"Requires Python 3.8+. Replaces litellm with portkey-ai package.",{"author":40,"version":41,"docs":42},"portkey-ai","1.0.0","https:\u002F\u002Fdocs.portkey.ai",{"type":44,"children":45},"root",[46,55,61,70,98,102,109,333,336,342,656,659,665,672,680,775,783,881,886,919,925,932,986,993,1061,1067,1074,1118,1125,1198,1204,1211,1255,1262,1389,1395,1402,1628,1635,1780,1786,1793,1836,1843,1922,1928,1935,1987,1996,2074,2080,2087,2117,2126,2193,2206,2209,2215,2236,2361,2374,2377,2383,2390,2452,2459,2545,2548,2554,2560,2573,2596,2623,2629,2681,2684,2690,2703,2777,2782,2785,2791,2804,2923,2935,3358,3364,3369,3531,3537,3542,3598,3618,3621,3627,3826,3829,3835,3900],{"type":47,"tag":48,"props":49,"children":51},"element","h1",{"id":50},"migrate-from-litellm-to-portkey",[52],{"type":53,"value":54},"text","Migrate from LiteLLM to Portkey",{"type":47,"tag":56,"props":57,"children":58},"p",{},[59],{"type":53,"value":60},"Step-by-step guide for migrating Python applications from LiteLLM to the Portkey AI Gateway SDK. Portkey uses an OpenAI-compatible client interface, so most code changes are structural (client instantiation and config) rather than call-signature changes.",{"type":47,"tag":56,"props":62,"children":63},{},[64],{"type":47,"tag":65,"props":66,"children":67},"strong",{},[68],{"type":53,"value":69},"Additional References:",{"type":47,"tag":71,"props":72,"children":73},"ul",{},[74,87],{"type":47,"tag":75,"props":76,"children":77},"li",{},[78,85],{"type":47,"tag":79,"props":80,"children":82},"a",{"href":81},"references\u002FMIGRATION_PATTERNS.md",[83],{"type":53,"value":84},"Detailed Migration Patterns",{"type":53,"value":86}," - Every LiteLLM pattern with before\u002Fafter code",{"type":47,"tag":75,"props":88,"children":89},{},[90,96],{"type":47,"tag":79,"props":91,"children":93},{"href":92},"references\u002FLITELLM_PROXY_MIGRATION.md",[94],{"type":53,"value":95},"Proxy & Config Migration",{"type":53,"value":97}," - LiteLLM Proxy → Portkey gateway migration",{"type":47,"tag":99,"props":100,"children":101},"hr",{},[],{"type":47,"tag":103,"props":104,"children":106},"h2",{"id":105},"pre-migration-checklist",[107],{"type":53,"value":108},"Pre-Migration Checklist",{"type":47,"tag":110,"props":111,"children":112},"ol",{},[113,131,167,228,298],{"type":47,"tag":75,"props":114,"children":115},{},[116,121,123],{"type":47,"tag":65,"props":117,"children":118},{},[119],{"type":53,"value":120},"Get a Portkey API key",{"type":53,"value":122}," at ",{"type":47,"tag":79,"props":124,"children":128},{"href":125,"rel":126},"https:\u002F\u002Fapp.portkey.ai\u002Fapi-keys",[127],"nofollow",[129],{"type":53,"value":130},"app.portkey.ai\u002Fapi-keys",{"type":47,"tag":75,"props":132,"children":133},{},[134,139,141,148,150,157,159,165],{"type":47,"tag":65,"props":135,"children":136},{},[137],{"type":53,"value":138},"Set up AI Providers",{"type":53,"value":140}," in the ",{"type":47,"tag":79,"props":142,"children":145},{"href":143,"rel":144},"https:\u002F\u002Fapp.portkey.ai",[127],[146],{"type":53,"value":147},"Model Catalog",{"type":53,"value":149}," — connect your LLM providers (OpenAI, Anthropic, Azure, Bedrock, etc.) and note each provider's slug (e.g., ",{"type":47,"tag":151,"props":152,"children":154},"code",{"className":153},[],[155],{"type":53,"value":156},"openai-prod",{"type":53,"value":158},", ",{"type":47,"tag":151,"props":160,"children":162},{"className":161},[],[163],{"type":53,"value":164},"anthropic-main",{"type":53,"value":166},")",{"type":47,"tag":75,"props":168,"children":169},{},[170,175,177],{"type":47,"tag":65,"props":171,"children":172},{},[173],{"type":53,"value":174},"Swap the package",{"type":53,"value":176},":\n",{"type":47,"tag":178,"props":179,"children":184},"pre",{"className":180,"code":181,"language":182,"meta":183,"style":183},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","pip uninstall litellm\npip install portkey-ai\n","bash","",[185],{"type":47,"tag":151,"props":186,"children":187},{"__ignoreMap":183},[188,210],{"type":47,"tag":189,"props":190,"children":192},"span",{"class":191,"line":26},"line",[193,199,205],{"type":47,"tag":189,"props":194,"children":196},{"style":195},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[197],{"type":53,"value":198},"pip",{"type":47,"tag":189,"props":200,"children":202},{"style":201},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[203],{"type":53,"value":204}," uninstall",{"type":47,"tag":189,"props":206,"children":207},{"style":201},[208],{"type":53,"value":209}," litellm\n",{"type":47,"tag":189,"props":211,"children":213},{"class":191,"line":212},2,[214,218,223],{"type":47,"tag":189,"props":215,"children":216},{"style":195},[217],{"type":53,"value":198},{"type":47,"tag":189,"props":219,"children":220},{"style":201},[221],{"type":53,"value":222}," install",{"type":47,"tag":189,"props":224,"children":225},{"style":201},[226],{"type":53,"value":227}," portkey-ai\n",{"type":47,"tag":75,"props":229,"children":230},{},[231,236,237],{"type":47,"tag":65,"props":232,"children":233},{},[234],{"type":53,"value":235},"Set environment variables",{"type":53,"value":176},{"type":47,"tag":178,"props":238,"children":240},{"className":180,"code":239,"language":182,"meta":183,"style":183},"export PORTKEY_API_KEY=\"your-portkey-api-key\"\n# Provider keys are now stored securely in the Model Catalog\n# You no longer need OPENAI_API_KEY, ANTHROPIC_API_KEY, etc. in your env\n",[241],{"type":47,"tag":151,"props":242,"children":243},{"__ignoreMap":183},[244,280,289],{"type":47,"tag":189,"props":245,"children":246},{"class":191,"line":26},[247,253,259,265,270,275],{"type":47,"tag":189,"props":248,"children":250},{"style":249},"--shiki-light:#9C3EDA;--shiki-default:#C792EA;--shiki-dark:#C792EA",[251],{"type":53,"value":252},"export",{"type":47,"tag":189,"props":254,"children":256},{"style":255},"--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8",[257],{"type":53,"value":258}," PORTKEY_API_KEY",{"type":47,"tag":189,"props":260,"children":262},{"style":261},"--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF",[263],{"type":53,"value":264},"=",{"type":47,"tag":189,"props":266,"children":267},{"style":261},[268],{"type":53,"value":269},"\"",{"type":47,"tag":189,"props":271,"children":272},{"style":201},[273],{"type":53,"value":274},"your-portkey-api-key",{"type":47,"tag":189,"props":276,"children":277},{"style":261},[278],{"type":53,"value":279},"\"\n",{"type":47,"tag":189,"props":281,"children":282},{"class":191,"line":212},[283],{"type":47,"tag":189,"props":284,"children":286},{"style":285},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[287],{"type":53,"value":288},"# Provider keys are now stored securely in the Model Catalog\n",{"type":47,"tag":189,"props":290,"children":292},{"class":191,"line":291},3,[293],{"type":47,"tag":189,"props":294,"children":295},{"style":285},[296],{"type":53,"value":297},"# You no longer need OPENAI_API_KEY, ANTHROPIC_API_KEY, etc. in your env\n",{"type":47,"tag":75,"props":299,"children":300},{},[301,306,308],{"type":47,"tag":65,"props":302,"children":303},{},[304],{"type":53,"value":305},"(Optional) Verify connectivity",{"type":53,"value":307}," with the Portkey CLI:\n",{"type":47,"tag":178,"props":309,"children":311},{"className":180,"code":310,"language":182,"meta":183,"style":183},"npx portkey verify\n",[312],{"type":47,"tag":151,"props":313,"children":314},{"__ignoreMap":183},[315],{"type":47,"tag":189,"props":316,"children":317},{"class":191,"line":26},[318,323,328],{"type":47,"tag":189,"props":319,"children":320},{"style":195},[321],{"type":53,"value":322},"npx",{"type":47,"tag":189,"props":324,"children":325},{"style":201},[326],{"type":53,"value":327}," portkey",{"type":47,"tag":189,"props":329,"children":330},{"style":201},[331],{"type":53,"value":332}," verify\n",{"type":47,"tag":99,"props":334,"children":335},{},[],{"type":47,"tag":103,"props":337,"children":339},{"id":338},"quick-migration-map",[340],{"type":53,"value":341},"Quick Migration Map",{"type":47,"tag":343,"props":344,"children":345},"table",{},[346,364],{"type":47,"tag":347,"props":348,"children":349},"thead",{},[350],{"type":47,"tag":351,"props":352,"children":353},"tr",{},[354,360],{"type":47,"tag":355,"props":356,"children":357},"th",{},[358],{"type":53,"value":359},"LiteLLM",{"type":47,"tag":355,"props":361,"children":362},{},[363],{"type":53,"value":9},{"type":47,"tag":365,"props":366,"children":367},"tbody",{},[368,390,411,432,453,474,495,516,537,558,589,610,631],{"type":47,"tag":351,"props":369,"children":370},{},[371,381],{"type":47,"tag":372,"props":373,"children":374},"td",{},[375],{"type":47,"tag":151,"props":376,"children":378},{"className":377},[],[379],{"type":53,"value":380},"from litellm import completion",{"type":47,"tag":372,"props":382,"children":383},{},[384],{"type":47,"tag":151,"props":385,"children":387},{"className":386},[],[388],{"type":53,"value":389},"from portkey_ai import Portkey",{"type":47,"tag":351,"props":391,"children":392},{},[393,402],{"type":47,"tag":372,"props":394,"children":395},{},[396],{"type":47,"tag":151,"props":397,"children":399},{"className":398},[],[400],{"type":53,"value":401},"completion(model=\"openai\u002Fgpt-4o\", ...)",{"type":47,"tag":372,"props":403,"children":404},{},[405],{"type":47,"tag":151,"props":406,"children":408},{"className":407},[],[409],{"type":53,"value":410},"client.chat.completions.create(model=\"@openai-prod\u002Fgpt-4o\", ...)",{"type":47,"tag":351,"props":412,"children":413},{},[414,423],{"type":47,"tag":372,"props":415,"children":416},{},[417],{"type":47,"tag":151,"props":418,"children":420},{"className":419},[],[421],{"type":53,"value":422},"litellm.acompletion(...)",{"type":47,"tag":372,"props":424,"children":425},{},[426],{"type":47,"tag":151,"props":427,"children":429},{"className":428},[],[430],{"type":53,"value":431},"await async_client.chat.completions.create(...)",{"type":47,"tag":351,"props":433,"children":434},{},[435,444],{"type":47,"tag":372,"props":436,"children":437},{},[438],{"type":47,"tag":151,"props":439,"children":441},{"className":440},[],[442],{"type":53,"value":443},"litellm.embedding(...)",{"type":47,"tag":372,"props":445,"children":446},{},[447],{"type":47,"tag":151,"props":448,"children":450},{"className":449},[],[451],{"type":53,"value":452},"client.embeddings.create(...)",{"type":47,"tag":351,"props":454,"children":455},{},[456,465],{"type":47,"tag":372,"props":457,"children":458},{},[459],{"type":47,"tag":151,"props":460,"children":462},{"className":461},[],[463],{"type":53,"value":464},"litellm.image_generation(...)",{"type":47,"tag":372,"props":466,"children":467},{},[468],{"type":47,"tag":151,"props":469,"children":471},{"className":470},[],[472],{"type":53,"value":473},"client.images.generate(...)",{"type":47,"tag":351,"props":475,"children":476},{},[477,486],{"type":47,"tag":372,"props":478,"children":479},{},[480],{"type":47,"tag":151,"props":481,"children":483},{"className":482},[],[484],{"type":53,"value":485},"Router(model_list=[...])",{"type":47,"tag":372,"props":487,"children":488},{},[489],{"type":47,"tag":151,"props":490,"children":492},{"className":491},[],[493],{"type":53,"value":494},"Portkey(config={\"strategy\": ..., \"targets\": [...]})",{"type":47,"tag":351,"props":496,"children":497},{},[498,507],{"type":47,"tag":372,"props":499,"children":500},{},[501],{"type":47,"tag":151,"props":502,"children":504},{"className":503},[],[505],{"type":53,"value":506},"completion(..., fallbacks=[...])",{"type":47,"tag":372,"props":508,"children":509},{},[510],{"type":47,"tag":151,"props":511,"children":513},{"className":512},[],[514],{"type":53,"value":515},"Portkey(config={\"strategy\": {\"mode\": \"fallback\"}, ...})",{"type":47,"tag":351,"props":517,"children":518},{},[519,528],{"type":47,"tag":372,"props":520,"children":521},{},[522],{"type":47,"tag":151,"props":523,"children":525},{"className":524},[],[526],{"type":53,"value":527},"completion(..., num_retries=3)",{"type":47,"tag":372,"props":529,"children":530},{},[531],{"type":47,"tag":151,"props":532,"children":534},{"className":533},[],[535],{"type":53,"value":536},"Portkey(config={\"retry\": {\"attempts\": 3, ...}})",{"type":47,"tag":351,"props":538,"children":539},{},[540,549],{"type":47,"tag":372,"props":541,"children":542},{},[543],{"type":47,"tag":151,"props":544,"children":546},{"className":545},[],[547],{"type":53,"value":548},"litellm.cache = Cache(...)",{"type":47,"tag":372,"props":550,"children":551},{},[552],{"type":47,"tag":151,"props":553,"children":555},{"className":554},[],[556],{"type":53,"value":557},"Portkey(config={\"cache\": {\"mode\": \"semantic\", ...}})",{"type":47,"tag":351,"props":559,"children":560},{},[561,570],{"type":47,"tag":372,"props":562,"children":563},{},[564],{"type":47,"tag":151,"props":565,"children":567},{"className":566},[],[568],{"type":53,"value":569},"litellm.success_callback = [...]",{"type":47,"tag":372,"props":571,"children":572},{},[573,575,581,583],{"type":53,"value":574},"Built-in dashboard + ",{"type":47,"tag":151,"props":576,"children":578},{"className":577},[],[579],{"type":53,"value":580},"trace_id",{"type":53,"value":582}," \u002F ",{"type":47,"tag":151,"props":584,"children":586},{"className":585},[],[587],{"type":53,"value":588},"metadata",{"type":47,"tag":351,"props":590,"children":591},{},[592,601],{"type":47,"tag":372,"props":593,"children":594},{},[595],{"type":47,"tag":151,"props":596,"children":598},{"className":597},[],[599],{"type":53,"value":600},"completion(..., timeout=30)",{"type":47,"tag":372,"props":602,"children":603},{},[604],{"type":47,"tag":151,"props":605,"children":607},{"className":606},[],[608],{"type":53,"value":609},"Portkey(..., request_timeout=30)",{"type":47,"tag":351,"props":611,"children":612},{},[613,622],{"type":47,"tag":372,"props":614,"children":615},{},[616],{"type":47,"tag":151,"props":617,"children":619},{"className":618},[],[620],{"type":53,"value":621},"completion(..., metadata={...})",{"type":47,"tag":372,"props":623,"children":624},{},[625],{"type":47,"tag":151,"props":626,"children":628},{"className":627},[],[629],{"type":53,"value":630},"Portkey(..., metadata={...})",{"type":47,"tag":351,"props":632,"children":633},{},[634,645],{"type":47,"tag":372,"props":635,"children":636},{},[637,643],{"type":47,"tag":151,"props":638,"children":640},{"className":639},[],[641],{"type":53,"value":642},"\"openai\u002Fgpt-4o\"",{"type":53,"value":644}," (provider\u002Fmodel)",{"type":47,"tag":372,"props":646,"children":647},{},[648,654],{"type":47,"tag":151,"props":649,"children":651},{"className":650},[],[652],{"type":53,"value":653},"\"@openai-prod\u002Fgpt-4o\"",{"type":53,"value":655}," (@provider-slug\u002Fmodel)",{"type":47,"tag":99,"props":657,"children":658},{},[],{"type":47,"tag":103,"props":660,"children":662},{"id":661},"core-migrations",[663],{"type":53,"value":664},"Core Migrations",{"type":47,"tag":666,"props":667,"children":669},"h3",{"id":668},"_1-basic-completion",[670],{"type":53,"value":671},"1. Basic Completion",{"type":47,"tag":56,"props":673,"children":674},{},[675],{"type":47,"tag":65,"props":676,"children":677},{},[678],{"type":53,"value":679},"LiteLLM:",{"type":47,"tag":178,"props":681,"children":684},{"className":682,"code":683,"language":22,"meta":183,"style":183},"language-python shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","from litellm import completion\nimport os\n\nos.environ[\"OPENAI_API_KEY\"] = \"sk-...\"\n\nresponse = completion(\n model=\"openai\u002Fgpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\nprint(response.choices[0].message.content)\n",[685],{"type":47,"tag":151,"props":686,"children":687},{"__ignoreMap":183},[688,696,704,713,722,730,739,748,757,766],{"type":47,"tag":189,"props":689,"children":690},{"class":191,"line":26},[691],{"type":47,"tag":189,"props":692,"children":693},{},[694],{"type":53,"value":695},"from litellm import completion\n",{"type":47,"tag":189,"props":697,"children":698},{"class":191,"line":212},[699],{"type":47,"tag":189,"props":700,"children":701},{},[702],{"type":53,"value":703},"import os\n",{"type":47,"tag":189,"props":705,"children":706},{"class":191,"line":291},[707],{"type":47,"tag":189,"props":708,"children":710},{"emptyLinePlaceholder":709},true,[711],{"type":53,"value":712},"\n",{"type":47,"tag":189,"props":714,"children":716},{"class":191,"line":715},4,[717],{"type":47,"tag":189,"props":718,"children":719},{},[720],{"type":53,"value":721},"os.environ[\"OPENAI_API_KEY\"] = \"sk-...\"\n",{"type":47,"tag":189,"props":723,"children":725},{"class":191,"line":724},5,[726],{"type":47,"tag":189,"props":727,"children":728},{"emptyLinePlaceholder":709},[729],{"type":53,"value":712},{"type":47,"tag":189,"props":731,"children":733},{"class":191,"line":732},6,[734],{"type":47,"tag":189,"props":735,"children":736},{},[737],{"type":53,"value":738},"response = completion(\n",{"type":47,"tag":189,"props":740,"children":742},{"class":191,"line":741},7,[743],{"type":47,"tag":189,"props":744,"children":745},{},[746],{"type":53,"value":747}," model=\"openai\u002Fgpt-4o\",\n",{"type":47,"tag":189,"props":749,"children":751},{"class":191,"line":750},8,[752],{"type":47,"tag":189,"props":753,"children":754},{},[755],{"type":53,"value":756}," messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n",{"type":47,"tag":189,"props":758,"children":760},{"class":191,"line":759},9,[761],{"type":47,"tag":189,"props":762,"children":763},{},[764],{"type":53,"value":765},")\n",{"type":47,"tag":189,"props":767,"children":769},{"class":191,"line":768},10,[770],{"type":47,"tag":189,"props":771,"children":772},{},[773],{"type":53,"value":774},"print(response.choices[0].message.content)\n",{"type":47,"tag":56,"props":776,"children":777},{},[778],{"type":47,"tag":65,"props":779,"children":780},{},[781],{"type":53,"value":782},"Portkey:",{"type":47,"tag":178,"props":784,"children":786},{"className":682,"code":785,"language":22,"meta":183,"style":183},"import os\nfrom portkey_ai import Portkey\n\nclient = Portkey(\n api_key=os.environ[\"PORTKEY_API_KEY\"]\n)\n\nresponse = client.chat.completions.create(\n model=\"@openai-prod\u002Fgpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\nprint(response.choices[0].message.content)\n",[787],{"type":47,"tag":151,"props":788,"children":789},{"__ignoreMap":183},[790,797,805,812,820,828,835,842,850,858,865,873],{"type":47,"tag":189,"props":791,"children":792},{"class":191,"line":26},[793],{"type":47,"tag":189,"props":794,"children":795},{},[796],{"type":53,"value":703},{"type":47,"tag":189,"props":798,"children":799},{"class":191,"line":212},[800],{"type":47,"tag":189,"props":801,"children":802},{},[803],{"type":53,"value":804},"from portkey_ai import Portkey\n",{"type":47,"tag":189,"props":806,"children":807},{"class":191,"line":291},[808],{"type":47,"tag":189,"props":809,"children":810},{"emptyLinePlaceholder":709},[811],{"type":53,"value":712},{"type":47,"tag":189,"props":813,"children":814},{"class":191,"line":715},[815],{"type":47,"tag":189,"props":816,"children":817},{},[818],{"type":53,"value":819},"client = Portkey(\n",{"type":47,"tag":189,"props":821,"children":822},{"class":191,"line":724},[823],{"type":47,"tag":189,"props":824,"children":825},{},[826],{"type":53,"value":827}," api_key=os.environ[\"PORTKEY_API_KEY\"]\n",{"type":47,"tag":189,"props":829,"children":830},{"class":191,"line":732},[831],{"type":47,"tag":189,"props":832,"children":833},{},[834],{"type":53,"value":765},{"type":47,"tag":189,"props":836,"children":837},{"class":191,"line":741},[838],{"type":47,"tag":189,"props":839,"children":840},{"emptyLinePlaceholder":709},[841],{"type":53,"value":712},{"type":47,"tag":189,"props":843,"children":844},{"class":191,"line":750},[845],{"type":47,"tag":189,"props":846,"children":847},{},[848],{"type":53,"value":849},"response = client.chat.completions.create(\n",{"type":47,"tag":189,"props":851,"children":852},{"class":191,"line":759},[853],{"type":47,"tag":189,"props":854,"children":855},{},[856],{"type":53,"value":857}," model=\"@openai-prod\u002Fgpt-4o\",\n",{"type":47,"tag":189,"props":859,"children":860},{"class":191,"line":768},[861],{"type":47,"tag":189,"props":862,"children":863},{},[864],{"type":53,"value":756},{"type":47,"tag":189,"props":866,"children":868},{"class":191,"line":867},11,[869],{"type":47,"tag":189,"props":870,"children":871},{},[872],{"type":53,"value":765},{"type":47,"tag":189,"props":874,"children":876},{"class":191,"line":875},12,[877],{"type":47,"tag":189,"props":878,"children":879},{},[880],{"type":53,"value":774},{"type":47,"tag":56,"props":882,"children":883},{},[884],{"type":53,"value":885},"Key differences:",{"type":47,"tag":71,"props":887,"children":888},{},[889,894,914],{"type":47,"tag":75,"props":890,"children":891},{},[892],{"type":53,"value":893},"Stateless function → stateful client object",{"type":47,"tag":75,"props":895,"children":896},{},[897,899,904,906,912],{"type":53,"value":898},"Model format changes from ",{"type":47,"tag":151,"props":900,"children":902},{"className":901},[],[903],{"type":53,"value":642},{"type":53,"value":905}," to ",{"type":47,"tag":151,"props":907,"children":909},{"className":908},[],[910],{"type":53,"value":911},"\"@provider-slug\u002Fgpt-4o\"",{"type":53,"value":913}," where the provider slug comes from your Model Catalog",{"type":47,"tag":75,"props":915,"children":916},{},[917],{"type":53,"value":918},"Provider API keys no longer needed in env — credentials are stored in the Model Catalog",{"type":47,"tag":666,"props":920,"children":922},{"id":921},"_2-streaming",[923],{"type":53,"value":924},"2. Streaming",{"type":47,"tag":56,"props":926,"children":927},{},[928],{"type":47,"tag":65,"props":929,"children":930},{},[931],{"type":53,"value":679},{"type":47,"tag":178,"props":933,"children":935},{"className":682,"code":934,"language":22,"meta":183,"style":183},"for chunk in completion(\n model=\"openai\u002Fgpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Write a poem\"}],\n stream=True,\n):\n print(chunk.choices[0].delta.content or \"\", end=\"\")\n",[936],{"type":47,"tag":151,"props":937,"children":938},{"__ignoreMap":183},[939,947,954,962,970,978],{"type":47,"tag":189,"props":940,"children":941},{"class":191,"line":26},[942],{"type":47,"tag":189,"props":943,"children":944},{},[945],{"type":53,"value":946},"for chunk in completion(\n",{"type":47,"tag":189,"props":948,"children":949},{"class":191,"line":212},[950],{"type":47,"tag":189,"props":951,"children":952},{},[953],{"type":53,"value":747},{"type":47,"tag":189,"props":955,"children":956},{"class":191,"line":291},[957],{"type":47,"tag":189,"props":958,"children":959},{},[960],{"type":53,"value":961}," messages=[{\"role\": \"user\", \"content\": \"Write a poem\"}],\n",{"type":47,"tag":189,"props":963,"children":964},{"class":191,"line":715},[965],{"type":47,"tag":189,"props":966,"children":967},{},[968],{"type":53,"value":969}," stream=True,\n",{"type":47,"tag":189,"props":971,"children":972},{"class":191,"line":724},[973],{"type":47,"tag":189,"props":974,"children":975},{},[976],{"type":53,"value":977},"):\n",{"type":47,"tag":189,"props":979,"children":980},{"class":191,"line":732},[981],{"type":47,"tag":189,"props":982,"children":983},{},[984],{"type":53,"value":985}," print(chunk.choices[0].delta.content or \"\", end=\"\")\n",{"type":47,"tag":56,"props":987,"children":988},{},[989],{"type":47,"tag":65,"props":990,"children":991},{},[992],{"type":53,"value":782},{"type":47,"tag":178,"props":994,"children":996},{"className":682,"code":995,"language":22,"meta":183,"style":183},"stream = client.chat.completions.create(\n model=\"@openai-prod\u002Fgpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Write a poem\"}],\n stream=True\n)\nfor chunk in stream:\n if chunk.choices[0].delta.content:\n print(chunk.choices[0].delta.content, end=\"\", flush=True)\n",[997],{"type":47,"tag":151,"props":998,"children":999},{"__ignoreMap":183},[1000,1008,1015,1022,1030,1037,1045,1053],{"type":47,"tag":189,"props":1001,"children":1002},{"class":191,"line":26},[1003],{"type":47,"tag":189,"props":1004,"children":1005},{},[1006],{"type":53,"value":1007},"stream = client.chat.completions.create(\n",{"type":47,"tag":189,"props":1009,"children":1010},{"class":191,"line":212},[1011],{"type":47,"tag":189,"props":1012,"children":1013},{},[1014],{"type":53,"value":857},{"type":47,"tag":189,"props":1016,"children":1017},{"class":191,"line":291},[1018],{"type":47,"tag":189,"props":1019,"children":1020},{},[1021],{"type":53,"value":961},{"type":47,"tag":189,"props":1023,"children":1024},{"class":191,"line":715},[1025],{"type":47,"tag":189,"props":1026,"children":1027},{},[1028],{"type":53,"value":1029}," stream=True\n",{"type":47,"tag":189,"props":1031,"children":1032},{"class":191,"line":724},[1033],{"type":47,"tag":189,"props":1034,"children":1035},{},[1036],{"type":53,"value":765},{"type":47,"tag":189,"props":1038,"children":1039},{"class":191,"line":732},[1040],{"type":47,"tag":189,"props":1041,"children":1042},{},[1043],{"type":53,"value":1044},"for chunk in stream:\n",{"type":47,"tag":189,"props":1046,"children":1047},{"class":191,"line":741},[1048],{"type":47,"tag":189,"props":1049,"children":1050},{},[1051],{"type":53,"value":1052}," if chunk.choices[0].delta.content:\n",{"type":47,"tag":189,"props":1054,"children":1055},{"class":191,"line":750},[1056],{"type":47,"tag":189,"props":1057,"children":1058},{},[1059],{"type":53,"value":1060}," print(chunk.choices[0].delta.content, end=\"\", flush=True)\n",{"type":47,"tag":666,"props":1062,"children":1064},{"id":1063},"_3-async",[1065],{"type":53,"value":1066},"3. Async",{"type":47,"tag":56,"props":1068,"children":1069},{},[1070],{"type":47,"tag":65,"props":1071,"children":1072},{},[1073],{"type":53,"value":679},{"type":47,"tag":178,"props":1075,"children":1077},{"className":682,"code":1076,"language":22,"meta":183,"style":183},"import litellm\nresponse = await litellm.acompletion(\n model=\"openai\u002Fgpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\n",[1078],{"type":47,"tag":151,"props":1079,"children":1080},{"__ignoreMap":183},[1081,1089,1097,1104,1111],{"type":47,"tag":189,"props":1082,"children":1083},{"class":191,"line":26},[1084],{"type":47,"tag":189,"props":1085,"children":1086},{},[1087],{"type":53,"value":1088},"import litellm\n",{"type":47,"tag":189,"props":1090,"children":1091},{"class":191,"line":212},[1092],{"type":47,"tag":189,"props":1093,"children":1094},{},[1095],{"type":53,"value":1096},"response = await litellm.acompletion(\n",{"type":47,"tag":189,"props":1098,"children":1099},{"class":191,"line":291},[1100],{"type":47,"tag":189,"props":1101,"children":1102},{},[1103],{"type":53,"value":747},{"type":47,"tag":189,"props":1105,"children":1106},{"class":191,"line":715},[1107],{"type":47,"tag":189,"props":1108,"children":1109},{},[1110],{"type":53,"value":756},{"type":47,"tag":189,"props":1112,"children":1113},{"class":191,"line":724},[1114],{"type":47,"tag":189,"props":1115,"children":1116},{},[1117],{"type":53,"value":765},{"type":47,"tag":56,"props":1119,"children":1120},{},[1121],{"type":47,"tag":65,"props":1122,"children":1123},{},[1124],{"type":53,"value":782},{"type":47,"tag":178,"props":1126,"children":1128},{"className":682,"code":1127,"language":22,"meta":183,"style":183},"from portkey_ai import AsyncPortkey\n\nclient = AsyncPortkey(\n api_key=os.environ[\"PORTKEY_API_KEY\"]\n)\nresponse = await client.chat.completions.create(\n model=\"@openai-prod\u002Fgpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\n",[1129],{"type":47,"tag":151,"props":1130,"children":1131},{"__ignoreMap":183},[1132,1140,1147,1155,1162,1169,1177,1184,1191],{"type":47,"tag":189,"props":1133,"children":1134},{"class":191,"line":26},[1135],{"type":47,"tag":189,"props":1136,"children":1137},{},[1138],{"type":53,"value":1139},"from portkey_ai import AsyncPortkey\n",{"type":47,"tag":189,"props":1141,"children":1142},{"class":191,"line":212},[1143],{"type":47,"tag":189,"props":1144,"children":1145},{"emptyLinePlaceholder":709},[1146],{"type":53,"value":712},{"type":47,"tag":189,"props":1148,"children":1149},{"class":191,"line":291},[1150],{"type":47,"tag":189,"props":1151,"children":1152},{},[1153],{"type":53,"value":1154},"client = AsyncPortkey(\n",{"type":47,"tag":189,"props":1156,"children":1157},{"class":191,"line":715},[1158],{"type":47,"tag":189,"props":1159,"children":1160},{},[1161],{"type":53,"value":827},{"type":47,"tag":189,"props":1163,"children":1164},{"class":191,"line":724},[1165],{"type":47,"tag":189,"props":1166,"children":1167},{},[1168],{"type":53,"value":765},{"type":47,"tag":189,"props":1170,"children":1171},{"class":191,"line":732},[1172],{"type":47,"tag":189,"props":1173,"children":1174},{},[1175],{"type":53,"value":1176},"response = await client.chat.completions.create(\n",{"type":47,"tag":189,"props":1178,"children":1179},{"class":191,"line":741},[1180],{"type":47,"tag":189,"props":1181,"children":1182},{},[1183],{"type":53,"value":857},{"type":47,"tag":189,"props":1185,"children":1186},{"class":191,"line":750},[1187],{"type":47,"tag":189,"props":1188,"children":1189},{},[1190],{"type":53,"value":756},{"type":47,"tag":189,"props":1192,"children":1193},{"class":191,"line":759},[1194],{"type":47,"tag":189,"props":1195,"children":1196},{},[1197],{"type":53,"value":765},{"type":47,"tag":666,"props":1199,"children":1201},{"id":1200},"_4-fallbacks",[1202],{"type":53,"value":1203},"4. Fallbacks",{"type":47,"tag":56,"props":1205,"children":1206},{},[1207],{"type":47,"tag":65,"props":1208,"children":1209},{},[1210],{"type":53,"value":679},{"type":47,"tag":178,"props":1212,"children":1214},{"className":682,"code":1213,"language":22,"meta":183,"style":183},"response = completion(\n model=\"openai\u002Fgpt-4o\",\n messages=messages,\n fallbacks=[\"anthropic\u002Fclaude-3-5-sonnet-20241022\"]\n)\n",[1215],{"type":47,"tag":151,"props":1216,"children":1217},{"__ignoreMap":183},[1218,1225,1232,1240,1248],{"type":47,"tag":189,"props":1219,"children":1220},{"class":191,"line":26},[1221],{"type":47,"tag":189,"props":1222,"children":1223},{},[1224],{"type":53,"value":738},{"type":47,"tag":189,"props":1226,"children":1227},{"class":191,"line":212},[1228],{"type":47,"tag":189,"props":1229,"children":1230},{},[1231],{"type":53,"value":747},{"type":47,"tag":189,"props":1233,"children":1234},{"class":191,"line":291},[1235],{"type":47,"tag":189,"props":1236,"children":1237},{},[1238],{"type":53,"value":1239}," messages=messages,\n",{"type":47,"tag":189,"props":1241,"children":1242},{"class":191,"line":715},[1243],{"type":47,"tag":189,"props":1244,"children":1245},{},[1246],{"type":53,"value":1247}," fallbacks=[\"anthropic\u002Fclaude-3-5-sonnet-20241022\"]\n",{"type":47,"tag":189,"props":1249,"children":1250},{"class":191,"line":724},[1251],{"type":47,"tag":189,"props":1252,"children":1253},{},[1254],{"type":53,"value":765},{"type":47,"tag":56,"props":1256,"children":1257},{},[1258],{"type":47,"tag":65,"props":1259,"children":1260},{},[1261],{"type":53,"value":782},{"type":47,"tag":178,"props":1263,"children":1265},{"className":682,"code":1264,"language":22,"meta":183,"style":183},"client = Portkey(\n api_key=os.environ[\"PORTKEY_API_KEY\"],\n config={\n \"strategy\": {\"mode\": \"fallback\"},\n \"targets\": [\n {\n \"override_params\": {\"model\": \"@openai-prod\u002Fgpt-4o\"}\n },\n {\n \"override_params\": {\"model\": \"@anthropic-main\u002Fclaude-3-5-sonnet-20241022\"}\n }\n ]\n }\n)\nresponse = client.chat.completions.create(messages=messages)\n",[1266],{"type":47,"tag":151,"props":1267,"children":1268},{"__ignoreMap":183},[1269,1276,1284,1292,1300,1308,1316,1324,1332,1339,1347,1355,1363,1372,1380],{"type":47,"tag":189,"props":1270,"children":1271},{"class":191,"line":26},[1272],{"type":47,"tag":189,"props":1273,"children":1274},{},[1275],{"type":53,"value":819},{"type":47,"tag":189,"props":1277,"children":1278},{"class":191,"line":212},[1279],{"type":47,"tag":189,"props":1280,"children":1281},{},[1282],{"type":53,"value":1283}," api_key=os.environ[\"PORTKEY_API_KEY\"],\n",{"type":47,"tag":189,"props":1285,"children":1286},{"class":191,"line":291},[1287],{"type":47,"tag":189,"props":1288,"children":1289},{},[1290],{"type":53,"value":1291}," config={\n",{"type":47,"tag":189,"props":1293,"children":1294},{"class":191,"line":715},[1295],{"type":47,"tag":189,"props":1296,"children":1297},{},[1298],{"type":53,"value":1299}," \"strategy\": {\"mode\": \"fallback\"},\n",{"type":47,"tag":189,"props":1301,"children":1302},{"class":191,"line":724},[1303],{"type":47,"tag":189,"props":1304,"children":1305},{},[1306],{"type":53,"value":1307}," \"targets\": [\n",{"type":47,"tag":189,"props":1309,"children":1310},{"class":191,"line":732},[1311],{"type":47,"tag":189,"props":1312,"children":1313},{},[1314],{"type":53,"value":1315}," {\n",{"type":47,"tag":189,"props":1317,"children":1318},{"class":191,"line":741},[1319],{"type":47,"tag":189,"props":1320,"children":1321},{},[1322],{"type":53,"value":1323}," \"override_params\": {\"model\": \"@openai-prod\u002Fgpt-4o\"}\n",{"type":47,"tag":189,"props":1325,"children":1326},{"class":191,"line":750},[1327],{"type":47,"tag":189,"props":1328,"children":1329},{},[1330],{"type":53,"value":1331}," },\n",{"type":47,"tag":189,"props":1333,"children":1334},{"class":191,"line":759},[1335],{"type":47,"tag":189,"props":1336,"children":1337},{},[1338],{"type":53,"value":1315},{"type":47,"tag":189,"props":1340,"children":1341},{"class":191,"line":768},[1342],{"type":47,"tag":189,"props":1343,"children":1344},{},[1345],{"type":53,"value":1346}," \"override_params\": {\"model\": \"@anthropic-main\u002Fclaude-3-5-sonnet-20241022\"}\n",{"type":47,"tag":189,"props":1348,"children":1349},{"class":191,"line":867},[1350],{"type":47,"tag":189,"props":1351,"children":1352},{},[1353],{"type":53,"value":1354}," }\n",{"type":47,"tag":189,"props":1356,"children":1357},{"class":191,"line":875},[1358],{"type":47,"tag":189,"props":1359,"children":1360},{},[1361],{"type":53,"value":1362}," ]\n",{"type":47,"tag":189,"props":1364,"children":1366},{"class":191,"line":1365},13,[1367],{"type":47,"tag":189,"props":1368,"children":1369},{},[1370],{"type":53,"value":1371}," }\n",{"type":47,"tag":189,"props":1373,"children":1375},{"class":191,"line":1374},14,[1376],{"type":47,"tag":189,"props":1377,"children":1378},{},[1379],{"type":53,"value":765},{"type":47,"tag":189,"props":1381,"children":1383},{"class":191,"line":1382},15,[1384],{"type":47,"tag":189,"props":1385,"children":1386},{},[1387],{"type":53,"value":1388},"response = client.chat.completions.create(messages=messages)\n",{"type":47,"tag":666,"props":1390,"children":1392},{"id":1391},"_5-load-balancing-router",[1393],{"type":53,"value":1394},"5. Load Balancing (Router)",{"type":47,"tag":56,"props":1396,"children":1397},{},[1398],{"type":47,"tag":65,"props":1399,"children":1400},{},[1401],{"type":53,"value":679},{"type":47,"tag":178,"props":1403,"children":1405},{"className":682,"code":1404,"language":22,"meta":183,"style":183},"from litellm import Router\n\nmodel_list = [\n {\n \"model_name\": \"gpt-3.5-turbo\",\n \"litellm_params\": {\n \"model\": \"azure\u002Fchatgpt-v-2\",\n \"api_key\": os.getenv(\"AZURE_API_KEY\"),\n \"api_base\": os.getenv(\"AZURE_API_BASE\"),\n \"rpm\": 900\n }\n },\n {\n \"model_name\": \"gpt-3.5-turbo\",\n \"litellm_params\": {\n \"model\": \"openai\u002Fgpt-3.5-turbo\",\n \"api_key\": os.getenv(\"OPENAI_API_KEY\"),\n \"rpm\": 100\n }\n }\n]\n\nrouter = Router(model_list=model_list, routing_strategy=\"simple-shuffle\")\nresponse = await router.acompletion(\n model=\"gpt-3.5-turbo\",\n messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\n",[1406],{"type":47,"tag":151,"props":1407,"children":1408},{"__ignoreMap":183},[1409,1417,1424,1432,1440,1448,1456,1464,1472,1480,1488,1496,1504,1511,1518,1525,1534,1543,1552,1560,1568,1577,1585,1594,1603,1612,1620],{"type":47,"tag":189,"props":1410,"children":1411},{"class":191,"line":26},[1412],{"type":47,"tag":189,"props":1413,"children":1414},{},[1415],{"type":53,"value":1416},"from litellm import Router\n",{"type":47,"tag":189,"props":1418,"children":1419},{"class":191,"line":212},[1420],{"type":47,"tag":189,"props":1421,"children":1422},{"emptyLinePlaceholder":709},[1423],{"type":53,"value":712},{"type":47,"tag":189,"props":1425,"children":1426},{"class":191,"line":291},[1427],{"type":47,"tag":189,"props":1428,"children":1429},{},[1430],{"type":53,"value":1431},"model_list = [\n",{"type":47,"tag":189,"props":1433,"children":1434},{"class":191,"line":715},[1435],{"type":47,"tag":189,"props":1436,"children":1437},{},[1438],{"type":53,"value":1439}," {\n",{"type":47,"tag":189,"props":1441,"children":1442},{"class":191,"line":724},[1443],{"type":47,"tag":189,"props":1444,"children":1445},{},[1446],{"type":53,"value":1447}," \"model_name\": \"gpt-3.5-turbo\",\n",{"type":47,"tag":189,"props":1449,"children":1450},{"class":191,"line":732},[1451],{"type":47,"tag":189,"props":1452,"children":1453},{},[1454],{"type":53,"value":1455}," \"litellm_params\": {\n",{"type":47,"tag":189,"props":1457,"children":1458},{"class":191,"line":741},[1459],{"type":47,"tag":189,"props":1460,"children":1461},{},[1462],{"type":53,"value":1463}," \"model\": \"azure\u002Fchatgpt-v-2\",\n",{"type":47,"tag":189,"props":1465,"children":1466},{"class":191,"line":750},[1467],{"type":47,"tag":189,"props":1468,"children":1469},{},[1470],{"type":53,"value":1471}," \"api_key\": os.getenv(\"AZURE_API_KEY\"),\n",{"type":47,"tag":189,"props":1473,"children":1474},{"class":191,"line":759},[1475],{"type":47,"tag":189,"props":1476,"children":1477},{},[1478],{"type":53,"value":1479}," \"api_base\": os.getenv(\"AZURE_API_BASE\"),\n",{"type":47,"tag":189,"props":1481,"children":1482},{"class":191,"line":768},[1483],{"type":47,"tag":189,"props":1484,"children":1485},{},[1486],{"type":53,"value":1487}," \"rpm\": 900\n",{"type":47,"tag":189,"props":1489,"children":1490},{"class":191,"line":867},[1491],{"type":47,"tag":189,"props":1492,"children":1493},{},[1494],{"type":53,"value":1495}," }\n",{"type":47,"tag":189,"props":1497,"children":1498},{"class":191,"line":875},[1499],{"type":47,"tag":189,"props":1500,"children":1501},{},[1502],{"type":53,"value":1503}," },\n",{"type":47,"tag":189,"props":1505,"children":1506},{"class":191,"line":1365},[1507],{"type":47,"tag":189,"props":1508,"children":1509},{},[1510],{"type":53,"value":1439},{"type":47,"tag":189,"props":1512,"children":1513},{"class":191,"line":1374},[1514],{"type":47,"tag":189,"props":1515,"children":1516},{},[1517],{"type":53,"value":1447},{"type":47,"tag":189,"props":1519,"children":1520},{"class":191,"line":1382},[1521],{"type":47,"tag":189,"props":1522,"children":1523},{},[1524],{"type":53,"value":1455},{"type":47,"tag":189,"props":1526,"children":1528},{"class":191,"line":1527},16,[1529],{"type":47,"tag":189,"props":1530,"children":1531},{},[1532],{"type":53,"value":1533}," \"model\": \"openai\u002Fgpt-3.5-turbo\",\n",{"type":47,"tag":189,"props":1535,"children":1537},{"class":191,"line":1536},17,[1538],{"type":47,"tag":189,"props":1539,"children":1540},{},[1541],{"type":53,"value":1542}," \"api_key\": os.getenv(\"OPENAI_API_KEY\"),\n",{"type":47,"tag":189,"props":1544,"children":1546},{"class":191,"line":1545},18,[1547],{"type":47,"tag":189,"props":1548,"children":1549},{},[1550],{"type":53,"value":1551}," \"rpm\": 100\n",{"type":47,"tag":189,"props":1553,"children":1555},{"class":191,"line":1554},19,[1556],{"type":47,"tag":189,"props":1557,"children":1558},{},[1559],{"type":53,"value":1495},{"type":47,"tag":189,"props":1561,"children":1563},{"class":191,"line":1562},20,[1564],{"type":47,"tag":189,"props":1565,"children":1566},{},[1567],{"type":53,"value":1371},{"type":47,"tag":189,"props":1569,"children":1571},{"class":191,"line":1570},21,[1572],{"type":47,"tag":189,"props":1573,"children":1574},{},[1575],{"type":53,"value":1576},"]\n",{"type":47,"tag":189,"props":1578,"children":1580},{"class":191,"line":1579},22,[1581],{"type":47,"tag":189,"props":1582,"children":1583},{"emptyLinePlaceholder":709},[1584],{"type":53,"value":712},{"type":47,"tag":189,"props":1586,"children":1588},{"class":191,"line":1587},23,[1589],{"type":47,"tag":189,"props":1590,"children":1591},{},[1592],{"type":53,"value":1593},"router = Router(model_list=model_list, routing_strategy=\"simple-shuffle\")\n",{"type":47,"tag":189,"props":1595,"children":1597},{"class":191,"line":1596},24,[1598],{"type":47,"tag":189,"props":1599,"children":1600},{},[1601],{"type":53,"value":1602},"response = await router.acompletion(\n",{"type":47,"tag":189,"props":1604,"children":1606},{"class":191,"line":1605},25,[1607],{"type":47,"tag":189,"props":1608,"children":1609},{},[1610],{"type":53,"value":1611}," model=\"gpt-3.5-turbo\",\n",{"type":47,"tag":189,"props":1613,"children":1615},{"class":191,"line":1614},26,[1616],{"type":47,"tag":189,"props":1617,"children":1618},{},[1619],{"type":53,"value":756},{"type":47,"tag":189,"props":1621,"children":1623},{"class":191,"line":1622},27,[1624],{"type":47,"tag":189,"props":1625,"children":1626},{},[1627],{"type":53,"value":765},{"type":47,"tag":56,"props":1629,"children":1630},{},[1631],{"type":47,"tag":65,"props":1632,"children":1633},{},[1634],{"type":53,"value":782},{"type":47,"tag":178,"props":1636,"children":1638},{"className":682,"code":1637,"language":22,"meta":183,"style":183},"client = Portkey(\n api_key=os.environ[\"PORTKEY_API_KEY\"],\n config={\n \"strategy\": {\"mode\": \"loadbalance\"},\n \"targets\": [\n {\n \"override_params\": {\"model\": \"@azure-prod\u002Fgpt-3.5-turbo\"},\n \"weight\": 0.9\n },\n {\n \"override_params\": {\"model\": \"@openai-prod\u002Fgpt-3.5-turbo\"},\n \"weight\": 0.1\n }\n ]\n }\n)\nresponse = client.chat.completions.create(\n messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\n",[1639],{"type":47,"tag":151,"props":1640,"children":1641},{"__ignoreMap":183},[1642,1649,1656,1663,1671,1678,1685,1693,1701,1708,1715,1723,1731,1738,1745,1752,1759,1766,1773],{"type":47,"tag":189,"props":1643,"children":1644},{"class":191,"line":26},[1645],{"type":47,"tag":189,"props":1646,"children":1647},{},[1648],{"type":53,"value":819},{"type":47,"tag":189,"props":1650,"children":1651},{"class":191,"line":212},[1652],{"type":47,"tag":189,"props":1653,"children":1654},{},[1655],{"type":53,"value":1283},{"type":47,"tag":189,"props":1657,"children":1658},{"class":191,"line":291},[1659],{"type":47,"tag":189,"props":1660,"children":1661},{},[1662],{"type":53,"value":1291},{"type":47,"tag":189,"props":1664,"children":1665},{"class":191,"line":715},[1666],{"type":47,"tag":189,"props":1667,"children":1668},{},[1669],{"type":53,"value":1670}," \"strategy\": {\"mode\": \"loadbalance\"},\n",{"type":47,"tag":189,"props":1672,"children":1673},{"class":191,"line":724},[1674],{"type":47,"tag":189,"props":1675,"children":1676},{},[1677],{"type":53,"value":1307},{"type":47,"tag":189,"props":1679,"children":1680},{"class":191,"line":732},[1681],{"type":47,"tag":189,"props":1682,"children":1683},{},[1684],{"type":53,"value":1315},{"type":47,"tag":189,"props":1686,"children":1687},{"class":191,"line":741},[1688],{"type":47,"tag":189,"props":1689,"children":1690},{},[1691],{"type":53,"value":1692}," \"override_params\": {\"model\": \"@azure-prod\u002Fgpt-3.5-turbo\"},\n",{"type":47,"tag":189,"props":1694,"children":1695},{"class":191,"line":750},[1696],{"type":47,"tag":189,"props":1697,"children":1698},{},[1699],{"type":53,"value":1700}," \"weight\": 0.9\n",{"type":47,"tag":189,"props":1702,"children":1703},{"class":191,"line":759},[1704],{"type":47,"tag":189,"props":1705,"children":1706},{},[1707],{"type":53,"value":1331},{"type":47,"tag":189,"props":1709,"children":1710},{"class":191,"line":768},[1711],{"type":47,"tag":189,"props":1712,"children":1713},{},[1714],{"type":53,"value":1315},{"type":47,"tag":189,"props":1716,"children":1717},{"class":191,"line":867},[1718],{"type":47,"tag":189,"props":1719,"children":1720},{},[1721],{"type":53,"value":1722}," \"override_params\": {\"model\": \"@openai-prod\u002Fgpt-3.5-turbo\"},\n",{"type":47,"tag":189,"props":1724,"children":1725},{"class":191,"line":875},[1726],{"type":47,"tag":189,"props":1727,"children":1728},{},[1729],{"type":53,"value":1730}," \"weight\": 0.1\n",{"type":47,"tag":189,"props":1732,"children":1733},{"class":191,"line":1365},[1734],{"type":47,"tag":189,"props":1735,"children":1736},{},[1737],{"type":53,"value":1354},{"type":47,"tag":189,"props":1739,"children":1740},{"class":191,"line":1374},[1741],{"type":47,"tag":189,"props":1742,"children":1743},{},[1744],{"type":53,"value":1362},{"type":47,"tag":189,"props":1746,"children":1747},{"class":191,"line":1382},[1748],{"type":47,"tag":189,"props":1749,"children":1750},{},[1751],{"type":53,"value":1371},{"type":47,"tag":189,"props":1753,"children":1754},{"class":191,"line":1527},[1755],{"type":47,"tag":189,"props":1756,"children":1757},{},[1758],{"type":53,"value":765},{"type":47,"tag":189,"props":1760,"children":1761},{"class":191,"line":1536},[1762],{"type":47,"tag":189,"props":1763,"children":1764},{},[1765],{"type":53,"value":849},{"type":47,"tag":189,"props":1767,"children":1768},{"class":191,"line":1545},[1769],{"type":47,"tag":189,"props":1770,"children":1771},{},[1772],{"type":53,"value":756},{"type":47,"tag":189,"props":1774,"children":1775},{"class":191,"line":1554},[1776],{"type":47,"tag":189,"props":1777,"children":1778},{},[1779],{"type":53,"value":765},{"type":47,"tag":666,"props":1781,"children":1783},{"id":1782},"_6-retries",[1784],{"type":53,"value":1785},"6. Retries",{"type":47,"tag":56,"props":1787,"children":1788},{},[1789],{"type":47,"tag":65,"props":1790,"children":1791},{},[1792],{"type":53,"value":679},{"type":47,"tag":178,"props":1794,"children":1796},{"className":682,"code":1795,"language":22,"meta":183,"style":183},"response = completion(\n model=\"openai\u002Fgpt-4o\",\n messages=messages,\n num_retries=3\n)\n",[1797],{"type":47,"tag":151,"props":1798,"children":1799},{"__ignoreMap":183},[1800,1807,1814,1821,1829],{"type":47,"tag":189,"props":1801,"children":1802},{"class":191,"line":26},[1803],{"type":47,"tag":189,"props":1804,"children":1805},{},[1806],{"type":53,"value":738},{"type":47,"tag":189,"props":1808,"children":1809},{"class":191,"line":212},[1810],{"type":47,"tag":189,"props":1811,"children":1812},{},[1813],{"type":53,"value":747},{"type":47,"tag":189,"props":1815,"children":1816},{"class":191,"line":291},[1817],{"type":47,"tag":189,"props":1818,"children":1819},{},[1820],{"type":53,"value":1239},{"type":47,"tag":189,"props":1822,"children":1823},{"class":191,"line":715},[1824],{"type":47,"tag":189,"props":1825,"children":1826},{},[1827],{"type":53,"value":1828}," num_retries=3\n",{"type":47,"tag":189,"props":1830,"children":1831},{"class":191,"line":724},[1832],{"type":47,"tag":189,"props":1833,"children":1834},{},[1835],{"type":53,"value":765},{"type":47,"tag":56,"props":1837,"children":1838},{},[1839],{"type":47,"tag":65,"props":1840,"children":1841},{},[1842],{"type":53,"value":782},{"type":47,"tag":178,"props":1844,"children":1846},{"className":682,"code":1845,"language":22,"meta":183,"style":183},"client = Portkey(\n api_key=os.environ[\"PORTKEY_API_KEY\"],\n config={\n \"retry\": {\"attempts\": 3, \"on_status_codes\": [429, 500, 502, 503, 504]}\n }\n)\nresponse = client.chat.completions.create(\n model=\"@openai-prod\u002Fgpt-4o\",\n messages=messages\n)\n",[1847],{"type":47,"tag":151,"props":1848,"children":1849},{"__ignoreMap":183},[1850,1857,1864,1871,1879,1886,1893,1900,1907,1915],{"type":47,"tag":189,"props":1851,"children":1852},{"class":191,"line":26},[1853],{"type":47,"tag":189,"props":1854,"children":1855},{},[1856],{"type":53,"value":819},{"type":47,"tag":189,"props":1858,"children":1859},{"class":191,"line":212},[1860],{"type":47,"tag":189,"props":1861,"children":1862},{},[1863],{"type":53,"value":1283},{"type":47,"tag":189,"props":1865,"children":1866},{"class":191,"line":291},[1867],{"type":47,"tag":189,"props":1868,"children":1869},{},[1870],{"type":53,"value":1291},{"type":47,"tag":189,"props":1872,"children":1873},{"class":191,"line":715},[1874],{"type":47,"tag":189,"props":1875,"children":1876},{},[1877],{"type":53,"value":1878}," \"retry\": {\"attempts\": 3, \"on_status_codes\": [429, 500, 502, 503, 504]}\n",{"type":47,"tag":189,"props":1880,"children":1881},{"class":191,"line":724},[1882],{"type":47,"tag":189,"props":1883,"children":1884},{},[1885],{"type":53,"value":1371},{"type":47,"tag":189,"props":1887,"children":1888},{"class":191,"line":732},[1889],{"type":47,"tag":189,"props":1890,"children":1891},{},[1892],{"type":53,"value":765},{"type":47,"tag":189,"props":1894,"children":1895},{"class":191,"line":741},[1896],{"type":47,"tag":189,"props":1897,"children":1898},{},[1899],{"type":53,"value":849},{"type":47,"tag":189,"props":1901,"children":1902},{"class":191,"line":750},[1903],{"type":47,"tag":189,"props":1904,"children":1905},{},[1906],{"type":53,"value":857},{"type":47,"tag":189,"props":1908,"children":1909},{"class":191,"line":759},[1910],{"type":47,"tag":189,"props":1911,"children":1912},{},[1913],{"type":53,"value":1914}," messages=messages\n",{"type":47,"tag":189,"props":1916,"children":1917},{"class":191,"line":768},[1918],{"type":47,"tag":189,"props":1919,"children":1920},{},[1921],{"type":53,"value":765},{"type":47,"tag":666,"props":1923,"children":1925},{"id":1924},"_7-caching",[1926],{"type":53,"value":1927},"7. Caching",{"type":47,"tag":56,"props":1929,"children":1930},{},[1931],{"type":47,"tag":65,"props":1932,"children":1933},{},[1934],{"type":53,"value":679},{"type":47,"tag":178,"props":1936,"children":1938},{"className":682,"code":1937,"language":22,"meta":183,"style":183},"import litellm\nfrom litellm.caching.caching import Cache\n\nlitellm.cache = Cache(type=\"redis\", host=\"localhost\", port=6379)\n\nresponse = completion(model=\"openai\u002Fgpt-4o\", messages=messages, caching=True)\n",[1939],{"type":47,"tag":151,"props":1940,"children":1941},{"__ignoreMap":183},[1942,1949,1957,1964,1972,1979],{"type":47,"tag":189,"props":1943,"children":1944},{"class":191,"line":26},[1945],{"type":47,"tag":189,"props":1946,"children":1947},{},[1948],{"type":53,"value":1088},{"type":47,"tag":189,"props":1950,"children":1951},{"class":191,"line":212},[1952],{"type":47,"tag":189,"props":1953,"children":1954},{},[1955],{"type":53,"value":1956},"from litellm.caching.caching import Cache\n",{"type":47,"tag":189,"props":1958,"children":1959},{"class":191,"line":291},[1960],{"type":47,"tag":189,"props":1961,"children":1962},{"emptyLinePlaceholder":709},[1963],{"type":53,"value":712},{"type":47,"tag":189,"props":1965,"children":1966},{"class":191,"line":715},[1967],{"type":47,"tag":189,"props":1968,"children":1969},{},[1970],{"type":53,"value":1971},"litellm.cache = Cache(type=\"redis\", host=\"localhost\", port=6379)\n",{"type":47,"tag":189,"props":1973,"children":1974},{"class":191,"line":724},[1975],{"type":47,"tag":189,"props":1976,"children":1977},{"emptyLinePlaceholder":709},[1978],{"type":53,"value":712},{"type":47,"tag":189,"props":1980,"children":1981},{"class":191,"line":732},[1982],{"type":47,"tag":189,"props":1983,"children":1984},{},[1985],{"type":53,"value":1986},"response = completion(model=\"openai\u002Fgpt-4o\", messages=messages, caching=True)\n",{"type":47,"tag":56,"props":1988,"children":1989},{},[1990,1994],{"type":47,"tag":65,"props":1991,"children":1992},{},[1993],{"type":53,"value":782},{"type":53,"value":1995}," (no infrastructure to manage — caching is built-in)",{"type":47,"tag":178,"props":1997,"children":1999},{"className":682,"code":1998,"language":22,"meta":183,"style":183},"client = Portkey(\n api_key=os.environ[\"PORTKEY_API_KEY\"],\n config={\n \"cache\": {\"mode\": \"semantic\", \"max_age\": 3600}\n }\n)\nresponse = client.chat.completions.create(\n model=\"@openai-prod\u002Fgpt-4o\",\n messages=messages\n)\n",[2000],{"type":47,"tag":151,"props":2001,"children":2002},{"__ignoreMap":183},[2003,2010,2017,2024,2032,2039,2046,2053,2060,2067],{"type":47,"tag":189,"props":2004,"children":2005},{"class":191,"line":26},[2006],{"type":47,"tag":189,"props":2007,"children":2008},{},[2009],{"type":53,"value":819},{"type":47,"tag":189,"props":2011,"children":2012},{"class":191,"line":212},[2013],{"type":47,"tag":189,"props":2014,"children":2015},{},[2016],{"type":53,"value":1283},{"type":47,"tag":189,"props":2018,"children":2019},{"class":191,"line":291},[2020],{"type":47,"tag":189,"props":2021,"children":2022},{},[2023],{"type":53,"value":1291},{"type":47,"tag":189,"props":2025,"children":2026},{"class":191,"line":715},[2027],{"type":47,"tag":189,"props":2028,"children":2029},{},[2030],{"type":53,"value":2031}," \"cache\": {\"mode\": \"semantic\", \"max_age\": 3600}\n",{"type":47,"tag":189,"props":2033,"children":2034},{"class":191,"line":724},[2035],{"type":47,"tag":189,"props":2036,"children":2037},{},[2038],{"type":53,"value":1371},{"type":47,"tag":189,"props":2040,"children":2041},{"class":191,"line":732},[2042],{"type":47,"tag":189,"props":2043,"children":2044},{},[2045],{"type":53,"value":765},{"type":47,"tag":189,"props":2047,"children":2048},{"class":191,"line":741},[2049],{"type":47,"tag":189,"props":2050,"children":2051},{},[2052],{"type":53,"value":849},{"type":47,"tag":189,"props":2054,"children":2055},{"class":191,"line":750},[2056],{"type":47,"tag":189,"props":2057,"children":2058},{},[2059],{"type":53,"value":857},{"type":47,"tag":189,"props":2061,"children":2062},{"class":191,"line":759},[2063],{"type":47,"tag":189,"props":2064,"children":2065},{},[2066],{"type":53,"value":1914},{"type":47,"tag":189,"props":2068,"children":2069},{"class":191,"line":768},[2070],{"type":47,"tag":189,"props":2071,"children":2072},{},[2073],{"type":53,"value":765},{"type":47,"tag":666,"props":2075,"children":2077},{"id":2076},"_8-observability",[2078],{"type":53,"value":2079},"8. Observability",{"type":47,"tag":56,"props":2081,"children":2082},{},[2083],{"type":47,"tag":65,"props":2084,"children":2085},{},[2086],{"type":53,"value":679},{"type":47,"tag":178,"props":2088,"children":2090},{"className":682,"code":2089,"language":22,"meta":183,"style":183},"import litellm\nlitellm.success_callback = [\"langfuse\"]\nlitellm.failure_callback = [\"langfuse\"]\n",[2091],{"type":47,"tag":151,"props":2092,"children":2093},{"__ignoreMap":183},[2094,2101,2109],{"type":47,"tag":189,"props":2095,"children":2096},{"class":191,"line":26},[2097],{"type":47,"tag":189,"props":2098,"children":2099},{},[2100],{"type":53,"value":1088},{"type":47,"tag":189,"props":2102,"children":2103},{"class":191,"line":212},[2104],{"type":47,"tag":189,"props":2105,"children":2106},{},[2107],{"type":53,"value":2108},"litellm.success_callback = [\"langfuse\"]\n",{"type":47,"tag":189,"props":2110,"children":2111},{"class":191,"line":291},[2112],{"type":47,"tag":189,"props":2113,"children":2114},{},[2115],{"type":53,"value":2116},"litellm.failure_callback = [\"langfuse\"]\n",{"type":47,"tag":56,"props":2118,"children":2119},{},[2120,2124],{"type":47,"tag":65,"props":2121,"children":2122},{},[2123],{"type":53,"value":782},{"type":53,"value":2125}," (built-in — every request is logged automatically)",{"type":47,"tag":178,"props":2127,"children":2129},{"className":682,"code":2128,"language":22,"meta":183,"style":183},"client = Portkey(\n api_key=os.environ[\"PORTKEY_API_KEY\"],\n trace_id=\"session-123\",\n metadata={\n \"user_id\": \"user-456\",\n \"environment\": \"production\"\n }\n)\n",[2130],{"type":47,"tag":151,"props":2131,"children":2132},{"__ignoreMap":183},[2133,2140,2147,2155,2163,2171,2179,2186],{"type":47,"tag":189,"props":2134,"children":2135},{"class":191,"line":26},[2136],{"type":47,"tag":189,"props":2137,"children":2138},{},[2139],{"type":53,"value":819},{"type":47,"tag":189,"props":2141,"children":2142},{"class":191,"line":212},[2143],{"type":47,"tag":189,"props":2144,"children":2145},{},[2146],{"type":53,"value":1283},{"type":47,"tag":189,"props":2148,"children":2149},{"class":191,"line":291},[2150],{"type":47,"tag":189,"props":2151,"children":2152},{},[2153],{"type":53,"value":2154}," trace_id=\"session-123\",\n",{"type":47,"tag":189,"props":2156,"children":2157},{"class":191,"line":715},[2158],{"type":47,"tag":189,"props":2159,"children":2160},{},[2161],{"type":53,"value":2162}," metadata={\n",{"type":47,"tag":189,"props":2164,"children":2165},{"class":191,"line":724},[2166],{"type":47,"tag":189,"props":2167,"children":2168},{},[2169],{"type":53,"value":2170}," \"user_id\": \"user-456\",\n",{"type":47,"tag":189,"props":2172,"children":2173},{"class":191,"line":732},[2174],{"type":47,"tag":189,"props":2175,"children":2176},{},[2177],{"type":53,"value":2178}," \"environment\": \"production\"\n",{"type":47,"tag":189,"props":2180,"children":2181},{"class":191,"line":741},[2182],{"type":47,"tag":189,"props":2183,"children":2184},{},[2185],{"type":53,"value":1371},{"type":47,"tag":189,"props":2187,"children":2188},{"class":191,"line":750},[2189],{"type":47,"tag":189,"props":2190,"children":2191},{},[2192],{"type":53,"value":765},{"type":47,"tag":56,"props":2194,"children":2195},{},[2196,2198,2204],{"type":53,"value":2197},"View logs, traces, costs, and latency at ",{"type":47,"tag":79,"props":2199,"children":2201},{"href":143,"rel":2200},[127],[2202],{"type":53,"value":2203},"app.portkey.ai",{"type":53,"value":2205},". No third-party callback integration needed.",{"type":47,"tag":99,"props":2207,"children":2208},{},[],{"type":47,"tag":103,"props":2210,"children":2212},{"id":2211},"model-naming",[2213],{"type":53,"value":2214},"Model Naming",{"type":47,"tag":56,"props":2216,"children":2217},{},[2218,2220,2226,2228,2234],{"type":53,"value":2219},"LiteLLM uses ",{"type":47,"tag":151,"props":2221,"children":2223},{"className":2222},[],[2224],{"type":53,"value":2225},"provider\u002Fmodel",{"type":53,"value":2227}," format. Portkey uses ",{"type":47,"tag":151,"props":2229,"children":2231},{"className":2230},[],[2232],{"type":53,"value":2233},"@provider-slug\u002Fmodel",{"type":53,"value":2235}," where the provider slug comes from your Model Catalog:",{"type":47,"tag":343,"props":2237,"children":2238},{},[2239,2255],{"type":47,"tag":347,"props":2240,"children":2241},{},[2242],{"type":47,"tag":351,"props":2243,"children":2244},{},[2245,2250],{"type":47,"tag":355,"props":2246,"children":2247},{},[2248],{"type":53,"value":2249},"LiteLLM Model String",{"type":47,"tag":355,"props":2251,"children":2252},{},[2253],{"type":53,"value":2254},"Portkey Model String",{"type":47,"tag":365,"props":2256,"children":2257},{},[2258,2277,2298,2319,2340],{"type":47,"tag":351,"props":2259,"children":2260},{},[2261,2269],{"type":47,"tag":372,"props":2262,"children":2263},{},[2264],{"type":47,"tag":151,"props":2265,"children":2267},{"className":2266},[],[2268],{"type":53,"value":642},{"type":47,"tag":372,"props":2270,"children":2271},{},[2272],{"type":47,"tag":151,"props":2273,"children":2275},{"className":2274},[],[2276],{"type":53,"value":653},{"type":47,"tag":351,"props":2278,"children":2279},{},[2280,2289],{"type":47,"tag":372,"props":2281,"children":2282},{},[2283],{"type":47,"tag":151,"props":2284,"children":2286},{"className":2285},[],[2287],{"type":53,"value":2288},"\"anthropic\u002Fclaude-3-5-sonnet-20241022\"",{"type":47,"tag":372,"props":2290,"children":2291},{},[2292],{"type":47,"tag":151,"props":2293,"children":2295},{"className":2294},[],[2296],{"type":53,"value":2297},"\"@anthropic-main\u002Fclaude-3-5-sonnet-20241022\"",{"type":47,"tag":351,"props":2299,"children":2300},{},[2301,2310],{"type":47,"tag":372,"props":2302,"children":2303},{},[2304],{"type":47,"tag":151,"props":2305,"children":2307},{"className":2306},[],[2308],{"type":53,"value":2309},"\"azure\u002Fgpt-4\"",{"type":47,"tag":372,"props":2311,"children":2312},{},[2313],{"type":47,"tag":151,"props":2314,"children":2316},{"className":2315},[],[2317],{"type":53,"value":2318},"\"@azure-us-east\u002Fgpt-4\"",{"type":47,"tag":351,"props":2320,"children":2321},{},[2322,2331],{"type":47,"tag":372,"props":2323,"children":2324},{},[2325],{"type":47,"tag":151,"props":2326,"children":2328},{"className":2327},[],[2329],{"type":53,"value":2330},"\"bedrock\u002Fanthropic.claude-3-sonnet...\"",{"type":47,"tag":372,"props":2332,"children":2333},{},[2334],{"type":47,"tag":151,"props":2335,"children":2337},{"className":2336},[],[2338],{"type":53,"value":2339},"\"@bedrock-main\u002Fanthropic.claude-3-sonnet...\"",{"type":47,"tag":351,"props":2341,"children":2342},{},[2343,2352],{"type":47,"tag":372,"props":2344,"children":2345},{},[2346],{"type":47,"tag":151,"props":2347,"children":2349},{"className":2348},[],[2350],{"type":53,"value":2351},"\"vertex_ai\u002Fgemini-1.5-pro\"",{"type":47,"tag":372,"props":2353,"children":2354},{},[2355],{"type":47,"tag":151,"props":2356,"children":2358},{"className":2357},[],[2359],{"type":53,"value":2360},"\"@vertex-prod\u002Fgemini-1.5-pro\"",{"type":47,"tag":56,"props":2362,"children":2363},{},[2364,2366,2372],{"type":53,"value":2365},"The ",{"type":47,"tag":151,"props":2367,"children":2369},{"className":2368},[],[2370],{"type":53,"value":2371},"@slug",{"type":53,"value":2373}," prefix maps to an AI Provider in your Model Catalog. You control which models are available under each provider.",{"type":47,"tag":99,"props":2375,"children":2376},{},[],{"type":47,"tag":103,"props":2378,"children":2380},{"id":2379},"error-handling",[2381],{"type":53,"value":2382},"Error Handling",{"type":47,"tag":56,"props":2384,"children":2385},{},[2386],{"type":47,"tag":65,"props":2387,"children":2388},{},[2389],{"type":53,"value":679},{"type":47,"tag":178,"props":2391,"children":2393},{"className":682,"code":2392,"language":22,"meta":183,"style":183},"import litellm\ntry:\n response = completion(model=\"openai\u002Fgpt-4o\", messages=messages)\nexcept litellm.AuthenticationError:\n print(\"Bad API key\")\nexcept litellm.RateLimitError:\n print(\"Rate limited\")\n",[2394],{"type":47,"tag":151,"props":2395,"children":2396},{"__ignoreMap":183},[2397,2404,2412,2420,2428,2436,2444],{"type":47,"tag":189,"props":2398,"children":2399},{"class":191,"line":26},[2400],{"type":47,"tag":189,"props":2401,"children":2402},{},[2403],{"type":53,"value":1088},{"type":47,"tag":189,"props":2405,"children":2406},{"class":191,"line":212},[2407],{"type":47,"tag":189,"props":2408,"children":2409},{},[2410],{"type":53,"value":2411},"try:\n",{"type":47,"tag":189,"props":2413,"children":2414},{"class":191,"line":291},[2415],{"type":47,"tag":189,"props":2416,"children":2417},{},[2418],{"type":53,"value":2419}," response = completion(model=\"openai\u002Fgpt-4o\", messages=messages)\n",{"type":47,"tag":189,"props":2421,"children":2422},{"class":191,"line":715},[2423],{"type":47,"tag":189,"props":2424,"children":2425},{},[2426],{"type":53,"value":2427},"except litellm.AuthenticationError:\n",{"type":47,"tag":189,"props":2429,"children":2430},{"class":191,"line":724},[2431],{"type":47,"tag":189,"props":2432,"children":2433},{},[2434],{"type":53,"value":2435}," print(\"Bad API key\")\n",{"type":47,"tag":189,"props":2437,"children":2438},{"class":191,"line":732},[2439],{"type":47,"tag":189,"props":2440,"children":2441},{},[2442],{"type":53,"value":2443},"except litellm.RateLimitError:\n",{"type":47,"tag":189,"props":2445,"children":2446},{"class":191,"line":741},[2447],{"type":47,"tag":189,"props":2448,"children":2449},{},[2450],{"type":53,"value":2451}," print(\"Rate limited\")\n",{"type":47,"tag":56,"props":2453,"children":2454},{},[2455],{"type":47,"tag":65,"props":2456,"children":2457},{},[2458],{"type":53,"value":782},{"type":47,"tag":178,"props":2460,"children":2462},{"className":682,"code":2461,"language":22,"meta":183,"style":183},"try:\n response = client.chat.completions.create(\n model=\"@openai-prod\u002Fgpt-4o\", messages=messages\n )\nexcept Exception as e:\n status = getattr(e, 'status_code', None)\n if status == 401:\n print(\"Bad API key\")\n elif status == 429:\n print(\"Rate limited\")\n",[2463],{"type":47,"tag":151,"props":2464,"children":2465},{"__ignoreMap":183},[2466,2473,2481,2489,2497,2505,2513,2521,2529,2537],{"type":47,"tag":189,"props":2467,"children":2468},{"class":191,"line":26},[2469],{"type":47,"tag":189,"props":2470,"children":2471},{},[2472],{"type":53,"value":2411},{"type":47,"tag":189,"props":2474,"children":2475},{"class":191,"line":212},[2476],{"type":47,"tag":189,"props":2477,"children":2478},{},[2479],{"type":53,"value":2480}," response = client.chat.completions.create(\n",{"type":47,"tag":189,"props":2482,"children":2483},{"class":191,"line":291},[2484],{"type":47,"tag":189,"props":2485,"children":2486},{},[2487],{"type":53,"value":2488}," model=\"@openai-prod\u002Fgpt-4o\", messages=messages\n",{"type":47,"tag":189,"props":2490,"children":2491},{"class":191,"line":715},[2492],{"type":47,"tag":189,"props":2493,"children":2494},{},[2495],{"type":53,"value":2496}," )\n",{"type":47,"tag":189,"props":2498,"children":2499},{"class":191,"line":724},[2500],{"type":47,"tag":189,"props":2501,"children":2502},{},[2503],{"type":53,"value":2504},"except Exception as e:\n",{"type":47,"tag":189,"props":2506,"children":2507},{"class":191,"line":732},[2508],{"type":47,"tag":189,"props":2509,"children":2510},{},[2511],{"type":53,"value":2512}," status = getattr(e, 'status_code', None)\n",{"type":47,"tag":189,"props":2514,"children":2515},{"class":191,"line":741},[2516],{"type":47,"tag":189,"props":2517,"children":2518},{},[2519],{"type":53,"value":2520}," if status == 401:\n",{"type":47,"tag":189,"props":2522,"children":2523},{"class":191,"line":750},[2524],{"type":47,"tag":189,"props":2525,"children":2526},{},[2527],{"type":53,"value":2528}," print(\"Bad API key\")\n",{"type":47,"tag":189,"props":2530,"children":2531},{"class":191,"line":759},[2532],{"type":47,"tag":189,"props":2533,"children":2534},{},[2535],{"type":53,"value":2536}," elif status == 429:\n",{"type":47,"tag":189,"props":2538,"children":2539},{"class":191,"line":768},[2540],{"type":47,"tag":189,"props":2541,"children":2542},{},[2543],{"type":53,"value":2544}," print(\"Rate limited\")\n",{"type":47,"tag":99,"props":2546,"children":2547},{},[],{"type":47,"tag":103,"props":2549,"children":2551},{"id":2550},"ai-coding-agent-migrations",[2552],{"type":53,"value":2553},"AI Coding Agent Migrations",{"type":47,"tag":666,"props":2555,"children":2557},{"id":2556},"claude-code",[2558],{"type":53,"value":2559},"Claude Code",{"type":47,"tag":56,"props":2561,"children":2562},{},[2563,2565,2571],{"type":53,"value":2564},"If Claude Code is routed through a LiteLLM Proxy (via ",{"type":47,"tag":151,"props":2566,"children":2568},{"className":2567},[],[2569],{"type":53,"value":2570},"ANTHROPIC_BASE_URL",{"type":53,"value":2572},"), the fastest migration is:",{"type":47,"tag":178,"props":2574,"children":2576},{"className":180,"code":2575,"language":182,"meta":183,"style":183},"npx portkey setup\n",[2577],{"type":47,"tag":151,"props":2578,"children":2579},{"__ignoreMap":183},[2580],{"type":47,"tag":189,"props":2581,"children":2582},{"class":191,"line":26},[2583,2587,2591],{"type":47,"tag":189,"props":2584,"children":2585},{"style":195},[2586],{"type":53,"value":322},{"type":47,"tag":189,"props":2588,"children":2589},{"style":201},[2590],{"type":53,"value":327},{"type":47,"tag":189,"props":2592,"children":2593},{"style":201},[2594],{"type":53,"value":2595}," setup\n",{"type":47,"tag":56,"props":2597,"children":2598},{},[2599,2601,2606,2607,2613,2615,2621],{"type":53,"value":2600},"This replaces the LiteLLM Proxy env vars (",{"type":47,"tag":151,"props":2602,"children":2604},{"className":2603},[],[2605],{"type":53,"value":2570},{"type":53,"value":158},{"type":47,"tag":151,"props":2608,"children":2610},{"className":2609},[],[2611],{"type":53,"value":2612},"ANTHROPIC_API_KEY",{"type":53,"value":2614},") with Portkey equivalents. See ",{"type":47,"tag":79,"props":2616,"children":2618},{"href":2617},"references\u002FLITELLM_PROXY_MIGRATION.md#claude-code-litellm-proxy--portkey",[2619],{"type":53,"value":2620},"Proxy Migration — Claude Code section",{"type":53,"value":2622}," for manual setup, Bedrock\u002FVertex model mappings, and Config-based routing.",{"type":47,"tag":666,"props":2624,"children":2626},{"id":2625},"openai-codex-cli",[2627],{"type":53,"value":2628},"OpenAI Codex CLI",{"type":47,"tag":56,"props":2630,"children":2631},{},[2632,2634,2640,2642,2648,2650,2656,2658,2663,2665,2671,2673,2679],{"type":53,"value":2633},"If Codex CLI is routed through a LiteLLM Proxy (via ",{"type":47,"tag":151,"props":2635,"children":2637},{"className":2636},[],[2638],{"type":53,"value":2639},"openai_base_url",{"type":53,"value":2641}," or ",{"type":47,"tag":151,"props":2643,"children":2645},{"className":2644},[],[2646],{"type":53,"value":2647},"OPENAI_BASE_URL",{"type":53,"value":2649},"), update ",{"type":47,"tag":151,"props":2651,"children":2653},{"className":2652},[],[2654],{"type":53,"value":2655},"~\u002F.codex\u002Fconfig.json",{"type":53,"value":2657}," to point at Portkey with a ",{"type":47,"tag":151,"props":2659,"children":2661},{"className":2660},[],[2662],{"type":53,"value":8},{"type":53,"value":2664}," provider and ",{"type":47,"tag":151,"props":2666,"children":2668},{"className":2667},[],[2669],{"type":53,"value":2670},"model: \"@provider-slug\u002Fmodel\"",{"type":53,"value":2672}," format. See ",{"type":47,"tag":79,"props":2674,"children":2676},{"href":2675},"references\u002FLITELLM_PROXY_MIGRATION.md#openai-codex-cli-litellm-proxy--portkey",[2677],{"type":53,"value":2678},"Proxy Migration — Codex section",{"type":53,"value":2680}," for step-by-step setup.",{"type":47,"tag":99,"props":2682,"children":2683},{},[],{"type":47,"tag":103,"props":2685,"children":2687},{"id":2686},"zero-change-path-keep-litellm-route-through-portkey",[2688],{"type":53,"value":2689},"Zero-Change Path: Keep LiteLLM, Route Through Portkey",{"type":47,"tag":56,"props":2691,"children":2692},{},[2693,2695,2701],{"type":53,"value":2694},"If you want Portkey's observability and reliability without rewriting any code, you can keep using ",{"type":47,"tag":151,"props":2696,"children":2698},{"className":2697},[],[2699],{"type":53,"value":2700},"litellm.completion()",{"type":53,"value":2702}," and just point it at Portkey's gateway:",{"type":47,"tag":178,"props":2704,"children":2706},{"className":682,"code":2705,"language":22,"meta":183,"style":183},"from litellm import completion\n\nresponse = completion(\n model=\"gpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Hello!\"}],\n api_base=\"https:\u002F\u002Fapi.portkey.ai\u002Fv1\",\n api_key=os.environ[\"PORTKEY_API_KEY\"],\n extra_headers={\"x-portkey-provider\": \"@openai-prod\"}\n)\n",[2707],{"type":47,"tag":151,"props":2708,"children":2709},{"__ignoreMap":183},[2710,2717,2724,2731,2739,2747,2755,2762,2770],{"type":47,"tag":189,"props":2711,"children":2712},{"class":191,"line":26},[2713],{"type":47,"tag":189,"props":2714,"children":2715},{},[2716],{"type":53,"value":695},{"type":47,"tag":189,"props":2718,"children":2719},{"class":191,"line":212},[2720],{"type":47,"tag":189,"props":2721,"children":2722},{"emptyLinePlaceholder":709},[2723],{"type":53,"value":712},{"type":47,"tag":189,"props":2725,"children":2726},{"class":191,"line":291},[2727],{"type":47,"tag":189,"props":2728,"children":2729},{},[2730],{"type":53,"value":738},{"type":47,"tag":189,"props":2732,"children":2733},{"class":191,"line":715},[2734],{"type":47,"tag":189,"props":2735,"children":2736},{},[2737],{"type":53,"value":2738}," model=\"gpt-4o\",\n",{"type":47,"tag":189,"props":2740,"children":2741},{"class":191,"line":724},[2742],{"type":47,"tag":189,"props":2743,"children":2744},{},[2745],{"type":53,"value":2746}," messages=[{\"role\": \"user\", \"content\": \"Hello!\"}],\n",{"type":47,"tag":189,"props":2748,"children":2749},{"class":191,"line":732},[2750],{"type":47,"tag":189,"props":2751,"children":2752},{},[2753],{"type":53,"value":2754}," api_base=\"https:\u002F\u002Fapi.portkey.ai\u002Fv1\",\n",{"type":47,"tag":189,"props":2756,"children":2757},{"class":191,"line":741},[2758],{"type":47,"tag":189,"props":2759,"children":2760},{},[2761],{"type":53,"value":1283},{"type":47,"tag":189,"props":2763,"children":2764},{"class":191,"line":750},[2765],{"type":47,"tag":189,"props":2766,"children":2767},{},[2768],{"type":53,"value":2769}," extra_headers={\"x-portkey-provider\": \"@openai-prod\"}\n",{"type":47,"tag":189,"props":2771,"children":2772},{"class":191,"line":759},[2773],{"type":47,"tag":189,"props":2774,"children":2775},{},[2776],{"type":53,"value":765},{"type":47,"tag":56,"props":2778,"children":2779},{},[2780],{"type":53,"value":2781},"This gives you Portkey logging, caching, and guardrails immediately — then you can migrate call-by-call to the Portkey SDK at your own pace.",{"type":47,"tag":99,"props":2783,"children":2784},{},[],{"type":47,"tag":103,"props":2786,"children":2788},{"id":2787},"openai-client-approach-minimal-changes",[2789],{"type":53,"value":2790},"OpenAI Client Approach (Minimal Changes)",{"type":47,"tag":56,"props":2792,"children":2793},{},[2794,2796,2802],{"type":53,"value":2795},"If your codebase uses the OpenAI SDK to talk to LiteLLM's proxy, you can switch to Portkey by only changing ",{"type":47,"tag":151,"props":2797,"children":2799},{"className":2798},[],[2800],{"type":53,"value":2801},"base_url",{"type":53,"value":2803}," and adding headers — no import changes required:",{"type":47,"tag":178,"props":2805,"children":2807},{"className":682,"code":2806,"language":22,"meta":183,"style":183},"import openai\nfrom portkey_ai import PORTKEY_GATEWAY_URL, createHeaders\n\nclient = openai.OpenAI(\n api_key=\"not-used\",\n base_url=PORTKEY_GATEWAY_URL, # https:\u002F\u002Fapi.portkey.ai\u002Fv1\n default_headers=createHeaders(\n api_key=\"your-portkey-api-key\"\n )\n)\n\nresponse = client.chat.completions.create(\n model=\"@openai-prod\u002Fgpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\n",[2808],{"type":47,"tag":151,"props":2809,"children":2810},{"__ignoreMap":183},[2811,2819,2827,2834,2842,2850,2858,2866,2874,2881,2888,2895,2902,2909,2916],{"type":47,"tag":189,"props":2812,"children":2813},{"class":191,"line":26},[2814],{"type":47,"tag":189,"props":2815,"children":2816},{},[2817],{"type":53,"value":2818},"import openai\n",{"type":47,"tag":189,"props":2820,"children":2821},{"class":191,"line":212},[2822],{"type":47,"tag":189,"props":2823,"children":2824},{},[2825],{"type":53,"value":2826},"from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders\n",{"type":47,"tag":189,"props":2828,"children":2829},{"class":191,"line":291},[2830],{"type":47,"tag":189,"props":2831,"children":2832},{"emptyLinePlaceholder":709},[2833],{"type":53,"value":712},{"type":47,"tag":189,"props":2835,"children":2836},{"class":191,"line":715},[2837],{"type":47,"tag":189,"props":2838,"children":2839},{},[2840],{"type":53,"value":2841},"client = openai.OpenAI(\n",{"type":47,"tag":189,"props":2843,"children":2844},{"class":191,"line":724},[2845],{"type":47,"tag":189,"props":2846,"children":2847},{},[2848],{"type":53,"value":2849}," api_key=\"not-used\",\n",{"type":47,"tag":189,"props":2851,"children":2852},{"class":191,"line":732},[2853],{"type":47,"tag":189,"props":2854,"children":2855},{},[2856],{"type":53,"value":2857}," base_url=PORTKEY_GATEWAY_URL, # https:\u002F\u002Fapi.portkey.ai\u002Fv1\n",{"type":47,"tag":189,"props":2859,"children":2860},{"class":191,"line":741},[2861],{"type":47,"tag":189,"props":2862,"children":2863},{},[2864],{"type":53,"value":2865}," default_headers=createHeaders(\n",{"type":47,"tag":189,"props":2867,"children":2868},{"class":191,"line":750},[2869],{"type":47,"tag":189,"props":2870,"children":2871},{},[2872],{"type":53,"value":2873}," api_key=\"your-portkey-api-key\"\n",{"type":47,"tag":189,"props":2875,"children":2876},{"class":191,"line":759},[2877],{"type":47,"tag":189,"props":2878,"children":2879},{},[2880],{"type":53,"value":2496},{"type":47,"tag":189,"props":2882,"children":2883},{"class":191,"line":768},[2884],{"type":47,"tag":189,"props":2885,"children":2886},{},[2887],{"type":53,"value":765},{"type":47,"tag":189,"props":2889,"children":2890},{"class":191,"line":867},[2891],{"type":47,"tag":189,"props":2892,"children":2893},{"emptyLinePlaceholder":709},[2894],{"type":53,"value":712},{"type":47,"tag":189,"props":2896,"children":2897},{"class":191,"line":875},[2898],{"type":47,"tag":189,"props":2899,"children":2900},{},[2901],{"type":53,"value":849},{"type":47,"tag":189,"props":2903,"children":2904},{"class":191,"line":1365},[2905],{"type":47,"tag":189,"props":2906,"children":2907},{},[2908],{"type":53,"value":857},{"type":47,"tag":189,"props":2910,"children":2911},{"class":191,"line":1374},[2912],{"type":47,"tag":189,"props":2913,"children":2914},{},[2915],{"type":53,"value":756},{"type":47,"tag":189,"props":2917,"children":2918},{"class":191,"line":1382},[2919],{"type":47,"tag":189,"props":2920,"children":2921},{},[2922],{"type":53,"value":765},{"type":47,"tag":56,"props":2924,"children":2925},{},[2926,2928,2933],{"type":53,"value":2927},"This also works in ",{"type":47,"tag":65,"props":2929,"children":2930},{},[2931],{"type":53,"value":2932},"TypeScript\u002FJavaScript",{"type":53,"value":2934},":",{"type":47,"tag":178,"props":2936,"children":2940},{"className":2937,"code":2938,"language":2939,"meta":183,"style":183},"language-typescript shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","import OpenAI from \"openai\";\n\nconst client = new OpenAI({\n baseURL: \"https:\u002F\u002Fapi.portkey.ai\u002Fv1\",\n apiKey: process.env.PORTKEY_API_KEY,\n defaultHeaders: { \"x-portkey-provider\": \"@openai-prod\" }\n});\n\nconst response = await client.chat.completions.create({\n model: \"gpt-4o\",\n messages: [{ role: \"user\", content: \"Hello!\" }]\n});\n","typescript",[2941],{"type":47,"tag":151,"props":2942,"children":2943},{"__ignoreMap":183},[2944,2982,2989,3027,3058,3098,3150,3166,3173,3234,3263,3343],{"type":47,"tag":189,"props":2945,"children":2946},{"class":191,"line":26},[2947,2953,2958,2963,2968,2973,2977],{"type":47,"tag":189,"props":2948,"children":2950},{"style":2949},"--shiki-light:#39ADB5;--shiki-light-font-style:italic;--shiki-default:#89DDFF;--shiki-default-font-style:italic;--shiki-dark:#89DDFF;--shiki-dark-font-style:italic",[2951],{"type":53,"value":2952},"import",{"type":47,"tag":189,"props":2954,"children":2955},{"style":255},[2956],{"type":53,"value":2957}," OpenAI ",{"type":47,"tag":189,"props":2959,"children":2960},{"style":2949},[2961],{"type":53,"value":2962},"from",{"type":47,"tag":189,"props":2964,"children":2965},{"style":261},[2966],{"type":53,"value":2967}," \"",{"type":47,"tag":189,"props":2969,"children":2970},{"style":201},[2971],{"type":53,"value":2972},"openai",{"type":47,"tag":189,"props":2974,"children":2975},{"style":261},[2976],{"type":53,"value":269},{"type":47,"tag":189,"props":2978,"children":2979},{"style":261},[2980],{"type":53,"value":2981},";\n",{"type":47,"tag":189,"props":2983,"children":2984},{"class":191,"line":212},[2985],{"type":47,"tag":189,"props":2986,"children":2987},{"emptyLinePlaceholder":709},[2988],{"type":53,"value":712},{"type":47,"tag":189,"props":2990,"children":2991},{"class":191,"line":291},[2992,2997,3002,3006,3011,3017,3022],{"type":47,"tag":189,"props":2993,"children":2994},{"style":249},[2995],{"type":53,"value":2996},"const",{"type":47,"tag":189,"props":2998,"children":2999},{"style":255},[3000],{"type":53,"value":3001}," client ",{"type":47,"tag":189,"props":3003,"children":3004},{"style":261},[3005],{"type":53,"value":264},{"type":47,"tag":189,"props":3007,"children":3008},{"style":261},[3009],{"type":53,"value":3010}," new",{"type":47,"tag":189,"props":3012,"children":3014},{"style":3013},"--shiki-light:#6182B8;--shiki-default:#82AAFF;--shiki-dark:#82AAFF",[3015],{"type":53,"value":3016}," OpenAI",{"type":47,"tag":189,"props":3018,"children":3019},{"style":255},[3020],{"type":53,"value":3021},"(",{"type":47,"tag":189,"props":3023,"children":3024},{"style":261},[3025],{"type":53,"value":3026},"{\n",{"type":47,"tag":189,"props":3028,"children":3029},{"class":191,"line":715},[3030,3036,3040,3044,3049,3053],{"type":47,"tag":189,"props":3031,"children":3033},{"style":3032},"--shiki-light:#E53935;--shiki-default:#F07178;--shiki-dark:#F07178",[3034],{"type":53,"value":3035}," baseURL",{"type":47,"tag":189,"props":3037,"children":3038},{"style":261},[3039],{"type":53,"value":2934},{"type":47,"tag":189,"props":3041,"children":3042},{"style":261},[3043],{"type":53,"value":2967},{"type":47,"tag":189,"props":3045,"children":3046},{"style":201},[3047],{"type":53,"value":3048},"https:\u002F\u002Fapi.portkey.ai\u002Fv1",{"type":47,"tag":189,"props":3050,"children":3051},{"style":261},[3052],{"type":53,"value":269},{"type":47,"tag":189,"props":3054,"children":3055},{"style":261},[3056],{"type":53,"value":3057},",\n",{"type":47,"tag":189,"props":3059,"children":3060},{"class":191,"line":724},[3061,3066,3070,3075,3080,3085,3089,3094],{"type":47,"tag":189,"props":3062,"children":3063},{"style":3032},[3064],{"type":53,"value":3065}," apiKey",{"type":47,"tag":189,"props":3067,"children":3068},{"style":261},[3069],{"type":53,"value":2934},{"type":47,"tag":189,"props":3071,"children":3072},{"style":255},[3073],{"type":53,"value":3074}," process",{"type":47,"tag":189,"props":3076,"children":3077},{"style":261},[3078],{"type":53,"value":3079},".",{"type":47,"tag":189,"props":3081,"children":3082},{"style":255},[3083],{"type":53,"value":3084},"env",{"type":47,"tag":189,"props":3086,"children":3087},{"style":261},[3088],{"type":53,"value":3079},{"type":47,"tag":189,"props":3090,"children":3091},{"style":255},[3092],{"type":53,"value":3093},"PORTKEY_API_KEY",{"type":47,"tag":189,"props":3095,"children":3096},{"style":261},[3097],{"type":53,"value":3057},{"type":47,"tag":189,"props":3099,"children":3100},{"class":191,"line":732},[3101,3106,3110,3115,3119,3124,3128,3132,3136,3141,3145],{"type":47,"tag":189,"props":3102,"children":3103},{"style":3032},[3104],{"type":53,"value":3105}," defaultHeaders",{"type":47,"tag":189,"props":3107,"children":3108},{"style":261},[3109],{"type":53,"value":2934},{"type":47,"tag":189,"props":3111,"children":3112},{"style":261},[3113],{"type":53,"value":3114}," {",{"type":47,"tag":189,"props":3116,"children":3117},{"style":261},[3118],{"type":53,"value":2967},{"type":47,"tag":189,"props":3120,"children":3121},{"style":3032},[3122],{"type":53,"value":3123},"x-portkey-provider",{"type":47,"tag":189,"props":3125,"children":3126},{"style":261},[3127],{"type":53,"value":269},{"type":47,"tag":189,"props":3129,"children":3130},{"style":261},[3131],{"type":53,"value":2934},{"type":47,"tag":189,"props":3133,"children":3134},{"style":261},[3135],{"type":53,"value":2967},{"type":47,"tag":189,"props":3137,"children":3138},{"style":201},[3139],{"type":53,"value":3140},"@openai-prod",{"type":47,"tag":189,"props":3142,"children":3143},{"style":261},[3144],{"type":53,"value":269},{"type":47,"tag":189,"props":3146,"children":3147},{"style":261},[3148],{"type":53,"value":3149}," }\n",{"type":47,"tag":189,"props":3151,"children":3152},{"class":191,"line":741},[3153,3158,3162],{"type":47,"tag":189,"props":3154,"children":3155},{"style":261},[3156],{"type":53,"value":3157},"}",{"type":47,"tag":189,"props":3159,"children":3160},{"style":255},[3161],{"type":53,"value":166},{"type":47,"tag":189,"props":3163,"children":3164},{"style":261},[3165],{"type":53,"value":2981},{"type":47,"tag":189,"props":3167,"children":3168},{"class":191,"line":750},[3169],{"type":47,"tag":189,"props":3170,"children":3171},{"emptyLinePlaceholder":709},[3172],{"type":53,"value":712},{"type":47,"tag":189,"props":3174,"children":3175},{"class":191,"line":759},[3176,3180,3185,3189,3194,3199,3203,3208,3212,3217,3221,3226,3230],{"type":47,"tag":189,"props":3177,"children":3178},{"style":249},[3179],{"type":53,"value":2996},{"type":47,"tag":189,"props":3181,"children":3182},{"style":255},[3183],{"type":53,"value":3184}," response ",{"type":47,"tag":189,"props":3186,"children":3187},{"style":261},[3188],{"type":53,"value":264},{"type":47,"tag":189,"props":3190,"children":3191},{"style":2949},[3192],{"type":53,"value":3193}," await",{"type":47,"tag":189,"props":3195,"children":3196},{"style":255},[3197],{"type":53,"value":3198}," client",{"type":47,"tag":189,"props":3200,"children":3201},{"style":261},[3202],{"type":53,"value":3079},{"type":47,"tag":189,"props":3204,"children":3205},{"style":255},[3206],{"type":53,"value":3207},"chat",{"type":47,"tag":189,"props":3209,"children":3210},{"style":261},[3211],{"type":53,"value":3079},{"type":47,"tag":189,"props":3213,"children":3214},{"style":255},[3215],{"type":53,"value":3216},"completions",{"type":47,"tag":189,"props":3218,"children":3219},{"style":261},[3220],{"type":53,"value":3079},{"type":47,"tag":189,"props":3222,"children":3223},{"style":3013},[3224],{"type":53,"value":3225},"create",{"type":47,"tag":189,"props":3227,"children":3228},{"style":255},[3229],{"type":53,"value":3021},{"type":47,"tag":189,"props":3231,"children":3232},{"style":261},[3233],{"type":53,"value":3026},{"type":47,"tag":189,"props":3235,"children":3236},{"class":191,"line":768},[3237,3242,3246,3250,3255,3259],{"type":47,"tag":189,"props":3238,"children":3239},{"style":3032},[3240],{"type":53,"value":3241}," model",{"type":47,"tag":189,"props":3243,"children":3244},{"style":261},[3245],{"type":53,"value":2934},{"type":47,"tag":189,"props":3247,"children":3248},{"style":261},[3249],{"type":53,"value":2967},{"type":47,"tag":189,"props":3251,"children":3252},{"style":201},[3253],{"type":53,"value":3254},"gpt-4o",{"type":47,"tag":189,"props":3256,"children":3257},{"style":261},[3258],{"type":53,"value":269},{"type":47,"tag":189,"props":3260,"children":3261},{"style":261},[3262],{"type":53,"value":3057},{"type":47,"tag":189,"props":3264,"children":3265},{"class":191,"line":867},[3266,3271,3275,3280,3285,3290,3294,3298,3303,3307,3312,3317,3321,3325,3330,3334,3339],{"type":47,"tag":189,"props":3267,"children":3268},{"style":3032},[3269],{"type":53,"value":3270}," messages",{"type":47,"tag":189,"props":3272,"children":3273},{"style":261},[3274],{"type":53,"value":2934},{"type":47,"tag":189,"props":3276,"children":3277},{"style":255},[3278],{"type":53,"value":3279}," [",{"type":47,"tag":189,"props":3281,"children":3282},{"style":261},[3283],{"type":53,"value":3284},"{",{"type":47,"tag":189,"props":3286,"children":3287},{"style":3032},[3288],{"type":53,"value":3289}," role",{"type":47,"tag":189,"props":3291,"children":3292},{"style":261},[3293],{"type":53,"value":2934},{"type":47,"tag":189,"props":3295,"children":3296},{"style":261},[3297],{"type":53,"value":2967},{"type":47,"tag":189,"props":3299,"children":3300},{"style":201},[3301],{"type":53,"value":3302},"user",{"type":47,"tag":189,"props":3304,"children":3305},{"style":261},[3306],{"type":53,"value":269},{"type":47,"tag":189,"props":3308,"children":3309},{"style":261},[3310],{"type":53,"value":3311},",",{"type":47,"tag":189,"props":3313,"children":3314},{"style":3032},[3315],{"type":53,"value":3316}," content",{"type":47,"tag":189,"props":3318,"children":3319},{"style":261},[3320],{"type":53,"value":2934},{"type":47,"tag":189,"props":3322,"children":3323},{"style":261},[3324],{"type":53,"value":2967},{"type":47,"tag":189,"props":3326,"children":3327},{"style":201},[3328],{"type":53,"value":3329},"Hello!",{"type":47,"tag":189,"props":3331,"children":3332},{"style":261},[3333],{"type":53,"value":269},{"type":47,"tag":189,"props":3335,"children":3336},{"style":261},[3337],{"type":53,"value":3338}," }",{"type":47,"tag":189,"props":3340,"children":3341},{"style":255},[3342],{"type":53,"value":1576},{"type":47,"tag":189,"props":3344,"children":3345},{"class":191,"line":875},[3346,3350,3354],{"type":47,"tag":189,"props":3347,"children":3348},{"style":261},[3349],{"type":53,"value":3157},{"type":47,"tag":189,"props":3351,"children":3352},{"style":255},[3353],{"type":53,"value":166},{"type":47,"tag":189,"props":3355,"children":3356},{"style":261},[3357],{"type":53,"value":2981},{"type":47,"tag":666,"props":3359,"children":3361},{"id":3360},"quick-verification-with-curl",[3362],{"type":53,"value":3363},"Quick Verification with cURL",{"type":47,"tag":56,"props":3365,"children":3366},{},[3367],{"type":53,"value":3368},"Before changing any code, test the gateway connection:",{"type":47,"tag":178,"props":3370,"children":3372},{"className":180,"code":3371,"language":182,"meta":183,"style":183},"curl -s https:\u002F\u002Fapi.portkey.ai\u002Fv1\u002Fchat\u002Fcompletions \\\n -H \"Content-Type: application\u002Fjson\" \\\n -H \"x-portkey-api-key: $PORTKEY_API_KEY\" \\\n -H \"x-portkey-provider: @openai-prod\" \\\n -d '{\n \"model\": \"gpt-4o\",\n \"max_tokens\": 5,\n \"messages\": [{\"role\": \"user\", \"content\": \"Say ok\"}]\n }'\n",[3373],{"type":47,"tag":151,"props":3374,"children":3375},{"__ignoreMap":183},[3376,3399,3424,3453,3477,3494,3502,3510,3518],{"type":47,"tag":189,"props":3377,"children":3378},{"class":191,"line":26},[3379,3384,3389,3394],{"type":47,"tag":189,"props":3380,"children":3381},{"style":195},[3382],{"type":53,"value":3383},"curl",{"type":47,"tag":189,"props":3385,"children":3386},{"style":201},[3387],{"type":53,"value":3388}," -s",{"type":47,"tag":189,"props":3390,"children":3391},{"style":201},[3392],{"type":53,"value":3393}," https:\u002F\u002Fapi.portkey.ai\u002Fv1\u002Fchat\u002Fcompletions",{"type":47,"tag":189,"props":3395,"children":3396},{"style":255},[3397],{"type":53,"value":3398}," \\\n",{"type":47,"tag":189,"props":3400,"children":3401},{"class":191,"line":212},[3402,3407,3411,3416,3420],{"type":47,"tag":189,"props":3403,"children":3404},{"style":201},[3405],{"type":53,"value":3406}," -H",{"type":47,"tag":189,"props":3408,"children":3409},{"style":261},[3410],{"type":53,"value":2967},{"type":47,"tag":189,"props":3412,"children":3413},{"style":201},[3414],{"type":53,"value":3415},"Content-Type: application\u002Fjson",{"type":47,"tag":189,"props":3417,"children":3418},{"style":261},[3419],{"type":53,"value":269},{"type":47,"tag":189,"props":3421,"children":3422},{"style":255},[3423],{"type":53,"value":3398},{"type":47,"tag":189,"props":3425,"children":3426},{"class":191,"line":291},[3427,3431,3435,3440,3445,3449],{"type":47,"tag":189,"props":3428,"children":3429},{"style":201},[3430],{"type":53,"value":3406},{"type":47,"tag":189,"props":3432,"children":3433},{"style":261},[3434],{"type":53,"value":2967},{"type":47,"tag":189,"props":3436,"children":3437},{"style":201},[3438],{"type":53,"value":3439},"x-portkey-api-key: ",{"type":47,"tag":189,"props":3441,"children":3442},{"style":255},[3443],{"type":53,"value":3444},"$PORTKEY_API_KEY",{"type":47,"tag":189,"props":3446,"children":3447},{"style":261},[3448],{"type":53,"value":269},{"type":47,"tag":189,"props":3450,"children":3451},{"style":255},[3452],{"type":53,"value":3398},{"type":47,"tag":189,"props":3454,"children":3455},{"class":191,"line":715},[3456,3460,3464,3469,3473],{"type":47,"tag":189,"props":3457,"children":3458},{"style":201},[3459],{"type":53,"value":3406},{"type":47,"tag":189,"props":3461,"children":3462},{"style":261},[3463],{"type":53,"value":2967},{"type":47,"tag":189,"props":3465,"children":3466},{"style":201},[3467],{"type":53,"value":3468},"x-portkey-provider: @openai-prod",{"type":47,"tag":189,"props":3470,"children":3471},{"style":261},[3472],{"type":53,"value":269},{"type":47,"tag":189,"props":3474,"children":3475},{"style":255},[3476],{"type":53,"value":3398},{"type":47,"tag":189,"props":3478,"children":3479},{"class":191,"line":724},[3480,3485,3490],{"type":47,"tag":189,"props":3481,"children":3482},{"style":201},[3483],{"type":53,"value":3484}," -d",{"type":47,"tag":189,"props":3486,"children":3487},{"style":261},[3488],{"type":53,"value":3489}," '",{"type":47,"tag":189,"props":3491,"children":3492},{"style":201},[3493],{"type":53,"value":3026},{"type":47,"tag":189,"props":3495,"children":3496},{"class":191,"line":732},[3497],{"type":47,"tag":189,"props":3498,"children":3499},{"style":201},[3500],{"type":53,"value":3501}," \"model\": \"gpt-4o\",\n",{"type":47,"tag":189,"props":3503,"children":3504},{"class":191,"line":741},[3505],{"type":47,"tag":189,"props":3506,"children":3507},{"style":201},[3508],{"type":53,"value":3509}," \"max_tokens\": 5,\n",{"type":47,"tag":189,"props":3511,"children":3512},{"class":191,"line":750},[3513],{"type":47,"tag":189,"props":3514,"children":3515},{"style":201},[3516],{"type":53,"value":3517}," \"messages\": [{\"role\": \"user\", \"content\": \"Say ok\"}]\n",{"type":47,"tag":189,"props":3519,"children":3520},{"class":191,"line":759},[3521,3526],{"type":47,"tag":189,"props":3522,"children":3523},{"style":201},[3524],{"type":53,"value":3525}," }",{"type":47,"tag":189,"props":3527,"children":3528},{"style":261},[3529],{"type":53,"value":3530},"'\n",{"type":47,"tag":666,"props":3532,"children":3534},{"id":3533},"routing-via-headers",[3535],{"type":53,"value":3536},"Routing via Headers",{"type":47,"tag":56,"props":3538,"children":3539},{},[3540],{"type":53,"value":3541},"Portkey supports two routing modes, set via headers:",{"type":47,"tag":343,"props":3543,"children":3544},{},[3545,3561],{"type":47,"tag":347,"props":3546,"children":3547},{},[3548],{"type":47,"tag":351,"props":3549,"children":3550},{},[3551,3556],{"type":47,"tag":355,"props":3552,"children":3553},{},[3554],{"type":53,"value":3555},"Header",{"type":47,"tag":355,"props":3557,"children":3558},{},[3559],{"type":53,"value":3560},"Use Case",{"type":47,"tag":365,"props":3562,"children":3563},{},[3564,3581],{"type":47,"tag":351,"props":3565,"children":3566},{},[3567,3576],{"type":47,"tag":372,"props":3568,"children":3569},{},[3570],{"type":47,"tag":151,"props":3571,"children":3573},{"className":3572},[],[3574],{"type":53,"value":3575},"x-portkey-provider: @slug",{"type":47,"tag":372,"props":3577,"children":3578},{},[3579],{"type":53,"value":3580},"Route to a specific AI Provider from your Model Catalog",{"type":47,"tag":351,"props":3582,"children":3583},{},[3584,3593],{"type":47,"tag":372,"props":3585,"children":3586},{},[3587],{"type":47,"tag":151,"props":3588,"children":3590},{"className":3589},[],[3591],{"type":53,"value":3592},"x-portkey-config: pc-config-xxx",{"type":47,"tag":372,"props":3594,"children":3595},{},[3596],{"type":53,"value":3597},"Use a Config (with fallbacks, caching, etc.) from the dashboard",{"type":47,"tag":56,"props":3599,"children":3600},{},[3601,3603,3608,3610,3616],{"type":53,"value":3602},"With the ",{"type":47,"tag":151,"props":3604,"children":3606},{"className":3605},[],[3607],{"type":53,"value":2233},{"type":53,"value":3609}," format in the ",{"type":47,"tag":151,"props":3611,"children":3613},{"className":3612},[],[3614],{"type":53,"value":3615},"model",{"type":53,"value":3617}," parameter, you typically don't need to set these headers explicitly.",{"type":47,"tag":99,"props":3619,"children":3620},{},[],{"type":47,"tag":103,"props":3622,"children":3624},{"id":3623},"migration-workflow",[3625],{"type":53,"value":3626},"Migration Workflow",{"type":47,"tag":110,"props":3628,"children":3629},{},[3630,3661,3722,3747,3762,3772,3798,3808],{"type":47,"tag":75,"props":3631,"children":3632},{},[3633,3638,3640],{"type":47,"tag":65,"props":3634,"children":3635},{},[3636],{"type":53,"value":3637},"Verify",{"type":53,"value":3639}," — Test connectivity before touching code:\n",{"type":47,"tag":178,"props":3641,"children":3642},{"className":180,"code":310,"language":182,"meta":183,"style":183},[3643],{"type":47,"tag":151,"props":3644,"children":3645},{"__ignoreMap":183},[3646],{"type":47,"tag":189,"props":3647,"children":3648},{"class":191,"line":26},[3649,3653,3657],{"type":47,"tag":189,"props":3650,"children":3651},{"style":195},[3652],{"type":53,"value":322},{"type":47,"tag":189,"props":3654,"children":3655},{"style":201},[3656],{"type":53,"value":327},{"type":47,"tag":189,"props":3658,"children":3659},{"style":201},[3660],{"type":53,"value":332},{"type":47,"tag":75,"props":3662,"children":3663},{},[3664,3669,3671],{"type":47,"tag":65,"props":3665,"children":3666},{},[3667],{"type":53,"value":3668},"Audit",{"type":53,"value":3670}," — Grep your codebase for LiteLLM usage:\n",{"type":47,"tag":178,"props":3672,"children":3674},{"className":180,"code":3673,"language":182,"meta":183,"style":183},"grep -rn \"from litellm\\|import litellm\\|litellm\\.\" --include=\"*.py\"\n",[3675],{"type":47,"tag":151,"props":3676,"children":3677},{"__ignoreMap":183},[3678],{"type":47,"tag":189,"props":3679,"children":3680},{"class":191,"line":26},[3681,3686,3691,3695,3700,3704,3709,3713,3718],{"type":47,"tag":189,"props":3682,"children":3683},{"style":195},[3684],{"type":53,"value":3685},"grep",{"type":47,"tag":189,"props":3687,"children":3688},{"style":201},[3689],{"type":53,"value":3690}," -rn",{"type":47,"tag":189,"props":3692,"children":3693},{"style":261},[3694],{"type":53,"value":2967},{"type":47,"tag":189,"props":3696,"children":3697},{"style":201},[3698],{"type":53,"value":3699},"from litellm\\|import litellm\\|litellm\\.",{"type":47,"tag":189,"props":3701,"children":3702},{"style":261},[3703],{"type":53,"value":269},{"type":47,"tag":189,"props":3705,"children":3706},{"style":201},[3707],{"type":53,"value":3708}," --include=",{"type":47,"tag":189,"props":3710,"children":3711},{"style":261},[3712],{"type":53,"value":269},{"type":47,"tag":189,"props":3714,"children":3715},{"style":201},[3716],{"type":53,"value":3717},"*.py",{"type":47,"tag":189,"props":3719,"children":3720},{"style":261},[3721],{"type":53,"value":279},{"type":47,"tag":75,"props":3723,"children":3724},{},[3725,3730,3732,3738,3740,3746],{"type":47,"tag":65,"props":3726,"children":3727},{},[3728],{"type":53,"value":3729},"Quick win",{"type":53,"value":3731}," — Point existing LiteLLM calls at Portkey with ",{"type":47,"tag":151,"props":3733,"children":3735},{"className":3734},[],[3736],{"type":53,"value":3737},"api_base",{"type":53,"value":3739}," (see ",{"type":47,"tag":79,"props":3741,"children":3743},{"href":3742},"#zero-change-path-keep-litellm-route-through-portkey",[3744],{"type":53,"value":3745},"Zero-Change Path",{"type":53,"value":166},{"type":47,"tag":75,"props":3748,"children":3749},{},[3750,3755,3757],{"type":47,"tag":65,"props":3751,"children":3752},{},[3753],{"type":53,"value":3754},"Map",{"type":53,"value":3756}," — For each usage, find the Portkey equivalent using the table above or ",{"type":47,"tag":79,"props":3758,"children":3759},{"href":81},[3760],{"type":53,"value":3761},"MIGRATION_PATTERNS.md",{"type":47,"tag":75,"props":3763,"children":3764},{},[3765,3770],{"type":47,"tag":65,"props":3766,"children":3767},{},[3768],{"type":53,"value":3769},"Replace",{"type":53,"value":3771}," — Swap imports, instantiate a Portkey client, and update call sites",{"type":47,"tag":75,"props":3773,"children":3774},{},[3775,3780,3782,3788,3790,3797],{"type":47,"tag":65,"props":3776,"children":3777},{},[3778],{"type":53,"value":3779},"Configure",{"type":53,"value":3781}," — Move routing\u002Ffallback\u002Fcache logic from code into Portkey ",{"type":47,"tag":151,"props":3783,"children":3785},{"className":3784},[],[3786],{"type":53,"value":3787},"config",{"type":53,"value":3789}," dicts (or dashboard ",{"type":47,"tag":79,"props":3791,"children":3794},{"href":3792,"rel":3793},"https:\u002F\u002Fapp.portkey.ai\u002Fconfigs",[127],[3795],{"type":53,"value":3796},"Config IDs",{"type":53,"value":166},{"type":47,"tag":75,"props":3799,"children":3800},{},[3801,3806],{"type":47,"tag":65,"props":3802,"children":3803},{},[3804],{"type":53,"value":3805},"Test",{"type":53,"value":3807}," — Run your test suite; response formats are identical (OpenAI-compatible)",{"type":47,"tag":75,"props":3809,"children":3810},{},[3811,3816,3818,3824],{"type":47,"tag":65,"props":3812,"children":3813},{},[3814],{"type":53,"value":3815},"Clean up",{"type":53,"value":3817}," — Remove ",{"type":47,"tag":151,"props":3819,"children":3821},{"className":3820},[],[3822],{"type":53,"value":3823},"litellm",{"type":53,"value":3825}," from requirements, remove provider API key env vars",{"type":47,"tag":99,"props":3827,"children":3828},{},[],{"type":47,"tag":103,"props":3830,"children":3832},{"id":3831},"resources",[3833],{"type":53,"value":3834},"Resources",{"type":47,"tag":71,"props":3836,"children":3837},{},[3838,3854,3868,3884],{"type":47,"tag":75,"props":3839,"children":3840},{},[3841,3846,3848],{"type":47,"tag":65,"props":3842,"children":3843},{},[3844],{"type":53,"value":3845},"Portkey Docs",{"type":53,"value":3847},": ",{"type":47,"tag":79,"props":3849,"children":3851},{"href":42,"rel":3850},[127],[3852],{"type":53,"value":3853},"docs.portkey.ai",{"type":47,"tag":75,"props":3855,"children":3856},{},[3857,3862,3863],{"type":47,"tag":65,"props":3858,"children":3859},{},[3860],{"type":53,"value":3861},"Dashboard",{"type":53,"value":3847},{"type":47,"tag":79,"props":3864,"children":3866},{"href":143,"rel":3865},[127],[3867],{"type":53,"value":2203},{"type":47,"tag":75,"props":3869,"children":3870},{},[3871,3876,3877],{"type":47,"tag":65,"props":3872,"children":3873},{},[3874],{"type":53,"value":3875},"Python SDK",{"type":53,"value":3847},{"type":47,"tag":79,"props":3878,"children":3881},{"href":3879,"rel":3880},"https:\u002F\u002Fgithub.com\u002Fportkey-ai\u002Fportkey-python-sdk",[127],[3882],{"type":53,"value":3883},"github.com\u002Fportkey-ai\u002Fportkey-python-sdk",{"type":47,"tag":75,"props":3885,"children":3886},{},[3887,3892,3893],{"type":47,"tag":65,"props":3888,"children":3889},{},[3890],{"type":53,"value":3891},"Discord",{"type":53,"value":3847},{"type":47,"tag":79,"props":3894,"children":3897},{"href":3895,"rel":3896},"https:\u002F\u002Fportkey.ai\u002Fdiscord",[127],[3898],{"type":53,"value":3899},"portkey.ai\u002Fdiscord",{"type":47,"tag":3901,"props":3902,"children":3903},"style",{},[3904],{"type":53,"value":3905},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":3907,"total":291},[3908,3915,3933],{"slug":4,"name":4,"fn":5,"description":6,"org":3909,"tags":3910,"stars":26,"repoUrl":27,"updatedAt":28},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[3911,3912,3913,3914],{"name":24,"slug":25,"type":16},{"name":14,"slug":15,"type":16},{"name":18,"slug":19,"type":16},{"name":21,"slug":22,"type":16},{"slug":3916,"name":3916,"fn":3917,"description":3918,"org":3919,"tags":3920,"stars":26,"repoUrl":27,"updatedAt":3932},"portkey-python-sdk","build Python applications with Portkey SDK","Complete reference for the Portkey AI Gateway Python SDK with unified API access to 200+ LLMs, automatic fallbacks, caching, and full observability. Use when building Python applications that need LLM integration with production-grade reliability.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[3921,3924,3927,3928,3931],{"name":3922,"slug":3923,"type":16},"API Development","api-development",{"name":3925,"slug":3926,"type":16},"Caching","caching",{"name":14,"slug":15,"type":16},{"name":3929,"slug":3930,"type":16},"Observability","observability",{"name":21,"slug":22,"type":16},"2026-07-13T06:00:05.815338",{"slug":3934,"name":3934,"fn":3935,"description":3936,"org":3937,"tags":3938,"stars":26,"repoUrl":27,"updatedAt":3944},"portkey-typescript-sdk","integrate Portkey AI Gateway","Integrate Portkey AI Gateway into TypeScript\u002FJavaScript applications. Use when building LLM apps with observability, caching, fallbacks, load balancing, or routing across 200+ LLM providers.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[3939,3940,3941,3942],{"name":24,"slug":25,"type":16},{"name":14,"slug":15,"type":16},{"name":3929,"slug":3930,"type":16},{"name":3943,"slug":2939,"type":16},"TypeScript","2026-07-13T06:00:07.08252",{"items":3946,"total":291},[3947,3954,3962],{"slug":4,"name":4,"fn":5,"description":6,"org":3948,"tags":3949,"stars":26,"repoUrl":27,"updatedAt":28},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[3950,3951,3952,3953],{"name":24,"slug":25,"type":16},{"name":14,"slug":15,"type":16},{"name":18,"slug":19,"type":16},{"name":21,"slug":22,"type":16},{"slug":3916,"name":3916,"fn":3917,"description":3918,"org":3955,"tags":3956,"stars":26,"repoUrl":27,"updatedAt":3932},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[3957,3958,3959,3960,3961],{"name":3922,"slug":3923,"type":16},{"name":3925,"slug":3926,"type":16},{"name":14,"slug":15,"type":16},{"name":3929,"slug":3930,"type":16},{"name":21,"slug":22,"type":16},{"slug":3934,"name":3934,"fn":3935,"description":3936,"org":3963,"tags":3964,"stars":26,"repoUrl":27,"updatedAt":3944},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[3965,3966,3967,3968],{"name":24,"slug":25,"type":16},{"name":14,"slug":15,"type":16},{"name":3929,"slug":3930,"type":16},{"name":3943,"slug":2939,"type":16}]