[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-portkey-portkey-python-sdk":3,"mdc--y4huxe-key":39,"related-org-portkey-portkey-python-sdk":2249,"related-repo-portkey-portkey-python-sdk":2287},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":12,"stars":29,"repoUrl":30,"updatedAt":31,"license":32,"forks":29,"topics":33,"repo":34,"sourceUrl":37,"mdContent":38},"portkey-python-sdk","build Python applications with Portkey SDK","Complete reference for the Portkey AI Gateway Python SDK with unified API access to 200+ LLMs, automatic fallbacks, caching, and full observability. Use when building Python applications that need LLM integration with production-grade reliability.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},"portkey","Portkey","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Fportkey.png","Portkey-AI",[13,17,20,23,26],{"name":14,"slug":15,"type":16},"Observability","observability","tag",{"name":18,"slug":19,"type":16},"LLM","llm",{"name":21,"slug":22,"type":16},"Python","python",{"name":24,"slug":25,"type":16},"API Development","api-development",{"name":27,"slug":28,"type":16},"Caching","caching",1,"https:\u002F\u002Fgithub.com\u002FPortkey-AI\u002Fskills","2026-07-13T06:00:05.815338","Apache-2.0",[],{"repoUrl":30,"stars":29,"forks":29,"topics":35,"description":36},[],null,"https:\u002F\u002Fgithub.com\u002FPortkey-AI\u002Fskills\u002Ftree\u002FHEAD\u002Fskills\u002Fportkey-python-sdk","---\nname: portkey-python-sdk\ndescription: Complete reference for the Portkey AI Gateway Python SDK with unified API access to 200+ LLMs, automatic fallbacks, caching, and full observability. Use when building Python applications that need LLM integration with production-grade reliability.\nlicense: Apache-2.0\ncompatibility: Requires Python 3.8+. Works with pip, poetry, or uv.\nmetadata:\n  author: portkey-ai\n  version: \"1.0.0\"\n  sdk-version: \"2.1.0\"\n  sdk-repo: https:\u002F\u002Fgithub.com\u002Fportkey-ai\u002Fportkey-python-sdk\n  docs: https:\u002F\u002Fdocs.portkey.ai\n---\n\n# Portkey Python SDK\n\nThe Portkey Python SDK provides a unified interface to 200+ LLMs through the Portkey AI Gateway. Built on top of the OpenAI SDK for seamless compatibility, it adds production-grade features: automatic fallbacks, retries, load balancing, semantic caching, guardrails, and comprehensive observability.\n\n**Additional References:**\n- [API Reference](references\u002FAPI_REFERENCE.md) - Response structures, error handling\n- [Advanced Features](references\u002FADVANCED_FEATURES.md) - Tool calling, embeddings, audio, images\n- [Framework Integrations](references\u002FINTEGRATIONS.md) - LangChain, LlamaIndex, Strands, Google ADK\n- [Provider Configuration](references\u002FPROVIDERS.md) - Azure, AWS Bedrock, Vertex AI setup\n\n---\n\n## Installation\n\n```bash\npip install portkey-ai\n\n# Or with poetry\u002Fuv\npoetry add portkey-ai\nuv add portkey-ai\n```\n\n---\n\n## Quick Start\n\n```python\nimport os\nfrom portkey_ai import Portkey\n\nclient = Portkey(\n    api_key=os.environ[\"PORTKEY_API_KEY\"],\n    virtual_key=\"your-openai-virtual-key\"\n)\n\nresponse = client.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\n\nprint(response.choices[0].message.content)\n```\n\n---\n\n## Authentication\n\n### API Key + Virtual Key (Recommended)\n\nVirtual keys securely store provider API keys in Portkey's vault:\n\n```python\nimport os\nfrom portkey_ai import Portkey\n\nclient = Portkey(\n    api_key=os.environ[\"PORTKEY_API_KEY\"],  # From app.portkey.ai\n    virtual_key=\"openai-virtual-key-xxx\"     # From app.portkey.ai\u002Fvirtual-keys\n)\n```\n\n### Using Config IDs\n\nPre-configure routing, fallbacks, and caching in the dashboard:\n\n```python\nclient = Portkey(\n    api_key=os.environ[\"PORTKEY_API_KEY\"],\n    config=\"pc-config-xxx\"  # Config ID from dashboard\n)\n```\n\n---\n\n## Chat Completions\n\n### Basic Request\n\n```python\nresponse = client.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[\n        {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n        {\"role\": \"user\", \"content\": \"Explain quantum computing briefly.\"}\n    ]\n)\n\nprint(response.choices[0].message.content)\nprint(f\"Tokens used: {response.usage.total_tokens}\")\n```\n\n### Streaming\n\n```python\nstream = client.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"Write a short story\"}],\n    stream=True\n)\n\nfor chunk in stream:\n    if chunk.choices[0].delta.content:\n        print(chunk.choices[0].delta.content, end=\"\", flush=True)\n```\n\n### Async Support\n\n```python\nimport asyncio\nfrom portkey_ai import AsyncPortkey\n\nasync def main():\n    client = AsyncPortkey(\n        api_key=os.environ[\"PORTKEY_API_KEY\"],\n        virtual_key=\"openai-key\"\n    )\n    \n    response = await client.chat.completions.create(\n        model=\"gpt-4o\",\n        messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n    )\n    print(response.choices[0].message.content)\n\nasyncio.run(main())\n```\n\n### Async Streaming\n\n```python\nasync def stream_response():\n    client = AsyncPortkey(\n        api_key=os.environ[\"PORTKEY_API_KEY\"],\n        virtual_key=\"openai-key\"\n    )\n    \n    stream = await client.chat.completions.create(\n        model=\"gpt-4o\",\n        messages=[{\"role\": \"user\", \"content\": \"Write a poem\"}],\n        stream=True\n    )\n    \n    async for chunk in stream:\n        if chunk.choices[0].delta.content:\n            print(chunk.choices[0].delta.content, end=\"\", flush=True)\n```\n\n---\n\n## Gateway Features\n\n### Fallbacks\n\nAutomatic failover when a provider fails:\n\n```python\nclient = Portkey(\n    api_key=os.environ[\"PORTKEY_API_KEY\"],\n    config={\n        \"strategy\": {\"mode\": \"fallback\"},\n        \"targets\": [\n            {\n                \"virtual_key\": \"openai-key\",\n                \"override_params\": {\"model\": \"gpt-4o\"}\n            },\n            {\n                \"virtual_key\": \"anthropic-key\",\n                \"override_params\": {\"model\": \"claude-3-5-sonnet-20241022\"}\n            }\n        ]\n    }\n)\n\n# If OpenAI fails, automatically tries Anthropic\nresponse = client.chat.completions.create(\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\n```\n\n### Load Balancing\n\nDistribute traffic across providers:\n\n```python\nclient = Portkey(\n    api_key=os.environ[\"PORTKEY_API_KEY\"],\n    config={\n        \"strategy\": {\"mode\": \"loadbalance\"},\n        \"targets\": [\n            {\"virtual_key\": \"openai-key-1\", \"weight\": 0.7},\n            {\"virtual_key\": \"openai-key-2\", \"weight\": 0.3}\n        ]\n    }\n)\n```\n\n### Automatic Retries\n\n```python\nclient = Portkey(\n    api_key=os.environ[\"PORTKEY_API_KEY\"],\n    config={\n        \"retry\": {\n            \"attempts\": 3,\n            \"on_status_codes\": [429, 500, 502, 503, 504]\n        },\n        \"virtual_key\": \"openai-key\"\n    }\n)\n```\n\n### Semantic Caching\n\nReduce costs and latency with intelligent caching:\n\n```python\nclient = Portkey(\n    api_key=os.environ[\"PORTKEY_API_KEY\"],\n    config={\n        \"cache\": {\n            \"mode\": \"semantic\",  # or \"simple\" for exact match\n            \"max_age\": 3600      # TTL in seconds\n        },\n        \"virtual_key\": \"openai-key\"\n    }\n)\n\n# Similar queries return cached responses\nresponse1 = client.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"What is the capital of France?\"}]\n)\n\nresponse2 = client.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"Tell me France's capital\"}]\n)  # Returns cached response\n```\n\n### Request Timeout\n\n```python\nclient = Portkey(\n    api_key=os.environ[\"PORTKEY_API_KEY\"],\n    virtual_key=\"openai-key\",\n    request_timeout=30  # 30 seconds\n)\n```\n\n---\n\n## Observability\n\n### Trace IDs\n\nLink related requests for debugging:\n\n```python\nimport uuid\n\nclient = Portkey(\n    api_key=os.environ[\"PORTKEY_API_KEY\"],\n    virtual_key=\"openai-key\",\n    trace_id=str(uuid.uuid4())\n)\n```\n\n### Custom Metadata\n\nAdd searchable metadata to requests:\n\n```python\nclient = Portkey(\n    api_key=os.environ[\"PORTKEY_API_KEY\"],\n    virtual_key=\"openai-key\",\n    metadata={\n        \"user_id\": \"user-123\",\n        \"session_id\": \"session-456\",\n        \"environment\": \"production\"\n    }\n)\n```\n\n### Per-Request Options\n\n```python\nresponse = client.with_options(\n    trace_id=\"unique-trace-id\",\n    metadata={\"request_type\": \"summarization\"}\n).chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"Summarize this...\"}]\n)\n```\n\n---\n\n## Common Patterns\n\n### Multi-turn Conversation\n\n```python\nmessages = [\n    {\"role\": \"system\", \"content\": \"You are a helpful coding assistant.\"},\n    {\"role\": \"user\", \"content\": \"What is Python?\"},\n    {\"role\": \"assistant\", \"content\": \"Python is a high-level programming language...\"},\n    {\"role\": \"user\", \"content\": \"Show me a hello world example.\"}\n]\n\nresponse = client.chat.completions.create(model=\"gpt-4o\", messages=messages)\n```\n\n### JSON Output\n\n```python\nresponse = client.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[\n        {\"role\": \"system\", \"content\": \"Extract as JSON with name and age fields.\"},\n        {\"role\": \"user\", \"content\": \"John is 30 years old.\"}\n    ],\n    response_format={\"type\": \"json_object\"}\n)\n# Returns: {\"name\": \"John\", \"age\": 30}\n```\n\n### Production Setup with Fallbacks + Caching\n\n```python\ndef create_production_client():\n    return Portkey(\n        api_key=os.environ[\"PORTKEY_API_KEY\"],\n        config={\n            \"strategy\": {\"mode\": \"fallback\"},\n            \"targets\": [\n                {\n                    \"virtual_key\": os.environ[\"OPENAI_VIRTUAL_KEY\"],\n                    \"override_params\": {\"model\": \"gpt-4o\"},\n                    \"retry\": {\"attempts\": 2, \"on_status_codes\": [429, 500]}\n                },\n                {\n                    \"virtual_key\": os.environ[\"ANTHROPIC_VIRTUAL_KEY\"],\n                    \"override_params\": {\"model\": \"claude-3-5-sonnet-20241022\"}\n                }\n            ],\n            \"cache\": {\"mode\": \"semantic\", \"max_age\": 3600}\n        },\n        trace_id=\"production-session\",\n        metadata={\"environment\": \"production\"}\n    )\n```\n\n---\n\n## Best Practices\n\n1. **Use environment variables** - Never hardcode API keys\n2. **Implement fallbacks** - Always have backup providers for production\n3. **Use streaming** - Better UX for long responses\n4. **Add tracing** - Enable observability with trace IDs and metadata\n5. **Enable caching** - Reduce costs with semantic caching\n6. **Handle errors** - Implement retry logic with exponential backoff\n\n---\n\n## Resources\n\n- **Dashboard**: [app.portkey.ai](https:\u002F\u002Fapp.portkey.ai)\n- **Documentation**: [docs.portkey.ai](https:\u002F\u002Fdocs.portkey.ai)\n- **GitHub**: [github.com\u002Fportkey-ai\u002Fportkey-python-sdk](https:\u002F\u002Fgithub.com\u002Fportkey-ai\u002Fportkey-python-sdk)\n- **Discord**: [portkey.ai\u002Fdiscord](https:\u002F\u002Fportkey.ai\u002Fdiscord)\n",{"data":40,"body":48},{"name":4,"description":6,"license":32,"compatibility":41,"metadata":42},"Requires Python 3.8+. Works with pip, poetry, or uv.",{"author":43,"version":44,"sdk-version":45,"sdk-repo":46,"docs":47},"portkey-ai","1.0.0","2.1.0","https:\u002F\u002Fgithub.com\u002Fportkey-ai\u002Fportkey-python-sdk","https:\u002F\u002Fdocs.portkey.ai",{"type":49,"children":50},"root",[51,59,65,74,124,128,135,224,227,233,358,361,367,374,379,437,443,448,484,487,493,499,581,587,663,669,803,809,928,931,937,943,948,1120,1126,1131,1211,1217,1299,1305,1310,1473,1479,1523,1526,1530,1536,1541,1599,1605,1610,1684,1690,1751,1754,1760,1766,1836,1842,1917,1923,2094,2097,2103,2167,2170,2176,2243],{"type":52,"tag":53,"props":54,"children":55},"element","h1",{"id":4},[56],{"type":57,"value":58},"text","Portkey Python SDK",{"type":52,"tag":60,"props":61,"children":62},"p",{},[63],{"type":57,"value":64},"The Portkey Python SDK provides a unified interface to 200+ LLMs through the Portkey AI Gateway. Built on top of the OpenAI SDK for seamless compatibility, it adds production-grade features: automatic fallbacks, retries, load balancing, semantic caching, guardrails, and comprehensive observability.",{"type":52,"tag":60,"props":66,"children":67},{},[68],{"type":52,"tag":69,"props":70,"children":71},"strong",{},[72],{"type":57,"value":73},"Additional References:",{"type":52,"tag":75,"props":76,"children":77},"ul",{},[78,91,102,113],{"type":52,"tag":79,"props":80,"children":81},"li",{},[82,89],{"type":52,"tag":83,"props":84,"children":86},"a",{"href":85},"references\u002FAPI_REFERENCE.md",[87],{"type":57,"value":88},"API Reference",{"type":57,"value":90}," - Response structures, error handling",{"type":52,"tag":79,"props":92,"children":93},{},[94,100],{"type":52,"tag":83,"props":95,"children":97},{"href":96},"references\u002FADVANCED_FEATURES.md",[98],{"type":57,"value":99},"Advanced Features",{"type":57,"value":101}," - Tool calling, embeddings, audio, images",{"type":52,"tag":79,"props":103,"children":104},{},[105,111],{"type":52,"tag":83,"props":106,"children":108},{"href":107},"references\u002FINTEGRATIONS.md",[109],{"type":57,"value":110},"Framework Integrations",{"type":57,"value":112}," - LangChain, LlamaIndex, Strands, Google ADK",{"type":52,"tag":79,"props":114,"children":115},{},[116,122],{"type":52,"tag":83,"props":117,"children":119},{"href":118},"references\u002FPROVIDERS.md",[120],{"type":57,"value":121},"Provider Configuration",{"type":57,"value":123}," - Azure, AWS Bedrock, Vertex AI setup",{"type":52,"tag":125,"props":126,"children":127},"hr",{},[],{"type":52,"tag":129,"props":130,"children":132},"h2",{"id":131},"installation",[133],{"type":57,"value":134},"Installation",{"type":52,"tag":136,"props":137,"children":142},"pre",{"className":138,"code":139,"language":140,"meta":141,"style":141},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","pip install portkey-ai\n\n# Or with poetry\u002Fuv\npoetry add portkey-ai\nuv add portkey-ai\n","bash","",[143],{"type":52,"tag":144,"props":145,"children":146},"code",{"__ignoreMap":141},[147,169,179,189,207],{"type":52,"tag":148,"props":149,"children":151},"span",{"class":150,"line":29},"line",[152,158,164],{"type":52,"tag":148,"props":153,"children":155},{"style":154},"--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B",[156],{"type":57,"value":157},"pip",{"type":52,"tag":148,"props":159,"children":161},{"style":160},"--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D",[162],{"type":57,"value":163}," install",{"type":52,"tag":148,"props":165,"children":166},{"style":160},[167],{"type":57,"value":168}," portkey-ai\n",{"type":52,"tag":148,"props":170,"children":172},{"class":150,"line":171},2,[173],{"type":52,"tag":148,"props":174,"children":176},{"emptyLinePlaceholder":175},true,[177],{"type":57,"value":178},"\n",{"type":52,"tag":148,"props":180,"children":182},{"class":150,"line":181},3,[183],{"type":52,"tag":148,"props":184,"children":186},{"style":185},"--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic",[187],{"type":57,"value":188},"# Or with poetry\u002Fuv\n",{"type":52,"tag":148,"props":190,"children":192},{"class":150,"line":191},4,[193,198,203],{"type":52,"tag":148,"props":194,"children":195},{"style":154},[196],{"type":57,"value":197},"poetry",{"type":52,"tag":148,"props":199,"children":200},{"style":160},[201],{"type":57,"value":202}," add",{"type":52,"tag":148,"props":204,"children":205},{"style":160},[206],{"type":57,"value":168},{"type":52,"tag":148,"props":208,"children":210},{"class":150,"line":209},5,[211,216,220],{"type":52,"tag":148,"props":212,"children":213},{"style":154},[214],{"type":57,"value":215},"uv",{"type":52,"tag":148,"props":217,"children":218},{"style":160},[219],{"type":57,"value":202},{"type":52,"tag":148,"props":221,"children":222},{"style":160},[223],{"type":57,"value":168},{"type":52,"tag":125,"props":225,"children":226},{},[],{"type":52,"tag":129,"props":228,"children":230},{"id":229},"quick-start",[231],{"type":57,"value":232},"Quick Start",{"type":52,"tag":136,"props":234,"children":237},{"className":235,"code":236,"language":22,"meta":141,"style":141},"language-python shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","import os\nfrom portkey_ai import Portkey\n\nclient = Portkey(\n    api_key=os.environ[\"PORTKEY_API_KEY\"],\n    virtual_key=\"your-openai-virtual-key\"\n)\n\nresponse = client.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\n\nprint(response.choices[0].message.content)\n",[238],{"type":52,"tag":144,"props":239,"children":240},{"__ignoreMap":141},[241,249,257,264,272,280,289,298,306,315,324,333,341,349],{"type":52,"tag":148,"props":242,"children":243},{"class":150,"line":29},[244],{"type":52,"tag":148,"props":245,"children":246},{},[247],{"type":57,"value":248},"import os\n",{"type":52,"tag":148,"props":250,"children":251},{"class":150,"line":171},[252],{"type":52,"tag":148,"props":253,"children":254},{},[255],{"type":57,"value":256},"from portkey_ai import Portkey\n",{"type":52,"tag":148,"props":258,"children":259},{"class":150,"line":181},[260],{"type":52,"tag":148,"props":261,"children":262},{"emptyLinePlaceholder":175},[263],{"type":57,"value":178},{"type":52,"tag":148,"props":265,"children":266},{"class":150,"line":191},[267],{"type":52,"tag":148,"props":268,"children":269},{},[270],{"type":57,"value":271},"client = Portkey(\n",{"type":52,"tag":148,"props":273,"children":274},{"class":150,"line":209},[275],{"type":52,"tag":148,"props":276,"children":277},{},[278],{"type":57,"value":279},"    api_key=os.environ[\"PORTKEY_API_KEY\"],\n",{"type":52,"tag":148,"props":281,"children":283},{"class":150,"line":282},6,[284],{"type":52,"tag":148,"props":285,"children":286},{},[287],{"type":57,"value":288},"    virtual_key=\"your-openai-virtual-key\"\n",{"type":52,"tag":148,"props":290,"children":292},{"class":150,"line":291},7,[293],{"type":52,"tag":148,"props":294,"children":295},{},[296],{"type":57,"value":297},")\n",{"type":52,"tag":148,"props":299,"children":301},{"class":150,"line":300},8,[302],{"type":52,"tag":148,"props":303,"children":304},{"emptyLinePlaceholder":175},[305],{"type":57,"value":178},{"type":52,"tag":148,"props":307,"children":309},{"class":150,"line":308},9,[310],{"type":52,"tag":148,"props":311,"children":312},{},[313],{"type":57,"value":314},"response = client.chat.completions.create(\n",{"type":52,"tag":148,"props":316,"children":318},{"class":150,"line":317},10,[319],{"type":52,"tag":148,"props":320,"children":321},{},[322],{"type":57,"value":323},"    model=\"gpt-4o\",\n",{"type":52,"tag":148,"props":325,"children":327},{"class":150,"line":326},11,[328],{"type":52,"tag":148,"props":329,"children":330},{},[331],{"type":57,"value":332},"    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n",{"type":52,"tag":148,"props":334,"children":336},{"class":150,"line":335},12,[337],{"type":52,"tag":148,"props":338,"children":339},{},[340],{"type":57,"value":297},{"type":52,"tag":148,"props":342,"children":344},{"class":150,"line":343},13,[345],{"type":52,"tag":148,"props":346,"children":347},{"emptyLinePlaceholder":175},[348],{"type":57,"value":178},{"type":52,"tag":148,"props":350,"children":352},{"class":150,"line":351},14,[353],{"type":52,"tag":148,"props":354,"children":355},{},[356],{"type":57,"value":357},"print(response.choices[0].message.content)\n",{"type":52,"tag":125,"props":359,"children":360},{},[],{"type":52,"tag":129,"props":362,"children":364},{"id":363},"authentication",[365],{"type":57,"value":366},"Authentication",{"type":52,"tag":368,"props":369,"children":371},"h3",{"id":370},"api-key-virtual-key-recommended",[372],{"type":57,"value":373},"API Key + Virtual Key (Recommended)",{"type":52,"tag":60,"props":375,"children":376},{},[377],{"type":57,"value":378},"Virtual keys securely store provider API keys in Portkey's vault:",{"type":52,"tag":136,"props":380,"children":382},{"className":235,"code":381,"language":22,"meta":141,"style":141},"import os\nfrom portkey_ai import Portkey\n\nclient = Portkey(\n    api_key=os.environ[\"PORTKEY_API_KEY\"],  # From app.portkey.ai\n    virtual_key=\"openai-virtual-key-xxx\"     # From app.portkey.ai\u002Fvirtual-keys\n)\n",[383],{"type":52,"tag":144,"props":384,"children":385},{"__ignoreMap":141},[386,393,400,407,414,422,430],{"type":52,"tag":148,"props":387,"children":388},{"class":150,"line":29},[389],{"type":52,"tag":148,"props":390,"children":391},{},[392],{"type":57,"value":248},{"type":52,"tag":148,"props":394,"children":395},{"class":150,"line":171},[396],{"type":52,"tag":148,"props":397,"children":398},{},[399],{"type":57,"value":256},{"type":52,"tag":148,"props":401,"children":402},{"class":150,"line":181},[403],{"type":52,"tag":148,"props":404,"children":405},{"emptyLinePlaceholder":175},[406],{"type":57,"value":178},{"type":52,"tag":148,"props":408,"children":409},{"class":150,"line":191},[410],{"type":52,"tag":148,"props":411,"children":412},{},[413],{"type":57,"value":271},{"type":52,"tag":148,"props":415,"children":416},{"class":150,"line":209},[417],{"type":52,"tag":148,"props":418,"children":419},{},[420],{"type":57,"value":421},"    api_key=os.environ[\"PORTKEY_API_KEY\"],  # From app.portkey.ai\n",{"type":52,"tag":148,"props":423,"children":424},{"class":150,"line":282},[425],{"type":52,"tag":148,"props":426,"children":427},{},[428],{"type":57,"value":429},"    virtual_key=\"openai-virtual-key-xxx\"     # From app.portkey.ai\u002Fvirtual-keys\n",{"type":52,"tag":148,"props":431,"children":432},{"class":150,"line":291},[433],{"type":52,"tag":148,"props":434,"children":435},{},[436],{"type":57,"value":297},{"type":52,"tag":368,"props":438,"children":440},{"id":439},"using-config-ids",[441],{"type":57,"value":442},"Using Config IDs",{"type":52,"tag":60,"props":444,"children":445},{},[446],{"type":57,"value":447},"Pre-configure routing, fallbacks, and caching in the dashboard:",{"type":52,"tag":136,"props":449,"children":451},{"className":235,"code":450,"language":22,"meta":141,"style":141},"client = Portkey(\n    api_key=os.environ[\"PORTKEY_API_KEY\"],\n    config=\"pc-config-xxx\"  # Config ID from dashboard\n)\n",[452],{"type":52,"tag":144,"props":453,"children":454},{"__ignoreMap":141},[455,462,469,477],{"type":52,"tag":148,"props":456,"children":457},{"class":150,"line":29},[458],{"type":52,"tag":148,"props":459,"children":460},{},[461],{"type":57,"value":271},{"type":52,"tag":148,"props":463,"children":464},{"class":150,"line":171},[465],{"type":52,"tag":148,"props":466,"children":467},{},[468],{"type":57,"value":279},{"type":52,"tag":148,"props":470,"children":471},{"class":150,"line":181},[472],{"type":52,"tag":148,"props":473,"children":474},{},[475],{"type":57,"value":476},"    config=\"pc-config-xxx\"  # Config ID from dashboard\n",{"type":52,"tag":148,"props":478,"children":479},{"class":150,"line":191},[480],{"type":52,"tag":148,"props":481,"children":482},{},[483],{"type":57,"value":297},{"type":52,"tag":125,"props":485,"children":486},{},[],{"type":52,"tag":129,"props":488,"children":490},{"id":489},"chat-completions",[491],{"type":57,"value":492},"Chat Completions",{"type":52,"tag":368,"props":494,"children":496},{"id":495},"basic-request",[497],{"type":57,"value":498},"Basic Request",{"type":52,"tag":136,"props":500,"children":502},{"className":235,"code":501,"language":22,"meta":141,"style":141},"response = client.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[\n        {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n        {\"role\": \"user\", \"content\": \"Explain quantum computing briefly.\"}\n    ]\n)\n\nprint(response.choices[0].message.content)\nprint(f\"Tokens used: {response.usage.total_tokens}\")\n",[503],{"type":52,"tag":144,"props":504,"children":505},{"__ignoreMap":141},[506,513,520,528,536,544,552,559,566,573],{"type":52,"tag":148,"props":507,"children":508},{"class":150,"line":29},[509],{"type":52,"tag":148,"props":510,"children":511},{},[512],{"type":57,"value":314},{"type":52,"tag":148,"props":514,"children":515},{"class":150,"line":171},[516],{"type":52,"tag":148,"props":517,"children":518},{},[519],{"type":57,"value":323},{"type":52,"tag":148,"props":521,"children":522},{"class":150,"line":181},[523],{"type":52,"tag":148,"props":524,"children":525},{},[526],{"type":57,"value":527},"    messages=[\n",{"type":52,"tag":148,"props":529,"children":530},{"class":150,"line":191},[531],{"type":52,"tag":148,"props":532,"children":533},{},[534],{"type":57,"value":535},"        {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",{"type":52,"tag":148,"props":537,"children":538},{"class":150,"line":209},[539],{"type":52,"tag":148,"props":540,"children":541},{},[542],{"type":57,"value":543},"        {\"role\": \"user\", \"content\": \"Explain quantum computing briefly.\"}\n",{"type":52,"tag":148,"props":545,"children":546},{"class":150,"line":282},[547],{"type":52,"tag":148,"props":548,"children":549},{},[550],{"type":57,"value":551},"    ]\n",{"type":52,"tag":148,"props":553,"children":554},{"class":150,"line":291},[555],{"type":52,"tag":148,"props":556,"children":557},{},[558],{"type":57,"value":297},{"type":52,"tag":148,"props":560,"children":561},{"class":150,"line":300},[562],{"type":52,"tag":148,"props":563,"children":564},{"emptyLinePlaceholder":175},[565],{"type":57,"value":178},{"type":52,"tag":148,"props":567,"children":568},{"class":150,"line":308},[569],{"type":52,"tag":148,"props":570,"children":571},{},[572],{"type":57,"value":357},{"type":52,"tag":148,"props":574,"children":575},{"class":150,"line":317},[576],{"type":52,"tag":148,"props":577,"children":578},{},[579],{"type":57,"value":580},"print(f\"Tokens used: {response.usage.total_tokens}\")\n",{"type":52,"tag":368,"props":582,"children":584},{"id":583},"streaming",[585],{"type":57,"value":586},"Streaming",{"type":52,"tag":136,"props":588,"children":590},{"className":235,"code":589,"language":22,"meta":141,"style":141},"stream = client.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"Write a short story\"}],\n    stream=True\n)\n\nfor chunk in stream:\n    if chunk.choices[0].delta.content:\n        print(chunk.choices[0].delta.content, end=\"\", flush=True)\n",[591],{"type":52,"tag":144,"props":592,"children":593},{"__ignoreMap":141},[594,602,609,617,625,632,639,647,655],{"type":52,"tag":148,"props":595,"children":596},{"class":150,"line":29},[597],{"type":52,"tag":148,"props":598,"children":599},{},[600],{"type":57,"value":601},"stream = client.chat.completions.create(\n",{"type":52,"tag":148,"props":603,"children":604},{"class":150,"line":171},[605],{"type":52,"tag":148,"props":606,"children":607},{},[608],{"type":57,"value":323},{"type":52,"tag":148,"props":610,"children":611},{"class":150,"line":181},[612],{"type":52,"tag":148,"props":613,"children":614},{},[615],{"type":57,"value":616},"    messages=[{\"role\": \"user\", \"content\": \"Write a short story\"}],\n",{"type":52,"tag":148,"props":618,"children":619},{"class":150,"line":191},[620],{"type":52,"tag":148,"props":621,"children":622},{},[623],{"type":57,"value":624},"    stream=True\n",{"type":52,"tag":148,"props":626,"children":627},{"class":150,"line":209},[628],{"type":52,"tag":148,"props":629,"children":630},{},[631],{"type":57,"value":297},{"type":52,"tag":148,"props":633,"children":634},{"class":150,"line":282},[635],{"type":52,"tag":148,"props":636,"children":637},{"emptyLinePlaceholder":175},[638],{"type":57,"value":178},{"type":52,"tag":148,"props":640,"children":641},{"class":150,"line":291},[642],{"type":52,"tag":148,"props":643,"children":644},{},[645],{"type":57,"value":646},"for chunk in stream:\n",{"type":52,"tag":148,"props":648,"children":649},{"class":150,"line":300},[650],{"type":52,"tag":148,"props":651,"children":652},{},[653],{"type":57,"value":654},"    if chunk.choices[0].delta.content:\n",{"type":52,"tag":148,"props":656,"children":657},{"class":150,"line":308},[658],{"type":52,"tag":148,"props":659,"children":660},{},[661],{"type":57,"value":662},"        print(chunk.choices[0].delta.content, end=\"\", flush=True)\n",{"type":52,"tag":368,"props":664,"children":666},{"id":665},"async-support",[667],{"type":57,"value":668},"Async Support",{"type":52,"tag":136,"props":670,"children":672},{"className":235,"code":671,"language":22,"meta":141,"style":141},"import asyncio\nfrom portkey_ai import AsyncPortkey\n\nasync def main():\n    client = AsyncPortkey(\n        api_key=os.environ[\"PORTKEY_API_KEY\"],\n        virtual_key=\"openai-key\"\n    )\n    \n    response = await client.chat.completions.create(\n        model=\"gpt-4o\",\n        messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n    )\n    print(response.choices[0].message.content)\n\nasyncio.run(main())\n",[673],{"type":52,"tag":144,"props":674,"children":675},{"__ignoreMap":141},[676,684,692,699,707,715,723,731,739,747,755,763,771,778,786,794],{"type":52,"tag":148,"props":677,"children":678},{"class":150,"line":29},[679],{"type":52,"tag":148,"props":680,"children":681},{},[682],{"type":57,"value":683},"import asyncio\n",{"type":52,"tag":148,"props":685,"children":686},{"class":150,"line":171},[687],{"type":52,"tag":148,"props":688,"children":689},{},[690],{"type":57,"value":691},"from portkey_ai import AsyncPortkey\n",{"type":52,"tag":148,"props":693,"children":694},{"class":150,"line":181},[695],{"type":52,"tag":148,"props":696,"children":697},{"emptyLinePlaceholder":175},[698],{"type":57,"value":178},{"type":52,"tag":148,"props":700,"children":701},{"class":150,"line":191},[702],{"type":52,"tag":148,"props":703,"children":704},{},[705],{"type":57,"value":706},"async def main():\n",{"type":52,"tag":148,"props":708,"children":709},{"class":150,"line":209},[710],{"type":52,"tag":148,"props":711,"children":712},{},[713],{"type":57,"value":714},"    client = AsyncPortkey(\n",{"type":52,"tag":148,"props":716,"children":717},{"class":150,"line":282},[718],{"type":52,"tag":148,"props":719,"children":720},{},[721],{"type":57,"value":722},"        api_key=os.environ[\"PORTKEY_API_KEY\"],\n",{"type":52,"tag":148,"props":724,"children":725},{"class":150,"line":291},[726],{"type":52,"tag":148,"props":727,"children":728},{},[729],{"type":57,"value":730},"        virtual_key=\"openai-key\"\n",{"type":52,"tag":148,"props":732,"children":733},{"class":150,"line":300},[734],{"type":52,"tag":148,"props":735,"children":736},{},[737],{"type":57,"value":738},"    )\n",{"type":52,"tag":148,"props":740,"children":741},{"class":150,"line":308},[742],{"type":52,"tag":148,"props":743,"children":744},{},[745],{"type":57,"value":746},"    \n",{"type":52,"tag":148,"props":748,"children":749},{"class":150,"line":317},[750],{"type":52,"tag":148,"props":751,"children":752},{},[753],{"type":57,"value":754},"    response = await client.chat.completions.create(\n",{"type":52,"tag":148,"props":756,"children":757},{"class":150,"line":326},[758],{"type":52,"tag":148,"props":759,"children":760},{},[761],{"type":57,"value":762},"        model=\"gpt-4o\",\n",{"type":52,"tag":148,"props":764,"children":765},{"class":150,"line":335},[766],{"type":52,"tag":148,"props":767,"children":768},{},[769],{"type":57,"value":770},"        messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n",{"type":52,"tag":148,"props":772,"children":773},{"class":150,"line":343},[774],{"type":52,"tag":148,"props":775,"children":776},{},[777],{"type":57,"value":738},{"type":52,"tag":148,"props":779,"children":780},{"class":150,"line":351},[781],{"type":52,"tag":148,"props":782,"children":783},{},[784],{"type":57,"value":785},"    print(response.choices[0].message.content)\n",{"type":52,"tag":148,"props":787,"children":789},{"class":150,"line":788},15,[790],{"type":52,"tag":148,"props":791,"children":792},{"emptyLinePlaceholder":175},[793],{"type":57,"value":178},{"type":52,"tag":148,"props":795,"children":797},{"class":150,"line":796},16,[798],{"type":52,"tag":148,"props":799,"children":800},{},[801],{"type":57,"value":802},"asyncio.run(main())\n",{"type":52,"tag":368,"props":804,"children":806},{"id":805},"async-streaming",[807],{"type":57,"value":808},"Async Streaming",{"type":52,"tag":136,"props":810,"children":812},{"className":235,"code":811,"language":22,"meta":141,"style":141},"async def stream_response():\n    client = AsyncPortkey(\n        api_key=os.environ[\"PORTKEY_API_KEY\"],\n        virtual_key=\"openai-key\"\n    )\n    \n    stream = await client.chat.completions.create(\n        model=\"gpt-4o\",\n        messages=[{\"role\": \"user\", \"content\": \"Write a poem\"}],\n        stream=True\n    )\n    \n    async for chunk in stream:\n        if chunk.choices[0].delta.content:\n            print(chunk.choices[0].delta.content, end=\"\", flush=True)\n",[813],{"type":52,"tag":144,"props":814,"children":815},{"__ignoreMap":141},[816,824,831,838,845,852,859,867,874,882,890,897,904,912,920],{"type":52,"tag":148,"props":817,"children":818},{"class":150,"line":29},[819],{"type":52,"tag":148,"props":820,"children":821},{},[822],{"type":57,"value":823},"async def stream_response():\n",{"type":52,"tag":148,"props":825,"children":826},{"class":150,"line":171},[827],{"type":52,"tag":148,"props":828,"children":829},{},[830],{"type":57,"value":714},{"type":52,"tag":148,"props":832,"children":833},{"class":150,"line":181},[834],{"type":52,"tag":148,"props":835,"children":836},{},[837],{"type":57,"value":722},{"type":52,"tag":148,"props":839,"children":840},{"class":150,"line":191},[841],{"type":52,"tag":148,"props":842,"children":843},{},[844],{"type":57,"value":730},{"type":52,"tag":148,"props":846,"children":847},{"class":150,"line":209},[848],{"type":52,"tag":148,"props":849,"children":850},{},[851],{"type":57,"value":738},{"type":52,"tag":148,"props":853,"children":854},{"class":150,"line":282},[855],{"type":52,"tag":148,"props":856,"children":857},{},[858],{"type":57,"value":746},{"type":52,"tag":148,"props":860,"children":861},{"class":150,"line":291},[862],{"type":52,"tag":148,"props":863,"children":864},{},[865],{"type":57,"value":866},"    stream = await client.chat.completions.create(\n",{"type":52,"tag":148,"props":868,"children":869},{"class":150,"line":300},[870],{"type":52,"tag":148,"props":871,"children":872},{},[873],{"type":57,"value":762},{"type":52,"tag":148,"props":875,"children":876},{"class":150,"line":308},[877],{"type":52,"tag":148,"props":878,"children":879},{},[880],{"type":57,"value":881},"        messages=[{\"role\": \"user\", \"content\": \"Write a poem\"}],\n",{"type":52,"tag":148,"props":883,"children":884},{"class":150,"line":317},[885],{"type":52,"tag":148,"props":886,"children":887},{},[888],{"type":57,"value":889},"        stream=True\n",{"type":52,"tag":148,"props":891,"children":892},{"class":150,"line":326},[893],{"type":52,"tag":148,"props":894,"children":895},{},[896],{"type":57,"value":738},{"type":52,"tag":148,"props":898,"children":899},{"class":150,"line":335},[900],{"type":52,"tag":148,"props":901,"children":902},{},[903],{"type":57,"value":746},{"type":52,"tag":148,"props":905,"children":906},{"class":150,"line":343},[907],{"type":52,"tag":148,"props":908,"children":909},{},[910],{"type":57,"value":911},"    async for chunk in stream:\n",{"type":52,"tag":148,"props":913,"children":914},{"class":150,"line":351},[915],{"type":52,"tag":148,"props":916,"children":917},{},[918],{"type":57,"value":919},"        if chunk.choices[0].delta.content:\n",{"type":52,"tag":148,"props":921,"children":922},{"class":150,"line":788},[923],{"type":52,"tag":148,"props":924,"children":925},{},[926],{"type":57,"value":927},"            print(chunk.choices[0].delta.content, end=\"\", flush=True)\n",{"type":52,"tag":125,"props":929,"children":930},{},[],{"type":52,"tag":129,"props":932,"children":934},{"id":933},"gateway-features",[935],{"type":57,"value":936},"Gateway Features",{"type":52,"tag":368,"props":938,"children":940},{"id":939},"fallbacks",[941],{"type":57,"value":942},"Fallbacks",{"type":52,"tag":60,"props":944,"children":945},{},[946],{"type":57,"value":947},"Automatic failover when a provider fails:",{"type":52,"tag":136,"props":949,"children":951},{"className":235,"code":950,"language":22,"meta":141,"style":141},"client = Portkey(\n    api_key=os.environ[\"PORTKEY_API_KEY\"],\n    config={\n        \"strategy\": {\"mode\": \"fallback\"},\n        \"targets\": [\n            {\n                \"virtual_key\": \"openai-key\",\n                \"override_params\": {\"model\": \"gpt-4o\"}\n            },\n            {\n                \"virtual_key\": \"anthropic-key\",\n                \"override_params\": {\"model\": \"claude-3-5-sonnet-20241022\"}\n            }\n        ]\n    }\n)\n\n# If OpenAI fails, automatically tries Anthropic\nresponse = client.chat.completions.create(\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\n",[952],{"type":52,"tag":144,"props":953,"children":954},{"__ignoreMap":141},[955,962,969,977,985,993,1001,1009,1017,1025,1032,1040,1048,1056,1064,1072,1079,1087,1096,1104,1112],{"type":52,"tag":148,"props":956,"children":957},{"class":150,"line":29},[958],{"type":52,"tag":148,"props":959,"children":960},{},[961],{"type":57,"value":271},{"type":52,"tag":148,"props":963,"children":964},{"class":150,"line":171},[965],{"type":52,"tag":148,"props":966,"children":967},{},[968],{"type":57,"value":279},{"type":52,"tag":148,"props":970,"children":971},{"class":150,"line":181},[972],{"type":52,"tag":148,"props":973,"children":974},{},[975],{"type":57,"value":976},"    config={\n",{"type":52,"tag":148,"props":978,"children":979},{"class":150,"line":191},[980],{"type":52,"tag":148,"props":981,"children":982},{},[983],{"type":57,"value":984},"        \"strategy\": {\"mode\": \"fallback\"},\n",{"type":52,"tag":148,"props":986,"children":987},{"class":150,"line":209},[988],{"type":52,"tag":148,"props":989,"children":990},{},[991],{"type":57,"value":992},"        \"targets\": [\n",{"type":52,"tag":148,"props":994,"children":995},{"class":150,"line":282},[996],{"type":52,"tag":148,"props":997,"children":998},{},[999],{"type":57,"value":1000},"            {\n",{"type":52,"tag":148,"props":1002,"children":1003},{"class":150,"line":291},[1004],{"type":52,"tag":148,"props":1005,"children":1006},{},[1007],{"type":57,"value":1008},"                \"virtual_key\": \"openai-key\",\n",{"type":52,"tag":148,"props":1010,"children":1011},{"class":150,"line":300},[1012],{"type":52,"tag":148,"props":1013,"children":1014},{},[1015],{"type":57,"value":1016},"                \"override_params\": {\"model\": \"gpt-4o\"}\n",{"type":52,"tag":148,"props":1018,"children":1019},{"class":150,"line":308},[1020],{"type":52,"tag":148,"props":1021,"children":1022},{},[1023],{"type":57,"value":1024},"            },\n",{"type":52,"tag":148,"props":1026,"children":1027},{"class":150,"line":317},[1028],{"type":52,"tag":148,"props":1029,"children":1030},{},[1031],{"type":57,"value":1000},{"type":52,"tag":148,"props":1033,"children":1034},{"class":150,"line":326},[1035],{"type":52,"tag":148,"props":1036,"children":1037},{},[1038],{"type":57,"value":1039},"                \"virtual_key\": \"anthropic-key\",\n",{"type":52,"tag":148,"props":1041,"children":1042},{"class":150,"line":335},[1043],{"type":52,"tag":148,"props":1044,"children":1045},{},[1046],{"type":57,"value":1047},"                \"override_params\": {\"model\": \"claude-3-5-sonnet-20241022\"}\n",{"type":52,"tag":148,"props":1049,"children":1050},{"class":150,"line":343},[1051],{"type":52,"tag":148,"props":1052,"children":1053},{},[1054],{"type":57,"value":1055},"            }\n",{"type":52,"tag":148,"props":1057,"children":1058},{"class":150,"line":351},[1059],{"type":52,"tag":148,"props":1060,"children":1061},{},[1062],{"type":57,"value":1063},"        ]\n",{"type":52,"tag":148,"props":1065,"children":1066},{"class":150,"line":788},[1067],{"type":52,"tag":148,"props":1068,"children":1069},{},[1070],{"type":57,"value":1071},"    }\n",{"type":52,"tag":148,"props":1073,"children":1074},{"class":150,"line":796},[1075],{"type":52,"tag":148,"props":1076,"children":1077},{},[1078],{"type":57,"value":297},{"type":52,"tag":148,"props":1080,"children":1082},{"class":150,"line":1081},17,[1083],{"type":52,"tag":148,"props":1084,"children":1085},{"emptyLinePlaceholder":175},[1086],{"type":57,"value":178},{"type":52,"tag":148,"props":1088,"children":1090},{"class":150,"line":1089},18,[1091],{"type":52,"tag":148,"props":1092,"children":1093},{},[1094],{"type":57,"value":1095},"# If OpenAI fails, automatically tries Anthropic\n",{"type":52,"tag":148,"props":1097,"children":1099},{"class":150,"line":1098},19,[1100],{"type":52,"tag":148,"props":1101,"children":1102},{},[1103],{"type":57,"value":314},{"type":52,"tag":148,"props":1105,"children":1107},{"class":150,"line":1106},20,[1108],{"type":52,"tag":148,"props":1109,"children":1110},{},[1111],{"type":57,"value":332},{"type":52,"tag":148,"props":1113,"children":1115},{"class":150,"line":1114},21,[1116],{"type":52,"tag":148,"props":1117,"children":1118},{},[1119],{"type":57,"value":297},{"type":52,"tag":368,"props":1121,"children":1123},{"id":1122},"load-balancing",[1124],{"type":57,"value":1125},"Load Balancing",{"type":52,"tag":60,"props":1127,"children":1128},{},[1129],{"type":57,"value":1130},"Distribute traffic across providers:",{"type":52,"tag":136,"props":1132,"children":1134},{"className":235,"code":1133,"language":22,"meta":141,"style":141},"client = Portkey(\n    api_key=os.environ[\"PORTKEY_API_KEY\"],\n    config={\n        \"strategy\": {\"mode\": \"loadbalance\"},\n        \"targets\": [\n            {\"virtual_key\": \"openai-key-1\", \"weight\": 0.7},\n            {\"virtual_key\": \"openai-key-2\", \"weight\": 0.3}\n        ]\n    }\n)\n",[1135],{"type":52,"tag":144,"props":1136,"children":1137},{"__ignoreMap":141},[1138,1145,1152,1159,1167,1174,1182,1190,1197,1204],{"type":52,"tag":148,"props":1139,"children":1140},{"class":150,"line":29},[1141],{"type":52,"tag":148,"props":1142,"children":1143},{},[1144],{"type":57,"value":271},{"type":52,"tag":148,"props":1146,"children":1147},{"class":150,"line":171},[1148],{"type":52,"tag":148,"props":1149,"children":1150},{},[1151],{"type":57,"value":279},{"type":52,"tag":148,"props":1153,"children":1154},{"class":150,"line":181},[1155],{"type":52,"tag":148,"props":1156,"children":1157},{},[1158],{"type":57,"value":976},{"type":52,"tag":148,"props":1160,"children":1161},{"class":150,"line":191},[1162],{"type":52,"tag":148,"props":1163,"children":1164},{},[1165],{"type":57,"value":1166},"        \"strategy\": {\"mode\": \"loadbalance\"},\n",{"type":52,"tag":148,"props":1168,"children":1169},{"class":150,"line":209},[1170],{"type":52,"tag":148,"props":1171,"children":1172},{},[1173],{"type":57,"value":992},{"type":52,"tag":148,"props":1175,"children":1176},{"class":150,"line":282},[1177],{"type":52,"tag":148,"props":1178,"children":1179},{},[1180],{"type":57,"value":1181},"            {\"virtual_key\": \"openai-key-1\", \"weight\": 0.7},\n",{"type":52,"tag":148,"props":1183,"children":1184},{"class":150,"line":291},[1185],{"type":52,"tag":148,"props":1186,"children":1187},{},[1188],{"type":57,"value":1189},"            {\"virtual_key\": \"openai-key-2\", \"weight\": 0.3}\n",{"type":52,"tag":148,"props":1191,"children":1192},{"class":150,"line":300},[1193],{"type":52,"tag":148,"props":1194,"children":1195},{},[1196],{"type":57,"value":1063},{"type":52,"tag":148,"props":1198,"children":1199},{"class":150,"line":308},[1200],{"type":52,"tag":148,"props":1201,"children":1202},{},[1203],{"type":57,"value":1071},{"type":52,"tag":148,"props":1205,"children":1206},{"class":150,"line":317},[1207],{"type":52,"tag":148,"props":1208,"children":1209},{},[1210],{"type":57,"value":297},{"type":52,"tag":368,"props":1212,"children":1214},{"id":1213},"automatic-retries",[1215],{"type":57,"value":1216},"Automatic Retries",{"type":52,"tag":136,"props":1218,"children":1220},{"className":235,"code":1219,"language":22,"meta":141,"style":141},"client = Portkey(\n    api_key=os.environ[\"PORTKEY_API_KEY\"],\n    config={\n        \"retry\": {\n            \"attempts\": 3,\n            \"on_status_codes\": [429, 500, 502, 503, 504]\n        },\n        \"virtual_key\": \"openai-key\"\n    }\n)\n",[1221],{"type":52,"tag":144,"props":1222,"children":1223},{"__ignoreMap":141},[1224,1231,1238,1245,1253,1261,1269,1277,1285,1292],{"type":52,"tag":148,"props":1225,"children":1226},{"class":150,"line":29},[1227],{"type":52,"tag":148,"props":1228,"children":1229},{},[1230],{"type":57,"value":271},{"type":52,"tag":148,"props":1232,"children":1233},{"class":150,"line":171},[1234],{"type":52,"tag":148,"props":1235,"children":1236},{},[1237],{"type":57,"value":279},{"type":52,"tag":148,"props":1239,"children":1240},{"class":150,"line":181},[1241],{"type":52,"tag":148,"props":1242,"children":1243},{},[1244],{"type":57,"value":976},{"type":52,"tag":148,"props":1246,"children":1247},{"class":150,"line":191},[1248],{"type":52,"tag":148,"props":1249,"children":1250},{},[1251],{"type":57,"value":1252},"        \"retry\": {\n",{"type":52,"tag":148,"props":1254,"children":1255},{"class":150,"line":209},[1256],{"type":52,"tag":148,"props":1257,"children":1258},{},[1259],{"type":57,"value":1260},"            \"attempts\": 3,\n",{"type":52,"tag":148,"props":1262,"children":1263},{"class":150,"line":282},[1264],{"type":52,"tag":148,"props":1265,"children":1266},{},[1267],{"type":57,"value":1268},"            \"on_status_codes\": [429, 500, 502, 503, 504]\n",{"type":52,"tag":148,"props":1270,"children":1271},{"class":150,"line":291},[1272],{"type":52,"tag":148,"props":1273,"children":1274},{},[1275],{"type":57,"value":1276},"        },\n",{"type":52,"tag":148,"props":1278,"children":1279},{"class":150,"line":300},[1280],{"type":52,"tag":148,"props":1281,"children":1282},{},[1283],{"type":57,"value":1284},"        \"virtual_key\": \"openai-key\"\n",{"type":52,"tag":148,"props":1286,"children":1287},{"class":150,"line":308},[1288],{"type":52,"tag":148,"props":1289,"children":1290},{},[1291],{"type":57,"value":1071},{"type":52,"tag":148,"props":1293,"children":1294},{"class":150,"line":317},[1295],{"type":52,"tag":148,"props":1296,"children":1297},{},[1298],{"type":57,"value":297},{"type":52,"tag":368,"props":1300,"children":1302},{"id":1301},"semantic-caching",[1303],{"type":57,"value":1304},"Semantic Caching",{"type":52,"tag":60,"props":1306,"children":1307},{},[1308],{"type":57,"value":1309},"Reduce costs and latency with intelligent caching:",{"type":52,"tag":136,"props":1311,"children":1313},{"className":235,"code":1312,"language":22,"meta":141,"style":141},"client = Portkey(\n    api_key=os.environ[\"PORTKEY_API_KEY\"],\n    config={\n        \"cache\": {\n            \"mode\": \"semantic\",  # or \"simple\" for exact match\n            \"max_age\": 3600      # TTL in seconds\n        },\n        \"virtual_key\": \"openai-key\"\n    }\n)\n\n# Similar queries return cached responses\nresponse1 = client.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"What is the capital of France?\"}]\n)\n\nresponse2 = client.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"Tell me France's capital\"}]\n)  # Returns cached response\n",[1314],{"type":52,"tag":144,"props":1315,"children":1316},{"__ignoreMap":141},[1317,1324,1331,1338,1346,1354,1362,1369,1376,1383,1390,1397,1405,1413,1420,1428,1435,1442,1450,1457,1465],{"type":52,"tag":148,"props":1318,"children":1319},{"class":150,"line":29},[1320],{"type":52,"tag":148,"props":1321,"children":1322},{},[1323],{"type":57,"value":271},{"type":52,"tag":148,"props":1325,"children":1326},{"class":150,"line":171},[1327],{"type":52,"tag":148,"props":1328,"children":1329},{},[1330],{"type":57,"value":279},{"type":52,"tag":148,"props":1332,"children":1333},{"class":150,"line":181},[1334],{"type":52,"tag":148,"props":1335,"children":1336},{},[1337],{"type":57,"value":976},{"type":52,"tag":148,"props":1339,"children":1340},{"class":150,"line":191},[1341],{"type":52,"tag":148,"props":1342,"children":1343},{},[1344],{"type":57,"value":1345},"        \"cache\": {\n",{"type":52,"tag":148,"props":1347,"children":1348},{"class":150,"line":209},[1349],{"type":52,"tag":148,"props":1350,"children":1351},{},[1352],{"type":57,"value":1353},"            \"mode\": \"semantic\",  # or \"simple\" for exact match\n",{"type":52,"tag":148,"props":1355,"children":1356},{"class":150,"line":282},[1357],{"type":52,"tag":148,"props":1358,"children":1359},{},[1360],{"type":57,"value":1361},"            \"max_age\": 3600      # TTL in seconds\n",{"type":52,"tag":148,"props":1363,"children":1364},{"class":150,"line":291},[1365],{"type":52,"tag":148,"props":1366,"children":1367},{},[1368],{"type":57,"value":1276},{"type":52,"tag":148,"props":1370,"children":1371},{"class":150,"line":300},[1372],{"type":52,"tag":148,"props":1373,"children":1374},{},[1375],{"type":57,"value":1284},{"type":52,"tag":148,"props":1377,"children":1378},{"class":150,"line":308},[1379],{"type":52,"tag":148,"props":1380,"children":1381},{},[1382],{"type":57,"value":1071},{"type":52,"tag":148,"props":1384,"children":1385},{"class":150,"line":317},[1386],{"type":52,"tag":148,"props":1387,"children":1388},{},[1389],{"type":57,"value":297},{"type":52,"tag":148,"props":1391,"children":1392},{"class":150,"line":326},[1393],{"type":52,"tag":148,"props":1394,"children":1395},{"emptyLinePlaceholder":175},[1396],{"type":57,"value":178},{"type":52,"tag":148,"props":1398,"children":1399},{"class":150,"line":335},[1400],{"type":52,"tag":148,"props":1401,"children":1402},{},[1403],{"type":57,"value":1404},"# Similar queries return cached responses\n",{"type":52,"tag":148,"props":1406,"children":1407},{"class":150,"line":343},[1408],{"type":52,"tag":148,"props":1409,"children":1410},{},[1411],{"type":57,"value":1412},"response1 = client.chat.completions.create(\n",{"type":52,"tag":148,"props":1414,"children":1415},{"class":150,"line":351},[1416],{"type":52,"tag":148,"props":1417,"children":1418},{},[1419],{"type":57,"value":323},{"type":52,"tag":148,"props":1421,"children":1422},{"class":150,"line":788},[1423],{"type":52,"tag":148,"props":1424,"children":1425},{},[1426],{"type":57,"value":1427},"    messages=[{\"role\": \"user\", \"content\": \"What is the capital of France?\"}]\n",{"type":52,"tag":148,"props":1429,"children":1430},{"class":150,"line":796},[1431],{"type":52,"tag":148,"props":1432,"children":1433},{},[1434],{"type":57,"value":297},{"type":52,"tag":148,"props":1436,"children":1437},{"class":150,"line":1081},[1438],{"type":52,"tag":148,"props":1439,"children":1440},{"emptyLinePlaceholder":175},[1441],{"type":57,"value":178},{"type":52,"tag":148,"props":1443,"children":1444},{"class":150,"line":1089},[1445],{"type":52,"tag":148,"props":1446,"children":1447},{},[1448],{"type":57,"value":1449},"response2 = client.chat.completions.create(\n",{"type":52,"tag":148,"props":1451,"children":1452},{"class":150,"line":1098},[1453],{"type":52,"tag":148,"props":1454,"children":1455},{},[1456],{"type":57,"value":323},{"type":52,"tag":148,"props":1458,"children":1459},{"class":150,"line":1106},[1460],{"type":52,"tag":148,"props":1461,"children":1462},{},[1463],{"type":57,"value":1464},"    messages=[{\"role\": \"user\", \"content\": \"Tell me France's capital\"}]\n",{"type":52,"tag":148,"props":1466,"children":1467},{"class":150,"line":1114},[1468],{"type":52,"tag":148,"props":1469,"children":1470},{},[1471],{"type":57,"value":1472},")  # Returns cached response\n",{"type":52,"tag":368,"props":1474,"children":1476},{"id":1475},"request-timeout",[1477],{"type":57,"value":1478},"Request Timeout",{"type":52,"tag":136,"props":1480,"children":1482},{"className":235,"code":1481,"language":22,"meta":141,"style":141},"client = Portkey(\n    api_key=os.environ[\"PORTKEY_API_KEY\"],\n    virtual_key=\"openai-key\",\n    request_timeout=30  # 30 seconds\n)\n",[1483],{"type":52,"tag":144,"props":1484,"children":1485},{"__ignoreMap":141},[1486,1493,1500,1508,1516],{"type":52,"tag":148,"props":1487,"children":1488},{"class":150,"line":29},[1489],{"type":52,"tag":148,"props":1490,"children":1491},{},[1492],{"type":57,"value":271},{"type":52,"tag":148,"props":1494,"children":1495},{"class":150,"line":171},[1496],{"type":52,"tag":148,"props":1497,"children":1498},{},[1499],{"type":57,"value":279},{"type":52,"tag":148,"props":1501,"children":1502},{"class":150,"line":181},[1503],{"type":52,"tag":148,"props":1504,"children":1505},{},[1506],{"type":57,"value":1507},"    virtual_key=\"openai-key\",\n",{"type":52,"tag":148,"props":1509,"children":1510},{"class":150,"line":191},[1511],{"type":52,"tag":148,"props":1512,"children":1513},{},[1514],{"type":57,"value":1515},"    request_timeout=30  # 30 seconds\n",{"type":52,"tag":148,"props":1517,"children":1518},{"class":150,"line":209},[1519],{"type":52,"tag":148,"props":1520,"children":1521},{},[1522],{"type":57,"value":297},{"type":52,"tag":125,"props":1524,"children":1525},{},[],{"type":52,"tag":129,"props":1527,"children":1528},{"id":15},[1529],{"type":57,"value":14},{"type":52,"tag":368,"props":1531,"children":1533},{"id":1532},"trace-ids",[1534],{"type":57,"value":1535},"Trace IDs",{"type":52,"tag":60,"props":1537,"children":1538},{},[1539],{"type":57,"value":1540},"Link related requests for debugging:",{"type":52,"tag":136,"props":1542,"children":1544},{"className":235,"code":1543,"language":22,"meta":141,"style":141},"import uuid\n\nclient = Portkey(\n    api_key=os.environ[\"PORTKEY_API_KEY\"],\n    virtual_key=\"openai-key\",\n    trace_id=str(uuid.uuid4())\n)\n",[1545],{"type":52,"tag":144,"props":1546,"children":1547},{"__ignoreMap":141},[1548,1556,1563,1570,1577,1584,1592],{"type":52,"tag":148,"props":1549,"children":1550},{"class":150,"line":29},[1551],{"type":52,"tag":148,"props":1552,"children":1553},{},[1554],{"type":57,"value":1555},"import uuid\n",{"type":52,"tag":148,"props":1557,"children":1558},{"class":150,"line":171},[1559],{"type":52,"tag":148,"props":1560,"children":1561},{"emptyLinePlaceholder":175},[1562],{"type":57,"value":178},{"type":52,"tag":148,"props":1564,"children":1565},{"class":150,"line":181},[1566],{"type":52,"tag":148,"props":1567,"children":1568},{},[1569],{"type":57,"value":271},{"type":52,"tag":148,"props":1571,"children":1572},{"class":150,"line":191},[1573],{"type":52,"tag":148,"props":1574,"children":1575},{},[1576],{"type":57,"value":279},{"type":52,"tag":148,"props":1578,"children":1579},{"class":150,"line":209},[1580],{"type":52,"tag":148,"props":1581,"children":1582},{},[1583],{"type":57,"value":1507},{"type":52,"tag":148,"props":1585,"children":1586},{"class":150,"line":282},[1587],{"type":52,"tag":148,"props":1588,"children":1589},{},[1590],{"type":57,"value":1591},"    trace_id=str(uuid.uuid4())\n",{"type":52,"tag":148,"props":1593,"children":1594},{"class":150,"line":291},[1595],{"type":52,"tag":148,"props":1596,"children":1597},{},[1598],{"type":57,"value":297},{"type":52,"tag":368,"props":1600,"children":1602},{"id":1601},"custom-metadata",[1603],{"type":57,"value":1604},"Custom Metadata",{"type":52,"tag":60,"props":1606,"children":1607},{},[1608],{"type":57,"value":1609},"Add searchable metadata to requests:",{"type":52,"tag":136,"props":1611,"children":1613},{"className":235,"code":1612,"language":22,"meta":141,"style":141},"client = Portkey(\n    api_key=os.environ[\"PORTKEY_API_KEY\"],\n    virtual_key=\"openai-key\",\n    metadata={\n        \"user_id\": \"user-123\",\n        \"session_id\": \"session-456\",\n        \"environment\": \"production\"\n    }\n)\n",[1614],{"type":52,"tag":144,"props":1615,"children":1616},{"__ignoreMap":141},[1617,1624,1631,1638,1646,1654,1662,1670,1677],{"type":52,"tag":148,"props":1618,"children":1619},{"class":150,"line":29},[1620],{"type":52,"tag":148,"props":1621,"children":1622},{},[1623],{"type":57,"value":271},{"type":52,"tag":148,"props":1625,"children":1626},{"class":150,"line":171},[1627],{"type":52,"tag":148,"props":1628,"children":1629},{},[1630],{"type":57,"value":279},{"type":52,"tag":148,"props":1632,"children":1633},{"class":150,"line":181},[1634],{"type":52,"tag":148,"props":1635,"children":1636},{},[1637],{"type":57,"value":1507},{"type":52,"tag":148,"props":1639,"children":1640},{"class":150,"line":191},[1641],{"type":52,"tag":148,"props":1642,"children":1643},{},[1644],{"type":57,"value":1645},"    metadata={\n",{"type":52,"tag":148,"props":1647,"children":1648},{"class":150,"line":209},[1649],{"type":52,"tag":148,"props":1650,"children":1651},{},[1652],{"type":57,"value":1653},"        \"user_id\": \"user-123\",\n",{"type":52,"tag":148,"props":1655,"children":1656},{"class":150,"line":282},[1657],{"type":52,"tag":148,"props":1658,"children":1659},{},[1660],{"type":57,"value":1661},"        \"session_id\": \"session-456\",\n",{"type":52,"tag":148,"props":1663,"children":1664},{"class":150,"line":291},[1665],{"type":52,"tag":148,"props":1666,"children":1667},{},[1668],{"type":57,"value":1669},"        \"environment\": \"production\"\n",{"type":52,"tag":148,"props":1671,"children":1672},{"class":150,"line":300},[1673],{"type":52,"tag":148,"props":1674,"children":1675},{},[1676],{"type":57,"value":1071},{"type":52,"tag":148,"props":1678,"children":1679},{"class":150,"line":308},[1680],{"type":52,"tag":148,"props":1681,"children":1682},{},[1683],{"type":57,"value":297},{"type":52,"tag":368,"props":1685,"children":1687},{"id":1686},"per-request-options",[1688],{"type":57,"value":1689},"Per-Request Options",{"type":52,"tag":136,"props":1691,"children":1693},{"className":235,"code":1692,"language":22,"meta":141,"style":141},"response = client.with_options(\n    trace_id=\"unique-trace-id\",\n    metadata={\"request_type\": \"summarization\"}\n).chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[{\"role\": \"user\", \"content\": \"Summarize this...\"}]\n)\n",[1694],{"type":52,"tag":144,"props":1695,"children":1696},{"__ignoreMap":141},[1697,1705,1713,1721,1729,1736,1744],{"type":52,"tag":148,"props":1698,"children":1699},{"class":150,"line":29},[1700],{"type":52,"tag":148,"props":1701,"children":1702},{},[1703],{"type":57,"value":1704},"response = client.with_options(\n",{"type":52,"tag":148,"props":1706,"children":1707},{"class":150,"line":171},[1708],{"type":52,"tag":148,"props":1709,"children":1710},{},[1711],{"type":57,"value":1712},"    trace_id=\"unique-trace-id\",\n",{"type":52,"tag":148,"props":1714,"children":1715},{"class":150,"line":181},[1716],{"type":52,"tag":148,"props":1717,"children":1718},{},[1719],{"type":57,"value":1720},"    metadata={\"request_type\": \"summarization\"}\n",{"type":52,"tag":148,"props":1722,"children":1723},{"class":150,"line":191},[1724],{"type":52,"tag":148,"props":1725,"children":1726},{},[1727],{"type":57,"value":1728},").chat.completions.create(\n",{"type":52,"tag":148,"props":1730,"children":1731},{"class":150,"line":209},[1732],{"type":52,"tag":148,"props":1733,"children":1734},{},[1735],{"type":57,"value":323},{"type":52,"tag":148,"props":1737,"children":1738},{"class":150,"line":282},[1739],{"type":52,"tag":148,"props":1740,"children":1741},{},[1742],{"type":57,"value":1743},"    messages=[{\"role\": \"user\", \"content\": \"Summarize this...\"}]\n",{"type":52,"tag":148,"props":1745,"children":1746},{"class":150,"line":291},[1747],{"type":52,"tag":148,"props":1748,"children":1749},{},[1750],{"type":57,"value":297},{"type":52,"tag":125,"props":1752,"children":1753},{},[],{"type":52,"tag":129,"props":1755,"children":1757},{"id":1756},"common-patterns",[1758],{"type":57,"value":1759},"Common Patterns",{"type":52,"tag":368,"props":1761,"children":1763},{"id":1762},"multi-turn-conversation",[1764],{"type":57,"value":1765},"Multi-turn Conversation",{"type":52,"tag":136,"props":1767,"children":1769},{"className":235,"code":1768,"language":22,"meta":141,"style":141},"messages = [\n    {\"role\": \"system\", \"content\": \"You are a helpful coding assistant.\"},\n    {\"role\": \"user\", \"content\": \"What is Python?\"},\n    {\"role\": \"assistant\", \"content\": \"Python is a high-level programming language...\"},\n    {\"role\": \"user\", \"content\": \"Show me a hello world example.\"}\n]\n\nresponse = client.chat.completions.create(model=\"gpt-4o\", messages=messages)\n",[1770],{"type":52,"tag":144,"props":1771,"children":1772},{"__ignoreMap":141},[1773,1781,1789,1797,1805,1813,1821,1828],{"type":52,"tag":148,"props":1774,"children":1775},{"class":150,"line":29},[1776],{"type":52,"tag":148,"props":1777,"children":1778},{},[1779],{"type":57,"value":1780},"messages = [\n",{"type":52,"tag":148,"props":1782,"children":1783},{"class":150,"line":171},[1784],{"type":52,"tag":148,"props":1785,"children":1786},{},[1787],{"type":57,"value":1788},"    {\"role\": \"system\", \"content\": \"You are a helpful coding assistant.\"},\n",{"type":52,"tag":148,"props":1790,"children":1791},{"class":150,"line":181},[1792],{"type":52,"tag":148,"props":1793,"children":1794},{},[1795],{"type":57,"value":1796},"    {\"role\": \"user\", \"content\": \"What is Python?\"},\n",{"type":52,"tag":148,"props":1798,"children":1799},{"class":150,"line":191},[1800],{"type":52,"tag":148,"props":1801,"children":1802},{},[1803],{"type":57,"value":1804},"    {\"role\": \"assistant\", \"content\": \"Python is a high-level programming language...\"},\n",{"type":52,"tag":148,"props":1806,"children":1807},{"class":150,"line":209},[1808],{"type":52,"tag":148,"props":1809,"children":1810},{},[1811],{"type":57,"value":1812},"    {\"role\": \"user\", \"content\": \"Show me a hello world example.\"}\n",{"type":52,"tag":148,"props":1814,"children":1815},{"class":150,"line":282},[1816],{"type":52,"tag":148,"props":1817,"children":1818},{},[1819],{"type":57,"value":1820},"]\n",{"type":52,"tag":148,"props":1822,"children":1823},{"class":150,"line":291},[1824],{"type":52,"tag":148,"props":1825,"children":1826},{"emptyLinePlaceholder":175},[1827],{"type":57,"value":178},{"type":52,"tag":148,"props":1829,"children":1830},{"class":150,"line":300},[1831],{"type":52,"tag":148,"props":1832,"children":1833},{},[1834],{"type":57,"value":1835},"response = client.chat.completions.create(model=\"gpt-4o\", messages=messages)\n",{"type":52,"tag":368,"props":1837,"children":1839},{"id":1838},"json-output",[1840],{"type":57,"value":1841},"JSON Output",{"type":52,"tag":136,"props":1843,"children":1845},{"className":235,"code":1844,"language":22,"meta":141,"style":141},"response = client.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=[\n        {\"role\": \"system\", \"content\": \"Extract as JSON with name and age fields.\"},\n        {\"role\": \"user\", \"content\": \"John is 30 years old.\"}\n    ],\n    response_format={\"type\": \"json_object\"}\n)\n# Returns: {\"name\": \"John\", \"age\": 30}\n",[1846],{"type":52,"tag":144,"props":1847,"children":1848},{"__ignoreMap":141},[1849,1856,1863,1870,1878,1886,1894,1902,1909],{"type":52,"tag":148,"props":1850,"children":1851},{"class":150,"line":29},[1852],{"type":52,"tag":148,"props":1853,"children":1854},{},[1855],{"type":57,"value":314},{"type":52,"tag":148,"props":1857,"children":1858},{"class":150,"line":171},[1859],{"type":52,"tag":148,"props":1860,"children":1861},{},[1862],{"type":57,"value":323},{"type":52,"tag":148,"props":1864,"children":1865},{"class":150,"line":181},[1866],{"type":52,"tag":148,"props":1867,"children":1868},{},[1869],{"type":57,"value":527},{"type":52,"tag":148,"props":1871,"children":1872},{"class":150,"line":191},[1873],{"type":52,"tag":148,"props":1874,"children":1875},{},[1876],{"type":57,"value":1877},"        {\"role\": \"system\", \"content\": \"Extract as JSON with name and age fields.\"},\n",{"type":52,"tag":148,"props":1879,"children":1880},{"class":150,"line":209},[1881],{"type":52,"tag":148,"props":1882,"children":1883},{},[1884],{"type":57,"value":1885},"        {\"role\": \"user\", \"content\": \"John is 30 years old.\"}\n",{"type":52,"tag":148,"props":1887,"children":1888},{"class":150,"line":282},[1889],{"type":52,"tag":148,"props":1890,"children":1891},{},[1892],{"type":57,"value":1893},"    ],\n",{"type":52,"tag":148,"props":1895,"children":1896},{"class":150,"line":291},[1897],{"type":52,"tag":148,"props":1898,"children":1899},{},[1900],{"type":57,"value":1901},"    response_format={\"type\": \"json_object\"}\n",{"type":52,"tag":148,"props":1903,"children":1904},{"class":150,"line":300},[1905],{"type":52,"tag":148,"props":1906,"children":1907},{},[1908],{"type":57,"value":297},{"type":52,"tag":148,"props":1910,"children":1911},{"class":150,"line":308},[1912],{"type":52,"tag":148,"props":1913,"children":1914},{},[1915],{"type":57,"value":1916},"# Returns: {\"name\": \"John\", \"age\": 30}\n",{"type":52,"tag":368,"props":1918,"children":1920},{"id":1919},"production-setup-with-fallbacks-caching",[1921],{"type":57,"value":1922},"Production Setup with Fallbacks + Caching",{"type":52,"tag":136,"props":1924,"children":1926},{"className":235,"code":1925,"language":22,"meta":141,"style":141},"def create_production_client():\n    return Portkey(\n        api_key=os.environ[\"PORTKEY_API_KEY\"],\n        config={\n            \"strategy\": {\"mode\": \"fallback\"},\n            \"targets\": [\n                {\n                    \"virtual_key\": os.environ[\"OPENAI_VIRTUAL_KEY\"],\n                    \"override_params\": {\"model\": \"gpt-4o\"},\n                    \"retry\": {\"attempts\": 2, \"on_status_codes\": [429, 500]}\n                },\n                {\n                    \"virtual_key\": os.environ[\"ANTHROPIC_VIRTUAL_KEY\"],\n                    \"override_params\": {\"model\": \"claude-3-5-sonnet-20241022\"}\n                }\n            ],\n            \"cache\": {\"mode\": \"semantic\", \"max_age\": 3600}\n        },\n        trace_id=\"production-session\",\n        metadata={\"environment\": \"production\"}\n    )\n",[1927],{"type":52,"tag":144,"props":1928,"children":1929},{"__ignoreMap":141},[1930,1938,1946,1953,1961,1969,1977,1985,1993,2001,2009,2017,2024,2032,2040,2048,2056,2064,2071,2079,2087],{"type":52,"tag":148,"props":1931,"children":1932},{"class":150,"line":29},[1933],{"type":52,"tag":148,"props":1934,"children":1935},{},[1936],{"type":57,"value":1937},"def create_production_client():\n",{"type":52,"tag":148,"props":1939,"children":1940},{"class":150,"line":171},[1941],{"type":52,"tag":148,"props":1942,"children":1943},{},[1944],{"type":57,"value":1945},"    return Portkey(\n",{"type":52,"tag":148,"props":1947,"children":1948},{"class":150,"line":181},[1949],{"type":52,"tag":148,"props":1950,"children":1951},{},[1952],{"type":57,"value":722},{"type":52,"tag":148,"props":1954,"children":1955},{"class":150,"line":191},[1956],{"type":52,"tag":148,"props":1957,"children":1958},{},[1959],{"type":57,"value":1960},"        config={\n",{"type":52,"tag":148,"props":1962,"children":1963},{"class":150,"line":209},[1964],{"type":52,"tag":148,"props":1965,"children":1966},{},[1967],{"type":57,"value":1968},"            \"strategy\": {\"mode\": \"fallback\"},\n",{"type":52,"tag":148,"props":1970,"children":1971},{"class":150,"line":282},[1972],{"type":52,"tag":148,"props":1973,"children":1974},{},[1975],{"type":57,"value":1976},"            \"targets\": [\n",{"type":52,"tag":148,"props":1978,"children":1979},{"class":150,"line":291},[1980],{"type":52,"tag":148,"props":1981,"children":1982},{},[1983],{"type":57,"value":1984},"                {\n",{"type":52,"tag":148,"props":1986,"children":1987},{"class":150,"line":300},[1988],{"type":52,"tag":148,"props":1989,"children":1990},{},[1991],{"type":57,"value":1992},"                    \"virtual_key\": os.environ[\"OPENAI_VIRTUAL_KEY\"],\n",{"type":52,"tag":148,"props":1994,"children":1995},{"class":150,"line":308},[1996],{"type":52,"tag":148,"props":1997,"children":1998},{},[1999],{"type":57,"value":2000},"                    \"override_params\": {\"model\": \"gpt-4o\"},\n",{"type":52,"tag":148,"props":2002,"children":2003},{"class":150,"line":317},[2004],{"type":52,"tag":148,"props":2005,"children":2006},{},[2007],{"type":57,"value":2008},"                    \"retry\": {\"attempts\": 2, \"on_status_codes\": [429, 500]}\n",{"type":52,"tag":148,"props":2010,"children":2011},{"class":150,"line":326},[2012],{"type":52,"tag":148,"props":2013,"children":2014},{},[2015],{"type":57,"value":2016},"                },\n",{"type":52,"tag":148,"props":2018,"children":2019},{"class":150,"line":335},[2020],{"type":52,"tag":148,"props":2021,"children":2022},{},[2023],{"type":57,"value":1984},{"type":52,"tag":148,"props":2025,"children":2026},{"class":150,"line":343},[2027],{"type":52,"tag":148,"props":2028,"children":2029},{},[2030],{"type":57,"value":2031},"                    \"virtual_key\": os.environ[\"ANTHROPIC_VIRTUAL_KEY\"],\n",{"type":52,"tag":148,"props":2033,"children":2034},{"class":150,"line":351},[2035],{"type":52,"tag":148,"props":2036,"children":2037},{},[2038],{"type":57,"value":2039},"                    \"override_params\": {\"model\": \"claude-3-5-sonnet-20241022\"}\n",{"type":52,"tag":148,"props":2041,"children":2042},{"class":150,"line":788},[2043],{"type":52,"tag":148,"props":2044,"children":2045},{},[2046],{"type":57,"value":2047},"                }\n",{"type":52,"tag":148,"props":2049,"children":2050},{"class":150,"line":796},[2051],{"type":52,"tag":148,"props":2052,"children":2053},{},[2054],{"type":57,"value":2055},"            ],\n",{"type":52,"tag":148,"props":2057,"children":2058},{"class":150,"line":1081},[2059],{"type":52,"tag":148,"props":2060,"children":2061},{},[2062],{"type":57,"value":2063},"            \"cache\": {\"mode\": \"semantic\", \"max_age\": 3600}\n",{"type":52,"tag":148,"props":2065,"children":2066},{"class":150,"line":1089},[2067],{"type":52,"tag":148,"props":2068,"children":2069},{},[2070],{"type":57,"value":1276},{"type":52,"tag":148,"props":2072,"children":2073},{"class":150,"line":1098},[2074],{"type":52,"tag":148,"props":2075,"children":2076},{},[2077],{"type":57,"value":2078},"        trace_id=\"production-session\",\n",{"type":52,"tag":148,"props":2080,"children":2081},{"class":150,"line":1106},[2082],{"type":52,"tag":148,"props":2083,"children":2084},{},[2085],{"type":57,"value":2086},"        metadata={\"environment\": \"production\"}\n",{"type":52,"tag":148,"props":2088,"children":2089},{"class":150,"line":1114},[2090],{"type":52,"tag":148,"props":2091,"children":2092},{},[2093],{"type":57,"value":738},{"type":52,"tag":125,"props":2095,"children":2096},{},[],{"type":52,"tag":129,"props":2098,"children":2100},{"id":2099},"best-practices",[2101],{"type":57,"value":2102},"Best Practices",{"type":52,"tag":2104,"props":2105,"children":2106},"ol",{},[2107,2117,2127,2137,2147,2157],{"type":52,"tag":79,"props":2108,"children":2109},{},[2110,2115],{"type":52,"tag":69,"props":2111,"children":2112},{},[2113],{"type":57,"value":2114},"Use environment variables",{"type":57,"value":2116}," - Never hardcode API keys",{"type":52,"tag":79,"props":2118,"children":2119},{},[2120,2125],{"type":52,"tag":69,"props":2121,"children":2122},{},[2123],{"type":57,"value":2124},"Implement fallbacks",{"type":57,"value":2126}," - Always have backup providers for production",{"type":52,"tag":79,"props":2128,"children":2129},{},[2130,2135],{"type":52,"tag":69,"props":2131,"children":2132},{},[2133],{"type":57,"value":2134},"Use streaming",{"type":57,"value":2136}," - Better UX for long responses",{"type":52,"tag":79,"props":2138,"children":2139},{},[2140,2145],{"type":52,"tag":69,"props":2141,"children":2142},{},[2143],{"type":57,"value":2144},"Add tracing",{"type":57,"value":2146}," - Enable observability with trace IDs and metadata",{"type":52,"tag":79,"props":2148,"children":2149},{},[2150,2155],{"type":52,"tag":69,"props":2151,"children":2152},{},[2153],{"type":57,"value":2154},"Enable caching",{"type":57,"value":2156}," - Reduce costs with semantic caching",{"type":52,"tag":79,"props":2158,"children":2159},{},[2160,2165],{"type":52,"tag":69,"props":2161,"children":2162},{},[2163],{"type":57,"value":2164},"Handle errors",{"type":57,"value":2166}," - Implement retry logic with exponential backoff",{"type":52,"tag":125,"props":2168,"children":2169},{},[],{"type":52,"tag":129,"props":2171,"children":2173},{"id":2172},"resources",[2174],{"type":57,"value":2175},"Resources",{"type":52,"tag":75,"props":2177,"children":2178},{},[2179,2197,2212,2227],{"type":52,"tag":79,"props":2180,"children":2181},{},[2182,2187,2189],{"type":52,"tag":69,"props":2183,"children":2184},{},[2185],{"type":57,"value":2186},"Dashboard",{"type":57,"value":2188},": ",{"type":52,"tag":83,"props":2190,"children":2194},{"href":2191,"rel":2192},"https:\u002F\u002Fapp.portkey.ai",[2193],"nofollow",[2195],{"type":57,"value":2196},"app.portkey.ai",{"type":52,"tag":79,"props":2198,"children":2199},{},[2200,2205,2206],{"type":52,"tag":69,"props":2201,"children":2202},{},[2203],{"type":57,"value":2204},"Documentation",{"type":57,"value":2188},{"type":52,"tag":83,"props":2207,"children":2209},{"href":47,"rel":2208},[2193],[2210],{"type":57,"value":2211},"docs.portkey.ai",{"type":52,"tag":79,"props":2213,"children":2214},{},[2215,2220,2221],{"type":52,"tag":69,"props":2216,"children":2217},{},[2218],{"type":57,"value":2219},"GitHub",{"type":57,"value":2188},{"type":52,"tag":83,"props":2222,"children":2224},{"href":46,"rel":2223},[2193],[2225],{"type":57,"value":2226},"github.com\u002Fportkey-ai\u002Fportkey-python-sdk",{"type":52,"tag":79,"props":2228,"children":2229},{},[2230,2235,2236],{"type":52,"tag":69,"props":2231,"children":2232},{},[2233],{"type":57,"value":2234},"Discord",{"type":57,"value":2188},{"type":52,"tag":83,"props":2237,"children":2240},{"href":2238,"rel":2239},"https:\u002F\u002Fportkey.ai\u002Fdiscord",[2193],[2241],{"type":57,"value":2242},"portkey.ai\u002Fdiscord",{"type":52,"tag":2244,"props":2245,"children":2246},"style",{},[2247],{"type":57,"value":2248},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"items":2250,"total":181},[2251,2266,2274],{"slug":2252,"name":2252,"fn":2253,"description":2254,"org":2255,"tags":2256,"stars":29,"repoUrl":30,"updatedAt":2265},"migrate-litellm-to-portkey","migrate Python apps to Portkey","Migrate Python applications from LiteLLM to Portkey AI Gateway. Covers litellm.completion, Router, Proxy, fallbacks, caching, retries, callbacks, embeddings, and image generation. Use when the user wants to replace LiteLLM with Portkey or switch from LiteLLM to Portkey.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[2257,2260,2261,2264],{"name":2258,"slug":2259,"type":16},"AI Infrastructure","ai-infrastructure",{"name":18,"slug":19,"type":16},{"name":2262,"slug":2263,"type":16},"Migration","migration",{"name":21,"slug":22,"type":16},"2026-07-13T06:00:08.528744",{"slug":4,"name":4,"fn":5,"description":6,"org":2267,"tags":2268,"stars":29,"repoUrl":30,"updatedAt":31},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[2269,2270,2271,2272,2273],{"name":24,"slug":25,"type":16},{"name":27,"slug":28,"type":16},{"name":18,"slug":19,"type":16},{"name":14,"slug":15,"type":16},{"name":21,"slug":22,"type":16},{"slug":2275,"name":2275,"fn":2276,"description":2277,"org":2278,"tags":2279,"stars":29,"repoUrl":30,"updatedAt":2286},"portkey-typescript-sdk","integrate Portkey AI Gateway","Integrate Portkey AI Gateway into TypeScript\u002FJavaScript applications. Use when building LLM apps with observability, caching, fallbacks, load balancing, or routing across 200+ LLM providers.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[2280,2281,2282,2283],{"name":2258,"slug":2259,"type":16},{"name":18,"slug":19,"type":16},{"name":14,"slug":15,"type":16},{"name":2284,"slug":2285,"type":16},"TypeScript","typescript","2026-07-13T06:00:07.08252",{"items":2288,"total":181},[2289,2296,2304],{"slug":2252,"name":2252,"fn":2253,"description":2254,"org":2290,"tags":2291,"stars":29,"repoUrl":30,"updatedAt":2265},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[2292,2293,2294,2295],{"name":2258,"slug":2259,"type":16},{"name":18,"slug":19,"type":16},{"name":2262,"slug":2263,"type":16},{"name":21,"slug":22,"type":16},{"slug":4,"name":4,"fn":5,"description":6,"org":2297,"tags":2298,"stars":29,"repoUrl":30,"updatedAt":31},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[2299,2300,2301,2302,2303],{"name":24,"slug":25,"type":16},{"name":27,"slug":28,"type":16},{"name":18,"slug":19,"type":16},{"name":14,"slug":15,"type":16},{"name":21,"slug":22,"type":16},{"slug":2275,"name":2275,"fn":2276,"description":2277,"org":2305,"tags":2306,"stars":29,"repoUrl":30,"updatedAt":2286},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[2307,2308,2309,2310],{"name":2258,"slug":2259,"type":16},{"name":18,"slug":19,"type":16},{"name":14,"slug":15,"type":16},{"name":2284,"slug":2285,"type":16}]