[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill-together-ai-together-dedicated-model-inference":3,"mdc-mcjku4-key":34,"related-org-together-ai-together-dedicated-model-inference":1520,"related-repo-together-ai-together-dedicated-model-inference":1693},{"slug":4,"name":4,"fn":5,"description":6,"org":7,"tags":12,"stars":23,"repoUrl":24,"updatedAt":25,"license":26,"forks":27,"topics":28,"repo":29,"sourceUrl":32,"mdContent":33},"together-dedicated-model-inference","deploy and operate Together AI models","Deploy and operate models on dedicated GPUs with Together AI's Dedicated Model Inference (DMI, the v2 dedicated endpoints API): beta endpoints, deployments, deployment profiles and hardware configs, autoscaling, traffic splitting, A\u002FB tests, shadow experiments, Prometheus metrics, and custom model or LoRA adapter uploads. Reach for it whenever the user mentions together beta endpoints or tg beta commands, client.beta.endpoints, DMI resources like ep_\u002Fdep_\u002Fcr_\u002Fml_ IDs, or wants production model serving with traffic management on Together AI. This is the current dedicated-hosting API and also covers migrating off the retired legacy v1 endpoints API (non-beta client.endpoints \u002F together endpoints), whose create and restart now return HTTP 403.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},"together-ai","Together AI","https:\u002F\u002Fpexgzepcugksgbtrxkhf.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Forg-logos\u002Ftogether-ai.jpg","togethercomputer",[13,17,20],{"name":14,"slug":15,"type":16},"LLM","llm","tag",{"name":18,"slug":19,"type":16},"Deployment","deployment",{"name":21,"slug":22,"type":16},"AI Infrastructure","ai-infrastructure",31,"https:\u002F\u002Fgithub.com\u002Ftogethercomputer\u002Fskills","2026-07-26T06:08:44.044039",null,4,[],{"repoUrl":24,"stars":23,"forks":27,"topics":30,"description":31},[],"Skills to help your coding agents use Together AI products.","https:\u002F\u002Fgithub.com\u002Ftogethercomputer\u002Fskills\u002Ftree\u002FHEAD\u002Fskills\u002Ftogether-dedicated-model-inference","---\nname: together-dedicated-model-inference\ndescription: \"Deploy and operate models on dedicated GPUs with Together AI's Dedicated Model Inference (DMI, the v2 dedicated endpoints API): beta endpoints, deployments, deployment profiles and hardware configs, autoscaling, traffic splitting, A\u002FB tests, shadow experiments, Prometheus metrics, and custom model or LoRA adapter uploads. Reach for it whenever the user mentions together beta endpoints or tg beta commands, client.beta.endpoints, DMI resources like ep_\u002Fdep_\u002Fcr_\u002Fml_ IDs, or wants production model serving with traffic management on Together AI. This is the current dedicated-hosting API and also covers migrating off the retired legacy v1 endpoints API (non-beta client.endpoints \u002F together endpoints), whose create and restart now return HTTP 403.\"\n---\n\n# Together Dedicated Model Inference\n\n## Overview\n\nDedicated model inference (DMI) serves a model on reserved single-tenant GPUs. It bills per\nminute per running replica (by hardware, not by token), has no hard rate limits, and uses the\nsame inference API as serverless models.\n\nThe resource model has six parts: **Project → Model → Config → Endpoint → Deployment → Replica**.\n\n- **Endpoint** — stable inference name; routes traffic across its deployments by weight.\n- **Deployment** — binds one model + one config to an endpoint with an autoscaling policy;\n  runs the replicas.\n- **Config** — immutable published revision describing how the model runs (engine, GPU type\n  and count, optimization profile). The model determines the available configs.\n- **Replica** — one model instance on its own dedicated hardware.\n\nInference goes to `https:\u002F\u002Fapi-inference.together.ai\u002Fv1` with the **endpoint string**\n(`\u003Cproject_slug>\u002F\u003Cendpoint_name>`) as the `model` parameter. Management goes through the\nTogether CLI (`tg beta ...` — install with `uv tool install \"together[cli]\"`; `tg` and\n`together` are interchangeable), the `client.beta.*` SDK namespaces, or the `\u002Fv2` REST API at\n`https:\u002F\u002Fapi.together.ai`.\n\n> **The legacy v1 dedicated endpoints API is retired.** The old flow (`client.endpoints.create`\n> with `model=` + `hardware=`, hardware IDs like `1x_nvidia_h100_80gb_sxm`, the `together\n> endpoints` CLI *without* `beta`) no longer accepts new endpoints: `POST \u002Fv1\u002Fendpoints`,\n> `client.endpoints.create(...)`, and `tg endpoints create` return\n> `endpoints_v1_create_access_disabled` (HTTP 403), and a stopped\u002Fpaused v1 endpoint can't be\n> restarted. Already-running v1 endpoints keep serving until further notice; everything new —\n> and any redeploy of a stopped v1 endpoint — uses the v2 flow this skill describes.\n\n## When This Skill Wins\n\n- Deploying a model (public, fine-tuned, or uploaded) on dedicated GPUs via the v2 API\n- Anything involving `tg beta endpoints` \u002F `tg beta models` (a.k.a. `together beta ...`) CLI commands\n- Anything involving `client.beta.endpoints` \u002F `client.beta.models` SDK namespaces\n- Traffic management: splits, A\u002FB tests, shadow experiments\n- Autoscaling, scale-to-zero, deployment lifecycle, monitoring (dashboards, events, Prometheus scrape)\n- Uploading custom model weights or LoRA adapters for dedicated serving\n\n## Hand Off To Another Skill\n\n- Use `together-chat-completions` for serverless inference and request-shaping questions\n  (the request shape is identical once the endpoint is up).\n- Use `together-fine-tuning` for training the model you'll deploy here.\n- Use `together-dedicated-containers` for custom Docker runtimes (Sprocket\u002FJig).\n- Use `together-gpu-clusters` for raw multi-node compute.\n\n## Quick Routing\n\n- **Deploy a model end to end**\n  - Start with [scripts\u002Fdeploy_model.py](scripts\u002Fdeploy_model.py)\n  - Read [references\u002Fcli-reference.md](references\u002Fcli-reference.md) for the one-command CLI path\n- **Endpoint\u002Fdeployment lifecycle from the SDK or API** (create, poll, scale, stop, delete, list filters)\n  - Read [references\u002Fapi-reference.md](references\u002Fapi-reference.md)\n- **Split traffic, A\u002FB tests, shadow experiments**\n  - Read [references\u002Ftraffic-routing.md](references\u002Ftraffic-routing.md)\n- **Choose a model or config, pricing, upload custom weights or LoRA adapters**\n  - Read [references\u002Fmodels-and-configs.md](references\u002Fmodels-and-configs.md)\n  - Start with [scripts\u002Fupload_custom_model.py](scripts\u002Fupload_custom_model.py)\n- **Autoscaling and monitoring**\n  - Read [references\u002Fapi-reference.md](references\u002Fapi-reference.md) (Autoscaling, Monitoring sections)\n\n## Workflow\n\n1. Pick a model: `tg beta models public` (or upload custom weights first).\n2. Pick a config\u002Fprecision: read the model's `deploymentProfiles` (they carry `quantization`\n   like `BF16`\u002F`FP8`, `gpuCount`, and the exact `model`+`config` resource names). `tg beta\n   models configs \u003Cid>` is often empty for catalog architectures — see\n   [models-and-configs.md](references\u002Fmodels-and-configs.md). `deploy` auto-picks only when the\n   model has a single profile.\n3. Deploy: `tg beta endpoints deploy \u003Cmodel> --endpoint \u003Cname>` — creates the\n   endpoint, attaches a deployment, and routes 100% of traffic in one step. For a public\n   catalog model with one profile, pass its **name** (`Qwen\u002FQwen2.5-7B-Instruct`); the catalog\n   `ml_`\u002F`arch_` ID is owned by a platform project and won't resolve as a `deploy`\u002F`ab`\n   positional in your project. If the model has **multiple profiles** (e.g. BF16 + FP8), a\n   bare-name deploy errors — pass the chosen profile's full resolved `model` resource path as\n   the positional plus its `--config` (see models-and-configs.md).\n4. Poll until `status.state` is `DEPLOYMENT_STATE_READY`. The CLI `get` now accepts an\n   endpoint **or** deployment ID: `tg beta endpoints get dep_... --json | jq -r '.status.state'`\n   is the scripted polling loop; the SDK equivalent is\n   `client.beta.endpoints.deployments.retrieve(dep_id, project_id=..., endpoint_id=...)`.\n   First-time provisioning commonly takes up to ~20 minutes.\n5. Send requests to `https:\u002F\u002Fapi-inference.together.ai\u002Fv1` with the endpoint string as `model`.\n6. Scale, reconfigure, or set traffic weights with `tg beta endpoints update \u003Cdep_id>`\n   (`--min\u002F--max-replicas`, `--scaling-metric`\u002F`--scaling-target`, `--traffic-weight`); split traffic or\n   experiment as needed.\n7. Clean up: `tg beta endpoints update \u003Cdep_id> --min-replicas 0 --max-replicas 0` to stop\n   billing, or `tg beta endpoints rm \u003Cep_id> --force` to tear everything down (it scales\n   deployments to zero itself).\n\n## High-Signal Rules\n\n- **Billing runs while replicas run, and there is no automatic idle shutdown.** Per minute,\n  per replica, by hardware. The old `inactive_timeout` auto-stop was removed — always scale to\n  zero (`min_replicas: 0, max_replicas: 0`) or delete when the user is done; a forgotten\n  deployment bills until someone stops it.\n- **A `READY` deployment serves nothing until it's in the endpoint's traffic split.** The CLI's\n  `deploy` routes automatically; otherwise set a weight with `tg beta endpoints update \u003Cdep_id>\n  --traffic-weight N` (upserts one entry) or replace the split via SDK\n  `endpoints.update(traffic_split=[...])`. A `routing_error`\u002F503 on a READY deployment almost\n  always means a missing\u002Fzero weight.\n- **Management IDs vs endpoint string.** Management calls take IDs (`ep_`, `dep_`, `cr_`, `ml_`,\n  `abx_`, `exp_`); inference takes the endpoint string `\u003Cproject_slug>\u002F\u003Cendpoint_name>`.\n- **The SDK requires `project_id` on every method** — derive it with `client.whoami().project_id`.\n  The SDK also takes models\u002Fconfigs as resource names (`projects\u002F{p}\u002Fmodels\u002F{ml}`,\n  `projects\u002F{p}\u002Fconfigs\u002F{cr}`), while the CLI takes bare IDs. Use the config's own `projectId`\n  (often a platform project) when building its resource name.\n- **Traffic weights are relative capacity, not percentages.** Share = weight × ready replicas.\n  Prefer shifting traffic by changing replica counts; keep weights stable.\n- **Scale-to-zero is all-or-nothing:** `min_replicas: 0` with a positive `max_replicas` is\n  rejected. `0\u002F0` stops; on a running deployment the floor is otherwise `1`.\n- **Autoscaling metric names are their own catalog** (`gpu_utilization`, `inflight_requests`,\n  `ttft`, ...) — raw Prometheus series names are rejected in scaling policies. Charts live in\n  the dashboard (`https:\u002F\u002Fapi.together.ai\u002Fendpoints`); raw series can be scraped from the\n  org-scoped Prometheus-compatible metrics endpoint (beta — see api-reference.md).\n- **Deletion order matters:** stop the deployment (wait for `STOPPED`), delete it, then\n  delete the endpoint. The CLI's `rm` smart-deletes by ID prefix and auto-detaches from the\n  split. Deleting still requires a stopped deployment, but `rm` now scales down for you:\n  `rm dep_...` on a running deployment sets `0\u002F0` and asks you to retry once `STOPPED`, and\n  `rm ep_... --force` scales the endpoint's deployments to zero itself as part of teardown.\n- **To see which deployment\u002Freplica served a request, read the inference response headers.**\n  The response *body*'s `model` field only echoes the endpoint string — identical for\n  every deployment. The routing headers distinguish them: `x-cluster` is the per-deployment\n  cluster ID and `worker_url` (inside the `x-i-router-log-event` header) is the replica pod.\n  This is the only way to verify a split or A\u002FB empirically. See\n  [traffic-routing.md](references\u002Ftraffic-routing.md) (Observing routing).\n- **To replace a deployment on live traffic**, create the new deployment on the same\n  endpoint, wait for `READY`, then shift traffic gradually with `--traffic-weight` (and\n  replica counts), watching the dashboard between steps. Take the old deployment out with\n  `--traffic-weight 0`, then scale it down and delete it.\n- The `client.beta.*` SDK surface and `together beta` CLI are **beta**: pin a current SDK\n  release (`uv pip install --upgrade together`) and expect the surface to evolve.\n\n## Resource Map\n\n- **SDK\u002FAPI lifecycle reference**: [references\u002Fapi-reference.md](references\u002Fapi-reference.md)\n- **CLI reference**: [references\u002Fcli-reference.md](references\u002Fcli-reference.md)\n- **Traffic routing guide**: [references\u002Ftraffic-routing.md](references\u002Ftraffic-routing.md)\n- **Models, configs, pricing, uploads**: [references\u002Fmodels-and-configs.md](references\u002Fmodels-and-configs.md)\n- **Deploy workflow script**: [scripts\u002Fdeploy_model.py](scripts\u002Fdeploy_model.py)\n- **Custom model upload script**: [scripts\u002Fupload_custom_model.py](scripts\u002Fupload_custom_model.py)\n\n## Official Docs\n\n- [Dedicated Model Inference overview](https:\u002F\u002Fdocs.together.ai\u002Fdocs\u002Fdedicated-endpoints\u002Foverview)\n- [Quickstart](https:\u002F\u002Fdocs.together.ai\u002Fdocs\u002Fdedicated-endpoints\u002Fquickstart)\n- [Concepts](https:\u002F\u002Fdocs.together.ai\u002Fdocs\u002Fdedicated-endpoints\u002Fconcepts)\n- [CLI reference: beta endpoints](https:\u002F\u002Fdocs.together.ai\u002Freference\u002Fcli\u002Fendpoints-beta)\n- [CLI reference: beta models](https:\u002F\u002Fdocs.together.ai\u002Freference\u002Fcli\u002Fmodels-beta)\n- [Migrate from v1](https:\u002F\u002Fdocs.together.ai\u002Fdocs\u002Fdedicated-endpoints\u002Fmigrate-from-v1)\n",{"data":35,"body":36},{"name":4,"description":6},{"type":37,"children":38},"root",[39,47,54,60,73,117,209,311,317,388,394,446,452,581,587,898,904,1362,1368,1450,1456],{"type":40,"tag":41,"props":42,"children":43},"element","h1",{"id":4},[44],{"type":45,"value":46},"text","Together Dedicated Model Inference",{"type":40,"tag":48,"props":49,"children":51},"h2",{"id":50},"overview",[52],{"type":45,"value":53},"Overview",{"type":40,"tag":55,"props":56,"children":57},"p",{},[58],{"type":45,"value":59},"Dedicated model inference (DMI) serves a model on reserved single-tenant GPUs. It bills per\nminute per running replica (by hardware, not by token), has no hard rate limits, and uses the\nsame inference API as serverless models.",{"type":40,"tag":55,"props":61,"children":62},{},[63,65,71],{"type":45,"value":64},"The resource model has six parts: ",{"type":40,"tag":66,"props":67,"children":68},"strong",{},[69],{"type":45,"value":70},"Project → Model → Config → Endpoint → Deployment → Replica",{"type":45,"value":72},".",{"type":40,"tag":74,"props":75,"children":76},"ul",{},[77,88,97,107],{"type":40,"tag":78,"props":79,"children":80},"li",{},[81,86],{"type":40,"tag":66,"props":82,"children":83},{},[84],{"type":45,"value":85},"Endpoint",{"type":45,"value":87}," — stable inference name; routes traffic across its deployments by weight.",{"type":40,"tag":78,"props":89,"children":90},{},[91,95],{"type":40,"tag":66,"props":92,"children":93},{},[94],{"type":45,"value":18},{"type":45,"value":96}," — binds one model + one config to an endpoint with an autoscaling policy;\nruns the replicas.",{"type":40,"tag":78,"props":98,"children":99},{},[100,105],{"type":40,"tag":66,"props":101,"children":102},{},[103],{"type":45,"value":104},"Config",{"type":45,"value":106}," — immutable published revision describing how the model runs (engine, GPU type\nand count, optimization profile). The model determines the available configs.",{"type":40,"tag":78,"props":108,"children":109},{},[110,115],{"type":40,"tag":66,"props":111,"children":112},{},[113],{"type":45,"value":114},"Replica",{"type":45,"value":116}," — one model instance on its own dedicated hardware.",{"type":40,"tag":55,"props":118,"children":119},{},[120,122,129,131,136,138,144,146,152,154,160,162,168,170,176,178,184,186,192,194,200,202,208],{"type":45,"value":121},"Inference goes to ",{"type":40,"tag":123,"props":124,"children":126},"code",{"className":125},[],[127],{"type":45,"value":128},"https:\u002F\u002Fapi-inference.together.ai\u002Fv1",{"type":45,"value":130}," with the ",{"type":40,"tag":66,"props":132,"children":133},{},[134],{"type":45,"value":135},"endpoint string",{"type":45,"value":137},"\n(",{"type":40,"tag":123,"props":139,"children":141},{"className":140},[],[142],{"type":45,"value":143},"\u003Cproject_slug>\u002F\u003Cendpoint_name>",{"type":45,"value":145},") as the ",{"type":40,"tag":123,"props":147,"children":149},{"className":148},[],[150],{"type":45,"value":151},"model",{"type":45,"value":153}," parameter. Management goes through the\nTogether CLI (",{"type":40,"tag":123,"props":155,"children":157},{"className":156},[],[158],{"type":45,"value":159},"tg beta ...",{"type":45,"value":161}," — install with ",{"type":40,"tag":123,"props":163,"children":165},{"className":164},[],[166],{"type":45,"value":167},"uv tool install \"together[cli]\"",{"type":45,"value":169},"; ",{"type":40,"tag":123,"props":171,"children":173},{"className":172},[],[174],{"type":45,"value":175},"tg",{"type":45,"value":177}," and\n",{"type":40,"tag":123,"props":179,"children":181},{"className":180},[],[182],{"type":45,"value":183},"together",{"type":45,"value":185}," are interchangeable), the ",{"type":40,"tag":123,"props":187,"children":189},{"className":188},[],[190],{"type":45,"value":191},"client.beta.*",{"type":45,"value":193}," SDK namespaces, or the ",{"type":40,"tag":123,"props":195,"children":197},{"className":196},[],[198],{"type":45,"value":199},"\u002Fv2",{"type":45,"value":201}," REST API at\n",{"type":40,"tag":123,"props":203,"children":205},{"className":204},[],[206],{"type":45,"value":207},"https:\u002F\u002Fapi.together.ai",{"type":45,"value":72},{"type":40,"tag":210,"props":211,"children":212},"blockquote",{},[213],{"type":40,"tag":55,"props":214,"children":215},{},[216,221,223,229,231,237,239,245,247,253,255,261,263,269,271,277,279,285,287,293,295,301,303,309],{"type":40,"tag":66,"props":217,"children":218},{},[219],{"type":45,"value":220},"The legacy v1 dedicated endpoints API is retired.",{"type":45,"value":222}," The old flow (",{"type":40,"tag":123,"props":224,"children":226},{"className":225},[],[227],{"type":45,"value":228},"client.endpoints.create",{"type":45,"value":230},"\nwith ",{"type":40,"tag":123,"props":232,"children":234},{"className":233},[],[235],{"type":45,"value":236},"model=",{"type":45,"value":238}," + ",{"type":40,"tag":123,"props":240,"children":242},{"className":241},[],[243],{"type":45,"value":244},"hardware=",{"type":45,"value":246},", hardware IDs like ",{"type":40,"tag":123,"props":248,"children":250},{"className":249},[],[251],{"type":45,"value":252},"1x_nvidia_h100_80gb_sxm",{"type":45,"value":254},", the ",{"type":40,"tag":123,"props":256,"children":258},{"className":257},[],[259],{"type":45,"value":260},"together endpoints",{"type":45,"value":262}," CLI ",{"type":40,"tag":264,"props":265,"children":266},"em",{},[267],{"type":45,"value":268},"without",{"type":45,"value":270}," ",{"type":40,"tag":123,"props":272,"children":274},{"className":273},[],[275],{"type":45,"value":276},"beta",{"type":45,"value":278},") no longer accepts new endpoints: ",{"type":40,"tag":123,"props":280,"children":282},{"className":281},[],[283],{"type":45,"value":284},"POST \u002Fv1\u002Fendpoints",{"type":45,"value":286},",\n",{"type":40,"tag":123,"props":288,"children":290},{"className":289},[],[291],{"type":45,"value":292},"client.endpoints.create(...)",{"type":45,"value":294},", and ",{"type":40,"tag":123,"props":296,"children":298},{"className":297},[],[299],{"type":45,"value":300},"tg endpoints create",{"type":45,"value":302}," return\n",{"type":40,"tag":123,"props":304,"children":306},{"className":305},[],[307],{"type":45,"value":308},"endpoints_v1_create_access_disabled",{"type":45,"value":310}," (HTTP 403), and a stopped\u002Fpaused v1 endpoint can't be\nrestarted. Already-running v1 endpoints keep serving until further notice; everything new —\nand any redeploy of a stopped v1 endpoint — uses the v2 flow this skill describes.",{"type":40,"tag":48,"props":312,"children":314},{"id":313},"when-this-skill-wins",[315],{"type":45,"value":316},"When This Skill Wins",{"type":40,"tag":74,"props":318,"children":319},{},[320,325,354,373,378,383],{"type":40,"tag":78,"props":321,"children":322},{},[323],{"type":45,"value":324},"Deploying a model (public, fine-tuned, or uploaded) on dedicated GPUs via the v2 API",{"type":40,"tag":78,"props":326,"children":327},{},[328,330,336,338,344,346,352],{"type":45,"value":329},"Anything involving ",{"type":40,"tag":123,"props":331,"children":333},{"className":332},[],[334],{"type":45,"value":335},"tg beta endpoints",{"type":45,"value":337}," \u002F ",{"type":40,"tag":123,"props":339,"children":341},{"className":340},[],[342],{"type":45,"value":343},"tg beta models",{"type":45,"value":345}," (a.k.a. ",{"type":40,"tag":123,"props":347,"children":349},{"className":348},[],[350],{"type":45,"value":351},"together beta ...",{"type":45,"value":353},") CLI commands",{"type":40,"tag":78,"props":355,"children":356},{},[357,358,364,365,371],{"type":45,"value":329},{"type":40,"tag":123,"props":359,"children":361},{"className":360},[],[362],{"type":45,"value":363},"client.beta.endpoints",{"type":45,"value":337},{"type":40,"tag":123,"props":366,"children":368},{"className":367},[],[369],{"type":45,"value":370},"client.beta.models",{"type":45,"value":372}," SDK namespaces",{"type":40,"tag":78,"props":374,"children":375},{},[376],{"type":45,"value":377},"Traffic management: splits, A\u002FB tests, shadow experiments",{"type":40,"tag":78,"props":379,"children":380},{},[381],{"type":45,"value":382},"Autoscaling, scale-to-zero, deployment lifecycle, monitoring (dashboards, events, Prometheus scrape)",{"type":40,"tag":78,"props":384,"children":385},{},[386],{"type":45,"value":387},"Uploading custom model weights or LoRA adapters for dedicated serving",{"type":40,"tag":48,"props":389,"children":391},{"id":390},"hand-off-to-another-skill",[392],{"type":45,"value":393},"Hand Off To Another Skill",{"type":40,"tag":74,"props":395,"children":396},{},[397,410,422,434],{"type":40,"tag":78,"props":398,"children":399},{},[400,402,408],{"type":45,"value":401},"Use ",{"type":40,"tag":123,"props":403,"children":405},{"className":404},[],[406],{"type":45,"value":407},"together-chat-completions",{"type":45,"value":409}," for serverless inference and request-shaping questions\n(the request shape is identical once the endpoint is up).",{"type":40,"tag":78,"props":411,"children":412},{},[413,414,420],{"type":45,"value":401},{"type":40,"tag":123,"props":415,"children":417},{"className":416},[],[418],{"type":45,"value":419},"together-fine-tuning",{"type":45,"value":421}," for training the model you'll deploy here.",{"type":40,"tag":78,"props":423,"children":424},{},[425,426,432],{"type":45,"value":401},{"type":40,"tag":123,"props":427,"children":429},{"className":428},[],[430],{"type":45,"value":431},"together-dedicated-containers",{"type":45,"value":433}," for custom Docker runtimes (Sprocket\u002FJig).",{"type":40,"tag":78,"props":435,"children":436},{},[437,438,444],{"type":45,"value":401},{"type":40,"tag":123,"props":439,"children":441},{"className":440},[],[442],{"type":45,"value":443},"together-gpu-clusters",{"type":45,"value":445}," for raw multi-node compute.",{"type":40,"tag":48,"props":447,"children":449},{"id":448},"quick-routing",[450],{"type":45,"value":451},"Quick Routing",{"type":40,"tag":74,"props":453,"children":454},{},[455,489,511,531,560],{"type":40,"tag":78,"props":456,"children":457},{},[458,463],{"type":40,"tag":66,"props":459,"children":460},{},[461],{"type":45,"value":462},"Deploy a model end to end",{"type":40,"tag":74,"props":464,"children":465},{},[466,477],{"type":40,"tag":78,"props":467,"children":468},{},[469,471],{"type":45,"value":470},"Start with ",{"type":40,"tag":472,"props":473,"children":475},"a",{"href":474},"scripts\u002Fdeploy_model.py",[476],{"type":45,"value":474},{"type":40,"tag":78,"props":478,"children":479},{},[480,482,487],{"type":45,"value":481},"Read ",{"type":40,"tag":472,"props":483,"children":485},{"href":484},"references\u002Fcli-reference.md",[486],{"type":45,"value":484},{"type":45,"value":488}," for the one-command CLI path",{"type":40,"tag":78,"props":490,"children":491},{},[492,497,499],{"type":40,"tag":66,"props":493,"children":494},{},[495],{"type":45,"value":496},"Endpoint\u002Fdeployment lifecycle from the SDK or API",{"type":45,"value":498}," (create, poll, scale, stop, delete, list filters)\n",{"type":40,"tag":74,"props":500,"children":501},{},[502],{"type":40,"tag":78,"props":503,"children":504},{},[505,506],{"type":45,"value":481},{"type":40,"tag":472,"props":507,"children":509},{"href":508},"references\u002Fapi-reference.md",[510],{"type":45,"value":508},{"type":40,"tag":78,"props":512,"children":513},{},[514,519],{"type":40,"tag":66,"props":515,"children":516},{},[517],{"type":45,"value":518},"Split traffic, A\u002FB tests, shadow experiments",{"type":40,"tag":74,"props":520,"children":521},{},[522],{"type":40,"tag":78,"props":523,"children":524},{},[525,526],{"type":45,"value":481},{"type":40,"tag":472,"props":527,"children":529},{"href":528},"references\u002Ftraffic-routing.md",[530],{"type":45,"value":528},{"type":40,"tag":78,"props":532,"children":533},{},[534,539],{"type":40,"tag":66,"props":535,"children":536},{},[537],{"type":45,"value":538},"Choose a model or config, pricing, upload custom weights or LoRA adapters",{"type":40,"tag":74,"props":540,"children":541},{},[542,551],{"type":40,"tag":78,"props":543,"children":544},{},[545,546],{"type":45,"value":481},{"type":40,"tag":472,"props":547,"children":549},{"href":548},"references\u002Fmodels-and-configs.md",[550],{"type":45,"value":548},{"type":40,"tag":78,"props":552,"children":553},{},[554,555],{"type":45,"value":470},{"type":40,"tag":472,"props":556,"children":558},{"href":557},"scripts\u002Fupload_custom_model.py",[559],{"type":45,"value":557},{"type":40,"tag":78,"props":561,"children":562},{},[563,568],{"type":40,"tag":66,"props":564,"children":565},{},[566],{"type":45,"value":567},"Autoscaling and monitoring",{"type":40,"tag":74,"props":569,"children":570},{},[571],{"type":40,"tag":78,"props":572,"children":573},{},[574,575,579],{"type":45,"value":481},{"type":40,"tag":472,"props":576,"children":577},{"href":508},[578],{"type":45,"value":508},{"type":45,"value":580}," (Autoscaling, Monitoring sections)",{"type":40,"tag":48,"props":582,"children":584},{"id":583},"workflow",[585],{"type":45,"value":586},"Workflow",{"type":40,"tag":588,"props":589,"children":590},"ol",{},[591,604,687,766,818,836,877],{"type":40,"tag":78,"props":592,"children":593},{},[594,596,602],{"type":45,"value":595},"Pick a model: ",{"type":40,"tag":123,"props":597,"children":599},{"className":598},[],[600],{"type":45,"value":601},"tg beta models public",{"type":45,"value":603}," (or upload custom weights first).",{"type":40,"tag":78,"props":605,"children":606},{},[607,609,615,617,623,625,631,633,639,641,647,649,654,656,662,664,670,672,677,679,685],{"type":45,"value":608},"Pick a config\u002Fprecision: read the model's ",{"type":40,"tag":123,"props":610,"children":612},{"className":611},[],[613],{"type":45,"value":614},"deploymentProfiles",{"type":45,"value":616}," (they carry ",{"type":40,"tag":123,"props":618,"children":620},{"className":619},[],[621],{"type":45,"value":622},"quantization",{"type":45,"value":624},"\nlike ",{"type":40,"tag":123,"props":626,"children":628},{"className":627},[],[629],{"type":45,"value":630},"BF16",{"type":45,"value":632},"\u002F",{"type":40,"tag":123,"props":634,"children":636},{"className":635},[],[637],{"type":45,"value":638},"FP8",{"type":45,"value":640},", ",{"type":40,"tag":123,"props":642,"children":644},{"className":643},[],[645],{"type":45,"value":646},"gpuCount",{"type":45,"value":648},", and the exact ",{"type":40,"tag":123,"props":650,"children":652},{"className":651},[],[653],{"type":45,"value":151},{"type":45,"value":655},"+",{"type":40,"tag":123,"props":657,"children":659},{"className":658},[],[660],{"type":45,"value":661},"config",{"type":45,"value":663}," resource names). ",{"type":40,"tag":123,"props":665,"children":667},{"className":666},[],[668],{"type":45,"value":669},"tg beta models configs \u003Cid>",{"type":45,"value":671}," is often empty for catalog architectures — see\n",{"type":40,"tag":472,"props":673,"children":674},{"href":548},[675],{"type":45,"value":676},"models-and-configs.md",{"type":45,"value":678},". ",{"type":40,"tag":123,"props":680,"children":682},{"className":681},[],[683],{"type":45,"value":684},"deploy",{"type":45,"value":686}," auto-picks only when the\nmodel has a single profile.",{"type":40,"tag":78,"props":688,"children":689},{},[690,692,698,700,705,707,713,715,721,722,728,730,735,736,742,744,749,751,756,758,764],{"type":45,"value":691},"Deploy: ",{"type":40,"tag":123,"props":693,"children":695},{"className":694},[],[696],{"type":45,"value":697},"tg beta endpoints deploy \u003Cmodel> --endpoint \u003Cname>",{"type":45,"value":699}," — creates the\nendpoint, attaches a deployment, and routes 100% of traffic in one step. For a public\ncatalog model with one profile, pass its ",{"type":40,"tag":66,"props":701,"children":702},{},[703],{"type":45,"value":704},"name",{"type":45,"value":706}," (",{"type":40,"tag":123,"props":708,"children":710},{"className":709},[],[711],{"type":45,"value":712},"Qwen\u002FQwen2.5-7B-Instruct",{"type":45,"value":714},"); the catalog\n",{"type":40,"tag":123,"props":716,"children":718},{"className":717},[],[719],{"type":45,"value":720},"ml_",{"type":45,"value":632},{"type":40,"tag":123,"props":723,"children":725},{"className":724},[],[726],{"type":45,"value":727},"arch_",{"type":45,"value":729}," ID is owned by a platform project and won't resolve as a ",{"type":40,"tag":123,"props":731,"children":733},{"className":732},[],[734],{"type":45,"value":684},{"type":45,"value":632},{"type":40,"tag":123,"props":737,"children":739},{"className":738},[],[740],{"type":45,"value":741},"ab",{"type":45,"value":743},"\npositional in your project. If the model has ",{"type":40,"tag":66,"props":745,"children":746},{},[747],{"type":45,"value":748},"multiple profiles",{"type":45,"value":750}," (e.g. BF16 + FP8), a\nbare-name deploy errors — pass the chosen profile's full resolved ",{"type":40,"tag":123,"props":752,"children":754},{"className":753},[],[755],{"type":45,"value":151},{"type":45,"value":757}," resource path as\nthe positional plus its ",{"type":40,"tag":123,"props":759,"children":761},{"className":760},[],[762],{"type":45,"value":763},"--config",{"type":45,"value":765}," (see models-and-configs.md).",{"type":40,"tag":78,"props":767,"children":768},{},[769,771,777,779,785,787,793,795,800,802,808,810,816],{"type":45,"value":770},"Poll until ",{"type":40,"tag":123,"props":772,"children":774},{"className":773},[],[775],{"type":45,"value":776},"status.state",{"type":45,"value":778}," is ",{"type":40,"tag":123,"props":780,"children":782},{"className":781},[],[783],{"type":45,"value":784},"DEPLOYMENT_STATE_READY",{"type":45,"value":786},". The CLI ",{"type":40,"tag":123,"props":788,"children":790},{"className":789},[],[791],{"type":45,"value":792},"get",{"type":45,"value":794}," now accepts an\nendpoint ",{"type":40,"tag":66,"props":796,"children":797},{},[798],{"type":45,"value":799},"or",{"type":45,"value":801}," deployment ID: ",{"type":40,"tag":123,"props":803,"children":805},{"className":804},[],[806],{"type":45,"value":807},"tg beta endpoints get dep_... --json | jq -r '.status.state'",{"type":45,"value":809},"\nis the scripted polling loop; the SDK equivalent is\n",{"type":40,"tag":123,"props":811,"children":813},{"className":812},[],[814],{"type":45,"value":815},"client.beta.endpoints.deployments.retrieve(dep_id, project_id=..., endpoint_id=...)",{"type":45,"value":817},".\nFirst-time provisioning commonly takes up to ~20 minutes.",{"type":40,"tag":78,"props":819,"children":820},{},[821,823,828,830,835],{"type":45,"value":822},"Send requests to ",{"type":40,"tag":123,"props":824,"children":826},{"className":825},[],[827],{"type":45,"value":128},{"type":45,"value":829}," with the endpoint string as ",{"type":40,"tag":123,"props":831,"children":833},{"className":832},[],[834],{"type":45,"value":151},{"type":45,"value":72},{"type":40,"tag":78,"props":837,"children":838},{},[839,841,847,848,854,855,861,862,868,869,875],{"type":45,"value":840},"Scale, reconfigure, or set traffic weights with ",{"type":40,"tag":123,"props":842,"children":844},{"className":843},[],[845],{"type":45,"value":846},"tg beta endpoints update \u003Cdep_id>",{"type":45,"value":137},{"type":40,"tag":123,"props":849,"children":851},{"className":850},[],[852],{"type":45,"value":853},"--min\u002F--max-replicas",{"type":45,"value":640},{"type":40,"tag":123,"props":856,"children":858},{"className":857},[],[859],{"type":45,"value":860},"--scaling-metric",{"type":45,"value":632},{"type":40,"tag":123,"props":863,"children":865},{"className":864},[],[866],{"type":45,"value":867},"--scaling-target",{"type":45,"value":640},{"type":40,"tag":123,"props":870,"children":872},{"className":871},[],[873],{"type":45,"value":874},"--traffic-weight",{"type":45,"value":876},"); split traffic or\nexperiment as needed.",{"type":40,"tag":78,"props":878,"children":879},{},[880,882,888,890,896],{"type":45,"value":881},"Clean up: ",{"type":40,"tag":123,"props":883,"children":885},{"className":884},[],[886],{"type":45,"value":887},"tg beta endpoints update \u003Cdep_id> --min-replicas 0 --max-replicas 0",{"type":45,"value":889}," to stop\nbilling, or ",{"type":40,"tag":123,"props":891,"children":893},{"className":892},[],[894],{"type":45,"value":895},"tg beta endpoints rm \u003Cep_id> --force",{"type":45,"value":897}," to tear everything down (it scales\ndeployments to zero itself).",{"type":40,"tag":48,"props":899,"children":901},{"id":900},"high-signal-rules",[902],{"type":45,"value":903},"High-Signal Rules",{"type":40,"tag":74,"props":905,"children":906},{},[907,933,982,1040,1089,1099,1139,1178,1241,1296,1328],{"type":40,"tag":78,"props":908,"children":909},{},[910,915,917,923,925,931],{"type":40,"tag":66,"props":911,"children":912},{},[913],{"type":45,"value":914},"Billing runs while replicas run, and there is no automatic idle shutdown.",{"type":45,"value":916}," Per minute,\nper replica, by hardware. The old ",{"type":40,"tag":123,"props":918,"children":920},{"className":919},[],[921],{"type":45,"value":922},"inactive_timeout",{"type":45,"value":924}," auto-stop was removed — always scale to\nzero (",{"type":40,"tag":123,"props":926,"children":928},{"className":927},[],[929],{"type":45,"value":930},"min_replicas: 0, max_replicas: 0",{"type":45,"value":932},") or delete when the user is done; a forgotten\ndeployment bills until someone stops it.",{"type":40,"tag":78,"props":934,"children":935},{},[936,949,951,956,958,964,966,972,974,980],{"type":40,"tag":66,"props":937,"children":938},{},[939,941,947],{"type":45,"value":940},"A ",{"type":40,"tag":123,"props":942,"children":944},{"className":943},[],[945],{"type":45,"value":946},"READY",{"type":45,"value":948}," deployment serves nothing until it's in the endpoint's traffic split.",{"type":45,"value":950}," The CLI's\n",{"type":40,"tag":123,"props":952,"children":954},{"className":953},[],[955],{"type":45,"value":684},{"type":45,"value":957}," routes automatically; otherwise set a weight with ",{"type":40,"tag":123,"props":959,"children":961},{"className":960},[],[962],{"type":45,"value":963},"tg beta endpoints update \u003Cdep_id> --traffic-weight N",{"type":45,"value":965}," (upserts one entry) or replace the split via SDK\n",{"type":40,"tag":123,"props":967,"children":969},{"className":968},[],[970],{"type":45,"value":971},"endpoints.update(traffic_split=[...])",{"type":45,"value":973},". A ",{"type":40,"tag":123,"props":975,"children":977},{"className":976},[],[978],{"type":45,"value":979},"routing_error",{"type":45,"value":981},"\u002F503 on a READY deployment almost\nalways means a missing\u002Fzero weight.",{"type":40,"tag":78,"props":983,"children":984},{},[985,990,992,998,999,1005,1006,1012,1013,1018,1019,1025,1026,1032,1034,1039],{"type":40,"tag":66,"props":986,"children":987},{},[988],{"type":45,"value":989},"Management IDs vs endpoint string.",{"type":45,"value":991}," Management calls take IDs (",{"type":40,"tag":123,"props":993,"children":995},{"className":994},[],[996],{"type":45,"value":997},"ep_",{"type":45,"value":640},{"type":40,"tag":123,"props":1000,"children":1002},{"className":1001},[],[1003],{"type":45,"value":1004},"dep_",{"type":45,"value":640},{"type":40,"tag":123,"props":1007,"children":1009},{"className":1008},[],[1010],{"type":45,"value":1011},"cr_",{"type":45,"value":640},{"type":40,"tag":123,"props":1014,"children":1016},{"className":1015},[],[1017],{"type":45,"value":720},{"type":45,"value":286},{"type":40,"tag":123,"props":1020,"children":1022},{"className":1021},[],[1023],{"type":45,"value":1024},"abx_",{"type":45,"value":640},{"type":40,"tag":123,"props":1027,"children":1029},{"className":1028},[],[1030],{"type":45,"value":1031},"exp_",{"type":45,"value":1033},"); inference takes the endpoint string ",{"type":40,"tag":123,"props":1035,"children":1037},{"className":1036},[],[1038],{"type":45,"value":143},{"type":45,"value":72},{"type":40,"tag":78,"props":1041,"children":1042},{},[1043,1056,1058,1064,1066,1072,1073,1079,1081,1087],{"type":40,"tag":66,"props":1044,"children":1045},{},[1046,1048,1054],{"type":45,"value":1047},"The SDK requires ",{"type":40,"tag":123,"props":1049,"children":1051},{"className":1050},[],[1052],{"type":45,"value":1053},"project_id",{"type":45,"value":1055}," on every method",{"type":45,"value":1057}," — derive it with ",{"type":40,"tag":123,"props":1059,"children":1061},{"className":1060},[],[1062],{"type":45,"value":1063},"client.whoami().project_id",{"type":45,"value":1065},".\nThe SDK also takes models\u002Fconfigs as resource names (",{"type":40,"tag":123,"props":1067,"children":1069},{"className":1068},[],[1070],{"type":45,"value":1071},"projects\u002F{p}\u002Fmodels\u002F{ml}",{"type":45,"value":286},{"type":40,"tag":123,"props":1074,"children":1076},{"className":1075},[],[1077],{"type":45,"value":1078},"projects\u002F{p}\u002Fconfigs\u002F{cr}",{"type":45,"value":1080},"), while the CLI takes bare IDs. Use the config's own ",{"type":40,"tag":123,"props":1082,"children":1084},{"className":1083},[],[1085],{"type":45,"value":1086},"projectId",{"type":45,"value":1088},"\n(often a platform project) when building its resource name.",{"type":40,"tag":78,"props":1090,"children":1091},{},[1092,1097],{"type":40,"tag":66,"props":1093,"children":1094},{},[1095],{"type":45,"value":1096},"Traffic weights are relative capacity, not percentages.",{"type":45,"value":1098}," Share = weight × ready replicas.\nPrefer shifting traffic by changing replica counts; keep weights stable.",{"type":40,"tag":78,"props":1100,"children":1101},{},[1102,1107,1108,1114,1116,1122,1124,1130,1132,1138],{"type":40,"tag":66,"props":1103,"children":1104},{},[1105],{"type":45,"value":1106},"Scale-to-zero is all-or-nothing:",{"type":45,"value":270},{"type":40,"tag":123,"props":1109,"children":1111},{"className":1110},[],[1112],{"type":45,"value":1113},"min_replicas: 0",{"type":45,"value":1115}," with a positive ",{"type":40,"tag":123,"props":1117,"children":1119},{"className":1118},[],[1120],{"type":45,"value":1121},"max_replicas",{"type":45,"value":1123}," is\nrejected. ",{"type":40,"tag":123,"props":1125,"children":1127},{"className":1126},[],[1128],{"type":45,"value":1129},"0\u002F0",{"type":45,"value":1131}," stops; on a running deployment the floor is otherwise ",{"type":40,"tag":123,"props":1133,"children":1135},{"className":1134},[],[1136],{"type":45,"value":1137},"1",{"type":45,"value":72},{"type":40,"tag":78,"props":1140,"children":1141},{},[1142,1147,1148,1154,1155,1161,1162,1168,1170,1176],{"type":40,"tag":66,"props":1143,"children":1144},{},[1145],{"type":45,"value":1146},"Autoscaling metric names are their own catalog",{"type":45,"value":706},{"type":40,"tag":123,"props":1149,"children":1151},{"className":1150},[],[1152],{"type":45,"value":1153},"gpu_utilization",{"type":45,"value":640},{"type":40,"tag":123,"props":1156,"children":1158},{"className":1157},[],[1159],{"type":45,"value":1160},"inflight_requests",{"type":45,"value":286},{"type":40,"tag":123,"props":1163,"children":1165},{"className":1164},[],[1166],{"type":45,"value":1167},"ttft",{"type":45,"value":1169},", ...) — raw Prometheus series names are rejected in scaling policies. Charts live in\nthe dashboard (",{"type":40,"tag":123,"props":1171,"children":1173},{"className":1172},[],[1174],{"type":45,"value":1175},"https:\u002F\u002Fapi.together.ai\u002Fendpoints",{"type":45,"value":1177},"); raw series can be scraped from the\norg-scoped Prometheus-compatible metrics endpoint (beta — see api-reference.md).",{"type":40,"tag":78,"props":1179,"children":1180},{},[1181,1186,1188,1194,1196,1202,1204,1209,1211,1217,1219,1224,1226,1231,1233,1239],{"type":40,"tag":66,"props":1182,"children":1183},{},[1184],{"type":45,"value":1185},"Deletion order matters:",{"type":45,"value":1187}," stop the deployment (wait for ",{"type":40,"tag":123,"props":1189,"children":1191},{"className":1190},[],[1192],{"type":45,"value":1193},"STOPPED",{"type":45,"value":1195},"), delete it, then\ndelete the endpoint. The CLI's ",{"type":40,"tag":123,"props":1197,"children":1199},{"className":1198},[],[1200],{"type":45,"value":1201},"rm",{"type":45,"value":1203}," smart-deletes by ID prefix and auto-detaches from the\nsplit. Deleting still requires a stopped deployment, but ",{"type":40,"tag":123,"props":1205,"children":1207},{"className":1206},[],[1208],{"type":45,"value":1201},{"type":45,"value":1210}," now scales down for you:\n",{"type":40,"tag":123,"props":1212,"children":1214},{"className":1213},[],[1215],{"type":45,"value":1216},"rm dep_...",{"type":45,"value":1218}," on a running deployment sets ",{"type":40,"tag":123,"props":1220,"children":1222},{"className":1221},[],[1223],{"type":45,"value":1129},{"type":45,"value":1225}," and asks you to retry once ",{"type":40,"tag":123,"props":1227,"children":1229},{"className":1228},[],[1230],{"type":45,"value":1193},{"type":45,"value":1232},", and\n",{"type":40,"tag":123,"props":1234,"children":1236},{"className":1235},[],[1237],{"type":45,"value":1238},"rm ep_... --force",{"type":45,"value":1240}," scales the endpoint's deployments to zero itself as part of teardown.",{"type":40,"tag":78,"props":1242,"children":1243},{},[1244,1249,1251,1256,1258,1263,1265,1271,1273,1279,1281,1287,1289,1294],{"type":40,"tag":66,"props":1245,"children":1246},{},[1247],{"type":45,"value":1248},"To see which deployment\u002Freplica served a request, read the inference response headers.",{"type":45,"value":1250},"\nThe response ",{"type":40,"tag":264,"props":1252,"children":1253},{},[1254],{"type":45,"value":1255},"body",{"type":45,"value":1257},"'s ",{"type":40,"tag":123,"props":1259,"children":1261},{"className":1260},[],[1262],{"type":45,"value":151},{"type":45,"value":1264}," field only echoes the endpoint string — identical for\nevery deployment. The routing headers distinguish them: ",{"type":40,"tag":123,"props":1266,"children":1268},{"className":1267},[],[1269],{"type":45,"value":1270},"x-cluster",{"type":45,"value":1272}," is the per-deployment\ncluster ID and ",{"type":40,"tag":123,"props":1274,"children":1276},{"className":1275},[],[1277],{"type":45,"value":1278},"worker_url",{"type":45,"value":1280}," (inside the ",{"type":40,"tag":123,"props":1282,"children":1284},{"className":1283},[],[1285],{"type":45,"value":1286},"x-i-router-log-event",{"type":45,"value":1288}," header) is the replica pod.\nThis is the only way to verify a split or A\u002FB empirically. See\n",{"type":40,"tag":472,"props":1290,"children":1291},{"href":528},[1292],{"type":45,"value":1293},"traffic-routing.md",{"type":45,"value":1295}," (Observing routing).",{"type":40,"tag":78,"props":1297,"children":1298},{},[1299,1304,1306,1311,1313,1318,1320,1326],{"type":40,"tag":66,"props":1300,"children":1301},{},[1302],{"type":45,"value":1303},"To replace a deployment on live traffic",{"type":45,"value":1305},", create the new deployment on the same\nendpoint, wait for ",{"type":40,"tag":123,"props":1307,"children":1309},{"className":1308},[],[1310],{"type":45,"value":946},{"type":45,"value":1312},", then shift traffic gradually with ",{"type":40,"tag":123,"props":1314,"children":1316},{"className":1315},[],[1317],{"type":45,"value":874},{"type":45,"value":1319}," (and\nreplica counts), watching the dashboard between steps. Take the old deployment out with\n",{"type":40,"tag":123,"props":1321,"children":1323},{"className":1322},[],[1324],{"type":45,"value":1325},"--traffic-weight 0",{"type":45,"value":1327},", then scale it down and delete it.",{"type":40,"tag":78,"props":1329,"children":1330},{},[1331,1333,1338,1340,1346,1348,1352,1354,1360],{"type":45,"value":1332},"The ",{"type":40,"tag":123,"props":1334,"children":1336},{"className":1335},[],[1337],{"type":45,"value":191},{"type":45,"value":1339}," SDK surface and ",{"type":40,"tag":123,"props":1341,"children":1343},{"className":1342},[],[1344],{"type":45,"value":1345},"together beta",{"type":45,"value":1347}," CLI are ",{"type":40,"tag":66,"props":1349,"children":1350},{},[1351],{"type":45,"value":276},{"type":45,"value":1353},": pin a current SDK\nrelease (",{"type":40,"tag":123,"props":1355,"children":1357},{"className":1356},[],[1358],{"type":45,"value":1359},"uv pip install --upgrade together",{"type":45,"value":1361},") and expect the surface to evolve.",{"type":40,"tag":48,"props":1363,"children":1365},{"id":1364},"resource-map",[1366],{"type":45,"value":1367},"Resource Map",{"type":40,"tag":74,"props":1369,"children":1370},{},[1371,1385,1398,1411,1424,1437],{"type":40,"tag":78,"props":1372,"children":1373},{},[1374,1379,1381],{"type":40,"tag":66,"props":1375,"children":1376},{},[1377],{"type":45,"value":1378},"SDK\u002FAPI lifecycle reference",{"type":45,"value":1380},": ",{"type":40,"tag":472,"props":1382,"children":1383},{"href":508},[1384],{"type":45,"value":508},{"type":40,"tag":78,"props":1386,"children":1387},{},[1388,1393,1394],{"type":40,"tag":66,"props":1389,"children":1390},{},[1391],{"type":45,"value":1392},"CLI reference",{"type":45,"value":1380},{"type":40,"tag":472,"props":1395,"children":1396},{"href":484},[1397],{"type":45,"value":484},{"type":40,"tag":78,"props":1399,"children":1400},{},[1401,1406,1407],{"type":40,"tag":66,"props":1402,"children":1403},{},[1404],{"type":45,"value":1405},"Traffic routing guide",{"type":45,"value":1380},{"type":40,"tag":472,"props":1408,"children":1409},{"href":528},[1410],{"type":45,"value":528},{"type":40,"tag":78,"props":1412,"children":1413},{},[1414,1419,1420],{"type":40,"tag":66,"props":1415,"children":1416},{},[1417],{"type":45,"value":1418},"Models, configs, pricing, uploads",{"type":45,"value":1380},{"type":40,"tag":472,"props":1421,"children":1422},{"href":548},[1423],{"type":45,"value":548},{"type":40,"tag":78,"props":1425,"children":1426},{},[1427,1432,1433],{"type":40,"tag":66,"props":1428,"children":1429},{},[1430],{"type":45,"value":1431},"Deploy workflow script",{"type":45,"value":1380},{"type":40,"tag":472,"props":1434,"children":1435},{"href":474},[1436],{"type":45,"value":474},{"type":40,"tag":78,"props":1438,"children":1439},{},[1440,1445,1446],{"type":40,"tag":66,"props":1441,"children":1442},{},[1443],{"type":45,"value":1444},"Custom model upload script",{"type":45,"value":1380},{"type":40,"tag":472,"props":1447,"children":1448},{"href":557},[1449],{"type":45,"value":557},{"type":40,"tag":48,"props":1451,"children":1453},{"id":1452},"official-docs",[1454],{"type":45,"value":1455},"Official Docs",{"type":40,"tag":74,"props":1457,"children":1458},{},[1459,1470,1480,1490,1500,1510],{"type":40,"tag":78,"props":1460,"children":1461},{},[1462],{"type":40,"tag":472,"props":1463,"children":1467},{"href":1464,"rel":1465},"https:\u002F\u002Fdocs.together.ai\u002Fdocs\u002Fdedicated-endpoints\u002Foverview",[1466],"nofollow",[1468],{"type":45,"value":1469},"Dedicated Model Inference overview",{"type":40,"tag":78,"props":1471,"children":1472},{},[1473],{"type":40,"tag":472,"props":1474,"children":1477},{"href":1475,"rel":1476},"https:\u002F\u002Fdocs.together.ai\u002Fdocs\u002Fdedicated-endpoints\u002Fquickstart",[1466],[1478],{"type":45,"value":1479},"Quickstart",{"type":40,"tag":78,"props":1481,"children":1482},{},[1483],{"type":40,"tag":472,"props":1484,"children":1487},{"href":1485,"rel":1486},"https:\u002F\u002Fdocs.together.ai\u002Fdocs\u002Fdedicated-endpoints\u002Fconcepts",[1466],[1488],{"type":45,"value":1489},"Concepts",{"type":40,"tag":78,"props":1491,"children":1492},{},[1493],{"type":40,"tag":472,"props":1494,"children":1497},{"href":1495,"rel":1496},"https:\u002F\u002Fdocs.together.ai\u002Freference\u002Fcli\u002Fendpoints-beta",[1466],[1498],{"type":45,"value":1499},"CLI reference: beta endpoints",{"type":40,"tag":78,"props":1501,"children":1502},{},[1503],{"type":40,"tag":472,"props":1504,"children":1507},{"href":1505,"rel":1506},"https:\u002F\u002Fdocs.together.ai\u002Freference\u002Fcli\u002Fmodels-beta",[1466],[1508],{"type":45,"value":1509},"CLI reference: beta models",{"type":40,"tag":78,"props":1511,"children":1512},{},[1513],{"type":40,"tag":472,"props":1514,"children":1517},{"href":1515,"rel":1516},"https:\u002F\u002Fdocs.together.ai\u002Fdocs\u002Fdedicated-endpoints\u002Fmigrate-from-v1",[1466],[1518],{"type":45,"value":1519},"Migrate from v1",{"items":1521,"total":1692},[1522,1543,1555,1574,1588,1601,1615,1621,1636,1650,1659,1673],{"slug":1523,"name":1523,"fn":1524,"description":1525,"org":1526,"tags":1527,"stars":1540,"repoUrl":1541,"updatedAt":1542},"blog-post","write structured blog posts","Write structured long-form blog posts with a consistent structure and SEO optimization.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1528,1531,1534,1537],{"name":1529,"slug":1530,"type":16},"Content Creation","content-creation",{"name":1532,"slug":1533,"type":16},"Marketing","marketing",{"name":1535,"slug":1536,"type":16},"SEO","seo",{"name":1538,"slug":1539,"type":16},"Writing","writing",1151,"https:\u002F\u002Fgithub.com\u002Ftogethercomputer\u002Ftogether-cookbook","2026-07-17T06:07:39.475433",{"slug":1544,"name":1544,"fn":1545,"description":1546,"org":1547,"tags":1548,"stars":1540,"repoUrl":1541,"updatedAt":1554},"social-media","create social media content","Create social media content including Twitter\u002FX threads, LinkedIn posts, and short-form updates.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1549,1550,1551,1553],{"name":1529,"slug":1530,"type":16},{"name":1532,"slug":1533,"type":16},{"name":1552,"slug":1544,"type":16},"Social Media",{"name":1538,"slug":1539,"type":16},"2026-07-17T06:07:38.455058",{"slug":1556,"name":1556,"fn":1557,"description":1558,"org":1559,"tags":1560,"stars":23,"repoUrl":24,"updatedAt":1573},"together-audio","process audio with Together AI","Text-to-speech and speech-to-text via Together AI, including REST, streaming, and realtime WebSocket TTS, plus transcription, translation, diarization, timestamps, and live STT. Reach for it whenever the user needs audio in or audio out on Together AI rather than chat generation, image or video creation, or model training.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1561,1564,1567,1570],{"name":1562,"slug":1563,"type":16},"Audio","audio",{"name":1565,"slug":1566,"type":16},"Speech","speech",{"name":1568,"slug":1569,"type":16},"Text-to-Speech","text-to-speech",{"name":1571,"slug":1572,"type":16},"Transcription","transcription","2026-07-26T05:49:08.246858",{"slug":1575,"name":1575,"fn":1576,"description":1577,"org":1578,"tags":1579,"stars":23,"repoUrl":24,"updatedAt":1587},"together-batch-inference","run asynchronous batch inference jobs","High-volume, asynchronous offline inference at up to 50% lower cost via Together AI's Batch API. Prepare JSONL inputs, upload files, create jobs, poll status, and download outputs. Reach for it whenever the user needs non-interactive bulk inference rather than real-time chat or evaluation jobs.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1580,1583,1584],{"name":1581,"slug":1582,"type":16},"Automation","automation",{"name":14,"slug":15,"type":16},{"name":1585,"slug":1586,"type":16},"Performance","performance","2026-07-17T06:08:23.919602",{"slug":407,"name":407,"fn":1589,"description":1590,"org":1591,"tags":1592,"stars":23,"repoUrl":24,"updatedAt":1600},"generate text with Together AI","Real-time and streaming text generation via Together AI's OpenAI-compatible chat\u002Fcompletions API, including multi-turn conversations, tool and function calling, structured JSON outputs, and reasoning models. Reach for it whenever the user wants to build or debug text generation on Together AI, unless they specifically need batch jobs, embeddings, fine-tuning, dedicated endpoints, dedicated containers, or GPU clusters.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1593,1596,1599],{"name":1594,"slug":1595,"type":16},"AI","ai",{"name":1597,"slug":1598,"type":16},"API Development","api-development",{"name":14,"slug":15,"type":16},"2026-07-26T05:49:11.244104",{"slug":431,"name":431,"fn":1602,"description":1603,"org":1604,"tags":1605,"stars":23,"repoUrl":24,"updatedAt":1614},"deploy custom inference containers on Together AI","Custom Dockerized inference workers on Together AI's managed GPU infrastructure. Build with Sprocket SDK, configure with Jig CLI, submit async queue jobs, and poll results. Reach for it whenever the user needs container-level control rather than a standard model endpoint or raw cluster.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1606,1607,1608,1611],{"name":21,"slug":22,"type":16},{"name":18,"slug":19,"type":16},{"name":1609,"slug":1610,"type":16},"Docker","docker",{"name":1612,"slug":1613,"type":16},"Machine Learning","machine-learning","2026-07-26T05:49:09.267892",{"slug":4,"name":4,"fn":5,"description":6,"org":1616,"tags":1617,"stars":23,"repoUrl":24,"updatedAt":25},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1618,1619,1620],{"name":21,"slug":22,"type":16},{"name":18,"slug":19,"type":16},{"name":14,"slug":15,"type":16},{"slug":1622,"name":1622,"fn":1623,"description":1624,"org":1625,"tags":1626,"stars":23,"repoUrl":24,"updatedAt":1635},"together-embeddings","generate embeddings and build RAG pipelines","Dense vector embeddings, semantic search, RAG pipelines, and reranking via Together AI. Generate embeddings with open-source models and rerank results behind dedicated endpoints. Reach for it whenever the user needs vector representations or retrieval quality improvements rather than direct text generation.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1627,1628,1629,1632],{"name":1594,"slug":1595,"type":16},{"name":14,"slug":15,"type":16},{"name":1630,"slug":1631,"type":16},"RAG","rag",{"name":1633,"slug":1634,"type":16},"Search","search","2026-07-26T05:49:06.247906",{"slug":1637,"name":1637,"fn":1638,"description":1639,"org":1640,"tags":1641,"stars":23,"repoUrl":24,"updatedAt":1649},"together-evaluations","evaluate LLM outputs with Together AI","LLM-as-a-judge evaluation framework on Together AI. Classify, score, and compare model outputs, select judge models, use external-provider judges or targets, poll results and download reports. Reach for it whenever the user wants to benchmark outputs, grade responses, compare A\u002FB variants, or operationalize automated evaluations.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1642,1645,1648],{"name":1643,"slug":1644,"type":16},"Benchmarking","benchmarking",{"name":1646,"slug":1647,"type":16},"Evals","evals",{"name":14,"slug":15,"type":16},"2026-07-26T05:49:07.241553",{"slug":419,"name":419,"fn":1651,"description":1652,"org":1653,"tags":1654,"stars":23,"repoUrl":24,"updatedAt":1658},"fine-tune and adapt models on Together AI","LoRA, full fine-tuning, DPO preference tuning, VLM training, function-calling tuning, reasoning tuning, and BYOM uploads on Together AI. Reach for it whenever the user wants to adapt a model on custom data rather than only run inference, evaluate outputs, or host an existing model.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1655,1656,1657],{"name":1594,"slug":1595,"type":16},{"name":14,"slug":15,"type":16},{"name":1612,"slug":1613,"type":16},"2026-07-26T05:49:10.243114",{"slug":443,"name":443,"fn":1660,"description":1661,"org":1662,"tags":1663,"stars":23,"repoUrl":24,"updatedAt":1672},"orchestrate GPU clusters on Together AI","On-demand and reserved GPU clusters (H100, H200, B200) on Together AI with Kubernetes or Slurm orchestration, shared storage, credential management, and cluster scaling for ML and HPC jobs. Reach for it when the user needs multi-node compute or infrastructure control rather than a managed model endpoint.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1664,1665,1668,1671],{"name":21,"slug":22,"type":16},{"name":1666,"slug":1667,"type":16},"Cloud","cloud",{"name":1669,"slug":1670,"type":16},"Kubernetes","kubernetes",{"name":1612,"slug":1613,"type":16},"2026-07-26T05:49:05.252646",{"slug":1674,"name":1674,"fn":1675,"description":1676,"org":1677,"tags":1678,"stars":23,"repoUrl":24,"updatedAt":1691},"together-images","generate and edit images with Together AI","Text-to-image generation and image editing via Together AI, including FLUX and Kontext models, LoRA-based styling, reference-image guidance, and local image downloads. Reach for it whenever the user wants to generate or edit images on Together AI rather than create videos or build text-only chat applications.",{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1679,1682,1685,1688],{"name":1680,"slug":1681,"type":16},"Creative","creative",{"name":1683,"slug":1684,"type":16},"Design","design",{"name":1686,"slug":1687,"type":16},"Image Generation","image-generation",{"name":1689,"slug":1690,"type":16},"Multimodal","multimodal","2026-07-17T06:04:23.067755",16,{"items":1694,"total":1740},[1695,1702,1708,1714,1721,1727,1734],{"slug":1556,"name":1556,"fn":1557,"description":1558,"org":1696,"tags":1697,"stars":23,"repoUrl":24,"updatedAt":1573},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1698,1699,1700,1701],{"name":1562,"slug":1563,"type":16},{"name":1565,"slug":1566,"type":16},{"name":1568,"slug":1569,"type":16},{"name":1571,"slug":1572,"type":16},{"slug":1575,"name":1575,"fn":1576,"description":1577,"org":1703,"tags":1704,"stars":23,"repoUrl":24,"updatedAt":1587},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1705,1706,1707],{"name":1581,"slug":1582,"type":16},{"name":14,"slug":15,"type":16},{"name":1585,"slug":1586,"type":16},{"slug":407,"name":407,"fn":1589,"description":1590,"org":1709,"tags":1710,"stars":23,"repoUrl":24,"updatedAt":1600},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1711,1712,1713],{"name":1594,"slug":1595,"type":16},{"name":1597,"slug":1598,"type":16},{"name":14,"slug":15,"type":16},{"slug":431,"name":431,"fn":1602,"description":1603,"org":1715,"tags":1716,"stars":23,"repoUrl":24,"updatedAt":1614},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1717,1718,1719,1720],{"name":21,"slug":22,"type":16},{"name":18,"slug":19,"type":16},{"name":1609,"slug":1610,"type":16},{"name":1612,"slug":1613,"type":16},{"slug":4,"name":4,"fn":5,"description":6,"org":1722,"tags":1723,"stars":23,"repoUrl":24,"updatedAt":25},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1724,1725,1726],{"name":21,"slug":22,"type":16},{"name":18,"slug":19,"type":16},{"name":14,"slug":15,"type":16},{"slug":1622,"name":1622,"fn":1623,"description":1624,"org":1728,"tags":1729,"stars":23,"repoUrl":24,"updatedAt":1635},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1730,1731,1732,1733],{"name":1594,"slug":1595,"type":16},{"name":14,"slug":15,"type":16},{"name":1630,"slug":1631,"type":16},{"name":1633,"slug":1634,"type":16},{"slug":1637,"name":1637,"fn":1638,"description":1639,"org":1735,"tags":1736,"stars":23,"repoUrl":24,"updatedAt":1649},{"slug":8,"name":9,"logoUrl":10,"githubOrg":11},[1737,1738,1739],{"name":1643,"slug":1644,"type":16},{"name":1646,"slug":1647,"type":16},{"name":14,"slug":15,"type":16},14]