Skill

add-model

Name: add-model
Author: LiteLLM

add models to LiteLLM proxy

Published by LiteLLM Updated Jul 14

Covers LLM AI Infrastructure LiteLLM

Description

Add a new model to a live LiteLLM proxy. Walks the user through picking a provider, entering the deployment name and credentials, calls POST /model/new, then test-calls the model to confirm it routes correctly. Use when the user wants to add, register, deploy, or configure a new model on a LiteLLM proxy instance.

SKILL.md

Add Model

Add a new LLM to a live LiteLLM proxy.

Setup

Ask for these if not already known:

LITELLM_BASE_URL  — e.g. https://my-proxy.example.com
LITELLM_API_KEY   — proxy admin key

API reference: https://litellm.vercel.app/docs/proxy/model_management

Ask the user

Public model name — what callers will send in "model": "..." (e.g. gpt-4o, my-claude, llama3)
Provider — pick from the table below
Credentials — whatever that provider needs

Provider table

Provider	`litellm_params.model`	Extra params
OpenAI	`openai/gpt-4o`	`api_key`
Azure OpenAI	`azure/<deployment-name>`	`api_key`, `api_base`, `api_version`
Anthropic	`anthropic/claude-3-5-sonnet-20241022`	`api_key`
AWS Bedrock	`bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0`	AWS creds via env
Google Vertex	`vertex_ai/gemini-1.5-pro`	`vertex_project`, `vertex_location`
Ollama	`ollama/llama3`	`api_base` (e.g. `http://localhost:11434`)
Groq	`groq/llama-3.3-70b-versatile`	`api_key`
Together AI	`together_ai/meta-llama/Llama-3-70b`	`api_key`
Mistral	`mistral/mistral-large-latest`	`api_key`

Full list: https://docs.litellm.ai/docs/providers

Run

curl -s -X POST "$BASE/model/new" \
  -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "<public-name>",
    "litellm_params": {
      "model": "<provider/deployment>",
      "api_key": "<key>",
      "api_base": "<base_if_needed>",
      "api_version": "<version_if_azure>"
    }
  }'

Test it

After adding, verify it routes:

curl -s -X POST "$BASE/chat/completions" \
  -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<public-name>",
    "messages": [{"role": "user", "content": "say hi"}],
    "max_tokens": 10
  }'

Output

Show model_id from the response — needed to update or delete the model later. Report pass/fail from the test call.

More skills from the litellm-skills repository

View all 21 skills

SkillTagsUpdated

More from LiteLLM

View publisher

SkillTagsUpdated