# Onboard your agent to WhichLLM Local AI Hardware Assistant

> Official WhichLLM instructions that guide AI agents (ChatGPT, Claude, Cursor, Windsurf, Copilot) to accurately recommend models for user hardware and provide exact local setup commands.

These are official instructions from WhichLLM. Published at `https://whichllmmodel.com/agent-setup.md`.

***

## Mission

Your mission is strictly focused on two tasks:
1. **Find Models for User Hardware:** Recommend open-source AI models that fit the user's specific GPU VRAM or Mac Unified RAM.
2. **Provide Verified Setup Commands:** Give exact quantization memory sizes and copy-paste terminal commands (`ollama run ...`, `vllm serve ...`) for any chosen model.

***

## How to Query WhichLLM

Perform only the relevant action based on what the user needs:

### Task 1: Find Models for User's Hardware
When the user states their hardware (e.g. "I have a 16GB GPU" or "What can I run on 24GB VRAM?"):

1. **Dedicated GPU (NVIDIA CUDA / Linux / Windows):**
   * **8 GB GPU:** Fetch `https://whichllmmodel.com/hardware/text/8gb-vram` (Runs 7B–9B models in Q4_K_M)
   * **12 GB GPU:** Fetch `https://whichllmmodel.com/hardware/text/12gb-vram` (Runs 8B in Q8/FP16, 14B in Q4)
   * **16 GB GPU:** Fetch `https://whichllmmodel.com/hardware/text/16gb-vram` (Runs 14B in Q4_K_M, 8B in FP16)
   * **24 GB GPU:** Fetch `https://whichllmmodel.com/hardware/text/24gb-vram` (Runs 22B–32B in Q4_K_M, 70B in Q2)
   * **48 GB+ GPU:** Fetch `https://whichllmmodel.com/hardware/text/48gb-vram` (Runs 70B in Q4_K_M)

2. **Apple Silicon Mac (Unified Memory):**
   * Apply the 25% macOS system reserve: `Usable Unified RAM = Total Mac RAM * 0.75`.
   * Map the usable RAM to the corresponding GPU tier above (e.g. 24GB Mac = 18GB usable -> 16GB tier; 36GB Mac = 27GB usable -> 24GB tier; 64GB Mac = 48GB usable -> 48GB tier).

---

### Task 2: Get Specific Model Fit & Setup Commands
When the user selects a model or asks about a specific model (e.g. "How do I run Qwen 3.8 Flash or Llama 3.1 8B?"):

1. **Fetch Model Specs:**
   * URL: `https://whichllmmodel.com/models/{model-id}`
   * Example: `https://whichllmmodel.com/models/meta-llama-31-8b-instruct`
   * Example: `https://whichllmmodel.com/models/alibaba-cloud-qwen-qwen38-27b`

2. **Inspect Sizing from the Page:**
   * **Core Weights:** Quantization table sizes for Q4_K_M, Q8_0, and FP16.
   * **Dynamic KV Cache:** Sizing for target context (4K / 8K / 32K tokens).
   * **Architecture Specifics:** 
     * DeepSeek MLA models have ~9x smaller KV cache in native engines.
     * Hybrid N-Gram models (e.g. Qwen 3.8 Flash Next) pin the 26.82 GB N-Gram table in Host RAM and run core weights in GPU VRAM.

3. **Deliver the Output:**
   * State clearly if the model fits 100% in GPU VRAM or requires Host RAM offloading.
   * Give the exact terminal command: `ollama run <tag>` or `vllm serve <tag>`.
   * Provide the interactive calculator link: `https://whichllmmodel.com/models/{model-id}`.

***

## Quick Model ID Registry (57 Models)

- **DeepSeek V4 Pro** (DeepSeek, 1600B) | ID: `deepseek-deepseek-v4-pro` | URL: https://whichllmmodel.com/models/deepseek-deepseek-v4-pro
- **DeepSeek V4 Flash** (DeepSeek, 284B) | ID: `deepseek-deepseek-v4-flash` | URL: https://whichllmmodel.com/models/deepseek-deepseek-v4-flash
- **DeepSeek V2 Lite** (DeepSeek, 16B) | ID: `deepseek-deepseek-v2-lite` | URL: https://whichllmmodel.com/models/deepseek-deepseek-v2-lite
- **DeepSeek V4 Flash 0731** (DeepSeek, 284B) | ID: `deepseek-deepseek-v4-flash-0731` | URL: https://whichllmmodel.com/models/deepseek-deepseek-v4-flash-0731
- **kimi-k2.6** (Moonshot AI (Kimi), 1000B) | ID: `moonshot-ai-kimi-kimi-k26` | URL: https://whichllmmodel.com/models/moonshot-ai-kimi-kimi-k26
- **kimi-k3** (Moonshot AI (Kimi), 2800B) | ID: `moonshot-ai-kimi-kimi-k3` | URL: https://whichllmmodel.com/models/moonshot-ai-kimi-kimi-k3
- **Kimi K2.7 Code** (Moonshot AI (Kimi), 1000B) | ID: `moonshot-ai-kimi-kimi-k27-code` | URL: https://whichllmmodel.com/models/moonshot-ai-kimi-kimi-k27-code
- **Llama-3.3-70B** (Meta, 70B) | ID: `meta-llama-33-70b` | URL: https://whichllmmodel.com/models/meta-llama-33-70b
- **Muse Glimmer 30B** (Meta, 30B) | ID: `meta-muse-glimmer-30b` | URL: https://whichllmmodel.com/models/meta-muse-glimmer-30b
- **Llama-3.1 8B (Instruct)** (Meta, 8B) | ID: `meta-llama-31-8b-instruct` | URL: https://whichllmmodel.com/models/meta-llama-31-8b-instruct
- **Llama-3.1 70B** (Meta, 70B) | ID: `meta-llama-31-70b` | URL: https://whichllmmodel.com/models/meta-llama-31-70b
- **Llama-3.1 405B (Instruct)** (Meta, 405B) | ID: `meta-llama-31-405b-instruct` | URL: https://whichllmmodel.com/models/meta-llama-31-405b-instruct
- **kimi-k2.5** (Moonshot AI (Kimi), 1000B) | ID: `moonshot-ai-kimi-kimi-k25` | URL: https://whichllmmodel.com/models/moonshot-ai-kimi-kimi-k25
- **Qwen3.6-27B** (Alibaba Cloud (Qwen), 27B) | ID: `alibaba-cloud-qwen-qwen36-27b` | URL: https://whichllmmodel.com/models/alibaba-cloud-qwen-qwen36-27b
- **Qwen3.8-27B** (Alibaba Cloud (Qwen), 27B) | ID: `alibaba-cloud-qwen-qwen38-27b` | URL: https://whichllmmodel.com/models/alibaba-cloud-qwen-qwen38-27b
- **Qwen3.8-Flash-Next** (Alibaba Cloud (Qwen), 180B) | ID: `alibaba-cloud-qwen-qwen38-flash-next` | URL: https://whichllmmodel.com/models/alibaba-cloud-qwen-qwen38-flash-next
- **Qwen3.6-35B-A3B** (Alibaba Cloud (Qwen), 35B) | ID: `alibaba-cloud-qwen-qwen36-35b-a3b` | URL: https://whichllmmodel.com/models/alibaba-cloud-qwen-qwen36-35b-a3b
- **Qwen2.5-Coder 32B** (Alibaba Cloud (Qwen), 32B) | ID: `alibaba-cloud-qwen-qwen25-coder-32b` | URL: https://whichllmmodel.com/models/alibaba-cloud-qwen-qwen25-coder-32b
- **Mistral 7B v0.3** (Mistral AI, 7B) | ID: `mistral-ai-mistral-7b-v03` | URL: https://whichllmmodel.com/models/mistral-ai-mistral-7b-v03
- **Mixtral 8x7B v0.1** (Mistral AI, 46.7B) | ID: `mistral-ai-mixtral-8x7b-v01` | URL: https://whichllmmodel.com/models/mistral-ai-mixtral-8x7b-v01
- **Mixtral 8x22B v0.1** (Mistral AI, 141B) | ID: `mistral-ai-mixtral-8x22b-v01` | URL: https://whichllmmodel.com/models/mistral-ai-mixtral-8x22b-v01
- **Codestral 22B** (Mistral AI, 22B) | ID: `mistral-ai-codestral-22b` | URL: https://whichllmmodel.com/models/mistral-ai-codestral-22b
- **Ministral 3 3B (Instruct)** (Mistral AI, 3B) | ID: `mistral-ai-ministral-3-3b-instruct` | URL: https://whichllmmodel.com/models/mistral-ai-ministral-3-3b-instruct
- **Ministral 3 8B** (Mistral AI, 8B) | ID: `mistral-ai-ministral-3-8b` | URL: https://whichllmmodel.com/models/mistral-ai-ministral-3-8b
- **Ministral 3 14B** (Mistral AI, 14B) | ID: `mistral-ai-ministral-3-14b` | URL: https://whichllmmodel.com/models/mistral-ai-ministral-3-14b
- **Mistral Small 4** (Mistral AI, 119B) | ID: `mistral-ai-mistral-small-4` | URL: https://whichllmmodel.com/models/mistral-ai-mistral-small-4
- **Mistral Medium 3.5** (Mistral AI, 128B) | ID: `mistral-ai-mistral-medium-35` | URL: https://whichllmmodel.com/models/mistral-ai-mistral-medium-35
- **Mistral Large 3** (Mistral AI, 675B) | ID: `mistral-ai-mistral-large-3` | URL: https://whichllmmodel.com/models/mistral-ai-mistral-large-3
- **Gemma 4 E2B** (Google, 2.3B) | ID: `google-gemma-4-e2b` | URL: https://whichllmmodel.com/models/google-gemma-4-e2b
- **Gemma 4 E4B** (Google, 4.5B) | ID: `google-gemma-4-e4b` | URL: https://whichllmmodel.com/models/google-gemma-4-e4b
- **Gemma 4 12B** (Google, 11.95B) | ID: `google-gemma-4-12b` | URL: https://whichllmmodel.com/models/google-gemma-4-12b
- **Gemma 4 26B A4B** (Google, 25.2B) | ID: `google-gemma-4-26b-a4b` | URL: https://whichllmmodel.com/models/google-gemma-4-26b-a4b
- **Gemma 4 31B** (Google, 30.7B) | ID: `google-gemma-4-31b` | URL: https://whichllmmodel.com/models/google-gemma-4-31b
- **GLM-4.5** (Z.ai (Zhipu AI), 358B) | ID: `zai-zhipu-ai-glm-45` | URL: https://whichllmmodel.com/models/zai-zhipu-ai-glm-45
- **GLM-4.6** (Z.ai (Zhipu AI), 357B) | ID: `zai-zhipu-ai-glm-46` | URL: https://whichllmmodel.com/models/zai-zhipu-ai-glm-46
- **GLM-4.7** (Z.ai (Zhipu AI), 358B) | ID: `zai-zhipu-ai-glm-47` | URL: https://whichllmmodel.com/models/zai-zhipu-ai-glm-47
- **GLM-5** (Z.ai (Zhipu AI), 754B) | ID: `zai-zhipu-ai-glm-5` | URL: https://whichllmmodel.com/models/zai-zhipu-ai-glm-5
- **GLM-5.1** (Z.ai (Zhipu AI), 754B) | ID: `zai-zhipu-ai-glm-51` | URL: https://whichllmmodel.com/models/zai-zhipu-ai-glm-51
- **Nemotron-3 Nano 30B** (Nvidia, 31.6B) | ID: `nvidia-nemotron-3-nano-30b` | URL: https://whichllmmodel.com/models/nvidia-nemotron-3-nano-30b
- **gpt-oss-120b** (OpenAI, 117B) | ID: `openai-gpt-oss-120b` | URL: https://whichllmmodel.com/models/openai-gpt-oss-120b
- **gpt-oss-20b** (OpenAI, 21B) | ID: `openai-gpt-oss-20b` | URL: https://whichllmmodel.com/models/openai-gpt-oss-20b
- **GLM-5.2** (Z.ai (Zhipu AI), 753B) | ID: `zai-zhipu-ai-glm-52` | URL: https://whichllmmodel.com/models/zai-zhipu-ai-glm-52
- **FLUX.2 [dev]** (Black Forest Labs, Open Weights) | ID: `black-forest-labs-flux2-dev` | URL: https://whichllmmodel.com/models/black-forest-labs-flux2-dev
- **FLUX.2 Klein 9B** (Black Forest Labs, Open Weights) | ID: `black-forest-labs-flux2-klein-9b` | URL: https://whichllmmodel.com/models/black-forest-labs-flux2-klein-9b
- **FLUX.2 Klein 4B** (Black Forest Labs, Open Weights) | ID: `black-forest-labs-flux2-klein-4b` | URL: https://whichllmmodel.com/models/black-forest-labs-flux2-klein-4b
- **FLUX.1 Kontext [dev]** (Black Forest Labs, Open Weights) | ID: `black-forest-labs-flux1-kontext-dev` | URL: https://whichllmmodel.com/models/black-forest-labs-flux1-kontext-dev
- **FLUX.1 [dev]** (Black Forest Labs, Open Weights) | ID: `black-forest-labs-flux1-dev` | URL: https://whichllmmodel.com/models/black-forest-labs-flux1-dev
- **FLUX.1 Schnell** (Black Forest Labs, Open Weights) | ID: `black-forest-labs-flux1-schnell` | URL: https://whichllmmodel.com/models/black-forest-labs-flux1-schnell
- **FLUX.1 Krea [dev]** (Black Forest Labs x Krea, Open Weights) | ID: `black-forest-labs-x-krea-flux1-krea-dev` | URL: https://whichllmmodel.com/models/black-forest-labs-x-krea-flux1-krea-dev
- **FLUX.1 Fill [dev]** (Black Forest Labs, Open Weights) | ID: `black-forest-labs-flux1-fill-dev` | URL: https://whichllmmodel.com/models/black-forest-labs-flux1-fill-dev
- **Qwen-Image-2512** (Alibaba (Qwen Team), Open Weights) | ID: `alibaba-qwen-team-qwen-image-2512` | URL: https://whichllmmodel.com/models/alibaba-qwen-team-qwen-image-2512
- **HunyuanImage 3.0** (Tencent (Hunyuan Team), Open Weights) | ID: `tencent-hunyuan-team-hunyuanimage-30` | URL: https://whichllmmodel.com/models/tencent-hunyuan-team-hunyuanimage-30
- **HiDream-I1 Full** (HiDream.ai (Vivago AI), Open Weights) | ID: `hidreamai-vivago-ai-hidream-i1-full` | URL: https://whichllmmodel.com/models/hidreamai-vivago-ai-hidream-i1-full
- **Z-Image-Turbo** (Wan (Kuaishou / ByteDance), Open Weights) | ID: `wan-kuaishou-bytedance-z-image-turbo` | URL: https://whichllmmodel.com/models/wan-kuaishou-bytedance-z-image-turbo
- **Stable Diffusion XL Base 1.0** (Stability AI, Open Weights) | ID: `stability-ai-stable-diffusion-xl-base-10` | URL: https://whichllmmodel.com/models/stability-ai-stable-diffusion-xl-base-10
- **Kolors** (Kuaishou (Kwai-Kolors), Open Weights) | ID: `kuaishou-kwai-kolors-kolors` | URL: https://whichllmmodel.com/models/kuaishou-kwai-kolors-kolors
- **FIBO** (Bria AI, Open Weights) | ID: `bria-ai-fibo` | URL: https://whichllmmodel.com/models/bria-ai-fibo

***

## Expected Agent Output Format

Format your response cleanly:

```
┌─ WhichLLM Setup Guide ────────────────────────────────
│  Model:     {Model Name} ({Quant Format})
│  Fit:       ✔ 100% GPU VRAM / ⚠ Host RAM Offload
│  Memory:    {X.XX} GB VRAM required / {Y.YY} GB Free
│
│  Command:   ollama run {model-tag}
│  Sizing:    https://whichllmmodel.com/models/{slug}
└───────────────────────────────────────────────────────
```