llm-install

Use this skill when user asks to install an LLM from a URL (Hugging Face or Ollama) and wants it ready for local use.

Invocation

• /llm-install <URL>
• Examples:
  • /llm-install https://huggingface.co/HauhauCS/Qwen3.5-4B-Uncensored-HauhauCS-Aggressive
  • /llm-install https://huggingface.co/TrevorJS/gemma-4-E2B-it-uncensored-GGUF
  • /llm-install https://ollama.com/library/qwen2.5-coder

Environment Defaults

• Ollama endpoint: http://172.31.192.1:11434
• Local hardware target: single-user local coding workflow on RTX 3060 12GB VRAM.
• Optimize for stability + quality per GB, not maximum precision.

Decision Policy (Most Logical Variant)

Given a repository with multiple GGUF quantizations, choose in this order:

1. Prefer Q6_K / Q6_K_P when available (best quality/speed balance for local use).
2. Else use Q5_K_M / Q5_K_S.
3. Else use Q4_K_M (safe fallback for constrained disk/VRAM).
4. Use Q8_0 only when user explicitly asks for maximum quality and space allows.
5. Avoid BF16 by default on this host (large footprint).
6. Avoid Q2 / Q3 unless they are the only options.

For repos without GGUF files:

1. Try direct pull if Ollama supports repo format.
2. If repo is Safetensors-only and fails, find a GGUF mirror of the same base model.
3. Tell user exactly which mirror was used and why.

Auth Policy (HF Token)

• First try public pull with no token.
• If pull fails due to auth/private/gated access:
  1. Read token from .env keys in this order: HF_TOKEN, HUGGINGFACEHUB_API_TOKEN, HUGGING_FACE_HUB_TOKEN.
  2. Retry with token in request/command environment.
  3. If token missing or invalid, ask user for intervention.

Install + Verify Steps

1. Resolve target model name and chosen quant.
2. Pull model to Ollama.
3. Verify model appears in /api/tags.
4. Run a minimal health call (/api/chat non-stream) to confirm runtime usability.
5. Confirm model is available to endpoint clients (API/OpenWebUI use same Ollama tags).

Continue Config Sync (Required)

After each successful install, sync both files to current installed models:

• ~/.continue/config.yaml
• .continue/config.yaml (workspace)

Sync rules:

1. Include all models currently returned by Ollama /api/tags.
2. Assign roles:
   • model name contains embed -> embed
   • qwen2.5-coder:7b-Fallback_Coding -> autocomplete
   • everything else -> chat
3. Keep tabAutocompleteModel set to qwen2.5-coder:7b-Fallback_Coding when present.
4. Preserve endpoint as http://172.31.192.1:11434.

Output Contract

Always return:

1. URL provided
2. Selected quant + reason
3. Installed model tag(s)
4. Verification result (/api/tags and chat probe)
5. Continue config sync result
6. Any fallback/mirror used
7. Any required user intervention