How to Run an LLM Locally in Copilot

Somatirtha

GitHub Copilot: GitHub Copilot supports local models through its app, CLI and compatible providers; this guide uses Ollama as a local runtime.

Ollama: Ollama provides a straightforward way to download and run open-source language models locally on Windows, macOS and Linux desktop computers.

Local Model Selection: Choose a coding-capable model that fits your available RAM or GPU memory; larger models need more resources to run smoothly.

Qwen2.5-Coder: Open a terminal after installing Ollama, then download your chosen coding model locally with this following command: ollama pull qwen2.5-coder:7b.

Ollama API: Start the Ollama application and confirm its local service is running; its default API endpoint is http://localhost:11434 on your computer.

Copilot Model Providers: In the GitHub Copilot app, open Settings, choose Model providers, select Add provider, and pick Ollama from the available options.

Provider Configuration: Enter the provider details requested by Copilot, including the local endpoint and model information, then save your provider configuration successfully.

Copilot Model Picker: Open a Copilot session and use the model picker to select your configured local model before sending coding prompts locally.

Privacy And Hardware Settings: Remember that local inference has hardware limits; Copilot's cloud features may still communicate online, so check privacy and routing settings.

Read More Stories!
Join our WhatsApp Channel to get the latest news, exclusives and videos on WhatsApp