How to Configure CodeGPT with Ollama for Completely Local Code Assistance

Engineering teams working in financial technology, healthcare, defense, and enterprise software face a strict security mandate: proprietary source code and intellectual property must never leave the organization's boundary. While cloud-based AI programming assistants deliver impressive productivity gains, sending internal codebase context, proprietary algorithms, and sensitive database schemas across external SaaS APIs violates compliance frameworks and corporate data policies.

Developers can eliminate external data exposure without sacrificing in-editor AI assistance by pairing the CodeGPT extension in Visual Studio Code with Ollama. CodeGPT acts as the IDE-native interface for code generation, chat, and inline completions, while Ollama serves open-weight models locally via an on-device HTTP endpoint (http://localhost:11434). Every token generated and every line analyzed stays entirely on local hardware.

Architecture: Cloud AI vs. Local Ollama Execution

Understanding the difference between conventional cloud AI coding assistants and a local CodeGPT/Ollama pipeline highlights why this setup satisfies air-gapped and high-compliance requirements.

When CodeGPT is pointed to Ollama, the extension sends standard JSON requests to http://localhost:11434/api/chat or completion endpoints. Ollama loads model weights directly into local RAM or GPU VRAM, processes the context, and streams responses back to VS Code through the loopback interface.

Choosing the Right Local Coding Model

Model selection dictates both inference speed and coding accuracy. Match the model parameter size to your workstation's hardware specifications:

1. Qwen2.5-Coder (7B / 14B): Exceptional multilingual coding accuracy, competitive with larger cloud models on code refactoring, complex logic generation, and unit test synthesis. The 7B variant runs smoothly on 16 GB RAM or 8 GB VRAM.
2. DeepSeek-Coder (6.7B / 1.3B): Highly regarded for algorithmic precision and repository comprehension.
3. Codestral (22B): High-capacity programming model from Mistral, ideal for dedicated developer workstations with 24 GB+ VRAM or unified memory Macs (32 GB+).
4. Qwen2.5-Coder (1.5B / 3B): Ultra-fast inference with minimal memory footprint, making it the premier choice for instantaneous inline ghost-text autocomplete.

Step 1: Install Ollama and Pull the Coding Model

Install Ollama according to your operating system:

• macOS / Linux: Download from the official website or install via package managers / terminal scripts.
• Windows: Download the Windows installer from ollama.com.

Once installed, verify that the Ollama background daemon is operational:

ollama --version

Download your selected coding model. Use ollama pull to download weights directly without launching an interactive terminal session:

# Pull the standard balanced programming model
ollama pull qwen2.5-coder:7b

# (Optional) Pull a lightweight model specifically for fast inline autocomplete
ollama pull qwen2.5-coder:1.5b

Verify that the model downloaded successfully:

ollama list

Step 2: Tune Context Length with a Custom Modelfile

Coding tasks require substantial context window capacity so the model can digest surrounding imports, type definitions, and function signatures. By default, many local models initialize with a conservative context size (typically 2,048 or 4,096 tokens) to conserve memory.

For coding assistance, expand the context window to 8,192 or 16,384 tokens using a custom Modelfile:

1. Create a plain text file named Modelfile in any local folder:

FROM qwen2.5-coder:7b

# Expand context window to 8192 tokens
PARAMETER num_ctx 8192

# Lower temperature for deterministic, syntactically correct code
PARAMETER temperature 0.2

# Tune top_p for focused sampling
PARAMETER top_p 0.95

2. Build the tailored local model in your terminal:

ollama create my-local-coder -f ./Modelfile

Alternatively, you can export OLLAMA_CONTEXT_LENGTH=8192 into your shell environment before launching ollama serve. Keep in mind that larger context windows allocate additional KV cache in VRAM or RAM.

Step 3: Install and Configure CodeGPT in Visual Studio Code

1. Launch Visual Studio Code.
2. Open the Extensions Marketplace (Ctrl+Shift+X on Windows/Linux, Cmd+Shift+X on macOS).
3. Search for CodeGPT (DanielSanMedium.dscodegpt) and click Install.
4. Open the CodeGPT interface by clicking the CodeGPT icon in the left Activity Bar or pressing Ctrl+Shift+P / Cmd+Shift+P and executing CodeGPT: Focus on Chat View.
5. Access model settings by clicking the model selector / gear icon at the top of the CodeGPT chat pane (or select Manage my AI Models).
6. Under the category selection, switch to Local LLMs.
7. In the Provider dropdown, select Ollama.
8. In the Server URL / API Base URL field, verify or enter:

http://localhost:11434

(If Ollama is deployed on a dedicated GPU workstation within your isolated enterprise intranet, enter that machine's private IP, such as http://192.168.1.100:11434).

9. In the Model dropdown list, select your pulled model (e.g., qwen2.5-coder:7b or my-local-coder).
10. Click outside the panel or confirm to connect. CodeGPT will establish a local handshake with Ollama's API.

Step 4: Configure Autocomplete and Chat Capabilities

CodeGPT supports two core workflows: conversational coding assistance in the sidebar and real-time ghost-text autocomplete in the editor.

Conversational Assistance and Code Actions

You can now highlight code in your active editor, open CodeGPT, and run contextual commands:

• Refactoring routines and explaining complex business logic.
• Generating unit test suites for functions without external API calls.
• Debugging compiler or runtime stack traces locally.

Setting Up Inline Autocomplete

1. Open the CodeGPT sidebar and locate the Autocomplete settings toggle.
2. Switch Autocomplete to ON.
3. Select your preferred local model. For instant, low-latency completions while typing, select a smaller parameter model like qwen2.5-coder:1.5b or codellama:code.
4. Adjust the Suggestion Delay and Max Tokens based on your workstation's hardware responsiveness.

Step 5: Enterprise Hardening and Zero-Egress Verification

To ensure total compliance in security-sensitive or regulated environments, apply these verification steps:

1. Disable Visual Studio Code Telemetry

Ensure the editor itself does not dispatch usage metadata to external telemetry endpoints:

1. Open VS Code Settings (Ctrl+, or Cmd+,).
2. Search for telemetry.telemetryLevel.
3. Set the value to off.

2. Lock Ollama to Loopback

By default, Ollama binds only to 127.0.0.1:11434. Ensure you have not exposed the port externally by avoiding setting OLLAMA_HOST=0.0.0.0 unless intentionally sharing the server across an internal subnet. Keeping it on 127.0.0.1 prevents other hosts on your local network from querying your machine's instance.

3. Conduct an Offline Test

1. Disconnect your developer machine from Wi-Fi and unplug any Ethernet cable (or apply an outbound blocking rule via your host firewall).
2. Open a code file in VS Code.
3. Prompt CodeGPT with a refactoring question and trigger an inline completion.
4. Verify that CodeGPT and Ollama respond normally without error. If responses generate successfully while offline, your coding assistance pipeline is completely decoupled from external networks.

Troubleshooting Common Configurations

• Connection Refused / Failed to Fetch: Ensure Ollama is running in the background. On macOS/Windows, verify that the Ollama tray application is active. On Linux, run systemctl status ollama or launch ollama serve in a terminal.
• CORS Restrictions: If CodeGPT webview indicates a cross-origin error, set OLLAMA_ORIGINS="*" in your environment variables and restart Ollama.
• High Latency or Memory Starvation: If token generation slows to a crawl, check your GPU VRAM allocation. When a model exceeds physical VRAM, Ollama offloads layers to system CPU RAM, causing significant throughput drops. Reduce parameter size (switch from 14B to 7B, or from 7B to 1.5B/3B) or decrease num_ctx in your Modelfile.

References

• Ollama | CodeGPT
• CodeGPT: AI Coding Agents & Chat - Visual Studio Marketplace
• Ollama · Run AI models locally and in the cloud
• GitHub - ollama/ollama: Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models. · GitHub