How Enterprise Administrators Configure JetBrains AI Enterprise to Route Chat Requests to On-Premises OpenAI-Compatible Models

Organizations in banking, defense, healthcare, and critical infrastructure face strict compliance barriers when adopting AI developer tooling. Regulations such as GDPR, PCI-DSS, HIPAA, and defense export controls prohibit sending proprietary source code or confidential business logic to public multi-tenant AI endpoints like cloud-hosted OpenAI or Anthropic models.

JetBrains addresses this challenge through JetBrains AI Enterprise, an enterprise module managed via JetBrains IDE Services. By deploying IDE Services within an isolated or air-gapped corporate network, enterprise administrators can centrally configure OpenAI-compatible endpoints—such as those served by vLLM, llama.cpp, LM Studio, or private model gateways—and route developers' AI Assistant chat queries entirely through self-hosted infrastructure.

This guide provides a comprehensive, step-by-step walkthrough for configuring JetBrains AI Enterprise to route chat requests to on-premises OpenAI-compatible models, provisioning enterprise profiles, and securing the internal development perimeter.

Architectural Separation: AI Chat vs. Code Completion

Before configuring endpoints, administrators must understand how JetBrains AI partitions its capabilities:

1. Interactive AI Chat and Contextual Actions: AI Chat, code explanation, refactoring suggestions, unit test generation, and commit message drafting communicate with standard LLMs via OpenAI-compatible endpoints (/v1/chat/completions). Administrators can point these capabilities to any internal model server running open-weights models (such as Qwen 2.5 Coder, Llama 3.3, or DeepSeek Coder).
2. Native Code Completion: Full-line and multi-line code completion requires ultra-low latency. In an isolated enterprise environment, JetBrains AI Enterprise powers code completion through JetBrains Mellum, an on-premises completion engine running on internal NVIDIA GPU hardware (such as L40, H100, or H200).

By pairing an on-premises OpenAI-compatible chat server with a self-hosted Mellum instance, organizations achieve 100% air-gapped coverage across both chat and completion workflows without exposing a single line of code to the public internet.

Prerequisites

Before starting the configuration in JetBrains IDE Services, ensure the following components are operational:

• JetBrains IDE Services Server: Installed on-premises (via Kubernetes Helm charts or Docker) and accessible over the corporate network with valid TLS certificates.
• AI Enterprise Enabled: Activated within your organization's JetBrains Account and provisioned in the IDE Services instance under the License & Activation settings.
• On-Premises Model Serving Cluster: An active inference server exposing an OpenAI-compatible API endpoint (such as http://vllm-cluster.internal:8000/v1 or an internal API gateway https://ai-gateway.corp.internal/v1).
  • Supported serving backends include vLLM, llama.cpp, LM Studio, or custom API proxies.
  • The endpoint must support standard /v1/chat/completions request and response payloads.
  • The model must have an appropriate chat template configured in its tokenizer.
• Client Prerequisites: Developers must have JetBrains Toolbox App (version 2.5.1 or later) installed and connected to the corporate IDE Services Server.

Step 1: Deploy and Verify the On-Premises OpenAI-Compatible Server

Ensure your internal inference cluster is reachable by the JetBrains IDE Services server. For example, if deploying with vLLM in an on-premises Kubernetes cluster:

vllm serve Qwen/Qwen2.5-Coder-32B-Instruct \
  --host 0.0.0.0 \
  --port 8000 \
  --dtype auto \
  --tensor-parallel-size 2 \
  --api-key corp-internal-secret-token \
  --chat-template /path/to/chat_template.jinja

Verify that the inference server responds to a health check and an authenticated chat request from the host network running IDE Services:

curl -X POST https://ai-gateway.corp.internal/v1/chat/completions \
  -H "Authorization: Bearer corp-internal-secret-token" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen2.5-Coder-32B-Instruct",
    "messages": [{"role": "user", "content": "Ping"}]
  }'

A valid JSON completion confirming model availability verifies readiness for integration.

Step 2: Add the OpenAI-Compatible Provider in IDE Services

Configuration is managed centrally through the JetBrains IDE Services administrative Web UI:

1. Log in to the JetBrains IDE Services Web UI with an administrator account.
2. In the navigation sidebar, go to Configuration and open the License & Activation tab.
3. Scroll to the AI Enterprise section and click Settings.
4. Select the AI Providers tab.
5. Click Add provider and choose OpenAI Compatible from the list of provider types.
6. In the configuration dialog, populate the required fields:
   • Endpoint: Enter the internal base URL of your model router or inference server (for example, https://ai-gateway.corp.internal/v1 or http://vllm.internal:8000/v1). Do not include the trailing /chat/completions path, as the client appends standard API paths automatically.
   • Authentication / API Key: Enter the Bearer token or corporate secret configured on your inference gateway. If your internal gateway uses custom header authentication, specify the header name and value accordingly.
   • Model Name: Enter the exact model identifier exposed by your inference server (e.g., Qwen/Qwen2.5-Coder-32B-Instruct or meta-llama/Llama-3.3-70B-Instruct).
   • Token Limits: Define the Max input tokens (context window limit) and Max output tokens based on your on-premises GPU VRAM budget and model capabilities.
7. Click Test connection to verify that IDE Services can establish a secure network handshake and successfully ping the model endpoint.
8. Click Save to persist the provider settings.

Step 3: Designate the Provider as Default

When organizations host their own AI models to enforce data privacy, administrators typically designate the on-premises provider as the default so developers do not inadvertently route queries elsewhere:

1. On the AI Providers tab, locate the newly created OpenAI-compatible provider.
2. Mark the provider as Default.
3. If public providers (such as JetBrains AI or cloud-hosted OpenAI Platform) are listed, disable them or remove them from default selection. Note that if JetBrains AI is selected as default, it must be the sole default; setting custom providers as default requires JetBrains AI to be deselected.
4. If your organization also hosts JetBrains Mellum on-premises, switch to the Code Completion settings tab and set Mellum as your dedicated completion provider.
5. Click Save to apply global settings.

Step 4: Configure Profiles and Assign User Access

JetBrains IDE Services uses Profiles to distribute IDE versions, plugins, tool settings, and AI Enterprise permissions to developers:

1. In the IDE Services Web UI, navigate to the Profiles page.
2. Select an existing profile (for example, Core Engineering Profile) or click Create profile.
3. Under the profile configuration, locate the AI Enterprise section.
4. Ensure AI Enterprise is enabled for this profile.
5. In the provider selection dropdown, verify that Default provider (pointing to your configured on-premises OpenAI-compatible endpoint) is selected.
6. Assign the profile to the appropriate user groups or set it as the global default profile for all enterprise users.
7. Click Save changes.

Step 5: Validate the Developer Experience in JetBrains IDEs

Once the configuration is published on the IDE Services server, developer workstations sync the policy automatically:

1. Toolbox App Connection: The developer opens JetBrains Toolbox App and connects to the corporate IDE Services Server using the invitation URL or company SSO credentials.
2. Automated Provisioning: Toolbox App enters Enterprise Mode, pushes the assigned profile to the developer's installed IDEs (IntelliJ IDEA, PyCharm, WebStorm, etc.), and ensures the AI Assistant plugin is installed and activated.
3. IDE AI Chat Window:
   • When the developer opens the AI Assistant tool window in IntelliJ IDEA, the chat interface is automatically connected to the enterprise-provisioned model.
   • The developer can ask questions, generate unit tests, explain code snippets, and review refactoring suggestions.
   • All network traffic originating from the AI Chat panel flows directly to the internal IDE Services proxy or designated on-premises inference endpoint.
4. Administrative Lockdown: Under Settings | Tools | AI Assistant | Providers & API keys, individual Bring Your Own Key (BYOK) overrides and external model configurations remain restricted by corporate policy, preventing developers from manually routing corporate code to public cloud APIs.

Best Practices for Enterprise Security and Infrastructure

To maintain stability and compliance in production environments, implement the following operational safeguards:

Internal TLS and Certificate Management

In air-gapped corporate intranets, servers frequently use private internal Certificate Authorities (CAs). Ensure your custom CA root certificates are trusted by both the JetBrains IDE Services host container and developer workstations. Mismatched or untrusted certificates will cause connection test failures and blocked chat streaming.

GPU Sizing and Concurrency Control

Chat interactions in developer IDEs involve large context payloads containing project file excerpts and syntax trees.

• Size your on-premises GPU clusters with adequate VRAM overhead for KV-cache allocation.
• In vLLM, utilize PagedAttention, continuous batching, and chunked prefill to handle concurrent requests during peak business hours.
• Set appropriate Max input tokens in the IDE Services provider settings to prevent single queries from consuming excessive GPU memory.

Air-Gapped Network Isolation and Audit Logging

• Place your inference cluster behind an internal API gateway or reverse proxy (such as Envoy, Kong, or Nginx).
• Enforce strict mutual TLS (mTLS) or network segmentation between IDE Services and the model nodes.
• Log prompt metadata, user identity, and token usage at the gateway for audit trails while discarding raw prompt text to minimize internal sensitive data storage.

By combining JetBrains IDE Services with an internal OpenAI-compatible model cluster, enterprise engineering organizations successfully unlock the full productivity benefits of JetBrains AI Assistant while keeping proprietary code completely contained within their secure perimeter.

References

• AI Enterprise in isolated infrastructure | IDE Services Documentation
• Manage AI Enterprise | IDE Services Documentation
• JetBrains Mellum | IDE Services Documentation
• Profiles | IDE Services Documentation
• User Guide: Connect to IDE Services Server | IDE Services Documentation