How to Self-Host AgentGPT with Docker and Connect Local or Private LLM Endpoints

Deploying AgentGPT on your own infrastructure allows engineering and security teams to run autonomous AI agents without exposing proprietary prompts, internal project goals, or customer data to external cloud APIs. Because autonomous agent loops recursively generate subtasks, evaluate execution outcomes, and query language models dozens of times per run, self-hosting with local or private model endpoints also eliminates runaway cloud token expenses.

AgentGPT (developed by Reworkd) provides an open-source, web-based autonomous agent platform consisting of a Next.js frontend, a FastAPI platform backend, and a relational MySQL database. By default, it communicates with OpenAI's public endpoints. However, by properly configuring Docker networking, environment variables, and OpenAI-compatible inference servers like Ollama, LocalAI, or vLLM, you can establish an entirely air-gapped, privacy-compliant agent environment.

Architectural Overview & The Docker Networking Trap

AgentGPT's self-hosted architecture is managed via Docker Compose across three core services:

1. Frontend (next): Port 3000, providing the visual agent orchestration dashboard.
2. Backend Platform (platform): Port 8000, running FastAPI and managing agent execution loops, task planning, and LLM communication.
3. Database (db): Port 3306, MySQL storing agent run histories, task statuses, and user configurations.

+-------------------------------------------------------------------------+
| Docker Network (agentgpt-network)                                       |
|                                                                         |
|  [ Frontend: Next.js ] <---> [ Backend: FastAPI Platform ] <---> [ DB ] |
|       (:3000)                        (:8000)                   (:3306)  |
+------------------------------------------|------------------------------+
                                           | (REWORKD_PLATFORM_OPENAI_API_BASE)
                                           v
                     +--------------------------------------------+
                     | Host Machine / Private Network             |
                     | host.docker.internal:11434/v1              |
                     | [ Ollama / vLLM / LocalAI Private Gateway] |
                     +--------------------------------------------+

The Container-to-Host Network Boundary

The most common mistake when self-hosting AgentGPT with local LLMs is setting the API base URL to http://localhost:11434 or http://127.0.0.1:11434. Inside a Docker container, localhost resolves to the container's isolated network namespace, not the host operating system where your local inference server runs.

To bridge this connection:

• macOS and Windows (Docker Desktop): Use http://host.docker.internal:11434/v1.
• Linux: Add extra_hosts: ["host.docker.internal:host-gateway"] to the platform service definition in docker-compose.yml, allowing the container to resolve host.docker.internal to the host bridge IP.
• Shared Docker Network: If running Ollama or LocalAI inside Docker alongside AgentGPT, attach both to the same compose network and use the service name (e.g., http://ollama:11434/v1).

Step 1: Prepare the Local or Private LLM Endpoint

Your local LLM runner must expose an OpenAI-compatible HTTP REST endpoint (specifically supporting /v1/chat/completions and /v1/models).

Option A: Ollama (Recommended for Local Workstations)

Install Ollama and bind it to all network interfaces so Docker containers can reach it:

# On Linux, configure the systemd service or launch with OLLAMA_HOST
OLLAMA_HOST=0.0.0.0:11434 ollama serve

Pull a model optimized for instruction following and structured JSON reasoning:

ollama pull qwen2.5:14b
# or
ollama pull llama3.1:8b

Verify that the OpenAI-compatible endpoint responds locally:

curl http://localhost:11434/v1/models

Option B: vLLM or Enterprise Gateway (High-Throughput / GPU Clusters)

For private clusters or on-premise servers, start a vLLM OpenAI-compatible server:

python3 -m vllm.entrypoints.openai.api_server \
  --model meta-llama/Llama-3.1-8B-Instruct \
  --host 0.0.0.0 \
  --port 8000

Step 2: Clone and Configure AgentGPT

Clone the official repository:

git clone https://github.com/reworkd/AgentGPT.git
cd AgentGPT

AgentGPT relies on environment variables defined in the repository root and inside individual service directories. Copy the template files:

cp .env.example .env

Critical Environment Settings in .env

Edit .env (and platform/.env if using manual builds) with the following adjustments:

# ----------------------------------------
# LLM Endpoint Configuration
# ----------------------------------------
# Point to your local endpoint via host.docker.internal.
# CRITICAL: Do NOT append a trailing slash (/v1/, not /v1//).
REWORKD_PLATFORM_OPENAI_API_BASE=http://host.docker.internal:11434/v1

# OpenAI SDKs require a non-empty string for authentication validation.
# Local endpoints typically ignore this value.
OPENAI_API_KEY=local-secret-key

# ----------------------------------------
# Database Configuration
# ----------------------------------------
DATABASE_URL=mysql://root:password@db:3306/agentgpt
MYSQL_ROOT_PASSWORD=password
MYSQL_DATABASE=agentgpt

# ----------------------------------------
# NextAuth & Secret Tokens
# ----------------------------------------
NEXTAUTH_SECRET=generate-a-secure-random-32-byte-hex-key
NEXTAUTH_URL=http://localhost:3000

Syntax Tip: Ensure REWORKD_PLATFORM_OPENAI_API_BASE does not end with /chat/completions or a trailing slash /. AgentGPT's internal client appends /chat/completions dynamically; an extra slash will cause HTTP 404 or path resolution errors.

Step 3: Configure docker-compose.yml for Linux Host Resolution

If deploying on a Linux machine, update docker-compose.yml under the platform service to ensure host resolution succeeds:

services:
  platform:
    build:
      context: ./platform
      dockerfile: Dockerfile
    environment:
      - REWORKD_PLATFORM_OPENAI_API_BASE=http://host.docker.internal:11434/v1
      - OPENAI_API_KEY=local-secret-key
      - DATABASE_URL=mysql://root:password@db:3306/agentgpt
    ports:
      - "8000:8000"
    extra_hosts:
      - "host.docker.internal:host-gateway"
    depends_on:
      - db

  frontend:
    build:
      context: ./next
      dockerfile: Dockerfile
    ports:
      - "3000:3000"
    environment:
      - NEXTAUTH_URL=http://localhost:3000
      - NEXTAUTH_SECRET=generate-a-secure-random-32-byte-hex-key
    depends_on:
      - platform

  db:
    image: mysql:8.0
    environment:
      MYSQL_ROOT_PASSWORD: password
      MYSQL_DATABASE: agentgpt
    ports:
      - "3306:3306"
    volumes:
      - db_data:/var/lib/mysql

volumes:
  db_data:

Step 4: Build and Start Containers

Start all services using Docker Compose:

docker compose up -d --build

Monitor container initialization and verify the platform connects to the database:

docker compose logs -f platform

Once all containers indicate healthy statuses, open your browser and navigate to http://localhost:3000.

Step 5: Configure Autonomous Workflows in the UI

1. Access Settings: Click the Settings gear icon in the bottom-left navigation bar of the AgentGPT UI.
2. Model Selection: Select custom model input or provide the exact model tag running on your local inference server (e.g., qwen2.5:14b or llama3.1:8b).
3. Execution Parameters:
   • Temperature: Set to 0.2 – 0.4. Autonomous task generation demands deterministic structure and JSON parsing compliance. Higher temperatures often lead to formatting errors in agent subtask schemas.
   • Loop Threshold / Max Loops: Set an initial cap of 5 to 10 loops. Because local inference can take several seconds per step depending on hardware, constraining loops prevents prolonged queue locking.
4. Deploy an Agent:
   • Agent Name: Local-Market-Researcher
   • Goal: Analyze key adoption barriers for on-premise AI deployments and provide a prioritized list of five technical mitigations.
   • Click Deploy Agent.
5. Observe Execution: Watch the live dashboard as the agent breaks down the goal into tasks, executes each task sequentially via the local model endpoint, evaluates the results, and displays the final synthesized summary.

Critical Practical Considerations & Model Selection

1. Model Capability Threshold for Autonomous Agents

Autonomous agents put significantly higher demands on models than simple conversational chatbots. The model must:

• Maintain strict adherence to JSON schemas for task creation.
• Accurately evaluate whether an objective has been achieved without getting trapped in infinite repetition.
• Follow complex system prompts defining agent roles and constraints.

Models with fewer than 7 billion parameters (or models tuned purely for chat without function/tool calling optimization) frequently hallucinate invalid JSON or generate redundant tasks. For reliable self-hosted performance, prioritize:

• Recommended Minimum: Llama-3.1-8B-Instruct or Qwen-2.5-7B-Instruct.
• Optimal Production Baseline: Qwen-2.5-14B/32B or Llama-3.1-70B (quantized with AWQ/GGUF for multi-GPU or high-VRAM workstations).

2. Context Window and Token Truncation

As an agent executes multiple tasks, the accumulated message history expands rapidly. Ensure your inference engine allocates sufficient context length:

• For Ollama, create a custom Modelfile setting PARAMETER num_ctx 16384 to prevent silent prompt truncation during multi-step runs.

3. Upstream Maintenance Context

Note that the reworkd/AgentGPT repository was archived in January 2026. While the architecture remains a robust, functional reference implementation for browser-based autonomous agent workflows, development teams should maintain internal Docker image tags, pin package dependencies in package.json and requirements.txt, and audit local database secrets rather than relying on upstream updates.

References

• GitHub - reworkd/AgentGPT: 🤖 Assemble, configure, and deploy autonomous AI Agents in your browser.
• Docker Compose | Docker Docs
• GitHub - ollama/ollama: Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.