How to Connect Hugging Face Tokens and Custom Inference Endpoints to HuggingChat

Connecting private open-source models or proprietary fine-tunes to HuggingChat requires understanding how the platform is architected. Many developers search the public HuggingChat interface (hf.co/chat) expecting a settings panel where they can input an endpoint URL and an API token. However, public HuggingChat operates as a hosted, multi-tenant showcase strictly bound to Hugging Face's curated router infrastructure; it does not allow users to inject arbitrary private inference endpoints into the shared interface.

The official, production-ready way to run private models with HuggingChat is to deploy your own instance of Chat UI—the open-source SvelteKit application that powers HuggingChat. Chat UI connects to any OpenAI-compatible backend, allowing you to route chats directly to a dedicated Hugging Face Inference Endpoint authenticated with your Hugging Face User Access Token. You can deploy this setup in minutes via a private Hugging Face Space or on your local infrastructure.

Architectural Workflow

The complete integration links three components:

1. Hugging Face Inference Endpoints: Hosts your private or custom open-source model weights (e.g., Llama 3.3, Mistral, Qwen, or a private LoRA adaptation) using high-performance serving engines like Text Generation Inference (TGI) or vLLM.
2. User Access Token (HF_TOKEN): Authenticates requests sent to your private endpoint.
3. Chat UI (Private HuggingChat Instance): The web interface that consumes the endpoint via OpenAI-compatible REST routes (/v1/chat/completions).

+-------------------------------------------------------------+
|                  Private Chat UI Instance                   |
|              (Hugging Face Space / Local Docker)            |
+-------------------------------------------------------------+
                               |
               OPENAI_BASE_URL | OPENAI_API_KEY (hf_xxx)
                               v
+-------------------------------------------------------------+
|             Hugging Face Inference Endpoint                 |
|   https://<endpoint-id>.<region>.<cloud>.endpoints.huggingface.cloud/v1   |
|                 (Powered by TGI / vLLM)                     |
+-------------------------------------------------------------+
                               |
                               v
+-------------------------------------------------------------+
|             Private Repository / Fine-Tuned Weights         |
+-------------------------------------------------------------+

Step 1: Deploy Your Custom Model to an Inference Endpoint

To serve a private model, set up dedicated compute on Hugging Face:

1. Navigate to the Inference Endpoints Console.
2. Click Create New Endpoint.
3. Model Repository: Enter your private model repository ID (e.g., your-org/private-llama-3-8b-instruct) or choose an open-source hub model.
4. Cloud Provider & Region: Select AWS or Azure and choose a region close to your users.
5. Instance Type: Pick GPU hardware suited for your model size (e.g., 1x NVIDIA A10G for an 8B model, 4x A100/H100 for 70B models).
6. Framework & Engine: Ensure the container uses Text Generation Inference (TGI) or vLLM. Both expose standard OpenAI-compatible API schemas.
7. Click Create Endpoint.

Wait until the endpoint status changes from Building to Running. Note the unique endpoint URL provided in the console dashboard:

https://<endpoint-id>.<region>.<cloud>.endpoints.huggingface.cloud

Step 2: Generate an Account Access Token

Your dedicated endpoint requires authentication for all inference calls:

1. Go to Settings > Access Tokens (hf.co/settings/tokens).
2. Click Create new token.
3. Set the Token type:
   • Fine-grained token (Recommended): Grant permissions under Inference to "Make calls to your Inference Endpoints".
   • Read token: Standard read tokens also authenticate endpoint requests for personal endpoints.
4. Copy the generated token string (hf_...). Store it securely.

Step 3: Connect the Endpoint to Chat UI (HuggingChat)

Chat UI discovers models through OpenAI-compatible endpoints by appending /v1 to the base URL.

Option A: 1-Click Deployment via Hugging Face Spaces (Recommended)

Running Chat UI on a private Hugging Face Space gives you a fully managed, collaborative HuggingChat frontend without maintaining local servers:

1. Open the official Chat UI Space template.
2. Set Visibility to Private to protect your internal conversations.
3. In the Space Settings > Variables and Secrets, define the following environment variables:
   • OPENAI_BASE_URL: https://<endpoint-id>.<region>.<cloud>.endpoints.huggingface.cloud/v1
   • OPENAI_API_KEY: hf_************************ (your Hugging Face access token)
4. (Optional) Set MONGODB_URL if you want multi-session persistence across Space restarts. For single-user testing, Chat UI falls back to an embedded local database.
5. Restart the Space. Chat UI will query your endpoint's /v1/models route and immediately display the private model in the chat interface.

Option B: Local or Self-Hosted Deployment (Node.js / Docker)

If you prefer hosting the interface on your own workstation or on-premise Kubernetes cluster:

1. Clone the Chat UI repository:

   git clone https://github.com/huggingface/chat-ui.git
   cd chat-ui
   

2. Create a .env.local configuration file:

   # Endpoint configuration
   OPENAI_BASE_URL=https://<endpoint-id>.<region>.<cloud>.endpoints.huggingface.cloud/v1
   OPENAI_API_KEY=hf_your_token_here
   
   # Database (optional for local development, uses embedded db in ./db if omitted)
   # MONGODB_URL=mongodb://localhost:27017
   

3. If your custom endpoint does not implement the /v1/models discovery route, or if you need custom display names, parameters, or tool support, configure explicit model definitions in .env.local using the MODELS JSON variable:

   MODELS=`[
     {
       "id": "tgi",
       "name": "Private Custom Assistant",
       "multimodal": false,
       "supportsTools": false,
       "parameters": {
         "temperature": 0.7,
         "max_tokens": 4096
       }
     }
   ]`
   

4. Install dependencies and start the local server:

   npm install
   npm run dev -- --open
   

Chat UI will launch in your browser at http://localhost:5173, delivering the complete HuggingChat conversational experience connected directly to your private weights.

Pre-Flight Verification

Before testing through the browser, verify connectivity using curl to eliminate network or authorization issues:

curl -X POST "https://<endpoint-id>.<region>.<cloud>.endpoints.huggingface.cloud/v1/chat/completions" \
  -H "Authorization: Bearer hf_your_token_here" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "tgi",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Hello! Confirm endpoint readiness."}
    ],
    "max_tokens": 50
  }'

If the endpoint responds with valid JSON containing model-generated text, your credentials and serving container are configured properly.

Practical Considerations and Trade-offs

• Public Assistants vs. Dedicated Endpoints: Creating a custom "Assistant" on public hf.co/chat allows you to customize system prompts, temperature, and upload RAG knowledge documents, but the underlying LLM remains confined to the shared catalog of public models. To run private proprietary weights, self-hosting Chat UI with Inference Endpoints is mandatory.
• Scale to Zero (Cold Starts): If your Inference Endpoint is configured with automatic scale-to-zero to minimize compute costs during inactivity, sending a message from Chat UI while the endpoint is idle may result in an initial request timeout while the GPU container initializes. Keep at least 1 minimum replica active for interactive production workflows.
• Streaming Compatibility: Chat UI relies on Server-Sent Events (stream: true) to stream responses character by character. TGI and vLLM both support standard streaming out of the box; ensure no reverse proxy or intermediate gateway buffers chunked responses.

References

• Chat UI · Hugging Face
• Configuration Overview · Hugging Face
• Running Locally · Hugging Face
• Inference Endpoints · Hugging Face