> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/lumina-ai-inc/chunkr/llms.txt
> Use this file to discover all available pages before exploring further.

# Installation

> Deploy Chunkr locally with Docker - GPU and CPU support for all platforms

# Installation Guide

Chunkr runs as a collection of Docker services orchestrated with Docker Compose. This guide covers installation for GPU-accelerated deployments, CPU-only systems, and Mac ARM devices.

## Prerequisites

<Steps>
  <Step title="Install Docker">
    Install Docker Desktop or Docker Engine:

    * **Docker Desktop**: [Download here](https://docs.docker.com/get-docker/)
    * **Docker Engine**: For Linux servers

    Verify installation:

    ```bash theme={null}
    docker --version
    docker compose version
    ```
  </Step>

  <Step title="Install NVIDIA Container Toolkit (GPU Only)">
    For GPU acceleration, install the NVIDIA Container Toolkit:

    <Info>
      Skip this step if you're using CPU-only or Mac ARM deployment.
    </Info>

    ```bash theme={null}
    # Add the NVIDIA repository
    distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
    curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
    curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
      sudo tee /etc/apt/sources.list.d/nvidia-docker.list

    # Install nvidia-container-toolkit
    sudo apt-get update
    sudo apt-get install -y nvidia-container-toolkit

    # Restart Docker
    sudo systemctl restart docker
    ```

    [Full installation guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html)
  </Step>

  <Step title="Verify GPU Access (GPU Only)">
    Test that Docker can access your GPU:

    ```bash theme={null}
    docker run --rm --gpus all nvidia/cuda:12.0.0-base-ubuntu22.04 nvidia-smi
    ```

    You should see your GPU information.
  </Step>
</Steps>

## Quick Installation

<Steps>
  <Step title="Clone the Repository">
    ```bash theme={null}
    git clone https://github.com/lumina-ai-inc/chunkr.git
    cd chunkr
    ```
  </Step>

  <Step title="Set Up Environment">
    ```bash theme={null}
    # Copy environment template
    cp .env.example .env

    # Copy LLM models template
    cp models.example.yaml models.yaml
    ```
  </Step>

  <Step title="Configure LLM Models">
    Edit `models.yaml` with your LLM configuration. See [LLM Configuration](#llm-configuration) below.
  </Step>

  <Step title="Start Services">
    <CodeGroup>
      ```bash GPU (Default) theme={null}
      # Recommended: Uses NVIDIA GPUs for faster processing
      docker compose up -d
      ```

      ```bash CPU Only theme={null}
      # For systems without GPU or non-NVIDIA GPUs
      docker compose -f compose.yaml -f compose.cpu.yaml up -d
      ```

      ```bash Mac ARM (M1/M2/M3) theme={null}
      # Optimized for Apple Silicon
      docker compose -f compose.yaml -f compose.cpu.yaml -f compose.mac.yaml up -d
      ```
    </CodeGroup>

    <Info>
      First startup downloads several GB of models and may take 10-15 minutes.
    </Info>
  </Step>

  <Step title="Verify Installation">
    Check that all services are running:

    ```bash theme={null}
    docker compose ps
    ```

    All services should show "Up" status. Access:

    * **Web UI**: [http://localhost:5173](http://localhost:5173)
    * **API**: [http://localhost:8000](http://localhost:8000)
    * **API Docs**: [http://localhost:8000/docs](http://localhost:8000/docs)
  </Step>
</Steps>

## LLM Configuration

Chunkr requires at least one LLM for vision-language model processing. You can configure multiple models with fallbacks.

### Using models.yaml (Recommended)

The `models.yaml` file supports multiple LLM providers with advanced options:

```yaml models.yaml theme={null}
models:
  # OpenAI Configuration
  - id: gpt-4o
    model: gpt-4o
    provider_url: https://api.openai.com/v1/chat/completions
    api_key: "sk-your-openai-key-here"
    default: true
    rate-limit: 200  # requests per minute (optional)

  # Google AI Studio Configuration
  - id: gemini-2.0-flash-lite
    model: gemini-2.0-flash-lite
    provider_url: https://generativelanguage.googleapis.com/v1beta/openai/chat/completions
    api_key: "your-google-ai-key-here"
    fallback: true

  # OpenRouter Configuration
  - id: gemini-pro-1.5
    model: google/gemini-pro-1.5
    provider_url: https://openrouter.ai/api/v1/chat/completions
    api_key: "your-openrouter-key-here"

  # Self-hosted LLM (Ollama, vLLM, etc.)
  - id: local-llm
    model: mistral-7b
    provider_url: http://localhost:11434/v1/chat/completions
    api_key: ""  # Leave empty if not required
```

<Note>
  * **Exactly one** model must have `default: true`
  * **Exactly one** model must have `fallback: true` (can be the same as default)
  * Use `id` to reference models in API requests
  * `rate-limit` is optional and sets requests per minute cap
</Note>

### Using Environment Variables (Basic)

For simple single-LLM setups, use environment variables in `.env`:

```bash .env theme={null}
LLM__KEY=sk-your-api-key-here
LLM__MODEL=gpt-4o
LLM__URL=https://api.openai.com/v1/chat/completions
```

<Warning>
  Environment variables are overridden by `models.yaml`. If you use `models.yaml`, remove or comment out the `LLM__*` variables.
</Warning>

### Common LLM Providers

<Tabs>
  <Tab title="OpenAI">
    ```yaml theme={null}
    - id: gpt-4o
      model: gpt-4o
      provider_url: https://api.openai.com/v1/chat/completions
      api_key: "sk-your-key-here"
      default: true
    ```

    [Get API Key](https://platform.openai.com/api-keys) | [Documentation](https://platform.openai.com/docs)
  </Tab>

  <Tab title="Google AI Studio">
    ```yaml theme={null}
    - id: gemini-flash
      model: gemini-2.0-flash-lite
      provider_url: https://generativelanguage.googleapis.com/v1beta/openai/chat/completions
      api_key: "your-key-here"
      default: true
    ```

    [Get API Key](https://aistudio.google.com/app/apikey) | [Documentation](https://ai.google.dev/gemini-api/docs/openai)
  </Tab>

  <Tab title="OpenRouter">
    ```yaml theme={null}
    - id: openrouter-model
      model: google/gemini-pro-1.5
      provider_url: https://openrouter.ai/api/v1/chat/completions
      api_key: "your-key-here"
      default: true
    ```

    [Get API Key](https://openrouter.ai/keys) | [Browse Models](https://openrouter.ai/models)
  </Tab>

  <Tab title="Ollama (Local)">
    ```yaml theme={null}
    - id: ollama-llama
      model: llama3.2-vision
      provider_url: http://host.docker.internal:11434/v1/chat/completions
      api_key: ""
      default: true
    ```

    Install Ollama, then:

    ```bash theme={null}
    ollama pull llama3.2-vision
    ```

    [Installation](https://ollama.com) | [OpenAI Compatibility](https://ollama.com/blog/openai-compatibility)
  </Tab>

  <Tab title="vLLM (Self-hosted)">
    ```yaml theme={null}
    - id: vllm-model
      model: meta-llama/Llama-3.2-11B-Vision
      provider_url: http://your-vllm-server:8000/v1/chat/completions
      api_key: "your-key-if-required"
      default: true
    ```

    [vLLM Documentation](https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html)
  </Tab>
</Tabs>

## Service Architecture

Chunkr consists of multiple containerized services:

<AccordionGroup>
  <Accordion title="Core Services" icon="server">
    * **server**: Main API server (Rust/Actix-Web) on port 8000
    * **task**: Background worker pool (30 replicas for GPU, 10 for CPU)
    * **web**: React-based UI on port 5173
    * **postgres**: Database for metadata and task state
    * **redis**: Queue and cache for job processing
    * **minio**: S3-compatible object storage for files
  </Accordion>

  <Accordion title="Processing Services" icon="microchip">
    * **segmentation**: YOLO-based layout detection (6 replicas)
      * GPU: Uses NVIDIA GPU acceleration
      * CPU: Optimized for multi-core processing
    * **ocr**: DocTR OCR engine (3 replicas)
      * GPU: CUDA-accelerated inference
      * CPU: Uses smaller model variant
  </Accordion>

  <Accordion title="Supporting Services" icon="gears">
    * **keycloak**: Authentication and user management (port 8080)
    * **adminer**: Database admin UI (port 8082)
    * **nginx**: Load balancer for processing services
  </Accordion>
</AccordionGroup>

### Port Mappings

| Service       | Port | Description              |
| ------------- | ---- | ------------------------ |
| Web UI        | 5173 | React application        |
| API           | 8000 | REST API endpoint        |
| Segmentation  | 8001 | Layout detection service |
| OCR           | 8002 | Text recognition service |
| Keycloak      | 8080 | Authentication           |
| Adminer       | 8082 | Database UI              |
| PostgreSQL    | 5432 | Database                 |
| Redis         | 6379 | Cache/Queue              |
| MinIO         | 9000 | Object storage           |
| MinIO Console | 9001 | Storage admin UI         |

## GPU vs CPU Performance

Performance comparison for a typical 10-page PDF:

| Configuration | Processing Time  | Hardware Requirements     |
| ------------- | ---------------- | ------------------------- |
| **GPU**       | \~20-30 seconds  | NVIDIA GPU with 8GB+ VRAM |
| **CPU**       | \~60-120 seconds | 8+ CPU cores, 16GB+ RAM   |
| **Mac ARM**   | \~45-90 seconds  | M1/M2/M3 with 16GB+ RAM   |

<Info>
  GPU acceleration provides 3-4x speedup for segmentation and OCR operations.
</Info>

## Scaling Configuration

### Adjusting Worker Replicas

Edit `compose.yaml` to scale processing:

```yaml compose.yaml theme={null}
services:
  task:
    deploy:
      replicas: 30  # Reduce for less memory usage
  
  segmentation-backend:
    deploy:
      replicas: 6   # Scale based on GPU count
  
  ocr-backend:
    deploy:
      replicas: 3   # Scale based on available resources
```

### Resource Limits

For production, add resource constraints:

```yaml theme={null}
services:
  task:
    deploy:
      replicas: 30
      resources:
        limits:
          memory: 2G
        reservations:
          memory: 1G
```

## Stopping and Managing Services

<CodeGroup>
  ```bash Stop Services theme={null}
  # GPU deployment
  docker compose down

  # CPU deployment
  docker compose -f compose.yaml -f compose.cpu.yaml down

  # Mac ARM deployment
  docker compose -f compose.yaml -f compose.cpu.yaml -f compose.mac.yaml down
  ```

  ```bash View Logs theme={null}
  # All services
  docker compose logs -f

  # Specific service
  docker compose logs -f server

  # Recent logs only
  docker compose logs --tail=100 -f
  ```

  ```bash Restart Services theme={null}
  # Restart all
  docker compose restart

  # Restart specific service
  docker compose restart server
  ```

  ```bash Remove Volumes (Clean Reset) theme={null}
  # Warning: Deletes all data!
  docker compose down -v
  ```
</CodeGroup>

## Troubleshooting

<AccordionGroup>
  <Accordion title="Services won't start">
    **Check Docker daemon:**

    ```bash theme={null}
    sudo systemctl status docker
    ```

    **View startup errors:**

    ```bash theme={null}
    docker compose logs
    ```

    **Common issues:**

    * Port conflicts (8000, 5173, etc. already in use)
    * Insufficient memory (requires 16GB+ for full stack)
    * Missing `.env` or `models.yaml` files
  </Accordion>

  <Accordion title="GPU not detected">
    **Verify GPU access:**

    ```bash theme={null}
    docker run --rm --gpus all nvidia/cuda:12.0.0-base-ubuntu22.04 nvidia-smi
    ```

    **Check NVIDIA Container Toolkit:**

    ```bash theme={null}
    nvidia-ctk --version
    ```

    **Restart Docker after toolkit install:**

    ```bash theme={null}
    sudo systemctl restart docker
    ```
  </Accordion>

  <Accordion title="Out of memory errors">
    **Reduce worker replicas** in `compose.yaml`:

    ```yaml theme={null}
    task:
      deploy:
        replicas: 10  # Down from 30
    ```

    **Use CPU deployment** if GPU memory is limited:

    ```bash theme={null}
    docker compose -f compose.yaml -f compose.cpu.yaml up -d
    ```

    **Monitor resource usage:**

    ```bash theme={null}
    docker stats
    ```
  </Accordion>

  <Accordion title="LLM connection failures">
    **Test LLM endpoint manually:**

    ```bash theme={null}
    curl -X POST https://api.openai.com/v1/chat/completions \
      -H "Authorization: Bearer YOUR_KEY" \
      -H "Content-Type: application/json" \
      -d '{"model":"gpt-4o","messages":[{"role":"user","content":"test"}]}'
    ```

    **Check models.yaml syntax:**

    ```bash theme={null}
    # Validate YAML
    python -c "import yaml; yaml.safe_load(open('models.yaml'))"
    ```

    **View server logs:**

    ```bash theme={null}
    docker compose logs -f server
    ```
  </Accordion>

  <Accordion title="Slow processing on Mac ARM">
    **Ensure using Mac compose override:**

    ```bash theme={null}
    docker compose -f compose.yaml -f compose.cpu.yaml -f compose.mac.yaml up -d
    ```

    **Reduce concurrent tasks:**

    * Decrease `replicas` for `task`, `segmentation-backend`, `ocr-backend`
    * Process documents sequentially instead of parallel

    **Allocate more resources to Docker Desktop:**

    * Open Docker Desktop → Settings → Resources
    * Increase CPUs to 8+ and Memory to 16GB+
  </Accordion>
</AccordionGroup>

## Production Deployment

<Warning>
  The default configuration is designed for development. For production:

  1. **Enable authentication**: Configure Keycloak properly
  2. **Use HTTPS**: Set up reverse proxy with SSL/TLS
  3. **Secure secrets**: Use Docker secrets or environment encryption
  4. **Configure backups**: Back up PostgreSQL and MinIO data
  5. **Monitor resources**: Set up alerts for CPU, memory, disk usage
  6. **Rate limiting**: Configure per-model rate limits in `models.yaml`
  7. **Task expiration**: Set appropriate `expires_in` values
</Warning>

### Environment Variables Reference

Key configuration options in `.env`:

```bash theme={null}
# Database
PG__URL=postgresql://postgres:postgres@postgres:5432/chunkr

# Redis
REDIS__URL=redis://redis:6379

# Object Storage
AWS__ENDPOINT=http://minio:9000
AWS__ACCESS_KEY=minioadmin
AWS__SECRET_KEY=minioadmin

# LLM Configuration Path
LLM__MODELS_PATH=./models.yaml

# Worker URLs
WORKER__GENERAL_OCR_URL=http://ocr:8000
WORKER__SEGMENTATION_URL=http://segmentation:8000
WORKER__SERVER_URL=http://localhost:8000

# Authentication
AUTH__KEYCLOAK_URL=http://keycloak:8080
```

## Next Steps

<CardGroup cols={2}>
  <Card title="Quickstart" icon="rocket" href="/quickstart">
    Make your first API request
  </Card>

  <Card title="API Reference" icon="book" href="/api-reference">
    Explore the complete API
  </Card>

  <Card title="Configuration" icon="sliders" href="/configuration">
    Advanced configuration options
  </Card>

  <Card title="Examples" icon="code" href="/examples">
    Code examples and use cases
  </Card>
</CardGroup>