> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/lumina-ai-inc/chunkr/llms.txt
> Use this file to discover all available pages before exploring further.

# Segment Processing

> Configure how different document segments are processed, formatted, and generated

## Overview

Segment processing controls the post-processing, formatting, and generation of content for each segment type detected in your documents. You can configure output formats (HTML or Markdown), generation strategies (Auto or LLM), image cropping, and custom LLM prompts.

## Segment Types

Chunkr detects and processes these segment types:

* `Title` - Document titles
* `SectionHeader` - Section and subsection headers
* `Text` - Regular paragraph text
* `ListItem` - List items (bulleted or numbered)
* `Table` - Tables and tabular data
* `Picture` - Images, charts, diagrams
* `Caption` - Image and table captions
* `Formula` - Mathematical formulas and equations
* `Footnote` - Footnotes and references
* `PageHeader` - Page headers
* `PageFooter` - Page footers
* `Page` - Full page content (when using `Page` segmentation strategy)

## Configuration Parameters

Each segment type can be configured with these parameters:

<ParamField path="format" type="enum" default="Markdown">
  Output format for the segment:

  * `Html` - HTML formatted content
  * `Markdown` - Markdown formatted content

  For tables, default is `Html`.
</ParamField>

<ParamField path="strategy" type="enum" default="Auto">
  Content generation strategy:

  * `Auto` - Use heuristics and rules (fast, no LLM cost)
  * `LLM` - Use Chunkr fine-tuned models (higher quality, requires LLM)

  For `Table`, `Formula`, and `Page` segments, default is `LLM`.
</ParamField>

<ParamField path="crop_image" type="enum" default="Auto">
  Image cropping behavior:

  * `Auto` - Crop only when needed for post-processing
  * `All` - Always crop to segment bounding box
  * `None` - Never crop

  Cropped images are stored in the segment's `image` field.
</ParamField>

<ParamField path="llm" type="string">
  Custom prompt for LLM-based generation. Use this to provide specific instructions for how the segment should be processed.
</ParamField>

<ParamField path="embed_sources" type="array" default={["Content"]}>
  Which content sources to include in the chunk's `embed` field:

  * `Content` - The primary content (uses `format` setting)
  * `LLM` - LLM-generated content
  * `HTML` - (deprecated) HTML content
  * `Markdown` - (deprecated) Markdown content

  The order determines the sequence in the embed field.
</ParamField>

<ParamField path="extended_context" type="boolean" default={false}>
  Whether to provide the full page image as context for LLM generation. Useful for segments that need broader context.
</ParamField>

## Basic Examples

### Default Configuration

<CodeGroup>
  ```json Auto-Generated Markdown theme={null}
  {
    "segment_processing": {
      "text": {
        "format": "Markdown",
        "strategy": "Auto"
      }
    }
  }
  ```

  ```python Python SDK theme={null}
  chunkr.upload(
      file_path="document.pdf",
      segment_processing={
          "text": {
              "format": "Markdown",
              "strategy": "Auto"
          }
      }
  )
  ```
</CodeGroup>

### HTML Output

```json theme={null}
{
  "segment_processing": {
    "text": {
      "format": "Html",
      "strategy": "Auto"
    }
  }
}
```

### LLM-Based Generation

```json theme={null}
{
  "segment_processing": {
    "text": {
      "format": "Markdown",
      "strategy": "LLM"
    }
  }
}
```

## Segment-Specific Configuration

<Tabs>
  <Tab title="Tables">
    Tables default to HTML format with LLM generation for best quality:

    ```json theme={null}
    {
      "segment_processing": {
        "table": {
          "format": "Html",
          "strategy": "LLM",
          "crop_image": "Auto"
        }
      }
    }
    ```

    **Options:**

    * `Html` format preserves table structure better
    * `Markdown` format for simpler tables
    * `Auto` strategy for basic tables (faster, no LLM cost)
    * `LLM` strategy for complex tables with merged cells, etc.
  </Tab>

  <Tab title="Pictures">
    Pictures have special cropping options:

    ```json theme={null}
    {
      "segment_processing": {
        "picture": {
          "format": "Markdown",
          "strategy": "Auto",
          "crop_image": "All"
        }
      }
    }
    ```

    **Cropping strategies:**

    * `All` - Always crop images (recommended for pictures)
    * `Auto` - Crop only when needed
    * `None` - Use full page image

    **Output:**

    * `Auto` strategy generates image markdown: `![Image](url)`
    * `LLM` strategy can generate descriptions or analyze image content
  </Tab>

  <Tab title="Formulas">
    Formulas default to LLM generation for accurate LaTeX:

    ```json theme={null}
    {
      "segment_processing": {
        "formula": {
          "format": "Markdown",
          "strategy": "LLM",
          "crop_image": "Auto"
        }
      }
    }
    ```

    LLM strategy converts formulas to LaTeX notation for rendering.
  </Tab>

  <Tab title="Headers & Footers">
    Control whether headers and footers are included:

    ```json theme={null}
    {
      "chunk_processing": {
        "ignore_headers_and_footers": true
      },
      "segment_processing": {
        "page_header": {
          "format": "Markdown",
          "strategy": "Auto"
        },
        "page_footer": {
          "format": "Markdown",
          "strategy": "Auto"
        }
      }
    }
    ```

    <Note>
      Even with these configurations, headers/footers are excluded from chunks when `ignore_headers_and_footers: true`.
    </Note>
  </Tab>
</Tabs>

## Advanced Features

### Custom LLM Prompts

Provide specific instructions for how segments should be processed:

```json theme={null}
{
  "segment_processing": {
    "table": {
      "format": "Html",
      "strategy": "LLM",
      "llm": "Convert this table to HTML. Merge cells where appropriate and preserve all numerical precision."
    },
    "picture": {
      "format": "Markdown",
      "strategy": "LLM",
      "llm": "Describe this image in detail, focusing on any charts, graphs, or data visualizations. Include all visible data points and trends."
    }
  }
}
```

### Embedding Configuration

Control what content is included in chunk embeddings:

<Tabs>
  <Tab title="Content Only">
    ```json theme={null}
    {
      "segment_processing": {
        "text": {
          "embed_sources": ["Content"]
        }
      }
    }
    ```

    Use the primary content based on the `format` setting.
  </Tab>

  <Tab title="LLM Only">
    ```json theme={null}
    {
      "segment_processing": {
        "text": {
          "strategy": "LLM",
          "embed_sources": ["LLM"]
        }
      }
    }
    ```

    Use only LLM-generated content for embeddings.
  </Tab>

  <Tab title="Combined Sources">
    ```json theme={null}
    {
      "segment_processing": {
        "text": {
          "strategy": "LLM",
          "embed_sources": ["Content", "LLM"]
        }
      }
    }
    ```

    Combine primary content and LLM output. Order matters - content appears before LLM in the embed field.
  </Tab>
</Tabs>

### Extended Context

Provide full page context for better LLM understanding:

```json theme={null}
{
  "segment_processing": {
    "table": {
      "format": "Html",
      "strategy": "LLM",
      "extended_context": true
    }
  }
}
```

**Use cases:**

* Tables that reference surrounding content
* Formulas with context-dependent notation
* Images that need page layout understanding

**Trade-off:** Increases LLM token usage and processing time.

## Complete Configuration Example

```json theme={null}
{
  "segment_processing": {
    "title": {
      "format": "Markdown",
      "strategy": "Auto",
      "crop_image": "Auto",
      "embed_sources": ["Content"]
    },
    "section_header": {
      "format": "Markdown",
      "strategy": "Auto",
      "crop_image": "Auto",
      "embed_sources": ["Content"]
    },
    "text": {
      "format": "Markdown",
      "strategy": "Auto",
      "crop_image": "Auto",
      "embed_sources": ["Content"]
    },
    "list_item": {
      "format": "Markdown",
      "strategy": "Auto",
      "crop_image": "Auto",
      "embed_sources": ["Content"]
    },
    "table": {
      "format": "Html",
      "strategy": "LLM",
      "crop_image": "Auto",
      "embed_sources": ["Content"],
      "llm": "Convert to clean HTML table with proper structure"
    },
    "picture": {
      "format": "Markdown",
      "strategy": "Auto",
      "crop_image": "All",
      "embed_sources": ["Content"]
    },
    "caption": {
      "format": "Markdown",
      "strategy": "Auto",
      "crop_image": "Auto",
      "embed_sources": ["Content"]
    },
    "formula": {
      "format": "Markdown",
      "strategy": "LLM",
      "crop_image": "Auto",
      "embed_sources": ["Content"]
    },
    "footnote": {
      "format": "Markdown",
      "strategy": "Auto",
      "crop_image": "Auto",
      "embed_sources": ["Content"]
    },
    "page_header": {
      "format": "Markdown",
      "strategy": "Auto",
      "crop_image": "Auto",
      "embed_sources": ["Content"]
    },
    "page_footer": {
      "format": "Markdown",
      "strategy": "Auto",
      "crop_image": "Auto",
      "embed_sources": ["Content"]
    },
    "page": {
      "format": "Markdown",
      "strategy": "LLM",
      "crop_image": "Auto",
      "embed_sources": ["Content"]
    }
  }
}
```

## Output Fields

Each processed segment includes these fields:

```json theme={null}
{
  "segment_id": "uuid",
  "segment_type": "Table",
  "bbox": { "left": 0, "top": 0, "width": 100, "height": 50 },
  "page_number": 1,
  "content": "<table>...</table>",  // Based on 'format' setting
  "text": "Raw OCR text",
  "html": "<table>...</table>",      // Deprecated
  "markdown": "| Header |\n| --- |", // Deprecated
  "llm": "LLM-generated content",    // If LLM strategy or custom prompt used
  "image": "https://presigned-url",  // If crop_image enabled
  "ocr": [...],
  "confidence": 0.95
}
```

<Note>
  The `content` field contains the formatted output based on your `format` setting. The deprecated `html` and `markdown` fields are still available for backwards compatibility.
</Note>

## Best Practices

1. **Use Auto strategy for simple segments**
   * Faster processing
   * No LLM costs
   * Good for text, headers, lists

2. **Use LLM strategy for complex segments**
   * Tables with complex structure
   * Mathematical formulas
   * Images requiring description

3. **Match format to your use case**
   * `Html` for tables and structured content
   * `Markdown` for general text and readability

4. **Configure embed\_sources carefully**
   * Include only necessary sources
   * Reduces token usage for embeddings
   * Improves retrieval relevance

5. **Use extended\_context sparingly**
   * Higher LLM costs
   * Longer processing time
   * Only when context is critical

6. **Test custom prompts**
   * Start with default prompts
   * Iterate based on output quality
   * Be specific in instructions

## Deprecated Fields

<Warning>
  The `html` and `markdown` fields in configuration are deprecated. Use `format` and `strategy` instead:

  **Old (Deprecated):**

  ```json theme={null}
  {
    "table": {
      "html": "LLM",
      "markdown": "Auto"
    }
  }
  ```

  **New (Recommended):**

  ```json theme={null}
  {
    "table": {
      "format": "Html",
      "strategy": "LLM"
    }
  }
  ```
</Warning>