> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/lumina-ai-inc/chunkr/llms.txt
> Use this file to discover all available pages before exploring further.

# Migration Guide

> Migrate from legacy content generation to the new unified format system

This guide helps you migrate from Chunkr's legacy HTML/Markdown dual-generation system to the new unified format approach.

## What Changed

Chunkr has consolidated content generation to improve performance and simplify the API.

### Before (Legacy)

```python Python theme={null}
# Could generate both formats simultaneously
config = {
    "segment_processing": {
        "Table": {
            "html": "LLM",      # Generate HTML
            "markdown": "Auto"  # Also generate Markdown
        }
    }
}

# Content in separate fields
html_content = segment["html"]
markdown_content = segment["markdown"]
ocr_text = segment["content"]  # OCR text was here
```

### After (New)

```python Python theme={null}
# Choose one format per segment type
config = {
    "segment_processing": {
        "Table": {
            "format": "Html",     # Choose format
            "strategy": "LLM"     # Choose generation strategy
        }
    }
}

# Content in unified field
generated_content = segment["content"]  # HTML or Markdown (your choice)
ocr_text = segment["text"]              # OCR text moved here
```

## Key Changes

<Warning>
  The purpose of `segment.content` has completely changed - it now contains generated HTML/Markdown instead of OCR text
</Warning>

| Field              | Legacy Behavior         | New Behavior                             |
| ------------------ | ----------------------- | ---------------------------------------- |
| `segment.content`  | OCR-extracted text      | Generated content (HTML or Markdown)     |
| `segment.text`     | ❌ Didn't exist          | OCR-extracted text                       |
| `segment.html`     | HTML representation     | Still available (backward compatibility) |
| `segment.markdown` | Markdown representation | Still available (backward compatibility) |

## Migration Priority

### 🚨 CRITICAL - Immediate Action Required

#### 1. Pipeline.Chunkr Table Processing Default Change

**Impact:** Tables no longer generate AI-enhanced markdown by default

```python Python theme={null}
# ❌ BROKEN - Default config no longer generates AI markdown for tables
# Old behavior: Tables got both HTML and AI-enhanced markdown automatically

# ✅ FIXED - Explicitly configure AI table processing
config = {
    "segment_processing": {
        "Table": {
            "format": "Markdown",
            "strategy": "LLM"
        }
    }
}
```

**Who's affected:**

* Applications using `Pipeline.Chunkr` with default table config
* Expecting AI-generated `segment.markdown` for tables

**Who's not affected:**

* Applications with explicit table configuration

### ⚠️ MEDIUM - Action Recommended

#### 2. OCR Text Access Breaking Change

**Impact:** Applications accessing OCR text from `segment.content` will break

```python Python theme={null}
# ❌ BROKEN - Will now return HTML/Markdown instead of OCR text
ocr_text = segment["content"]

# ✅ FIXED - Update to use segment.text
ocr_text = segment["text"]
```

**Who's affected:**

* Applications reading `segment.content` expecting OCR text

### 💡 LOW - Performance Optimization

#### 3. Update to New Content Access Pattern

**Impact:** Performance improvement and future-proofing

```python Python theme={null}
# 📊 OLD - Accessing format-specific fields
html_content = segment["html"]
markdown_content = segment["markdown"]

# ⚡ NEW - Access unified content field
generated_content = segment["content"]  # Contains format you requested
```

**Who's affected:** All applications (optional upgrade)

## Quick Self-Assessment

**Am I affected by Critical Issue #1?**

* [ ] Do I use `Pipeline.Chunkr` (layout analysis)?
* [ ] Do I process tables without explicit configuration?
* [ ] Do I expect `segment.markdown` to contain AI-enhanced table content?
* [ ] **If yes to all**: Add explicit table configuration immediately

**Am I affected by Medium Issue #2?**

* [ ] Do I access `segment.content` anywhere in my code?
* [ ] Do I expect `segment.content` to contain OCR text?
* [ ] **If yes to both**: Update to `segment.text`

## Migration Steps

<Steps>
  <Step title="Update Configuration Format">
    Replace legacy `html` and `markdown` fields with `format` and `strategy`:

    ```python Python theme={null}
    # Legacy
    old_config = {
        "segment_processing": {
            "Page": {
                "html": "LLM",
                "markdown": "LLM",
                "embed_sources": ["Markdown"]
            }
        }
    }

    # New
    new_config = {
        "segment_processing": {
            "Page": {
                "format": "Markdown",
                "strategy": "LLM",
                "embed_sources": ["Content"]
            }
        }
    }
    ```
  </Step>

  <Step title="Update Content Access">
    Change how you access OCR text:

    ```python Python theme={null}
    # Legacy
    for segment in chunk["segments"]:
        ocr_text = segment["content"]
        html = segment["html"]
        markdown = segment["markdown"]

    # New
    for segment in chunk["segments"]:
        ocr_text = segment["text"]        # Changed!
        generated = segment["content"]     # Now contains HTML or Markdown
        html = segment["html"]             # Still available
        markdown = segment["markdown"]     # Still available
    ```
  </Step>

  <Step title="Update Embed Sources">
    Replace format-specific embed sources:

    ```python Python theme={null}
    # Legacy
    old_embed = {
        "embed_sources": ["HTML", "Markdown", "LLM"]
    }

    # New
    new_embed = {
        "embed_sources": ["Content", "LLM"]
    }
    ```

    <Note>
      `Content` refers to the generated content based on your chosen format
    </Note>
  </Step>

  <Step title="Test Thoroughly">
    Verify that:

    * Table processing generates expected content
    * OCR text is correctly accessed from `segment.text`
    * Embed fields contain expected content
    * Chunk lengths are calculated correctly
  </Step>
</Steps>

## Complete Migration Example

### Legacy Configuration

```python Python theme={null}
import requests

# Legacy approach
legacy_config = {
    "file": "https://example.com/document.pdf",
    "segment_processing": {
        "Page": {
            "html": "LLM",
            "markdown": "LLM",
            "embed_sources": ["Markdown"]
        },
        "Table": {
            "html": "Auto",
            "markdown": "LLM",
            "embed_sources": ["HTML", "Markdown"]
        },
        "Picture": {
            "html": "Auto",
            "markdown": "Auto",
            "embed_sources": ["Markdown"]
        }
    }
}

# Legacy content access
for chunk in task["output"]["chunks"]:
    for segment in chunk["segments"]:
        ocr_text = segment["content"]      # OCR text
        html = segment["html"]
        markdown = segment["markdown"]
```

### New Configuration

```python Python theme={null}
import requests

# New approach
new_config = {
    "file": "https://example.com/document.pdf",
    "segment_processing": {
        "Page": {
            "format": "Markdown",
            "strategy": "LLM",
            "embed_sources": ["Content"]
        },
        "Table": {
            "format": "Html",
            "strategy": "LLM",
            "embed_sources": ["Content"]
        },
        "Picture": {
            "format": "Markdown",
            "strategy": "Auto",
            "embed_sources": ["Content"]
        }
    }
}

response = requests.post(
    "https://api.chunkr.ai/api/v1/task/parse",
    headers={"Authorization": "YOUR_API_KEY"},
    json=new_config
)

# New content access
for chunk in task["output"]["chunks"]:
    for segment in chunk["segments"]:
        ocr_text = segment["text"]         # OCR text moved here!
        generated = segment["content"]      # HTML or Markdown based on format
        
        # Backward compatible fields still available
        html = segment["html"]
        markdown = segment["markdown"]
```

## Backward Compatibility

The API maintains backward compatibility:

### Configuration Deserialization

Legacy configurations still work:

```python Python theme={null}
# This legacy config still works
legacy = {
    "segment_processing": {
        "Table": {
            "html": "LLM",
            "markdown": "Auto"
        }
    }
}
```

**Resolution logic:**

1. If both `html` and `markdown` use `LLM`, prefer the default format for that segment type
2. If one uses `LLM` and one uses `Auto`, use the `LLM` one
3. If only one is set, use that format and strategy

### Response Fields

All fields remain populated:

```python Python theme={null}
# All fields are still available in responses
segment = {
    "content": "<table>...</table>",  # Generated content (NEW purpose)
    "text": "Raw OCR text",           # OCR text (NEW field)
    "html": "<table>...</table>",     # HTML (backward compatibility)
    "markdown": "| A | B |\n...",   # Markdown (backward compatibility)
}
```

## Benefits of Migration

<Accordion title="Performance Improvements">
  * **Reduced processing time**: Generate only the format you need
  * **Lower resource usage**: Single format generation vs dual format
  * **Faster API responses**: Less content to transfer
</Accordion>

<Accordion title="Cleaner Architecture">
  * **Simplified configuration**: Choose format once instead of multiple strategies
  * **Better resource allocation**: Focus processing on chosen format
  * **Clearer content contracts**: Know exactly what format you'll receive
</Accordion>

<Accordion title="Improved Developer Experience">
  * **Unified content access**: One field for generated content
  * **Clearer field purposes**: `content` for generated, `text` for OCR
  * **Easier embed configuration**: Use `Content` instead of format-specific sources
</Accordion>

## Common Migration Patterns

### Pattern 1: Tables with HTML

```python Python theme={null}
# Before
{"Table": {"html": "LLM", "markdown": "Auto"}}

# After
{"Table": {"format": "Html", "strategy": "LLM"}}
```

### Pattern 2: Text with Markdown

```python Python theme={null}
# Before
{"Text": {"html": "Auto", "markdown": "Auto"}}

# After
{"Text": {"format": "Markdown", "strategy": "Auto"}}
```

### Pattern 3: Mixed Formats

```python Python theme={null}
# Before
{
    "segment_processing": {
        "Table": {"html": "LLM", "markdown": "Auto"},
        "Text": {"html": "Auto", "markdown": "Auto"}
    }
}

# After
{
    "segment_processing": {
        "Table": {"format": "Html", "strategy": "LLM"},
        "Text": {"format": "Markdown", "strategy": "Auto"}
    }
}
```

## Troubleshooting

<Accordion title="Tables not generating AI-enhanced content">
  **Problem:** Using default config, tables aren't processed with LLM

  **Solution:** Explicitly configure table processing:

  ```python theme={null}
  {
      "segment_processing": {
          "Table": {
              "format": "Html",
              "strategy": "LLM"
          }
      }
  }
  ```
</Accordion>

<Accordion title="Getting wrong content from segment.content">
  **Problem:** Expected OCR text but getting HTML/Markdown

  **Solution:** Use `segment.text` for OCR text:

  ```python theme={null}
  ocr_text = segment["text"]  # Not segment["content"]
  ```
</Accordion>

<Accordion title="Embed sources not working">
  **Problem:** Using deprecated `HTML` or `Markdown` embed sources

  **Solution:** Update to `Content`:

  ```python theme={null}
  {"embed_sources": ["Content", "LLM"]}  # Not ["HTML", "Markdown"]
  ```
</Accordion>

## Need Help?

If you encounter issues during migration:

* Check the [processing documents](/guides/processing-documents) guide for current API usage
* Review [custom chunking](/guides/custom-chunking) for segment configuration
* See [VLM processing](/guides/using-vlm) for LLM strategy details
* Contact support at [mehul@chunkr.ai](mailto:mehul@chunkr.ai)

## Migration Checklist

* [ ] Updated all `segment_processing` configs to use `format` and `strategy`
* [ ] Changed OCR text access from `segment.content` to `segment.text`
* [ ] Updated `embed_sources` from `HTML`/`Markdown` to `Content`
* [ ] Added explicit table configuration if using default settings
* [ ] Tested with sample documents
* [ ] Verified chunk embed content is correct
* [ ] Updated documentation and examples
* [ ] Deployed and monitored in production