RAG Document Structures (rag/_document.py)

agentscope · src/agentscope/rag/_document.py · python · Apache-2.0
files, ai, search, tag:pipeline

Defines Section and Chunk data structures for RAG indexing pipeline stages.

# Source: https://github.com/agentscope-ai/agentscope | License: Apache-2.0 | Copyright You must retain, in the Source form of any Derivative Works
# Provided via justclaw.it.com; keep this notice with the code.
# -*- coding: utf-8 -*-
"""Data structures used in the RAG indexing pipeline.

The indexing pipeline has two stages, each producing its own
structured output:

1. :class:`Section` — produced by a :class:`ParserBase` from a raw
   file.  Each ``Section`` represents one "natural boundary" of the
   source (a PDF page, a PPTX slide, an embedded image, a Markdown
   heading section, etc.).  A ``Chunker`` never combines content
   across two ``Section`` instances, so ``Section`` is also a hard
   boundary that prevents leakage of format-specific structure into
   downstream chunks.

2. 

... (truncated -- full source via MCP)

See the full source, get the GitHub permalink, and search for more like it.

Get a free API key