rag.splitters.recursive

Split text recursively by a prioritized list of separators.

class RecursiveChunker(chunk_size=1000, chunk_overlap=200, metadata=None, separators=None, keep_separator=True)[source]

Recursive chunking that splits hierarchically by separators.

Parameters:

chunk_size (int) – Maximum characters per chunk.
chunk_overlap (int) – Overlap between consecutive chunks.
metadata (dict[str, Any] | None) – Default metadata for all chunks.
separators (list[str] | None) – List of separators in order of preference.
keep_separator (bool) – Whether to keep the separator with chunks.

DEFAULT_SEPARATORS = ['\n\n\n', '\n\n', '\n', '. ', '! ', '? ', '; ', ', ', ' ', '']

chunk(text, metadata=None)[source]

Split text recursively using hierarchical separators.

Parameters:

text (str) – The text to split.
metadata (dict[str, Any] | None) – Optional metadata to attach to each chunk.

Returns:

List of TextChunk objects.

Return type:

list[TextChunk]

Classes

class RecursiveChunker(chunk_size=1000, chunk_overlap=200, metadata=None, separators=None, keep_separator=True)[source]

Recursive chunking that splits hierarchically by separators.

Parameters:

chunk_size (int) – Maximum characters per chunk.
chunk_overlap (int) – Overlap between consecutive chunks.
metadata (dict[str, Any] | None) – Default metadata for all chunks.
separators (list[str] | None) – List of separators in order of preference.
keep_separator (bool) – Whether to keep the separator with chunks.

DEFAULT_SEPARATORS = ['\n\n\n', '\n\n', '\n', '. ', '! ', '? ', '; ', ', ', ' ', '']

chunk(text, metadata=None)[source]

Split text recursively using hierarchical separators.

Parameters:

text (str) – The text to split.
metadata (dict[str, Any] | None) – Optional metadata to attach to each chunk.

Returns:

List of TextChunk objects.

Return type:

list[TextChunk]