rag.splitters.character

Split text into overlapping character-based chunks with optional word-boundary awareness.

class CharacterChunker(chunk_size=1000, chunk_overlap=200, metadata=None, respect_word_boundaries=True)[source]

Character-based chunking with word-boundary awareness.

Parameters:

chunk_size (int) – Maximum characters per chunk.
chunk_overlap (int) – Overlap between consecutive chunks.
metadata (dict[str, Any] | None) – Default metadata for all chunks.
respect_word_boundaries (bool) – Whether to prefer word boundaries for splits.

chunk(text, metadata=None)[source]

Split text into overlapping chunks.

Parameters:

text (str) – The text to split.
metadata (dict[str, Any] | None) – Optional metadata to attach to each chunk.

Returns:

List of TextChunk objects.

Return type:

list[TextChunk]

Classes

class CharacterChunker(chunk_size=1000, chunk_overlap=200, metadata=None, respect_word_boundaries=True)[source]

Character-based chunking with word-boundary awareness.

Parameters:

chunk_size (int) – Maximum characters per chunk.
chunk_overlap (int) – Overlap between consecutive chunks.
metadata (dict[str, Any] | None) – Default metadata for all chunks.
respect_word_boundaries (bool) – Whether to prefer word boundaries for splits.

chunk(text, metadata=None)[source]

Split text into overlapping chunks.

Parameters:

text (str) – The text to split.
metadata (dict[str, Any] | None) – Optional metadata to attach to each chunk.

Returns:

List of TextChunk objects.

Return type:

list[TextChunk]