rag.splitters.factory

Provide a registry and factory function for instantiating chunker strategies by name.

Provide a registry and factory function for instantiating chunker strategies by name.

get_chunker(strategy='recursive', chunk_size=1000, chunk_overlap=200, metadata=None, **kwargs)[source]

Get a chunker instance based on strategy name.

Parameters:

  • strategy (str) – Chunking strategy name (character, recursive, markdown, code, hierarchical).

  • chunk_size (int) – Maximum chunk size.

  • chunk_overlap (int) – Overlap between chunks.

  • metadata (dict[str, Any] | None) – Default metadata for chunks.

  • **kwargs (Any) – Additional strategy-specific parameters.

Returns:

Configured chunker instance.

Raises:

ValueError – If strategy name is not recognized.

Return type:

ChunkerBase

list_chunkers()[source]

List available chunking strategies.

Returns:

List of strategy names.

Return type:

list[str]

chunk_text(text, strategy='recursive', chunk_size=1000, chunk_overlap=200, metadata=None, **kwargs)[source]

Convenience function to chunk text with a single call.

Parameters:

  • text (str) – Text to chunk.

  • strategy (str) – Chunking strategy name.

  • chunk_size (int) – Maximum chunk size.

  • chunk_overlap (int) – Overlap between chunks.

  • metadata (dict[str, Any] | None) – Metadata to attach to chunks.

  • **kwargs (Any) – Additional strategy-specific parameters.

Returns:

List of TextChunk objects.

Return type:

list[TextChunk]

Functions

chunk_text(text, strategy='recursive', chunk_size=1000, chunk_overlap=200, metadata=None, **kwargs)[source]

Convenience function to chunk text with a single call.

Parameters:

  • text (str) – Text to chunk.

  • strategy (str) – Chunking strategy name.

  • chunk_size (int) – Maximum chunk size.

  • chunk_overlap (int) – Overlap between chunks.

  • metadata (dict[str, Any] | None) – Metadata to attach to chunks.

  • **kwargs (Any) – Additional strategy-specific parameters.

Returns:

List of TextChunk objects.

Return type:

list[TextChunk]

get_chunker(strategy='recursive', chunk_size=1000, chunk_overlap=200, metadata=None, **kwargs)[source]

Get a chunker instance based on strategy name.

Parameters:

  • strategy (str) – Chunking strategy name (character, recursive, markdown, code, hierarchical).

  • chunk_size (int) – Maximum chunk size.

  • chunk_overlap (int) – Overlap between chunks.

  • metadata (dict[str, Any] | None) – Default metadata for chunks.

  • **kwargs (Any) – Additional strategy-specific parameters.

Returns:

Configured chunker instance.

Raises:

ValueError – If strategy name is not recognized.

Return type:

ChunkerBase

list_chunkers()[source]

List available chunking strategies.

Returns:

List of strategy names.

Return type:

list[str]