DataSourceSemanticChunkingConfiguration

data class DataSourceSemanticChunkingConfiguration(val breakpointPercentileThreshold: Int, val bufferSize: Int, val maxTokens: Int)

Configurations for when you choose semantic chunking. If you set the chunkingStrategy as NONE, exclude this field.

Constructors

Link copied to clipboard
constructor(breakpointPercentileThreshold: Int, bufferSize: Int, maxTokens: Int)

Types

Link copied to clipboard
object Companion

Properties

Link copied to clipboard

The dissimilarity threshold for splitting chunks.

Link copied to clipboard

The buffer size.

Link copied to clipboard

The maximum number of tokens that a chunk can contain.