DataSourceFixedSizeChunkingConfiguration

data class DataSourceFixedSizeChunkingConfiguration(val maxTokens: Int, val overlapPercentage: Int)

Configurations for when you choose fixed-size chunking. If you set the chunkingStrategy as NONE, exclude this field.

Constructors

Link copied to clipboard
constructor(maxTokens: Int, overlapPercentage: Int)

Types

Link copied to clipboard
object Companion

Properties

Link copied to clipboard

The maximum number of tokens to include in a chunk.

Link copied to clipboard

The percentage of overlap between adjacent chunks of a data source.