DataSourceWebCrawlerLimits

data class DataSourceWebCrawlerLimits(val maxPages: Int? = null, val rateLimit: Int? = null)

Limit settings for the web crawler.

Constructors

Link copied to clipboard
constructor(maxPages: Int? = null, rateLimit: Int? = null)

Types

Link copied to clipboard
object Companion

Properties

Link copied to clipboard
val maxPages: Int? = null

Maximum number of pages the crawler can crawl.

Link copied to clipboard
val rateLimit: Int? = null

Rate of web URLs retrieved per minute.