Data
data class DataSourceWebCrawlerUrls(val seedUrlConfiguration: DataSourceWebCrawlerSeedUrlConfiguration? = null, val siteMapsConfiguration: DataSourceWebCrawlerSiteMapsConfiguration? = null)
Constructors
Link copied to clipboard
constructor(seedUrlConfiguration: DataSourceWebCrawlerSeedUrlConfiguration? = null, siteMapsConfiguration: DataSourceWebCrawlerSiteMapsConfiguration? = null)
Properties
Link copied to clipboard
Configuration of the seed or starting point URLs of the websites you want to crawl. You can choose to crawl only the website host names, or the website host names with subdomains, or the website host names with subdomains and other domains that the web pages link to. You can list up to 100 seed URLs.
Link copied to clipboard
Configuration of the sitemap URLs of the websites you want to crawl. Only URLs belonging to the same website host names are crawled. You can list up to three sitemap URLs.