Limit settings for the web crawler.
Maximum number of pages the crawler can crawl.
Rate of web URLs retrieved per minute.