Skip to main content
Version: 3.0

Crawl all links on a website

This example uses the enqueueLinks() method to add new links to the RequestQueue as the crawler navigates from page to page. If only the required parameters are defined, all links will be crawled.

Using CheerioCrawler:

import { Actor } from 'apify';
import { CheerioCrawler } from 'crawlee';

await Actor.init();

const crawler = new CheerioCrawler({
async requestHandler({ request, enqueueLinks }) {
console.log(request.url);
// Add all links from page to RequestQueue
await enqueueLinks();
},
maxRequestsPerCrawl: 10, // Limitation for only 10 requests (do not use if you want to crawl all links)
});

// Run the crawler
await crawler.run(['https://apify.com/']);

await Actor.exit();