Apify SDK
The Apify SDK is a toolkit for building Actors—serverless microservices running (not only) on the Apify platform. Apify comes with first-class support for JavaScript/TypeScript and Python, but you can run any containerized code on the Apify platform.
SDK for JavaScript
Toolkit for building Actors—serverless microservices running (not only) on the Apify platform.
npx apify-cli create my-crawler
// The Apify SDK makes it easy to initialize the actor on the platform with the Actor.init() method,
// and to save the scraped data from your Actors to a dataset by simply using the Actor.pushData() method.
import { Actor } from 'apify';
import { PlaywrightCrawler } from 'crawlee';
await Actor.init();
const crawler = new PlaywrightCrawler({
async requestHandler({ request, page, enqueueLinks }) {
const title = await page.title();
console.log(`Title of ${request.loadedUrl} is '${title}'`);
await Actor.pushData({ title, url: request.loadedUrl });
await enqueueLinks();
}
});
await crawler.run(['https://crawlee.dev']);
await Actor.exit();
SDK for Python
The Apify SDK for Python is the official library for creating Apify Actors in Python. It provides useful features like actor lifecycle management, local storage emulation, and actor event handling.
apify create my-python-actor
# The Apify SDK makes it easy to read the actor input with the Actor.get_input() method,
# and to save the scraped data from your Actors to a dataset by simply using the Actor.push_data() method.
from apify import Actor
from bs4 import BeautifulSoup
import requests
async def main():
async with Actor:
actor_input = await Actor.get_input()
response = requests.get(actor_input['url'])
soup = BeautifulSoup(response.content, 'html.parser')
await Actor.push_data({ 'url': actor_input['url'], 'title': soup.title.string })