# Development

In this section, you can find the documentation on developing and building your own actor.

*   [Input schema](/actors/development/input-schema) - definition of the input an actor can accept.
*   [Source code](/actors/development/source-code) - the location of an actor's source code.
*   [Builds](/actors/development/builds) - details on building your actor.
*   [Base Docker images](/actors/development/base-docker-images) - Docker images that can be used as a base for an actor.
*   [Environment variables](/actors/development/environment-variables) - variables that provide an actor with context.
*   [State persistence](/actors/development/state-persistence) - persisting an actor's state to preserve data in case of interruptions.


Read about the technical part of building Apify actors. Learn to define actor inputs, build new versions, persist actor state and choose base Docker images.

Rf0FORjzzglhH3gJ416MnFaG70f438P8IAQnLCPReU4=

Overview of the Apify platform's documentation. Make your first steps in web scraping and automating processes or dive deep into technical reference.

Home

47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=

Step-by-step tutorial that will help you get started with all Apify Scrapers. Learn the foundations of scraping the web with Apify and creating your own actors.

Getting started

Getting started with Apify Scrapers

m9vaN2PzzDK1jbyIAvs6coFyK2HxrCiT7B4hWYvoA+c=

Learn how to scrape a website using Apify's Web Scraper. Build an actor's page function, extract information from a web page and download your data.

Web Scraper

Learn how to scrape a website using Apify's Cheerio Scraper. Build an actor's page function, extract information from a web page and download your data.

Cheerio Scraper

Learn how to scrape a website using Apify's Puppeteer Scraper. Build an actor's page function, extract information from a web page and download your data.

Puppeteer Scraper

GQqUZbTZXWZXGzyLTrY8daJroYQnZfA/VIOoLLE59jI=

Discover Apify's ready-made web scraping and automation tools. Compare Web Scraper, Cheerio Scraper and Puppeteer Scraper to decide which is right for you.

Apify scrapers

/XHyk1PhMK8yAqm/xDZpWRBMjL1elLD2PAHPJa78adk=

Learn how to scrape the web and automate processes with Apify. From beginner guides for using actors to advanced topics like migrations and performance.

Tutorials

GW6nCZ0OC5mfmBhs237v810i309EWxBBdFD4qBEl2iE=

Explore the basics of web scraping prevention. Get around them with IP address rotation and proxies, emulate browser signatures or shared IP address sessions.

Anti-scraping techniques

Anti-scraping techniques and how to bypass them

RgfCfVSjz/3ZXxGPZuyENtgi6STG7b1SjFdz7WGccH8=

All you need to know about extracting structured data from web pages, the protections websites employ to prevent it, and how to bypass them.

Web Scraping 101

SgjKBP72EBa6tzjtXFSxLIzqUPbXL2geBBJxsxkzNYo=

Create your first Apify actor using code examples. A step-by-step introduction to web scraping with the Apify platform.

Quick start

hdjOX0NXsZ5RwgEEo9iveV2scjIVlYsLXHJAup1+VrE=

Configure your actor's input parameters using the Apify app, locally or via API. Access parameters in key-value stores from your actor's code.

Input

t/Oh+kb4g6ZdwAuwnCplq3jxe8Emlyi1Er4PTzBcQ8A=

Start your actors from the Apify app or via API. Learn about actor lifecycles, how to specify settings and version, provide input and resurrect finished runs.

Running

sLDtD2xrgh7nwXFuMKcs76WD+52aJqzAmrCJCbA3HiE=

Create custom versions of an actor for specific use cases. Set a task's name and input parameters.

Configure

d6TQvlwys4aLEYRr6jd6z0eD3jK5Z9YbRrLB3iDf3xE=

Learn how to set up reusable versions of your actors for specific use cases.

Create

S87dynVsMwG52LgS79qb/FKw8BQ1iWbFSHFFi1BXlvo=

Start your actor tasks from the Apify app or via API.

8rzw7exASJlANvytYsIVXV0mPnQq1k11yo2ZzOhuCl4=

Set up configurations of your Apify actors for simplified usage. Create multiple configurations of the same actor tailored to various use cases.

Tasks

UGu0V5+N7lwMC2u0t5smvU6OWVbl3SxlwbDUDFcZi88=

Learn about the Docker images you can use as your actor's base. Choose the correct base image for your actor's type and requirements.

Base Docker images

p8TYQel2zLO3KFfwEcWZjwfMqSEPdyC/BAqn+H5BQAM=

Learn how to provide your actor with context that determines its behavior.

Environment variables

haV31b5XZaHp5YimWCyLZawTrV1KObIiTs9dxFyLUyk=

Apify's conventions for actor build numbers. Learn to use a specific version of your actor in a run, understand an actor's lifecycle and manage its cache.

Builds

PHULHnxQvLhfLlwPoOyjfdizORU0CynqIMbfFwtD0Jo=

Learn how to define and easily validate a schema for your actor's input with code examples. Provide an autogenerated input UI to your actor's users.

Input schema

auTeHP7/ab1dpM/Q+dO4OAO2Om0dlxwnahDTb/rod68=

Learn all about an Apify actor's code. Compare deployment options, create a custom Dockerfile, set environment variables and metamorph your actor.

Source code

LspdNqk99R6hNBUCQyaR4IxMXbJSRmV7aqw68Swku5U=

Maintain a long-running actor's state to protect from unexpected restarts. See a code example on how to protect your run in case of server shutdown.

State persistence

9EROvX5Mrfe+pYTdG8yL0WpLnf/aNROHVfHVi8u4NrE=

Development

5yBYwebrjpiiUc0dSGP2vb8QThQH60u9LMG7xcTjAMc=

Apify's standards for actor naming. Learn how to choose the right name for scraping and non-scraping actors and how to optimize your actor for search engines.

Naming your actor

gd6JDIYJESLw1B06Tg6kDcLNv+gST9FI5mDo1qPqJQI=

Optimize your actors to get more relevant visits from Google and other search engines. Set search engine-friendly actor name, title and description examples.

SEO for actors

Ouytpd/3wsTXT6BHL0MpQ/JC5MDvba7PvMHZMyPHbRk=

Learn how to make your actor available to the public or keep it private. Prepare your actor for Apify Store with a description and README file.

Publishing

JquS//dbbBI78gothu2Cj4C8iGrsjoHb4HseuGKzXwI=

Read Apify's security whitepaper, which describes our best practices.

Security

Zec7v/RKM31JENADfka6icpgFTOTjrKlUHI1uM6DtSk=

Learn the Apify platform's resource capability and limitations such as max memory, disk size and number of actors/tasks per user or schedule.

Limits

R04YPQA/seFALnBpat183ofD8vSoO2J9Z1Phj2yuxKQ=

Discover the Apify platform's various features. This page contains introductory code for building actors, Dockerfiles and persisting your actor's state.

Examples

WAVoQICVogn6Nhu1jv0RjfCoZ+VerBLcoh0AniT+/2c=

Learn how to develop, run and share serverless cloud programs. Create your own web scraping and automation tools and publish them on the Apify platform.

Actors

Z/FkZY34iBJe4Rix81PYtjOfGP88H4RxYxnXJCI3xj4=

Learn how to automatically start your actor and task runs and the basics of *cron* expressions. Set up and manage your schedules from the Apify app or via API.

Schedules

NoqFxs9zcXOfoIamX0dqeuptusho2C0EQarqAWxIciU=

Store and export web scraping, crawling or data processing job results. Learn how to access and manage datasets in the Apify app or via API.

Dataset

2SYp5PnyiCMW7G3HqAT3ufWi7Udb4HkAiI6svZS7YfY=

Store anything from actor or task run results JSON documents or images. Learn how to access and manage key-value stores from the Apify app or via API.

Key-value store

nNYrRI8zyzhTfgLwNQQLsH+T8CqaXwRLakMiPQG1BM4=

Queue URLs for an actor to visit in its run. Learn how to share your queues between actor runs. Access and manage request queues from the Apify app or via API.

Request queue

DG8xFqPxmkKpeMU5SibkFi/YDRREghKQn94wBc2jCo0=

Store anything from images and key-value pairs to structured output data. Learn how to access and manage your stored data from the Apify platform or via API.

Storage

UbZCetJdeVc3tNcdDloDduyB+27U5skBnVyGZ//kdD4=

Learn how to connect your application to Apify Proxy. See the required parameters such as the correct username and password.

Connection settings

QflS3ivuNrdmWBACONXjYiQj0rocnp1LpnltqJHYl5Y=

Learn how to connect to Apify's datacenter proxies from your application using Node.js code examples. Configure proxy groups and IP address use.

NodeJS Examples

uElFiQ5m+dzNOTwKFdS7Q/SvhsUQZ7CmC30fBz8dXSs=

Learn how to connect to Apify's datacenter proxies from your application using PHP code examples. Configure proxy groups and IP address use.

PHP Examples

MlHBxn+xCYer60ScHzb3AOCJ9sHptow7eYmJbthNf3I=

Learn how to connect to Apify's datacenter proxies from your application using Python 2 and 3 code examples. Configure proxy groups and IP address use.

Python Examples

EcYaoCBpS7920SsdrcoSfeMcNOsZKsS9Y6abI0b7W+g=

Learn how to reduce blocking when web scraping using IP address rotation. See proxy parameters and learn to implement Apify Proxy in an application.

Datacenter proxy

RQuahEbBESzT+4T2FITm96eAPQ0y9//to4ngETYAbAU=

Learn how to connect to Apify's residential proxies from your application using PHP code examples. Configure proxy locations and IP address use.

IYL7JrFDLMWrIt55y0BNFXNSU4qS2evbTueMxuw5oE0=

Learn how to connect to Apify's residential proxies from your application using Node.js code examples. Configure proxy locations and IP address use.

kwTIWqSsK1bfgdhn1GWHQOmnJAf3/E3HU0oRmQglLZk=

Learn how to connect to Apify's residential proxies from your application using Python 2 and 3 code examples. Configure proxy locations and IP address use.

eK6bGkaenHJGXCrDfyRZb01dkjGwdoVc+wATsVU88/Y=

Helpful tips for using your application with Apify's residential proxies. Control traffic, deal with interrupted connections and manage expenses.

Tips and tricks

hc2AeukW6YMtu+wl0TYclAEnKIAeo4PeN/7kWe+y2gQ=

Achieve a higher level of anonymity using real IP addresses. Access a wider pool of proxies and reduce blocking by websites' anti-scraping measures.

Residential proxy

vJOQbZx1z78hVsoRvu8zgWV7s/fDaVPm6mc3a6mEq18=

Learn how to connect to Google SERP proxies from your application using Node.js code examples. Configure proxy locations and reduce blocking when web scraping.

H+byXPJKn+0M4ZF5MeY9p/BtvXmGVzMAdjHxQWmPjY0=

Learn how to connect to Google SERP proxies from your application using PHP code examples. Configure proxy locations and reduce blocking when web scraping.

q53gTkGkPr+/L1X9nIossUphpQdZsU2tvJueXQPCEs0=

Learn how to connect to Google SERP proxies from your application using Python 2 and 3 code examples. Configure proxy locations and reduce blocking.

otC6Psx8VpI6S0OQlYBHRqWbNow4R2lhfvodXuKSY9s=

Learn how to collect search results from Google Search-powered tools. Get search results from localised domains in multiple countries, e.g. the US and Germany.

Google SERP proxy

iiMzdwcCnEOnNDEoa/dDpFb9Z25098dct4uyJunFtf4=

Useful tips for debugging applications that use Apify Proxy. Check the status of your proxies and view information about the client IP address.

Troubleshooting

QaCQacQ7s2ao3vK+joGO8RfQxpGM4X+pW20kvdnIMNc=

Learn how to anonymously access websites when running web scraping or automation jobs. Prevent IP address-based blocking using IP address rotation.

Proxy

O0juD9aPRsi9SZj/UD1rd+MdXeJSJI51BYOL2Y3CJ1g=

Specify the types of events that trigger a webhook in an actor or task run. Trigger an action on actor or task run creation, success or failure.

Events

MfxLIXpn7cnh112PIP3F9JsXQVPuWYklUtMVtgyve+A=

Send a notification when a specific event occurs in your actor or task run. Dynamically add data to the notification payload when sending the notification.

Actions

vQW7QIts1nzeOlcXRjL2T7sRR6SeIiCHa3oXvL7L+5g=

Learn how to set one-off webhooks for actor runs started via the Apify API or from the actor's code. Trigger the event once the run reaches a desired state.

Ad hoc webhooks

yZDQCzNv8M1G1sTD/72XdAol2oyN48bLKY2vDm/psf4=

Learn how to integrate multiple Apify actors or external systems with your actor or task run. Send alerts when your actor run succeeds or fails.

Webhooks

zxnTweednbXfiA9q+ooEkqyckabWJ4CeTw59Dge33NI=

Configure your organization account by inviting new members and assigning their roles. Manage team members' access permissions to the organization's resources.

Setup

F5a+jlLeJiaHdYBnbRwm6d4k01LqyiirGX6eOK6x1Ts=

Learn to use and manage your organization account using the Apify app or API. View the organizations you are in and manage your memberships.

How to use

J26efX3rmKMQlFk/7gH5GKL8Fo8Udk2qbA/+mPsXzv4=

Create a specialized account for your organization to encourage collaboration and manage permissions efficiently. Convert an existing account or create one from scratch.

Organization account

8VMHCw0IIaMLSQUdkU4M5JOd6SLL7dKAEjJY5b6GztI=

Learn about the access rights you can grant to other users. See a list of all access options for Apify resources such as actors, actor runs/tasks and storage.

List of permissions

7r+IoYov5vTv4TGI/Rcx2u1NVwaRSI5m0VuWSiqSTfc=

Manage permissions for organizations or private resources such as actors, actor runs and storages. Allow other users to read, run, modify or build new versions.

Access rights

daEdpEyAJIa8b2VkCqSKcw8PaExcB6Qro80XNes/sHA=

Manage, build and run actors, access and manage your storages with the Apify REST API (v2). Test API endpoints and create URL requests using the Blueprint GUI.

API reference

Simplified access to the Apify API from any JavaScript or Node.js application. Manage your actors, tasks and storage via API with the apify-client NPM package.

JavaScript API client

AbpHGcgLb+kRsJGnwFEktk7uzpZOCcBY74+YBdrKVGs=

Create new actors from your computer's command line. Run actors locally or deploy them to the Apify platform. View the Apify CLI's command reference.

Command-line interface

Develop web scraping and automation tools using JavaScript/Node.js and headless Chrome and Puppeteer. Build Apify actors locally or upload to the Apify cloud.