Skip to main content
Version: Next

Error handling

Copy for LLM

When you run an Actor, exceptions come from a few layers: the Apify API client for failed API requests, the Apify SDK for misuse and invalid input, and the libraries you build on, such as Crawlee.

Errors from the Apify API

Every SDK operation that talks to the Apify API can raise ApifyApiError. Such operations include Actor.start, Actor.call, Actor.abort, Actor.metamorph, Actor.add_webhook, charging, and all storage operations on datasets, key-value stores, and request queues. The SDK raises these client exceptions as-is, so you keep the HTTP status code, the error type, and the response data on the exception.

ApifyApiError dispatches to a subclass based on the HTTP status code:

The client retries rate-limited and server errors on its own, so you only see RateLimitError or ServerError once those retries are exhausted. The apify.errors module re-exports the whole client error hierarchy, so you can import everything from one place:

from apify.errors import ApifyApiError, NotFoundError, RateLimitError

To handle any API failure in one place, catch ApifyApiError, then branch on the subclass or the HTTP status_code. To react to a specific failure, catch its subclass first:

Run on
import asyncio

from apify import Actor
from apify.errors import ApifyApiError, NotFoundError


async def main() -> None:
    async with Actor:
        try:
            run = await Actor.call('apify/web-scraper', run_input={'startUrls': []})
        except NotFoundError:
            # Catch a specific subclass first.
            Actor.log.error('The Actor to call does not exist.')
            return
        except ApifyApiError as exc:
            # Any other API failure, e.g. an invalid token or a server error.
            Actor.log.error(f'Calling the Actor failed: {exc} (HTTP {exc.status_code}).')
            return

        # `Actor.call` returns the finished run whatever its status, so check it.
        if run.status != 'SUCCEEDED':
            Actor.log.error(f'Run {run.id} ended with status {run.status}.')
            return

        Actor.log.info(f'Run {run.id} finished successfully.')


if __name__ == '__main__':
    asyncio.run(main())

Misuse and invalid input

The SDK raises standard Python exceptions when it's used incorrectly or given invalid input. These exceptions point to a bug or a bad argument in your code, so the fix is to correct the call rather than to catch the exception.

  • RuntimeError when an Actor method is used outside the async with Actor: block, either before initialization or after exit, or when the Actor is initialized twice.
  • ValueError for an invalid argument, such as a malformed timeout, an invalid proxy configuration, charging an automatically charged event by hand, or pushing data that is not JSON-serializable or is over the size limit.
  • TypeError for an argument of the wrong type.
  • ConnectionError when Actor.create_proxy_configuration verifies Apify Proxy access and the proxy reports that you have none.

Run failures

Actor.call and Actor.call_task wait for the run to finish and return it, whatever its final status. A finished run can be SUCCEEDED, FAILED, ABORTED, or TIMED-OUT, so check run.status before you rely on the run's output. A timed-out run is the one case where retrying can help, as long as you give it more time:

Run on
import asyncio
from datetime import timedelta

from apify import Actor


async def main() -> None:
    async with Actor:
        timeout = timedelta(minutes=5)
        max_attempts = 3

        for attempt in range(1, max_attempts + 1):
            run = await Actor.call('apify/web-scraper', timeout=timeout)

            if run.status != 'TIMED-OUT' or attempt == max_attempts:
                Actor.log.info(f'Run {run.id} ended with status {run.status}.')
                break

            timeout *= 2
            Actor.log.warning(f'Timed out, retrying with timeout {timeout}.')


if __name__ == '__main__':
    asyncio.run(main())

The pay-per-event charge limit

Reaching the pay-per-event charge limit doesn't raise an error. Instead, the SDK caps charging and data pushing, while your Actor keeps running. When a single Actor.charge call crosses the limit, only the part that fits within the budget is billed, and charged_count on the returned ChargeResult reports how many events went through. Actor.push_data behaves the same way when given a charged_event_name. It writes only the items that fit within the budget.

To detect the limit, check the event_charge_limit_reached field on the ChargeResult. It's a return value and not an exception, so you can read it in a tight charging loop and stop your work once the budget runs out. For details, see Pay-per-event monetization.

Errors while crawling

If your Actor runs a Crawlee crawler, failures inside request handlers surface as Crawlee exceptions. Crawlee handles the retries and session rotation around them, so a single failing request doesn't stop the crawl. API calls you make from inside a handler still raise ApifyApiError. For how to handle those errors, see Errors from the Apify API.

Conclusion

Most failures you handle at runtime are ApifyApiError from the API client. Catch it to cover any API failure, and reach for a subclass or the HTTP status_code when you need finer control. The standard RuntimeError, ValueError, and TypeError signal a bug or bad input, so correct the call rather than catch them. After Actor.call, check run.status to react to a failed run, and let Crawlee handle the errors raised inside a crawler.