Retries
The Apify client automatically retries requests that fail due to:
- Network errors
- Internal errors in the Apify API (HTTP status codes 500 and above)
- Rate limit errors (HTTP status code 429)
By default, the client retries a failed request up to 4 times. The retry intervals use an exponential backoff strategy:
- The first retry occurs after approximately 500 milliseconds.
- The second retry occurs after approximately 1,000 milliseconds, and so on.
You can customize this behavior using the following options in the ApifyClient constructor:
max_retries: Defines the maximum number of retry attempts. Set it to0to disable retries.min_delay_between_retries: Sets the minimum delay between retries as atimedelta.
Retries with exponential backoff help reduce the load on the server and increase the chances of a successful request.
- Async client
- Sync client
from datetime import timedelta
from apify_client import ApifyClientAsync
TOKEN = 'MY-APIFY-TOKEN'
async def main() -> None:
apify_client = ApifyClientAsync(
token=TOKEN,
max_retries=4,
min_delay_between_retries=timedelta(milliseconds=500),
timeout_short=timedelta(seconds=5),
timeout_medium=timedelta(seconds=30),
timeout_long=timedelta(seconds=360),
timeout_max=timedelta(seconds=360),
)
from datetime import timedelta
from apify_client import ApifyClient
TOKEN = 'MY-APIFY-TOKEN'
def main() -> None:
apify_client = ApifyClient(
token=TOKEN,
max_retries=4,
min_delay_between_retries=timedelta(milliseconds=500),
timeout_short=timedelta(seconds=5),
timeout_medium=timedelta(seconds=30),
timeout_long=timedelta(seconds=360),
timeout_max=timedelta(seconds=360),
)
Request statistics
The client counts its calls, HTTP requests, and rate limit errors. To read the counts, use the stats property. It returns a ClientStatistics object with these fields:
callsis the number of calls made through the client, including requests to Actor servers.requestsis the number of HTTP requests sent, including retries and requests to Actor servers.rate_limit_errorsmaps each attempt to the number of rate limit (429) errors it got. Key0is the first attempt, key1is the first retry, and so on. The field is adefaultdict, so reading a missing key adds it. To read a count without changing the dict, use.get(). Only API calls count here. A 429 from an Actor server called throughActorServerClient.requestis not counted.
With a custom HTTP client, stats returns the statistics of that client. For details, see HTTP clients.
- Async client
- Sync client
from apify_client import ApifyClientAsync
TOKEN = 'MY-APIFY-TOKEN'
async def main() -> None:
apify_client = ApifyClientAsync(TOKEN)
await apify_client.actor('username/actor-name').call()
stats = apify_client.stats
print(f'API calls: {stats.calls}, HTTP requests: {stats.requests}')
print(f'Rate limit errors per attempt: {dict(stats.rate_limit_errors)}')
from apify_client import ApifyClient
TOKEN = 'MY-APIFY-TOKEN'
def main() -> None:
apify_client = ApifyClient(TOKEN)
apify_client.actor('username/actor-name').call()
stats = apify_client.stats
print(f'API calls: {stats.calls}, HTTP requests: {stats.requests}')
print(f'Rate limit errors per attempt: {dict(stats.rate_limit_errors)}')
Wait for resources to start a run
Starting a run fails with an HTTP 402 error when the account doesn't have enough free memory for the run, or when it already runs as many Actors as its plan allows. The error type is actor-memory-limit-exceeded or concurrent-runs-limit-exceeded. The client doesn't retry these errors on its own, since they clear only after other runs or builds of the account finish.
To keep retrying the start until the resources free up, set the wait_for_resources argument of ActorClient.start, ActorClient.call, or the same methods of TaskClient. The client then retries the start every 10 seconds, and the argument value sets how long:
Trueretries until the run starts.- A
timedeltastops retrying after that time and raises the last error.
Any other error raises right away. A run that asks for more memory than the account's whole memory limit gets the same actor-memory-limit-exceeded error and never starts, so with True the client retries it forever. In call, the time spent retrying doesn't count toward wait_duration.
- Async client
- Sync client
from datetime import timedelta
from apify_client import ApifyClientAsync
TOKEN = 'MY-APIFY-TOKEN'
async def main() -> None:
apify_client = ApifyClientAsync(TOKEN)
# Retry the start until the account has the resources for the run.
run = await apify_client.actor('username/actor-name').call(wait_for_resources=True)
# Stop retrying after 10 minutes and raise the last error.
started_run = await apify_client.task('username~task-name').start(
wait_for_resources=timedelta(minutes=10),
)
from datetime import timedelta
from apify_client import ApifyClient
TOKEN = 'MY-APIFY-TOKEN'
def main() -> None:
apify_client = ApifyClient(TOKEN)
# Retry the start until the account has the resources for the run.
run = apify_client.actor('username/actor-name').call(wait_for_resources=True)
# Stop retrying after 10 minutes and raise the last error.
started_run = apify_client.task('username~task-name').start(
wait_for_resources=timedelta(minutes=10),
)