HTTP clients

Copy for LLM

The Apify API client uses a pluggable HTTP layer. It ships with an Impit-based default, offers HTTPX2 as an optional built-in alternative, and accepts custom synchronous or asynchronous implementations.

Default HTTP client

When you create an ApifyClient or ApifyClientAsync instance, it automatically uses the built-in ImpitHttpClient (or ImpitHttpClientAsync). This default client provides:

• Automatic retries with exponential backoff for network errors, HTTP 429, and HTTP 5xx responses.
• Configurable timeouts.
• Request compression and preparation of API-compatible data, query parameters, and headers, including authentication.
• API error handling, structured logging, and request statistics.

You can configure the default client through the ApifyClient or ApifyClientAsync constructor:

Async client

from datetime import timedelta

from apify_client import ApifyClientAsync

TOKEN = 'MY-APIFY-TOKEN'


async def main() -> None:
    client = ApifyClientAsync(
        token=TOKEN,
        max_retries=4,
        min_delay_between_retries=timedelta(milliseconds=500),
        timeout_medium=timedelta(seconds=360),
    )

Sync client

from datetime import timedelta

from apify_client import ApifyClient

TOKEN = 'MY-APIFY-TOKEN'


def main() -> None:
    client = ApifyClient(
        token=TOKEN,
        max_retries=4,
        min_delay_between_retries=timedelta(milliseconds=500),
        timeout_medium=timedelta(seconds=360),
    )

Built-in HTTPX2 client

The package also provides Httpx2HttpClient and Httpx2HttpClientAsync. They use the same request preparation, compression, retry policy, timeout tiers and growth, error handling, logging, and statistics as the default Impit clients, with HTTPX2 as the transport.

The httpx2 package, Pydantic's maintained continuation of HTTPX, is an optional dependency. Install apify-client[httpx2], then pass the appropriate client to ApifyClient.with_custom_http_client. Impit remains the default even when the extra is installed.

pip install "apify-client[httpx2]"
# or
uv add "apify-client[httpx2]"

Async client

import asyncio

from apify_client import ApifyClientAsync
from apify_client.http_clients import Httpx2HttpClientAsync

TOKEN = 'MY-APIFY-TOKEN'


async def main() -> None:
    async with Httpx2HttpClientAsync() as http_client:
        client = ApifyClientAsync.with_custom_http_client(
            token=TOKEN,
            http_client=http_client,
        )
        print(await client.actor('apify/hello-world').get())


if __name__ == '__main__':
    asyncio.run(main())

Sync client

from apify_client import ApifyClient
from apify_client.http_clients import Httpx2HttpClient

TOKEN = 'MY-APIFY-TOKEN'


def main() -> None:
    with Httpx2HttpClient() as http_client:
        client = ApifyClient.with_custom_http_client(
            token=TOKEN,
            http_client=http_client,
        )
        print(client.actor('apify/hello-world').get())


if __name__ == '__main__':
    main()

Configure retries, timeout tiers, default headers, and compression on the HTTPX2 client instance. The token passed to with_custom_http_client is applied automatically unless the HTTP client already has an Authorization header. The examples use the clients as context managers so their connection pools are closed deterministically. If a context manager doesn't fit your application's lifecycle, call close() on Httpx2HttpClient or await aclose() on Httpx2HttpClientAsync during shutdown.

Timeout values are passed to the selected transport. Impit enforces them as a deadline for the whole request, body included. HTTPX2 applies them to each socket operation instead, so a response whose body arrives slowly keeps resetting the timeout and can outlast both the requested timeout and timeout_max. The no_timeout option disables HTTPX2's timeouts.

Architecture

Internally, the HTTP client hierarchy has three layers:

• A common internal base contains configuration and utilities shared by synchronous and asynchronous clients, including headers, request-body preparation, parameters, compression, and timeout tiers. It isn't a public extension point.
• HttpClient and HttpClientAsync add the synchronous or asynchronous request pipeline, retry loop, transport hooks, and lifecycle interface.
• The built-in Impit and HTTPX2 classes inherit directly from the corresponding sync or async class and adapt the underlying transport.

HttpClient.is_timeout_error(exc) and HttpClientAsync.is_timeout_error(exc) are the public, transport-neutral way to tell whether an exception is a timeout, so code built on the client, such as streamed logs, doesn't need to know which transport raised it.

Responses have their own abstraction. HttpResponse is a runtime-checkable protocol that defines the expected response shape. Any object with the required attributes and methods satisfies the protocol, so no inheritance is needed.

Custom transport adapters implement the request, error-classification, and lifecycle hooks. They inherit request preparation, retries, timeout growth, API error conversion, logging, and statistics from the base.

The base classes and the response protocol are available from the apify_client.http_clients module:

from apify_client.http_clients import (
    HttpClient,
    HttpClientAsync,
    HttpResponse,
)

The transport contract

The public call method provides the shared request pipeline. A concrete transport implements these hooks:

• send_request(...) sends one prepared request and returns an HttpResponse, error statuses included. It receives the URL with the query parameters already encoded into it, the headers with the client's default headers already merged in, the body already serialized and compressed, and the timeout for this attempt in seconds. The inherited call needs it, so every transport adapter has to implement it. Let the HTTP library's exceptions propagate unwrapped, and leave status handling and ApifyApiError to call.
• is_retryable_transport_error(exc) classifies transport failures for the shared retry loop. The default classifies nothing as retryable, so a transport that doesn't override it gives up on the first connection failure.
• is_timeout_error(exc) identifies transport-specific timeout exceptions for higher-level client features. The default recognizes Python's TimeoutError. Timeout classification is independent of retryability, so a timeout the retry loop should retry has to be listed in is_retryable_transport_error too.
• close() or aclose() closes resources owned by the transport. The default does nothing, which is correct for a transport that owns no pool or session.

Decorate your implementations with @override, as the built-in Impit and HTTPX2 adapters do, so a type checker catches a misspelled or incompatible override.

The HTTP response protocol

HttpResponse is not a concrete class. Any object with the following attributes and methods will work:

Property / Method	Description
status_code: int	HTTP status code
text: str	Response body as text
content: bytes	Raw response body
headers: Mapping[str, str]	Response headers
json() -> Any	Parse body as JSON
read() -> bytes	Read entire response body
aread() -> bytes	Read entire response body (async)
close() -> None	Close the response
aclose() -> None	Close the response (async)
iter_bytes() -> Iterator[bytes]	Iterate body in chunks
aiter_bytes() -> AsyncIterator[bytes]	Iterate body in chunks (async)

note

Many HTTP libraries, including our default Impit or for example HTTPX2 already satisfy this protocol out of the box.

For a streamed response, consume the body inside the streaming context manager with iter_bytes() / aiter_bytes(), or call read() / aread() before accessing content. Some transports, including HTTPX2, intentionally reject content on an unread streamed response.

Plugging it in

Use the ApifyClient.with_custom_http_client (or ApifyClientAsync.with_custom_http_client) class method to create a client with your custom implementation:

Async client

from typing_extensions import override

from apify_client import ApifyClientAsync
from apify_client.http_clients import HttpClientAsync, HttpResponse

TOKEN = 'MY-APIFY-TOKEN'


class MyHttpClientAsync(HttpClientAsync):
    """Custom async HTTP client."""

    @override
    async def send_request(
        self,
        *,
        method: str,
        url: str,
        headers: dict[str, str],
        content: bytes | None,
        timeout: float | None,
        stream: bool,
    ) -> HttpResponse:
        """Send one request through the custom transport."""
        raise NotImplementedError

    @override
    def is_retryable_transport_error(self, exc: Exception) -> bool:
        # List the transport's transient failures here, e.g. its timeout
        # and connection errors. Returning False for everything opts out
        # of transport retries entirely.
        return isinstance(exc, TimeoutError)


async def main() -> None:
    client = ApifyClientAsync.with_custom_http_client(
        token=TOKEN,
        http_client=MyHttpClientAsync(),
    )

Sync client

from typing_extensions import override

from apify_client import ApifyClient
from apify_client.http_clients import HttpClient, HttpResponse

TOKEN = 'MY-APIFY-TOKEN'


class MyHttpClient(HttpClient):
    """Custom sync HTTP client."""

    @override
    def send_request(
        self,
        *,
        method: str,
        url: str,
        headers: dict[str, str],
        content: bytes | None,
        timeout: float | None,
        stream: bool,
    ) -> HttpResponse:
        """Send one request through the custom transport."""
        raise NotImplementedError

    @override
    def is_retryable_transport_error(self, exc: Exception) -> bool:
        # List the transport's transient failures here, e.g. its timeout
        # and connection errors. Returning False for everything opts out
        # of transport retries entirely.
        return isinstance(exc, TimeoutError)


def main() -> None:
    client = ApifyClient.with_custom_http_client(
        token=TOKEN,
        http_client=MyHttpClient(),
    )

After that, all API calls made through the client will go through your custom HTTP client.

warning

If you override call itself, your implementation becomes responsible for request preparation, retries, timeouts, API error conversion, logging, and statistics. Implementing the transport hooks and inheriting call keeps the shared behavior.

Use cases

Custom HTTP clients might be useful when the built-in Impit and HTTPX2 clients don't cover your requirements, for example when you need to:

• Use a different HTTP library - Integrate requests, aiohttp, or another transport.
• Route through a proxy - Add proxy support or request routing.
• Implement custom retry logic - Use different backoff strategies or retry conditions.
• Log requests and responses - Track API calls for debugging or auditing.
• Modify requests - Add custom fields, modify the body, or change headers.
• Collect custom metrics - Measure request latency, track error rates, or count API calls.

For complete synchronous and asynchronous implementations over a transport with a different response API, see Build a custom HTTP client. The HttpClient and HttpClientAsync API references document the full contract.