Skip to main content
Version: Next

abstractHttpClient

Base class for the HTTP clients used by ApifyClient.

It holds the shared request pipeline: call merges the default headers in, serializes and compresses the body, encodes the query parameters, retries transient failures with exponential backoff, grows the timeout with every attempt, decodes the response body by content type, records statistics and converts error statuses to ApifyApiError. A concrete client only has to implement sendRequest, the transport that moves one prepared request over the wire, and can override the classification and lifecycle hooks isTimeoutError, isRetryableTransportError and close.

Overriding call itself is also supported and bypasses the transport hooks entirely. Such a client has to send the default headers from defaultHeaders with every request, otherwise the Authorization header never reaches the API. The protected helpers prepareRequest(), buildUrl() and computeTimeoutMillis() stay available to it.

Hierarchy

import { ApifyClient, HttpClient } from 'apify-client';

class FetchHttpClient extends HttpClient {
    async sendRequest({ method, url, headers, body, timeoutMillis, signal }) {
        const signals = signal ? [signal] : [];
        if (timeoutMillis !== undefined) signals.push(AbortSignal.timeout(timeoutMillis));
        const response = await fetch(url, { method, headers, body, signal: AbortSignal.any(signals) });
        return {
            status: response.status,
            headers: Object.fromEntries(response.headers),
            body: Buffer.from(await response.arrayBuffer()),
        };
    }

    isRetryableTransportError(error) {
        // Without this the client gives up on the first connection failure.
        return this.isTimeoutError(error) || error instanceof TypeError;
    }
}

const client = ApifyClient.withCustomHttpClient({ token: 'my-token', httpClient: new FetchHttpClient() });

Index

Constructors

Properties

Methods

Constructors

constructor

  • Parameters

    • options: HttpClientOptions = {}

      Configuration of the pipeline. The retry and timeout options are validated the same way the ApifyClient constructor validates them.

    Returns HttpClient

    Throws - When a retry or timeout option is out of bounds.

Properties

readonlyhttpCompressor

httpCompressor: HttpCompressor

Compressor the request pipeline runs the bodies worth compressing through.

logger

logger: Log

Logger for the retry warnings.

maxRetries

maxRetries: number

How many times a failed request is retried at most.

minDelayBetweenRetriesMillis

minDelayBetweenRetriesMillis: number

Lower bound for the delay before the first retry in milliseconds. It doubles with every further retry.

stats

stats: Statistics

Statistics of the API calls made through this client.

timeoutMaxMillis

timeoutMaxMillis: number

Upper bound for the timeout of a single attempt, in milliseconds.

timeoutMillis

timeoutMillis: Record<TimeoutTier, number>

Duration of each timeout tier, in milliseconds.

Methods

call

  • Makes an API request with automatic retries and exponential backoff.

    Network errors the transport classifies as retryable, rate limits (HTTP 429) and server errors (HTTP 5xx) are retried up to maxRetries times. Any other error status is thrown as ApifyApiError right away. A request whose body is a Readable is never retried, since part of the stream has already been consumed by the time the failure shows. Aborting config.signal ends the attempt in flight, skips the remaining retries and rejects the call with the signal's reason.

    Parameters

    Returns Promise<ApifyResponse<T>>

    The successful response.

    Throws - When the API responds with an error status the retries could not fix.

close

  • close(): Promise<void>

  • Releases resources owned by the transport, such as a connection pool. The default does nothing.

    Returns Promise<void>

isRetryableTransportError

  • isRetryableTransportError(_error): boolean

  • Whether an error thrown by the transport is worth retrying.

    The default classifies nothing as retryable, so a transport that does not override it gives up on the first connection failure. Every transport should map its own transient errors here: connection resets, refused connections, timeouts and the like. Error responses are not the transport's concern, the pipeline decides on them from the status code.

    Parameters

    • _error: unknown

    Returns boolean

isTimeoutError

  • isTimeoutError(error): boolean

  • Whether an error thrown by the transport is a timeout.

    Recognizes errors named TimeoutError, which is what AbortSignal.timeout() produces. Transports extend it with the timeout errors their HTTP library throws. The classification is independent of retryability: a timeout the retry loop should retry has to be covered by isRetryableTransportError too.

    Parameters

    • error: unknown

    Returns boolean

sendRequest

  • Sends one prepared request through the underlying HTTP library.

    The inherited call needs it, so every transport has to implement it. Let the library's errors propagate unwrapped: call classifies them through isRetryableTransportError and isTimeoutError. Return error responses as they are too, the pipeline turns them into ApifyApiError and decides whether to retry.

    Parameters

    • request: HttpRequest

      The request to send, with the headers merged, the body serialized and the query encoded.

    Returns Promise<HttpResponse>

    The response, with the body unread when request.stream is set and as raw bytes otherwise.

setDefaultAuthorization

  • setDefaultAuthorization(token): void

  • Sets the Authorization header from the token, unless an authorization header is already configured.

    Parameters

    • token: string

      The Apify API token to send as the Bearer token.

    Returns void

Page Options