Skip to main content

ApifyRequestQueueSharedClient

Internal request queue client implementation for multi-consumer scenarios on the Apify platform.

This implementation is optimized for scenarios where multiple clients concurrently fetch and process requests from the same queue. It makes more frequent API calls to ensure consistency across all consumers and uses request locking to prevent duplicate processing.

This class is used internally by ApifyRequestQueueClient when access='shared' is specified.

Public methods are not individually documented as they implement the interface defined in RequestQueueClient.

Index

Methods

Methods

__init__

  • __init__(*, api_client, metadata, cache_size, metadata_getter): None

  • Initialize a new shared request queue client instance.

    Use ApifyRequestQueueClient.open(access='shared') instead of calling this directly.

    Parameters

    • keyword-onlyapi_client: RequestQueueClientAsync

      The Apify API client for request queue operations.

    • keyword-onlymetadata: RequestQueueMetadata

      Initial metadata for the request queue.

    • keyword-onlycache_size: int

      Maximum number of requests to cache locally.

    • keyword-onlymetadata_getter: Callable[[], Coroutine[Any, Any, ApifyRequestQueueMetadata]]

      Async function to fetch current metadata from the API.

    Returns None

add_batch_of_requests

  • async add_batch_of_requests(requests, *, forefront): AddRequestsResponse

  • Specific implementation of this method for the RQ shared access mode.

    Parameters

    • requests: Sequence[Request]
    • optionalkeyword-onlyforefront: bool = False

    Returns AddRequestsResponse

fetch_next_request

  • async fetch_next_request(): Request | None

  • Specific implementation of this method for the RQ shared access mode.

    Returns Request | None

get_request

  • async get_request(unique_key): Request | None

  • Specific implementation of this method for the RQ shared access mode.

    Parameters

    • unique_key: str

    Returns Request | None

is_empty

  • async is_empty(): bool

  • Specific implementation of this method for the RQ shared access mode.

    Returns bool

mark_request_as_handled

  • async mark_request_as_handled(request): ProcessedRequest | None

  • Specific implementation of this method for the RQ shared access mode.

    Parameters

    • request: Request

    Returns ProcessedRequest | None

reclaim_request

  • async reclaim_request(request, *, forefront): ProcessedRequest | None

  • Specific implementation of this method for the RQ shared access mode.

    Parameters

    • request: Request
    • optionalkeyword-onlyforefront: bool = False

    Returns ProcessedRequest | None

Page Options