Skip to main content
Version: Next

RequestQueueClient

Client for managing a specific Request queue.

Request queues store URLs to be crawled and their metadata. Each request in the queue has a unique ID and can be in various states (pending, handled). This client provides methods to add, get, update, and delete requests, as well as manage the queue itself.

@example
const client = new ApifyClient({ token: 'my-token' });
const queueClient = client.requestQueue('my-queue-id');

// Add a request to the queue
await queueClient.addRequest({
  url: 'https://example.com',
  uniqueKey: 'example-com'
});

// Get the next request from the queue
const request = await queueClient.listHead();

// Mark request as handled
await queueClient.updateRequest({
  id: request.id,
  handledAt: new Date().toISOString()
});
@see

https://docs.apify.com/platform/storage/request-queue

Hierarchy

  • ResourceClient
    • RequestQueueClient

Index

Properties

Methods

Properties

inheritedapifyClient

apifyClient: ApifyClient

inheritedbaseUrl

baseUrl: string

inheritedhttpClient

httpClient: HttpClient

optionalinheritedid

id?: string

optionalinheritedparams

params?: Record<string, unknown>

inheritedpublicBaseUrl

publicBaseUrl: string

inheritedresourcePath

resourcePath: string

optionalinheritedsafeId

safeId?: string

inheritedurl

url: string

Methods

addRequest

  • Adds a single request to the queue.

    If a request with the same uniqueKey already exists, the method will return information about the existing request without adding a duplicate. The uniqueKey is used for deduplication - typically it's the URL, but you can use any string to identify the request.

    @see

    https://docs.apify.com/api/v2/request-queue-requests-post

    @example
    const result = await client.requestQueue('my-queue').addRequest({
      url: 'https://example.com',
      uniqueKey: 'example-page',
      method: 'GET',
      userData: { label: 'START', depth: 0 }
    });
    console.log(`Request ID: ${result.requestId}`);
    console.log(`Already present: ${result.wasAlreadyPresent}`);
    console.log(`Already handled: ${result.wasAlreadyHandled}`);

    // Add urgent request to the front of the queue
    await client.requestQueue('my-queue').addRequest(
      { url: 'https://priority.com', uniqueKey: 'priority-page' },
      { forefront: true }
    );

    Parameters

    Returns Promise<RequestQueueClientAddRequestResult>

    Object with requestId, wasAlreadyPresent, and wasAlreadyHandled flags

batchAddRequests

  • Adds multiple requests to the queue in a single operation.

    This is significantly more efficient than calling addRequest multiple times, especially for large batches. The method automatically handles batching (max 25 requests per API call), retries on rate limiting, and parallel processing. Requests are sent in chunks respecting the API payload size limit, and any unprocessed requests due to rate limits are automatically retried with exponential backoff.

    @see

    https://docs.apify.com/api/v2/request-queue-requests-batch-post

    @example
    // Add a batch of URLs to crawl
    const requests = [
      { url: 'https://example.com', uniqueKey: 'page1', userData: { depth: 1 } },
      { url: 'https://example.com/2', uniqueKey: 'page2', userData: { depth: 1 } },
      { url: 'https://example.com/3', uniqueKey: 'page3', userData: { depth: 1 } }
    ];
    const result = await client.requestQueue('my-queue').batchAddRequests(requests);
    console.log(`Successfully added: ${result.processedRequests.length}`);
    console.log(`Failed: ${result.unprocessedRequests.length}`);

    // Batch add with custom retry settings
    const result = await client.requestQueue('my-queue').batchAddRequests(
      requests,
      { maxUnprocessedRequestsRetries: 5, maxParallel: 10 }
    );

    Parameters

    Returns Promise<RequestQueueClientBatchRequestsOperationResult>

    Object with processedRequests (successfully added) and unprocessedRequests (failed after all retries)

batchDeleteRequests

delete

  • delete(): Promise<void>

deleteRequest

  • deleteRequest(id): Promise<void>

  • Deletes a specific request from the queue.

    Parameters

    • id: string

      Request ID

    Returns Promise<void>

deleteRequestLock

  • deleteRequestLock(id, options): Promise<void>

get

getRequest

listAndLockHead

  • Gets and locks the next requests from the queue head for processing.

    This method retrieves requests from the beginning of the queue and locks them for the specified duration to prevent other clients from processing them simultaneously. This is the primary method used by distributed web crawlers to coordinate work across multiple workers. Locked requests won't be returned to other clients until the lock expires or is explicitly released using deleteRequestLock.

    @see

    https://docs.apify.com/api/v2/request-queue-head-lock-post

    @example
    // Get and lock up to 10 requests for 60 seconds
    const { items, lockSecs } = await client.requestQueue('my-queue').listAndLockHead({
      lockSecs: 60,
      limit: 10
    });

    // Process each locked request
    for (const request of items) {
      console.log(`Processing: ${request.url}`);
      // ... process request ...
      // Delete lock after successful processing
      await client.requestQueue('my-queue').deleteRequestLock(request.id);
    }

    Parameters

    Returns Promise<RequestQueueClientListAndLockHeadResult>

    Object containing items (locked requests), queueModifiedAt, hadMultipleClients, and lock information

listHead

listRequests

paginateRequests

prolongRequestLock

  • Prolongs the lock on a request to prevent it from being returned to other clients.

    This is useful when processing a request takes longer than expected and you need to extend the lock duration to prevent other workers from picking it up. The lock expiration time is reset to the current time plus the specified duration.

    @see

    https://docs.apify.com/api/v2/request-queue-request-lock-put

    @example
    // Lock request for initial processing
    const { items } = await client.requestQueue('my-queue').listAndLockHead({ lockSecs: 60, limit: 1 });
    const request = items[0];

    // Processing takes longer than expected, extend the lock
    await client.requestQueue('my-queue').prolongRequestLock(request.id, { lockSecs: 120 });

    Parameters

    Returns Promise<RequestQueueClientProlongRequestLockResult>

    Object with new lockExpiresAt timestamp

unlockRequests

update

updateRequest

Page Options