ApifyScheduler

A Scrapy scheduler that uses the Apify Request Queue to manage requests.

This scheduler requires the asyncio Twisted reactor to be installed.

Index

Methods

• __init__
• enqueue_request
• has_pending_requests
• next_request
• open

Methods

__init__

• __init__(): None

• Create a new instance.

  ---

  Returns None

enqueue_request

• enqueue_request(*, request): bool

• Add a request to the scheduler.

  This could be called from either from a spider or a downloader middleware (e.g. redirect, retry, ...).

  ---

  Parameters

  • optionalkeyword-onlyrequest: Request

    The request to add to the scheduler.

  Returns bool

has_pending_requests

• has_pending_requests(): bool

• Check if the scheduler has any pending requests.

  ---

  Returns bool

next_request

• next_request(): Request | None

• Fetch the next request from the scheduler.

  ---

  Returns Request | None

open

• open(*, spider): None

• Open the scheduler.

  ---

  Parameters

  • optionalkeyword-onlyspider: Spider

    The spider that the scheduler is associated with.

  Returns None