ApifyScheduler

apify.scrapy.scheduler.ApifyScheduler

A Scrapy scheduler that uses the Apify Request Queue to manage requests.

This scheduler requires the asyncio Twisted reactor to be installed.

Index

Constructors

• __init__

Methods

• enqueue_request
• has_pending_requests
• next_request
• open

Constructors

__init__

• __init__(): None

• Create a new instance.

  ---

  Returns None

Methods

enqueue_request

• enqueue_request(request): bool

• Add a request to the scheduler.

  This could be called from either from a spider or a downloader middleware (e.g. redirect, retry, …).

  ---

  Parameters

  • request: Request

  Returns bool

has_pending_requests

• has_pending_requests(): bool

• Check if the scheduler has any pending requests.

  ---

  Returns bool

next_request

• next_request(): Request | None

• Fetch the next request from the scheduler.

  ---

  Returns Request | None

open

• open(spider): None

• Open the scheduler.

  ---

  Parameters

  • spider: Spider

  Returns None