ApifyRequestList

Extends crawlee RequestList.

Method open is used to create RequestList from actor's requestListSources input.

Index

Methods

• open

Methods

open

• async open(name, request_list_sources_input, http_client): ApifyRequestList

• Initialize a new instance from request list source input.

  Usage

  example_input = [
      # Gather urls from response body.
      {'requestsFromUrl': 'https://crawlee.dev/file.txt', 'method': 'GET'},
      # Directly include this url.
      {'url': 'https://crawlee.dev', 'method': 'GET'}
  ]
  request_list = await RequestList.open(request_list_sources_input=example_input)

  ---

  Parameters

  • optionalname: str | None = None

    Name of the returned RequestList.

  • optionalrequest_list_sources_input: list[dict[str, Any]] | None = None

    List of dicts with either url key or requestsFromUrl key.

  • optionalhttp_client: HttpClient | None = None

    Client that will be used to send get request to urls defined by value of requestsFromUrl keys.

  Returns ApifyRequestList

  RequestList created from request_list_sources_input.