ApifyHttpProxyMiddleware

{"content": ["Apify HTTP proxy middleware for Scrapy.\n\nThis middleware enhances request processing by adding a 'proxy' field to the request's meta and an authentication\nheader. It draws inspiration from the HttpProxyMiddleware included by default in Scrapy projects. The proxy URL\nis sourced from the settings under the APIFY_PROXY_SETTINGS key. The value of this key, a dictionary, should be\nprovided by the Actor input. An example of the proxy settings:\n\nproxy_settings = {'useApifyProxy': true, 'apifyProxyGroups': []}"]}

Index

Constructors

• __init__

Methods

• from_crawler
• process_exception
• process_request

Constructors

__init__

• __init__(proxy_settings): None

• {"content": ["Create a new instance.\n", {"

  ---

  Parameters

  • proxy_settings: dict

  Returns None

Methods

from_crawler

• from_crawler(crawler): ApifyHttpProxyMiddleware

• {"content": ["Create an instance of ApifyHttpProxyMiddleware from a Scrapy Crawler.\n", {"

  ---

  Parameters

  • crawler: Crawler

  Returns ApifyHttpProxyMiddleware

process_exception

• process_exception(request, exception, spider): None

• {"content": ["Process an exception that occurs during request processing.\n", {"

  ---

  Parameters

  • request: Request

  • exception: Exception

  • spider: Spider

  Returns None

process_request

• async process_request(request, spider): None

• {"content": ["Process a Scrapy request by assigning a new proxy.\n", {"

  ---

  Parameters

  • request: Request

  • spider: Spider

  Returns None