Skip to main content
Version: 1.7

apify-sdk-python

Index

Methods

Other

Constants

Properties

Methods

apply_apify_settings

  • apply_apify_settings(*, settings, proxy_config): Settings

  • {"content": ["Integrates Apify configuration into a Scrapy project settings.\n\nNote: The function directly modifies the passed settings object and also returns it.\n", {"

    Parameters

    • keyword-onlysettings: Settings | None = None
    • keyword-onlyproxy_config: dict | None = None

    Returns Settings

budget_ow

  • budget_ow(value, predicate, value_name): None

  • {"content": ["Budget version of ow."]}

    Parameters

    • value: dict | str | float | bool
    • predicate: dict[str, tuple[type, bool]] | tuple[type, bool]
    • value_name: str | None = None

    Returns None

compute_short_hash

  • compute_short_hash(data, *, length): str

  • {"content": ["Computes a hexadecimal SHA-256 hash of the provided data and returns a substring (prefix) of it.\n", {"

    Parameters

    • data: bytes
    • keyword-onlylength: int = 8

    Returns str

compute_unique_key

  • compute_unique_key(url, method, payload, *, keep_url_fragment, use_extended_unique_key): str

  • {"content": ["Computes a unique key for caching & deduplication of requests.\n\nThis function computes a unique key by normalizing the provided URL and method.\nIf 'use_extended_unique_key' is True and a payload is provided, the payload is hashed and\nincluded in the key. Otherwise, the unique key is just the normalized URL.\n", {"

    Parameters

    • url: str
    • method: str = 'GET'
    • payload: bytes | None = None
    • keyword-onlykeep_url_fragment: bool = False
    • keyword-onlyuse_extended_unique_key: bool = False

    Returns str

crypto_random_object_id

  • crypto_random_object_id(length): str

  • {"content": ["Python reimplementation of cryptoRandomObjectId from @apify/utilities."]}

    Parameters

    • length: int = 17

    Returns str

decrypt_input_secrets

  • decrypt_input_secrets(private_key, input): Any

  • {"content": ["Decrypt input secrets."]}

    Parameters

    • private_key: rsa.RSAPrivateKey
    • input: Any

    Returns Any

force_remove

  • async force_remove(filename): None

  • {"content": ["JS-like rm(filename, { force: true })."]}

    Parameters

    • filename: str

    Returns None

force_rename

  • async force_rename(src_dir, dst_dir): None

  • {"content": ["Rename a directory. Checks for existence of source directory and removes destination directory if it exists."]}

    Parameters

    • src_dir: str
    • dst_dir: str

    Returns None

get_basic_auth_header

  • get_basic_auth_header(username, password, auth_encoding): bytes

  • {"content": ["Generate a basic authentication header for the given username and password."]}

    Parameters

    • username: str
    • password: str
    • auth_encoding: str = 'latin-1'

    Returns bytes

get_running_event_loop_id

  • get_running_event_loop_id(): int

  • {"content": ["Get the ID of the currently running event loop.\n\nIt could be useful mainly for debugging purposes.\n", {"

    Returns int

guess_file_extension

  • guess_file_extension(content_type): str | None

  • {"content": ["Guess the file extension based on content type."]}

    Parameters

    • content_type: str

    Returns str | None

is_url

  • is_url(url): bool

  • {"content": ["Check if the given string is a valid URL."]}

    Parameters

    • url: str

    Returns bool

normalize_url

  • normalize_url(url, *, keep_url_fragment): str

  • {"content": ["Normalizes a URL.\n\nThis function cleans and standardizes a URL by removing leading and trailing whitespaces,\nconverting the scheme and netloc to lower case, stripping unwanted tracking parameters\n(specifically those beginning with 'utm_'), sorting the remaining query parameters alphabetically,\nand optionally retaining the URL fragment. The goal is to ensure that URLs that are functionally\nidentical but differ in trivial ways (such as parameter order or casing) are treated as the same.\n", {"

    Parameters

    • url: str
    • keyword-onlykeep_url_fragment: bool = False

    Returns str

open_queue_with_custom_client

  • {"content": ["Open a Request Queue with custom Apify Client.\n\nTODO: add support for custom client to Actor.open_request_queue(), so that\nwe don't have to do this hacky workaround"]}

    Returns RequestQueue

to_apify_request

  • to_apify_request(scrapy_request, spider): dict | None

  • {"content": ["Convert a Scrapy request to an Apify request.\n", {"

    Parameters

    • scrapy_request: Request
    • spider: Spider

    Returns dict | None

to_scrapy_request

  • to_scrapy_request(apify_request, spider): Request

  • {"content": ["Convert an Apify request to a Scrapy request.\n", {"

    Parameters

    • apify_request: dict
    • spider: Spider

    Returns Request

unique_key_to_request_id

  • unique_key_to_request_id(unique_key): str

  • {"content": ["Generate request ID based on unique key in a deterministic way."]}

    Parameters

    • unique_key: str

    Returns str

Properties

ResourceClientType

ResourceClientType:

{"content": ["noqa: PLC0105"]}

Page Options