Skip to main content

SmartApifyStorageClient

SmartApifyStorageClient that delegates to cloud_storage_client or local_storage_client.

When running on Apify platform use cloud_storage_client, else use local_storage_client. This storage client is designed to work specifically in Actor context.

Index

Methods

Methods

__init__

  • __init__(*, cloud_storage_client, local_storage_client): None

  • Initialize the Apify storage client.

    Parameters

    • optionalkeyword-onlycloud_storage_client: ApifyStorageClient | None = None

      Client used to communicate with the Apify platform storage. Either through force_cloud argument when opening storages or automatically when running on the Apify platform.

    • optionalkeyword-onlylocal_storage_client: StorageClient | None = None

      Client used to communicate with the storage when not running on the Apify platform and not using force_cloud argument when opening storages.

    Returns None

__str__

  • __str__(): str

  • Returns str

create_dataset_client

  • async create_dataset_client(*, id, name, alias, configuration): DatasetClient

  • Parameters

    • optionalkeyword-onlyid: str | None = None
    • optionalkeyword-onlyname: str | None = None
    • optionalkeyword-onlyalias: str | None = None
    • optionalkeyword-onlyconfiguration: CrawleeConfiguration | None = None

    Returns DatasetClient

create_kvs_client

  • async create_kvs_client(*, id, name, alias, configuration): KeyValueStoreClient

  • Parameters

    • optionalkeyword-onlyid: str | None = None
    • optionalkeyword-onlyname: str | None = None
    • optionalkeyword-onlyalias: str | None = None
    • optionalkeyword-onlyconfiguration: CrawleeConfiguration | None = None

    Returns KeyValueStoreClient

create_rq_client

  • async create_rq_client(*, id, name, alias, configuration): RequestQueueClient

  • Parameters

    • optionalkeyword-onlyid: str | None = None
    • optionalkeyword-onlyname: str | None = None
    • optionalkeyword-onlyalias: str | None = None
    • optionalkeyword-onlyconfiguration: CrawleeConfiguration | None = None

    Returns RequestQueueClient

get_storage_client_cache_key

  • get_storage_client_cache_key(configuration): Hashable

  • Parameters

    • configuration: CrawleeConfiguration

    Returns Hashable

get_suitable_storage_client

  • get_suitable_storage_client(*, force_cloud): StorageClient

  • Get a suitable storage client based on the global configuration and the value of the force_cloud flag.

    Parameters

    • optionalkeyword-onlyforce_cloud: bool = False

      If True, return cloud_storage_client.

    Returns StorageClient

Page Options