Skip to main content
Version: 0.2

KeyValueStore

{"content": ["The KeyValueStore class represents a key-value store.\n\nYou can imagine it as a simple data storage that is used\nfor saving and reading data records or files. Each data record is\nrepresented by a unique key and associated with a MIME content type.\n\nDo not instantiate this class directly, use the Actor.open_key_value_store() function instead.\n\nEach crawler run is associated with a default key-value store, which is created exclusively\nfor the run. By convention, the crawler input and output are stored into the\ndefault key-value store under the INPUT and OUTPUT key, respectively.\nTypically, input and output are JSON files, although it can be any other format.\nTo access the default key-value store directly, you can use the\nKeyValueStore.get_value and KeyValueStore.set_value convenience functions.\n\nKeyValueStore stores its data either on local disk or in the Apify cloud,\ndepending on whether the APIFY_LOCAL_STORAGE_DIR or APIFY_TOKEN environment variables are set.\n\nIf the APIFY_LOCAL_STORAGE_DIR environment variable is set, the data is stored in\nthe local directory in the following files:\n``\n{APIFY_LOCAL_STORAGE_DIR}/key_value_stores/{STORE_ID}/{INDEX}.{EXT}\n``\nNote that {STORE_ID} is the name or ID of the key-value store. The default key-value store has ID: default,\nunless you override it by setting the APIFY_DEFAULT_KEY_VALUE_STORE_ID environment variable.\nThe {KEY} is the key of the record and {EXT} corresponds to the MIME content type of the data value.\n\nIf the APIFY_TOKEN environment variable is set but APIFY_LOCAL_STORAGE_DIR is not, the data is stored in the\nApify Key-value store cloud storage."]}

Index

Constructors

Methods

Constructors

__init__

  • __init__(id, name, client, config): None

  • {"content": ["Create a KeyValueStore instance.\n\nDo not use the constructor directly, use the Actor.open_key_value_store() function instead.\n\nArgs:\n id (str): ID of the key-value store.\n name (str, optional): Name of the key-value store.\n client (ApifyClientAsync or MemoryStorageClient): The storage client which should be used.\n config (Configuration): The configuration which should be used."]}

    Parameters

    • id: str
    • name: Optional[str]
    • client: Union[ApifyClientAsync, MemoryStorageClient]
    • config: Configuration

    Returns None

Methods

get_value

  • async get_value(key): Any

  • Parameters

    • key: str

    Returns Any

get_value

  • async get_value(key, default_value): T

  • Parameters

    • key: str
    • default_value: T

    Returns T

get_value

  • async get_value(key, default_value): Optional[T]

  • Parameters

    • key: str
    • default_value: Optional[T] = None

    Returns Optional[T]

get_value

  • async get_value(key, default_value): Optional[T]

  • {"content": ["Get a value from the key-value store.\n\nArgs:\n key (str): Key of the record to retrieve.\n default_value (Any, optional): Default value returned in case the record does not exist.\n\nReturns:\n Any: The value associated with the given key. default_value is used in case the record does not exist."]}

    Parameters

    • key: str
    • default_value: Optional[T] = None

    Returns Optional[T]

iterate_keys

  • async iterate_keys(exclusive_start_key): AsyncIterator[IterateKeysTuple]

  • {"content": ["Iterate over the keys in the key-value store.\n\nArgs:\n exclusive_start_key (str, optional): All keys up to this one (including) are skipped from the result.\n\nYields:\n IterateKeysTuple: A tuple (key, info),\n where key is the record key, and info is an object that contains a single property size\n indicating size of the record in bytes."]}

    Parameters

    • exclusive_start_key: Optional[str] = None

    Returns AsyncIterator[IterateKeysTuple]

set_value

  • async set_value(key, value, content_type): None

  • {"content": ["Set or delete a value in the key-value store.\n\nArgs:\n key (str): The key under which the value should be saved.\n value (Any, optional): The value to save. If the value is None, the corresponding key-value pair will be deleted.\n content_type (str, optional): The content type of the saved value."]}

    Parameters

    • key: str
    • value: Optional[T]
    • content_type: Optional[str] = None

    Returns None

get_public_url

  • async get_public_url(key): str

  • {"content": ["Get a URL for the given key that may be used to publicly access the value in the remote key-value store.\n\nArgs:\n key (str): The key for which the URL should be generated."]}

    Parameters

    • key: str

    Returns str

drop

  • async drop(): None

  • {"content": ["Remove the key-value store either from the Apify cloud storage or from the local directory."]}

    Returns None

open

  • async open(*, id, name, force_cloud, config): 'KeyValueStore'

  • {"content": ["Open a key-value store.\n\nKey-value stores are used to store records or files, along with their MIME content type.\nThe records are stored and retrieved using a unique key.\nThe actual data is stored either on a local filesystem or in the Apify cloud.\n\nArgs:\n id (str, optional): ID of the key-value store to be opened.\n If neither id nor name are provided, the method returns the default key-value store associated with the actor run.\n If the key-value store with the given ID does not exist, it raises an error.\n name (str, optional): Name of the key-value store to be opened.\n If neither id nor name are provided, the method returns the default key-value store associated with the actor run.\n If the key-value store with the given name does not exist, it is created.\n force_cloud (bool, optional): If set to True, it will open a key-value store on the Apify Platform even when running the actor locally.\n Defaults to False.\n config (Configuration, optional): A Configuration instance, uses global configuration if omitted.\n\nReturns:\n KeyValueStore: An instance of the KeyValueStore class for the given ID or name."]}

    Parameters

    • keyword-onlyid: Optional[str] = None
    • keyword-onlyname: Optional[str] = None
    • keyword-onlyforce_cloud: bool = False
    • keyword-onlyconfig: Optional[Configuration] = None

    Returns 'KeyValueStore'

Page Options