KeyValueStore
Index
Methods
drop
Remove the key-value store either from the Apify cloud storage or from the local directory.
Returns None
get_public_url
Get a URL for the given key that may be used to publicly access the value in the remote key-value store.
Parameters
key: str
The key for which the URL should be generated.
Returns str
get_value
Get a value from the key-value store.
Parameters
key: str
Key of the record to retrieve.
optionaldefault_value: T
Default value returned in case the record does not exist.
Returns T
iterate_keys
Iterate over the keys in the key-value store.
Parameters
optionalexclusive_start_key: str
All keys up to this one (including) are skipped from the result.
Yields: IterateKeysTuple: A tuple
(key, info)
, wherekey
is the record key, andinfo
is an object that contains a single propertysize
indicating size of the record in bytes.
Returns AsyncIterator[IterateKeysTuple]
open
Open a key-value store.
Key-value stores are used to store records or files, along with their MIME content type. The records are stored and retrieved using a unique key. The actual data is stored either on a local filesystem or in the Apify cloud.
Parameters
optionalid: str
ID of the key-value store to be opened. If neither
id
norname
are provided, the method returns the default key-value store associated with the actor run. If the key-value store with the given ID does not exist, it raises an error.optionalname: str
Name of the key-value store to be opened. If neither
id
norname
are provided, the method returns the default key-value store associated with the actor run. If the key-value store with the given name does not exist, it is created.force_cloud: bool
If set to True, it will open a key-value store on the Apify Platform even when running the actor locally. Defaults to False.
optionalconfig: Configuration
A
Configuration
instance, uses global configuration if omitted.
Returns 'KeyValueStore'
set_value
Set or delete a value in the key-value store.
Parameters
key: str
The key under which the value should be saved.
optionalvalue: T
The value to save. If the value is
None
, the corresponding key-value pair will be deleted.optionalcontent_type: str
The content type of the saved value.
Returns None
The
KeyValueStore
class represents a key-value store.You can imagine it as a simple data storage that is used for saving and reading data records or files. Each data record is represented by a unique key and associated with a MIME content type.
Do not instantiate this class directly, use the
Actor.open_key_value_store()
function instead.Each crawler run is associated with a default key-value store, which is created exclusively for the run. By convention, the crawler input and output are stored into the default key-value store under the
INPUT
andOUTPUT
key, respectively. Typically, input and output are JSON files, although it can be any other format. To access the default key-value store directly, you can use theKeyValueStore.get_value
andKeyValueStore.set_value
convenience functions.KeyValueStore
stores its data either on local disk or in the Apify cloud, depending on whether theAPIFY_LOCAL_STORAGE_DIR
orAPIFY_TOKEN
environment variables are set.If the
APIFY_LOCAL_STORAGE_DIR
environment variable is set, the data is stored in the local directory in the following files:Note that
{STORE_ID}
is the name or ID of the key-value store. The default key-value store has ID:default
, unless you override it by setting theAPIFY_DEFAULT_KEY_VALUE_STORE_ID
environment variable. The{KEY}
is the key of the record and{EXT}
corresponds to the MIME content type of the data value.If the
APIFY_TOKEN
environment variable is set butAPIFY_LOCAL_STORAGE_DIR
is not, the data is stored in the Apify Key-value store cloud storage.