ApifyFileSystemStorageOptions

Hierarchy

• FileSystemStorageOptions
  • ApifyFileSystemStorageOptions

Index

Properties

• inputKey
• localDataDirectory
• requestQueueAccess

Properties

optionalinputKey

inputKey?: string = ‘INPUT’

The key the Actor input is read from — Configuration.inputKey (ACTOR_INPUT_KEY).

INPUT is treated as an input key regardless of this setting, so a project's INPUT.json stays intact when the Apify CLI redirects the run to a temporary key.

externalinheritedlocalDataDirectory

localDataDirectory: string

Inherited from FileSystemStorageOptions.localDataDirectory

Path to directory where the data will be saved.

externaloptionalinheritedrequestQueueAccess

requestQueueAccess?: single | shared = single | shared

Inherited from FileSystemStorageOptions.requestQueueAccess

How the on-disk request queues opened by this backend are expected to be accessed.

With 'single' (the default), this process asserts it is the sole consumer of every request queue it opens: on open, any requests that a previous run left in progress (e.g. after a crash) are reclaimed immediately, so they become fetchable again right away. This is the right behavior for the common single-process crawl.

Use 'shared' if multiple processes share the same on-disk request queue concurrently (for example, the "Parallel Scraping Guide"). In that mode an in-progress request is treated as a potential live peer's lock and is only reclaimed once that lock expires on the wall clock, so two workers won't process the same request at once.