Skip to main content
Version: Next

ConfigurationOptions

Hierarchy

  • ConfigurationOptions
    • ConfigurationOptions

Index

Properties

Properties

optionalactorEventsWsUrl

actorEventsWsUrl?: string

optionalactorId

actorId?: string

optionalactorRunId

actorRunId?: string

optionalactorTaskId

actorTaskId?: string

optionalapiBaseUrl

apiBaseUrl?: string

externaloptionalinheritedavailableMemoryRatio

availableMemoryRatio?: number = 0.25

Sets the ratio, defining the amount of system memory that could be used by the AutoscaledPool. When the memory usage is more than the provided ratio, the memory is considered overloaded.

Alternative to CRAWLEE_AVAILABLE_MEMORY_RATIO environment variable.

externaloptionalinheritedchromeExecutablePath

chromeExecutablePath?: string

Defines a path to Chrome executable.

Alternative to CRAWLEE_CHROME_EXECUTABLE_PATH environment variable.

optionalcontainerPort

containerPort?: number

optionalcontainerUrl

containerUrl?: string

externaloptionalinheriteddefaultBrowserPath

defaultBrowserPath?: string

Defines a path to default browser executable.

Alternative to CRAWLEE_DEFAULT_BROWSER_PATH environment variable.

externaloptionalinheriteddefaultDatasetId

defaultDatasetId?: string = ‘default’

Default dataset id.

Alternative to CRAWLEE_DEFAULT_DATASET_ID environment variable.

externaloptionalinheriteddefaultKeyValueStoreId

defaultKeyValueStoreId?: string = ‘default’

Default key-value store id.

Alternative to CRAWLEE_DEFAULT_KEY_VALUE_STORE_ID environment variable.

externaloptionalinheriteddefaultRequestQueueId

defaultRequestQueueId?: string = ‘default’

Default request queue id.

Alternative to CRAWLEE_DEFAULT_REQUEST_QUEUE_ID environment variable.

externaloptionalinheriteddisableBrowserSandbox

disableBrowserSandbox?: boolean

Defines whether to disable browser sandbox by adding --no-sandbox flag to launchOptions.

Alternative to CRAWLEE_DISABLE_BROWSER_SANDBOX environment variable.

externaloptionalinheritedeventManager

eventManager?: EventManager = EventManager

Defines the Event Manager to be used.

externaloptionalinheritedheadless

headless?: boolean = true

Defines whether web browsers launched by Crawlee will run in the headless mode.

Alternative to CRAWLEE_HEADLESS environment variable.

externaloptionalinheritedinputKey

inputKey?: string = ‘INPUT’

Defines the default input key, i.e. the key that is used to get the crawler input value from the default KeyValueStore associated with the current crawler run.

Alternative to CRAWLEE_INPUT_KEY environment variable.

optionalinputSecretsPrivateKeyFile

inputSecretsPrivateKeyFile?: string

optionalinputSecretsPrivateKeyPassphrase

inputSecretsPrivateKeyPassphrase?: string

optionalisAtHome

isAtHome?: boolean

externaloptionalinheritedlogLevel

logLevel?: LogLevel | (radix?: number) => string | (fractionDigits?: number) => string | (fractionDigits?: number) => string | (precision?: number) => string | () => number | ({ (locales?: string | string[], options?: NumberFormatOptions): string; (locales?: LocalesArgument, options?: NumberFormatOptions): string }) = LogLevel | (radix?: number) => string | (fractionDigits?: number) => string | (fractionDigits?: number) => string | (precision?: number) => string | () => number | ({ (locales?: string | string[], options?: NumberFormatOptions): string; (locales?: LocalesArgument, options?: NumberFormatOptions): string })

Sets the log level to the given value.

Alternative to CRAWLEE_LOG_LEVEL environment variable.

externaloptionalinheritedmaxUsedCpuRatio

maxUsedCpuRatio?: number = 0.95

Sets the ratio, defining the maximum CPU usage. When the CPU usage is higher than the provided ratio, the CPU is considered overloaded.

externaloptionalinheritedmemoryMbytes

memoryMbytes?: number

Sets the amount of system memory in megabytes to be used by the AutoscaledPool. By default, the maximum memory is set to one quarter of total system memory.

Alternative to CRAWLEE_MEMORY_MBYTES environment variable.

optionalmetamorphAfterSleepMillis

metamorphAfterSleepMillis?: number

optionalmetaOrigin

metaOrigin?: DEVELOPMENT | WEB | API | SCHEDULER | TEST | WEBHOOK | ACTOR | CLI | STANDBY

externaloptionalinheritedpersistStateIntervalMillis

persistStateIntervalMillis?: number = 60_000

Defines the interval of emitting the persistState event.

Alternative to CRAWLEE_PERSIST_STATE_INTERVAL_MILLIS environment variable.

externaloptionalinheritedpersistStorage

persistStorage?: boolean

Defines whether the storage client used should persist the data it stores.

Alternative to CRAWLEE_PERSIST_STORAGE environment variable.

optionalproxyHostname

proxyHostname?: string

optionalproxyPassword

proxyPassword?: string

optionalproxyPort

proxyPort?: number

optionalproxyStatusUrl

proxyStatusUrl?: string

externaloptionalinheritedpurgeOnStart

purgeOnStart?: boolean = true

Defines whether to purge the default storage folders before starting the crawler run.

Alternative to CRAWLEE_PURGE_ON_START environment variable.

optionalstandbyPort

standbyPort?: number
@deprecated

use containerPort instead

optionalstandbyUrl

standbyUrl?: string

externaloptionalinheritedstorageClient

storageClient?: StorageClient = StorageClient

Defines storage client to be used.

externaloptionalinheritedstorageClientOptions

storageClientOptions?: Dictionary

Could be used to adjust the storage client behavior e.g. MemoryStorageOptions could be used to adjust the MemoryStorage behavior.

externaloptionalinheritedsystemInfoIntervalMillis

systemInfoIntervalMillis?: number = 1_000

Defines the interval of emitting the systemInfo event.

optionaltoken

token?: string

optionaluserId

userId?: string

externaloptionalinheritedxvfb

xvfb?: boolean = false

Defines whether to run X virtual framebuffer on the web browsers launched by Crawlee.

Alternative to CRAWLEE_XVFB environment variable.

Page Options