Skip to main content
Version: Next

DatasetClient <Data>

Client for managing a specific Dataset.

Datasets store structured data results from Actor runs. This client provides methods to push items, list and retrieve items, download items in various formats (JSON, CSV, Excel, etc.), and manage the dataset.

Hierarchy

  • ResourceClient
    • DatasetClient
const client = new ApifyClient({ token: 'my-token' });
const datasetClient = client.dataset('my-dataset-id');

// Push items to the dataset
await datasetClient.pushItems([
  { url: 'https://example.com', title: 'Example' },
  { url: 'https://test.com', title: 'Test' }
]);

// List all items
const { items } = await datasetClient.listItems();

// Download items as CSV
const buffer = await datasetClient.downloadItems('csv');

See more at https://docs.apify.com/platform/storage/dataset

Index

Properties

Methods

Properties

inheritedapifyClient

apifyClient: ApifyClient

inheritedbaseUrl

baseUrl: string

inheritedhttpClient

httpClient: HttpClient

optionalinheritedid

id?: string

optionalinheritedparams

params?: Record<string, unknown>

inheritedpublicBaseUrl

publicBaseUrl: string

Added in 2.17.0

inheritedresourcePath

resourcePath: string

optionalinheritedsafeId

safeId?: string

inheritedurl

url: string

Methods

createItemsPublicUrl

  • createItemsPublicUrl(options): Promise<string>

  • Generates a public URL for accessing dataset items.

    If the client has permission to access the dataset's URL signing key, the URL will include a cryptographic signature allowing access without authentication. This is useful for sharing dataset results with external services or users.

    Parameters

    Returns Promise<string>

    A public URL string for accessing the dataset items

    // Create a URL that expires in 1 hour with specific fields
    const url = await client.dataset('my-dataset').createItemsPublicUrl({
      expiresInSecs: 3600,
      fields: ['url', 'title'],
      limit: 100
    });
    console.log(`Share this URL: ${url}`);

    // Create a permanent public URL for clean items only
    const url = await client.dataset('my-dataset').createItemsPublicUrl({
      clean: true,
      skipEmpty: true
    });

    Added in 2.13.0

delete

  • delete(options): Promise<void>

downloadItems

  • downloadItems(format, options): Promise<Buffer<ArrayBufferLike>>
  • downloadItems(format, options): Promise<Readable>
  • downloadItems(format, options): Promise<Readable | Buffer<ArrayBufferLike>>

  • Downloads dataset items in a specific format.

    Unlike listItems which returns a PaginatedList with an array of individual dataset items, this method returns the items serialized to the provided format (JSON, CSV, Excel, etc.) as a Buffer. Useful for exporting data for further processing.

    In Node.js you can use the stream option to get a Readable stream, so a large export is processed in chunks without holding all of it in memory. The stream holds its connection open until you consume it to the end or destroy it. Iterating it with for await or passing it to pipeline() from node:stream/promises does either one for you.

    Parameters

    • format: json | jsonl | xml | html | csv | xlsx | rss

      Output format: 'json', 'jsonl', 'csv', 'xlsx', 'xml', 'rss', or 'html'

    • optionaloptions: DatasetClientDownloadItemsOptions & { stream?: false }

      Download and formatting options (extends all options from listItems)

    Returns Promise<Buffer<ArrayBufferLike>>

    Buffer containing the serialized data in the specified format, or a Readable stream of it with stream

    // Download as CSV with BOM for Excel compatibility
    const csvBuffer = await client.dataset('my-dataset').downloadItems('csv', { bom: true });
    require('fs').writeFileSync('output.csv', csvBuffer);

    // Download as Excel with custom options
    const xlsxBuffer = await client.dataset('my-dataset').downloadItems('xlsx', {
      fields: ['url', 'title', 'price'],
      skipEmpty: true,
      limit: 1000
    });

    // Download as XML with custom element names
    const xmlBuffer = await client.dataset('my-dataset').downloadItems('xml', {
      xmlRoot: 'products',
      xmlRow: 'product'
    });

    // Stream a CSV export straight to a file
    const { createWriteStream } = require('node:fs');
    const { pipeline } = require('node:stream/promises');
    const stream = await client.dataset('my-dataset').downloadItems('csv', { stream: true });
    await pipeline(stream, createWriteStream('output.csv'));

    See more at https://docs.apify.com/api/v2/dataset-items-get

get

getStatistics

listItems

  • listItems(options): PaginatedIterator<Data>

  • Lists items in the dataset.

    Returns a paginated list of dataset items. You can use pagination parameters to retrieve specific subsets of items, and various filtering and formatting options to customize the output.

    Parameters

    Returns PaginatedIterator<Data>

    A paginated list with items, total count, offset, count, and limit

    // Get first 100 items
    const { items, total } = await client.dataset('my-dataset').listItems({ limit: 100 });
    console.log(`Retrieved ${items.length} of ${total} total items`);

    // Get items with specific fields only
    const { items } = await client.dataset('my-dataset').listItems({
      fields: ['url', 'title'],
      skipEmpty: true,
      limit: 50
    });

    // Get items in descending order with pagination
    const { items } = await client.dataset('my-dataset').listItems({
      desc: true,
      offset: 100,
      limit: 50
    });

    See more at https://docs.apify.com/api/v2/dataset-items-get

pushItems

  • pushItems(items, options): Promise<void>

  • Stores one or more items into the dataset.

    Each item will be stored as a separate record in the dataset. Objects are automatically serialized to JSON. If you provide an array, all items will be stored in order. This method is idempotent - calling it multiple times with the same data will not create duplicates, but will append items each time.

    Parameters

    • items: string | Data | Data[]

      A single item, an array of items, or a string that is the JSON serialization of either - the API only accepts an object or an array of objects, so a plain string is not a valid item on its own.

    • options: TimeoutOptions = {}

      Request options

    Returns Promise<void>

    // Store a single object
    await client.dataset('my-dataset').pushItems({
      url: 'https://example.com',
      title: 'Example Page',
      extractedAt: new Date()
    });

    // Store multiple items at once
    await client.dataset('my-dataset').pushItems([
      { url: 'https://example.com', title: 'Example' },
      { url: 'https://test.com', title: 'Test' },
      { url: 'https://demo.com', title: 'Demo' }
    ]);

    See more at https://docs.apify.com/api/v2/dataset-items-post

update

Page Options