Retrieve Actor data

Copy for LLM

Actor output data is stored in datasets, which can be retrieved from individual Actor runs. Dataset items support pagination for efficient retrieval, and multiple datasets can be merged into a single dataset for further analysis. You can then export the merged dataset into formats such as CSV, JSON, XLSX, or XML. Integrations can automate such data workflows as well.

The following example shows how to fetch the datasets of an Actor's runs, read their items, and merge them into a single dataset:

import { ApifyClient } from 'apify-client';

// Client initialization with the API token.
const client = new ApifyClient({ token: 'MY-APIFY-TOKEN' });

const runsClient = client.actor('apify/instagram-hashtag-scraper').runs();

// List the last 20 runs. For how to get more, see the Pagination concept.
const actorRuns = await runsClient.list({ limit: 20 });

const mergingDataset = await client.datasets().getOrCreate('merge-dataset');

for (const run of actorRuns.items) {
    // Fetch the items of the run's default dataset. The items can be paginated.
    const { items } = await client.dataset(run.defaultDatasetId).listItems({ limit: 1000 });

    // Push the items to the merged dataset.
    await client.dataset(mergingDataset.id).pushItems(items);
}

Read items while the run is going

To process a run's output before the run finishes, use RunClient.iterateDatasetItems(). It yields the items of the run's default dataset shortly after the run pushes them, and it returns once the run has finished and every item is read.

Note that:

• Between polls, the iterator waits up to pollIntervalSecs (5 by default) for the run to finish. Once it finishes, the iterator reads the remaining items right away. An explicit timeoutSecs has to leave room for that wait.
• The item options are the same as in DatasetClient.listItems(), except for desc, signature, flatten and view.
• offset and limit count dataset rows, not the items you get back. With clean, skipEmpty or unwind, the iterator can yield fewer or more items than limit.
• A run that's ABORTING or TIMING-OUT can still push items, so the iterator keeps polling until the run reaches a terminal status.

The following example starts an Actor and prints its items as the run produces them:

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'MY-APIFY-TOKEN' });

// Start the Actor without waiting for it to finish.
const run = await client.actor('username/actor-name').start({ query: 'web scraping' });

// Each item arrives shortly after the run pushes it. The loop ends once the run
// has finished and every item is read.
for await (const item of client.run(run.id).iterateDatasetItems({ skipEmpty: true })) {
    console.log(item);
}