What is monitoring?
Learn how to check the performance of your actors, tasks and storages. Validate your data and receive notifications when your jobs succeed or fail.
Monitoring allows you to track and observe how software works. It enables you to measure and compare your programs' performance over time, and to be notified when something goes wrong.
You can use data you gain from monitoring to optimize your software and maximize its potential.
What can I gain from monitoring?
In essence, monitoring gives you more time. It allows you to automate processes that are complicated or tedious if you have to do them manually.
If you are scraping your competitor's website every day, you have already automated a part of the process. With monitoring, you can save even more time and ensure all your results are usable even before you need them.
And if the scraping job fails, you will receive a notification, so there will be no unpleasant surprises.
What is the Apify monitoring suite?
The monitoring suite is a collection of Apify actors that allows you to automate the monitoring of jobs you have running on the Apify platform.
You can use it to monitor anything from a single Actor to complex projects spanning multiple Actors, tasks or storages.
How does it work?
The monitoring suite is made up of a system of actors: checkers and reporters. Based on your configuration, the suite will create schedules and webhooks under your account and use those to run these actors.
As the names suggest, the checkers make sure your data is up to standard. Reporters create visualizations of how your actors perform over time and send notifications about failed runs.
How much does it cost?
You can use the monitoring suite free of charge, but under the hood, it is a system of actors that consume compute units (CUs). We have tried our best to make it as efficient as possible.
- Medium projects (single monitoring configuration with 5-20 daily running monitored tasks or actors) should consume around 10 CUs per month.
- Bigger projects (more monitoring configurations with more than 20 daily running tasks or actors) can exceed 50 CUs+ per month.
How can I set up monitoring?
We currently have video tutorials for setting up monitoring and adding data validation.
We also have written tutorials for the following use cases.
How do I turn it off or make changes?
To turn off a monitoring task, simply run it with the Mode (in the task's Input section) set to Delete configuration. It will clean up all its resources and turn off all monitoring activity. You will no longer receive notifications.
To make any changes to an existing configuration, choose the Update configuration mode in the task's Input, make your changes and run the monitoring suite task again.
Add the monitoring suite to your account
On the monitoring suite's page in Apify Store, click the Try for free button. This will take you to Apify Console. Add the suite to your list of Actors and create a new task.
Notifications
The suite will send notifications to the email address associated with your account by default.
You can disable email notifications or change the email address in a task Input's Email notifications section.
If you use Slack, we suggest using Slack notifications instead of email. More details and setup instructions here.
Example of a failed run
Below is an example of an email notification for a task run that did not return enough results.
To see what went wrong, click on the time next to Actor run to open your task's Runs dashboard. When you click on Details, you'll find a detailed monitoring report in JSON format.