Skip to main content

Examples

Connect to datacenter proxies

Learn how to connect to Apify's datacenter proxies from your application with Node.js (axios and got-scraping), Python 2 and 3 and PHP using code examples.

This page contains code examples for connecting to datacenter proxies using Apify Proxy.

See the connection settings page for connection parameters.

Using the Apify SDK and Crawlee​

If you are developing your own Apify actor using the Apify SDK and Crawlee, you can use Apify Proxy in:

The Apify SDK's ProxyConfiguration enables you to choose which proxies you use for all connections. You can inspect the current proxy's URL and other attributes using the proxyInfo property of crawling context of your crawler's requestHandler.

Rotate IP addresses​

IP addresses for each request are selected at random from all available proxy servers.

import { Actor } from 'apify';
import { PuppeteerCrawler } from 'crawlee';

await Actor.init();

const proxyConfiguration = await Actor.createProxyConfiguration();

const crawler = new PuppeteerCrawler({
    proxyConfiguration,
    async requestHandler({ page }) {
        console.log(await page.content())
    },
});

await crawler.run(['https://proxy.apify.com/?format=json']);

await Actor.exit();

Single IP address for multiple requests​

Use a single IP address until it fails (gets retired).

The maxPoolSize: 1 specified in sessionPoolOptions of PuppeteerCrawler (works the same with other crawler classes) means that a single IP will be used by all browsers until it fails. Then, all running browsers are retired, a new IP is selected and new browsers opened. The browsers all use the new IP.

import { Actor } from 'apify';
import { PuppeteerCrawler } from 'crawlee';

await Actor.init();

const proxyConfiguration = await Actor.createProxyConfiguration();

const crawler = new PuppeteerCrawler({
    proxyConfiguration,
    sessionPoolOptions: { maxPoolSize: 1 },
    async requestHandler({ page}) {
        console.log(await page.content());
    },
});

await crawler.run([
    'https://proxy.apify.com/?format=json',
    'https://proxy.apify.com',
]);

await Actor.exit();

How to use proxy groups​

For simplicity, the examples above use the automatic proxy configuration (no specific proxy groups are specified), which selects IP addresses from all available groups.

To use IP addresses from specific proxy groups, add the groups property to Actor.createProxyConfiguration() and specify the group names. For example:

import { Actor } from 'apify';

await Actor.init();
// ...
const proxyConfiguration = await Actor.createProxyConfiguration({
    groups: ['GROUP_NAME_1', 'GROUP_NAME_2'],
});
// ...
await Actor.exit();

Using standard libraries and languages​

You can find your proxy password on the Proxy page of the Apify Console.

The username field is not your Apify username.
Instead, you specify proxy settings (e.g. groups-BUYPROXIES94952, session-123).
Use auto for default settings.

For examples using PHP, you need to have the cURL extension enabled in your PHP installation. See installation instructions for more information.

Examples in Python 2 use the six library. Run pip install six to enable it.

Use IP rotation​

For each request, a random IP address is chosen from all available proxy groups. You can use random IP addresses from proxy groups by specifying the group(s) in the username parameter.

A random IP address will be used for each request.

import axios from 'axios';

const proxy = {
    protocol: 'http',
    host: 'proxy.apify.com',
    port: 8000,
    // Replace <YOUR_PROXY_PASSWORD> below with your password
    // found at https://console.apify.com/proxy
    auth: { username: 'auto', password: <YOUR_PROXY_PASSWORD> },
};

const url = 'http://proxy.apify.com/?format=json';

const { data } = await axios.get(url, { proxy });

console.log(data);

Multiple requests with the same IP address​

The IP address in the example is chosen at random from all available proxy groups.

To use this option, set a session name in the username parameter.

import axios from 'axios';
import { HttpsProxyAgent } from 'hpagent';

const httpsAgent = new HttpsProxyAgent({
    // Replace <YOUR_PROXY_PASSWORD> below with your password
    // found at https://console.apify.com/proxy
    proxy: 'http://session-my_session:<YOUR_PROXY_PASSWORD>@proxy.apify.com:8000',
});
const axiosWithProxy = axios.create({ httpsAgent });

const url = 'https://api.apify.com/v2/browser-info';

const response1 = await axiosWithProxy.get(url);
const response2 = await axiosWithProxy.get(url);
// Should return the same clientIp for both requests
console.log('clientIp1:', response1.data.clientIp);
console.log('clientIp2:', response2.data.clientIp);

Username examples​

Use randomly allocated IP addresses from the BUYPROXIES94952 group:

groups-BUYPROXIES94952

Use a randomly allocated IP address for multiple requests:

session-new_job_123

Use the same IP address from the SHADER and BUYPROXIES94952 groups for multiple requests:

groups-SHADER+BUYPROXIES94952,session-new_job_123

Set a session and select an IP from the BUYPROXIES94952 group located in the USA:

groups-BUYPROXIES94952,session-new_job_123,country-US