Edit

Connect to Google SERP proxies

Learn how to connect to Google SERP proxies from your applications with Node.js (axios and got), Python 2 and 3 and PHP using code examples.

This page contains code examples for connecting to Google SERP proxies using Apify Proxy.

See the connection settings page for connection parameters.

Using the Apify SDK

If you are developing your own Apify actor using the Apify SDK, the most efficient way to use Google SERP proxy is CheerioCrawler. This is because Google SERP proxy only returns a page's HTML. Alternatively, you can use the requestAsBrowser() function by specifying proxy configuration in the options.

Apify Proxy also works with PuppeteerCrawler and launchPuppeteer(). However, it is not efficient to load a full browser just to retrieve HTML.

Get a list of search results

Get a list of search results for the keyword wikipedia from the USA (google.com).

const Apify = require('apify');

Apify.main(async() => {
    const proxyConfiguration = await Apify.createProxyConfiguration({
        groups: ['GOOGLE_SERP'],
    });
    const url = 'http://www.google.co.uk/search?q=wikipedia';

    const requestList = await Apify.openRequestList('my-list', [url]);

    const crawler = new Apify.CheerioCrawler({
        requestList,
        proxyConfiguration,
        handlePageFunction: async ({ request, response, body }) => {
            // ...
            console.log(body)
        },
    });

    await crawler.run();
});

Get a list of shopping results

Get a list of shopping results for the query Apple iPhone XS 64GB from Great Britain (google.co.uk).

const Apify = require('apify');

Apify.main(async() => {
    const proxyConfiguration = await Apify.createProxyConfiguration({
        groups: ['GOOGLE_SERP'],
    });
    const query = encodeURI('Apple iPhone XS 64GB');
    const url = `http://www.google.co.uk/search?q=${query}&tbm=shop`;

    const requestList = await Apify.openRequestList('my-list', [url]);

    const crawler = new Apify.CheerioCrawler({
        requestList,
        proxyConfiguration,
        handlePageFunction: async ({ request, response, body }) => {
            // ...
            console.log(body)
        },
    });

    await crawler.run();
});

Using standard libraries and languages

You can find your proxy password on the Proxy page of the Apify app.

The username field is not your Apify username.
Instead, you specify proxy settings (e.g. groups-GOOGLE_SERP).
Use groups-GOOGLE_SERP to use proxies from all available countries.

For examples using PHP, you need to have the cURL extension enabled in your PHP installation. See installation instructions for more information.

Examples in Python 2 use the six library. Run pip install six to enable it.

HTML from search results

Get the HTML of search results for the keyword wikipedia from the USA (google.com).

Select this option by setting the username parameter to groups-GOOGLE_SERP. Add the item you want to search to the query parameter.

const HttpsProxyAgent = require("https-proxy-agent");
const axios = require("axios");

const httpsAgent = new HttpsProxyAgent({
    host: "proxy.apify.com",
    port: "8000",
    // Replace <YOUR_PROXY_PASSWORD> below with your password
    // found at https://my.apify.com/proxy
    auth: "groups-GOOGLE_SERP:<YOUR_PROXY_PASSWORD>"
});

const axiosWithProxy = axios.create({ httpsAgent });

async function useProxy() {
    const response = await axiosWithProxy.get(
      `http://www.google.com/search`,
      {
        params: {
          query: 'wikipedia',
        },
      }
    );
    console.log(response.data)
};

useProxy();

HTML from localized shopping results

Get HTML of shopping results for the query Apple iPhone XS 64GB from Great Britain (google.co.uk).

Select this option by setting the username parameter to groups-GOOGLE_SERP. In the query parameter, add the item you want to search and specify the shop page as a URL parameter.

Set the domain (your country of choice) in the URL (in the response variable).

const HttpsProxyAgent = require("https-proxy-agent");
const axios = require("axios");

const httpsAgent = new HttpsProxyAgent({
    host: "proxy.apify.com",
    port: "8000",
    // Replace <YOUR_PROXY_PASSWORD> below with your password
    // found at https://my.apify.com/proxy
    auth: "groups-GOOGLE_SERP:<YOUR_PROXY_PASSWORD>"
});

const axiosWithProxy = axios.create({ httpsAgent });

async function useProxy() {
    const response = await axiosWithProxy.get(
      `http://www.google.com/search`,
      {
        params: {
          query: 'Apple iPhone XS 64GB',
          tbm: 'shop',
        },
      }
    );
    console.log(response.data)
};

useProxy();