Skip to main content

Submitting forms on .ASPX pages

Apify users sometimes need to submit a form on pages created with ASP.NET (URL typically ends with .aspx). These pages have a different approach for how they submit forms and navigate through pages.

This tutorial shows you how to handle these kinds of pages. This approach is based on a blog post from Todd Hayton, where he explains how crawlers for ASP.NET pages should work.

First of all, you need to copy&paste this function to your Web Scraper Page function:

const enqueueAspxForm = async function (request, formSelector, submitButtonSelector, async) {
request.payload = $(formSelector).serialize();
if ($(submitButtonSelector).length) {
request.payload += decodeURIComponent(`&${$(submitButtonSelector).attr('name')}=${$(submitButtonSelector).attr('value')}`);
request.payload += decodeURIComponent(`&__ASYNCPOST=${async.toString()}`);
request.method = 'POST';
request.uniqueKey = Math.random();
await context.enqueueRequest(request);
return request;

The function has these parameters:

request - the object that describes the next request

formSelector - selector for a form to be submitted e.g 'form[name="test"]'

submitButtonSelector - selector for a button for submit form e.g. '#nextPageButton'

async - if true, request returns only params, not HTML content

Then you can use it in your Page function as follows:

await enqueueAspxForm({
url: '',
userData: { label: 'SEARCH-RESULT' },
}, 'form[name="aspnetForm"]', '#ctl00_ContentPlaceHolder1_btnSearch', false);