Skip to main content

12.08 Using the Headless browser node

tip

You can watch a training video here.

Let's create a scenario where:

  • The input data consists of currency names for which the exchange rates need to be obtained;
  • The output data is the exchange rates of the specified currencies to, for instance, the British pound.

The data source for currency exchange rates is the BoE website. he web page presents a table with the rates of various currencies against the British pound.

For the successful execution of the scenario, three nodes need to be created:

  • (1) A Trigger on Webhook node with a URL to which a POST request is sent, initiating the execution of the scenario and transmitting the names of the required currencies;

Example 1:

{
  "C1": "Hong Kong Dollar",
  "C2": "Polish Zloty"
}

Example 2:

{
  "C1": "US",
  "C2": "Euro"
}
  • (2) A Headless browser node to process the web page and retrieve the necessary parameters with the code below. Choose the parameters from the previous Trigger on Webhook node as const currencyCode1 and const currencyCode2:
async function run({execution_id, input, data, page}) {
// Retrieve currency codes from the data passed by node 3
const currencyCode1 = data["{{1.\`body\`.\`C1\`}}"]; // Variable for the first currency code
const currencyCode2 = data["{{1.\`body\`.\`C2\`}}"]; // Variable for the second currency code

// Set User-Agent to mimic browser requests
await page.setUserAgent('Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36');

// Navigate to the Bank of England's currency rates page
await page.goto('https://www.bankofengland.co.uk/boeapps/database/Rates.asp?Travel=NIxIRx&into=GBP');

// Wait for the currency rates table to load
await page.waitForXPath('//*[@id="editorial"]/table');

// Find the element with the currency rate corresponding to the first code and get its value
const currency1Element = await page.waitForXPath(\`//td[a[contains(@title, "${currencyCode1}")]]/following-sibling::td[1]\`, {timeout: 30000});
var currency1 = await currency1Element.evaluate(el => el.textContent.trim());

// Find the element with the currency rate corresponding to the second code and get its value
const currency2Element = await page.waitForXPath(\`//td[a[contains(@title, "${currencyCode2}")]]/following-sibling::td[1]\`, {timeout: 30000});
var currency2 = await currency2Element.evaluate(el => el.textContent.trim());

// Construct a JSON object with the desired structure
const result = {
    C1: \`${currencyCode1} - ${currency1}\`,
    C2: \`${currencyCode2} - ${currency2}\`,
};

// Return this object
return result;
}
  • (3) A Webhook response node to form the response to execute the scenario.

The result of executing the scenario is a JSON with the parameters of the British pound exchange rate according to the BoE table.

If the input parameters were a JSON, then:

{
  "C1": "Hong Kong Dollar",
  "C2": "Polish Zloty"
}

If the input parameters were a JSON, then:

{
  "C1": "US",
  "C2": "Euro"
}