12.07 Using the Headless Browser Node

note

Link to the scenario template

As an example, let's create a scenario that:

• Generates currency names whose exchange rates need to be retrieved.

• Returns the exchange rates of these currencies against the British Pound.

We will use the BoE website as the data source. The web page provides a table of various currency exchange rates against the British Pound.

To set up this scenario, you need to create four nodes:

• (1) Trigger on Run once: This node initiates the scenario when you click the Run Once button;

• (2) JavaScript: This node generates currency names in JSON format;

export default async function run({execution_id, input, data, store}) {
    return {
        C1: "US",
        C2: "Euro"
    };
}

• (3) Headless browser: This node processes the web page and retrieves the required parameters using the code below. Use const currencyCode1 and const currencyCode2 to select parameters from the previous JavaScript node:

async function run({execution_id, input, data, page}) {

// Retrieve currency codes from the data passed by node 3
const currencyCode1 = data["{{$2.C1}}"]; // Variable for the first currency code
const currencyCode2 = data["{{$2.C2}}"]; // Variable for the second currency code

// Set User-Agent to mimic browser requests
await page.setUserAgent('Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36');

// Navigate to the Bank of England's currency rates page
await page.goto('https://www.bankofengland.co.uk/boeapps/database/Rates.asp?Travel=NIxIRx&into=GBP');

// Wait for the currency rates table to load
await page.waitForXPath('//*[@id="editorial"]/table');

// Find the element with the currency rate corresponding to the first code and get its value
const currency1Element = await page.waitForXPath(\`//td[a[contains(@title, "${currencyCode1}")]]/following-sibling::td[1]\`, {timeout: 30000});
var currency1 = await currency1Element.evaluate(el => el.textContent.trim());

// Find the element with the currency rate corresponding to the second code and get its value
const currency2Element = await page.waitForXPath(\`//td[a[contains(@title, "${currencyCode2}")]]/following-sibling::td[1]\`, {timeout: 30000});
var currency2 = await currency2Element.evaluate(el => el.textContent.trim());

// Construct a JSON object with the desired structure
const result = {
    C1: \`${currencyCode1} - ${currency1}\`,
    C2: \`${currencyCode2} - ${currency2}\`,
};

// Return this object
return result;
}

• (4) Webhook response: This node formats the response for the scenario execution.

The result of this scenario is a string with the exchange rate parameters of the British Pound according to the BoE table.