Getting Started with Puppeteer

What is Puppeteer?

Puppeteer is a node based library from Google that provides a simple API for interacting with Chrome based applications. As the web is emerging with powerful javascript based applications, CSS and HTML are now injected into javascript instead of other ways such that we need not compromise with the speed of the applications. The use of JavaScript based frameworks have improved the performance of applications a lot but at the same time, it has been a challenge for the tester community. Puppeteer is introduced as a node library used to focus on Chrome browser testing.

Headless browser

Headless browsers, as the name suggests are the browsers without any UI elements. These are faster, lightweight and are more flexible. With the use of puppeteer web scraping, launching automated scripts, web crawling etc can be achieved with less effort and more stability.

Automation with Puppeteer

Compared to other popular tools like selenium, Puppeteer does not require real browser to be installed in the testing environment (cloud or local machine). Use of frameworks like Karma, Mocha along with Puppeteer has made automated testing far more easier and efficient. Puppeteer can be used for Visual UI Regression testing, performance testing, Screenshot testing, load testing, end to end testing etc.

Set up

Installing and getting started with puppeteer is quite simple. You need to have node.js installed (version 8+). It can be downloaded from https://nodejs.org/en/
Since puppeteer is a node library, it can be installed via npm tool:

npm -g i puppeteer --save

Above command will install Puppeteer. It also downloads a recent version of Chromium (approx 170MB on Mac) that will work with the API. With the above command, we are ready to start with Puppeteer.

Example: Open Google home page and capture screenshot

Let us start with a basic example. We are going to write a script that will launch google home page and capture a screenshot of the same.

First step is to create a JavaScript file in your project directory, say googlescreenshot.js and open it in any editor of your choice. Now, the first line of code in the googlescreenshot.js file should be importing puppeteer library in your script:
const puppeteer = require('puppeteer');
Now, we need to provide URL for Google home page:
const url = 'https://www.google.com';
Next step is to launch browser and open Google using Puppeteer library:

const browser = puppeteer.launch(); 
const page = browser.newPage(); 
page.goto(url); 

Since javascript performs asynchronous calls to the browser instance, we need to perform the above actions using async/await like:

async function run() {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();
    await page.goto(url);
}

With above lines of code, we are able to launch chrome browser using puppeteer library and the google home page has been opened.    Now, next step is to capture the screenshot:

page.screenshot({path: 'google.png'}); 

Final step is closing the browser instance after performing required operations.

browser.close();

So, the final code should look like:

async function run() {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();
    await page.goto(url);
    await page.screenshot({ path: 'google.png' });
    browser.close();
}
run();

To run the above code type below command in command line:

node run googlescreenshot.js

The screenshot will be saved in same directory with name ‘google.png’.

Example: Capture search suggestions from Google

Let us take another example, where we are going to open google home page, type in some text in the text box and then capture the suggestions list.    We will perform the same steps as mentioned above to launch the browser and open the desired URL.
Next step is to type some text in the text box. For that first step is to find an identifier for the text box and then enter text:

page.type('input.gLFyf.gsfi','what is puppeteer');

Next step is to find an identifier for the auto suggestions and capture the suggestions:

const items = page.evaluate(() => {
    const result = Array.from(document.querySelectorAll('ul.erkvQe li div div div span'))
    return result.map(anchor => anchor.textContent.trim());
})

So, the final code should be like:

const puppeteer = require('puppeteer');
const url = "https://google.com";

async function getAutoSuggestionsList() {
    return new Promise(async (resolve, reject) => {
        try {
            const browser = await puppeteer.launch({ headless: false });
            const page = await browser.newPage();
            await page.goto(url);
            await page.type('input.gLFyf.gsfi', 'what is puppeteer');
            await page.waitForSelector('ul.erkvQe li');

            const items = await page.evaluate(() => {
                const result = Array.from(document.querySelectorAll('ul.erkvQe li div div div span'))
                return result.map(anchor => anchor.textContent.trim());
            })
            await browser.close();
            return resolve(items);
        } catch (e) {
            console.log(e.message);
            return reject(e);
        }
    })
}
getAutoSuggestionsList().then(console.log).catch(console.error);