How to use SauceLabs with Nightwatch.js

September 28, 2020

#post

36

#software testing

18

#quality assurance

17

#nightwatchjs

15

#saucelabs

1

Using SauceLabs to run your Nightwatch.js Tests

When running your automated tests on your local workstation or server you can quickly get resource constrained by the number of tests you have to run when you multiply them by platforms, browsers, and versions in-between.

Nightwatch supports cloud testing platforms including SauceLabs which allows you to offload the running of the tests to their virtual machines instead of your own.

There are many advantages which this approach

• Nightwatch tests can be run in parallel across multiple virtual machines in the SauceLabs cloud
• No need to maintain local test servers
• Access to different operating systems and current and past browser configurations
• Video recordings of test runs

Disadvantages

• Much slower single test execution time
• Slightly more complicated initial setup

The video tutorial below show you how to configure Nightwatch to use SauceLabs to run your tests end to end.

Configuring Nightwatch to use SauceLabs

We will be using the saucelabs and nightwatch-saucelabs-endsauce packages for Node.js to remotely execute our automated test commands and send our test results to SauceLabs respectively.

To follow along you can download the source code from GitHub from the sauceLabsExample in my Nightwatch tutorials project. See the README.md file there for installation instructions.

In your nightwatch.js.conf (or nightwatch.json) configuration file the following entries will allow the two packages to connect to and authenticate with the remote SauceLabs cloud.

(partial section showing relevant parts)

{
...
    "plugins": ["nightwatch-saucelabs-endsauce"],
    "test_settings": {
        "default": {
            "use_ssl": true,
            "silent": true,
            "selenium": {
                "port": 443,
                "host": "ondemand.saucelabs.com"
            },
            "webdriver": {
                "start_process": false
            },
            "desiredCapabilities": {
                "sauce:options": {
                    "username": "${SAUCE_USERNAME}",
                    "accessKey": "${SAUCE_ACCESS_KEY}",
                    "region": "us-west-1", // or your region per your Sauce account if different
                    "screenResolution": "1920x1080"
                },
                "browserName": "chrome",
                "platformName": "Windows 11",
                "browserVersion": "latest",
                "javascriptEnabled": true,
                "acceptSslCerts": true,
                "timeZone": "New York" // optional
            }
        }
    }
...
}

The plugins property listed above will allow Nightwatch to find the nightwatch-saucelabs-endsauce package that will be called to send the test result information to SauceLabs on test end.

Use

npm install nightwatch-saucelabs-endsauce --save

to add it to your project.

(partial package.json)

{
   "dependencies": {
    "nightwatch": "^2.4.2",
    "nightwatch-saucelabs-endsauce": "^2.0.0",
    "saucelabs": "^7.2.0"
   }
}

The username and access key as shown will look for their values as environment variables. If you prefer to hardcode them you can replace the placeholder names in quotes with the username and access key from your SauceLabs account screen.

Setting environment variables on Windows 10

1. Type env into your run bar and hit enter
2. Click Environment Variables toward the bottom of System Properties
3. In user variables click New
4. Type SAUCE_USERNAME for the Variable Name and enter your SauceLabs username for the variable value.
5. Repeat for SAUCE_ACCESS_KEY

Setting environment variables on Mac (if bash is your default terminal)

1. Type sudo nano ~/.bash_profile into a terminal
2. Type export SAUCE_USERNAME= followed by your SauceLabs username
3. Below that type export SAUCE_ACCESS_KEY= followed by your SauceLabs access key
4. Quit and save.

Setting environment variables on Mac (if zsh is your default terminal)

1. Type sudo nano ~/.zshrc into a terminal
2. Type export SAUCE_USERNAME="saucelabs-username-here"
3. Below that type export SAUCE_ACCESS_KEY="saucelabs-accesskey-here"
4. Quit and save.

In either operating system you'll need to reopen any terminal windows for them to read the change.

Verify the settings are being read by your operating system by using echo ${SAUCE_USERNAME}

Sauce_region

The last setting in the package.json file above is the sauce_region. This corresponds to your data center region for your account. Valid settings are us-west-1, eu-central-1, and us-east-1 so choose the appropriate value that matches your account.

Writing a Nightwatch.js test that runs in SauceLabs

The trickiest part, sending in the test results, is already solved by using the nightwatch-saucelabs-endsauce custom command. It just needs to be called in the afterEach test hook in your Nightwatch test suite.

module.exports = {
    beforeEach: function (browser) {
        browser.page.blog().navigate();
    },
    afterEach: function (browser) {
        browser.endSauce();
        browser.end();
    },
    'Can see homepage': function (browser) {
        browser.page.blog().assert.title('David Mello')
    },
    'Can navigate to Software Testing': function (browser) {
        browser.page.blog().click('@softwareTesting')
            .expect.element('h1').text.to.equal('Software Testing and Quality Assurance')
    }
}

SauceLabs relies on the browser session id in order to keep track of the individual tests so the browser must be closed and reopened between test cases to generate a new session id.

Starting with Nightwatch 2.0 the session id is cleared after the browser is closed so call browser.endSauce() before browser.end()

I found that if you leave the browser open between test cases SauceLabs will treat the entire test suite as one test case--it needs a new session id per test.

For the individual tests to properly record use the afterEach test hook which runs after each test. Inside there browser.end() is called which closes the browser.

browser.endSauce then records the session id, test name, and pass/fail status back to SauceLabs.

When nightwatch is run in the terminal of the root of the SauceLabs example project you'll see output similar to the following.

Running:  Can navigate to Software Testing

i Connected to ondemand.saucelabs.com on port 443 (3427ms).
  Using: chrome (85.0.4183.83) on Windows platform.

√ Expected element <h1> text to equal: "Software Testing and Quality Assurance" (793ms)
SauceOnDemandSessionID=207dc6cf57824a72af4787dd49774aa8
job-name==Can navigate to Software Testing
passed=true

Nightwatch sends the commands over the saucelabs REST API to the remote environment and browser they have there in the cloud which then sends the results back to your local Nightwatch terminal window. The last 3 lines are the output from nightwatch-saucelabs-endsauce that sends the final test result back to SauceLabs.

Desired Capabilities, setting the browser

In the nightwatch.json file you can tell SauceLabs what browser and operating system you want your test automation to run under using the desired capabilities configuration section.

"test_settings": {
        "default": {
...
            "desiredCapabilities": {
                "browserName": "chrome",
                "screenResolution": "1280x1024",
                "browserVersion": "latest",
                "javascriptEnabled": true,
                "acceptSslCerts": true,
                "timeZone": "New York"
            }
        }
    }

The screen resolution is important to set a realistic window size for the test to run under. If you notice oddities around date and time ensure that your timeZone setting is set correctly here as well. I noticed javascript dates were wrong when running my test from the east coast while being on the west coast cloud before I explicitly set New York for the time zone here.

Parallelizing Nightwatch SauceLabs tests

The biggest drawback when using SauceLabs instead of local test execution is that it is considerably slower for them to run a per test basis. However, the time is quickly made up if you take advantage of parallel test execution.

Nightwatch has the capability to run your tests in parallel as long as they are in different .js files. It can spin up one test execution thread per .js file.

With SauceLabs you can take advantage of this and use as many virtual machines to parallelize your tests as your account allocation permits.

To enable parallel test execution in Nightwatch set the enabled flag equal to true under test_workers and configure the workers to auto or a set number of your choosing.

{
...
    "page_objects_path": "./page-objects",
    "test_workers": {
        "enabled": true,
        "workers": "auto"
    },

Final Thoughts

Initially it was a little hard to put together all the bits and pieces of tutorials out there to figure out how to get SauceLabs working with Nightwatch so I hope this article does a more comprehensive job for you. Nightwatch and SauceLabs seem to pair pretty well together so far and I've immediately appreciated the video recording capabilities SauceLabs offers as well as the parallel test execution offloaded from my local development machine.

If you have any questions or comments please reach out through my social links below 👇