BrowserStack Runner

BrowserStack Runner enables you to run JavaScript unit tests written in QUnit, Jasmine, or Mocha, in browsers on BrowserStack.

Installation

To install browserstack-runner as a command-line tool

npm install -g browserstack-runner

You may also install browserstack-runner as a dev-dependency in your project.

npm install --save-dev browserstack-runner

Configuration

• username: BrowserStack username (Or BROWSERSTACK_USERNAME environment variable)
• key: BrowserStack access key (Or BROWSERSTACK_KEY environment variable)
• test_path: Path to the test page which will run the tests when opened in a browser.
• test_framework: Specify test framework which will run the tests. Currently supporting qunit, jasmine, jasmine2 and mocha.
• timeout: Specify worker timeout with BrowserStack.
• browsers: A list of browsers on which tests are to be run. Find a list of all supported browsers and platforms on browerstack.com.
• build: A string to identify your test run in BrowserStack. In TRAVIS setup TRAVIS_COMMIT will be the default identifier.
• proxy: Specify a proxy to use for the local tunnel. Object with host, port, username and password properties.

For advanced use-cases in which test files are served from an external web server, the test_server property can be set to the root URL for the tests and test_path can contain the relative path to HTML test pages.

Basic Configuration

To run tests on BrowserStack infrastructure, you need to create a browserstack.json file in project's root directory (the directory from which tests are run), by running this command:

browserstack-runner init

This creates a browserstack.json file containing a set a browsers to help you get started. You may either set the username and key fields in browserstack.json or store your credentials in BROWSERSTACK_USERNAME and BROWSERSTACK_KEY environment variables.

Next, set the test_path property to a list paths to .html files that contain your tests.

To run your tests -

browserstack-runner

You may add the optional --verbose switch for detailed information.

Configuring Browsers

The browsers property in browserstack.json contains the list of browsers to run tests in. You may refer to this list of platforms and browsers (requires login) and documentation here.

Each browser object may contain the following properties:

• os: The operating system.
• os_version: The operating system version.
• browser: The browser name.
• browser_version: The browser version.
• device: The device name for testing on mobile emulators. Optional.

For desktop platforms, os, os_version, browser, browser_version are to be supplied. A special value of latest is supported for browser_version, which will use the latest stable version.

{
      "browser": "ie",
      "browser_version": "10.0",
      "os": "Windows",
      "os_version": "8"
}

For mobile platforms, os, os_version and device parameters are to be supplied.

{
      "os": "ios",
      "os_version": "7.0",
      "browser": "Mobile Safari",
      "device": "iPhone 5S"
}

How It Works

The following steps are performed by BrowserStack Runner to execute your tests:

• Starts a web server on your local machine on port 8888 to serve your test files
• Starts BrowserStack Local to establish a secure tunnel to BrowserStack's servers to make the local web server accessible to browsers on BrowserStack
• Launches browsers on BrowserStack by creating API workers
• As they are being served, test HTMLs are patched to include files that contain hooks for the specified test_framework
  • Patch files capture test run status in the browser and report data back to BrowserStack Runner
• Terminates workers after tests are complete

Working with Mobile Emulators

Mobile emulators, especially Android, can take a while to boot up. This requires increasing the worker timeout by setting the timeout property to 300 (or more) seconds.

Proxy Support

In case you need to supply proxy configuration to BrowserStack Local, you may set the proxy property:

"proxy": {
      "host": "localhost",
      "port": 3128,
      "username": "foo",
      "password": "bar"
}

Troubleshooting

Tests run but session doesn't end

Mocha - Ensure that the mocha.run(); is within window.onload = function () { ... }. This ensures that our patch files are executed before the Mocha tests are run in the browser.

Jasmine - For Jasmine v1, ensure that the jasmine.getEnv().execute(); call exists within window.onload = function () { ... } so that our patch files are executed before tests run.

EACCES Error For BrowserStackLocal

If you're getting an error EACCES open ... BrowserStackLocal, configure npm to install modules using something other than the default nobody user:

npm -g config set user [user]

Another case could be that the BrowserStackLocal binary has been partially downloaded. Try deleting node_modules/browserstack-runner/lib/BrowserStackLocal or node_modules/browserstack-runner/lib/BrowserStackLocal.exe in that case.

Debugging Steps

To run BrowserStack Local in debug mode

LOG_LEVEL=trace browserstack-runner --verbose

Development

Generating browserstack-util.js with webpack

Webpack is used to generate code in browserstack-util.js which contains browser-compatible code for node's util.inspect. This is used for serializing the user's console.log data and sending it to the server.

To update browserstack-util.js:

npm install -g webpack
npm run update-util

Running Tests

BrowserStack Runner is currently tested by running test cases defined in QUnit, Mocha, and Spine repositories.

To run tests:

npm test

To run a larger suite of tests ensuring compatibility with older versions of QUnit, etc.:

npm run test-ci

Tests are also run for every pull request, courtesy Travis CI.