Developing Selenium tests with JUnit 5

Selenium has become the de-facto standard framework for end-to-end web testing nowadays. JUnit 5 is the latest version of this popular testing framework and implements a brand-new programming and extension model named Jupiter. This talk presents Selenium-Jupiter (https://bonigarcia.github.io/selenium-jupiter/), an open-source JUnit 5 extension aimed to provide seamless integration with Selenium. At first glance, it allows to create Selenium tests using local or remote browsers, reducing the required boilerplate thanks to the dependency injection mechanism supported in JUnit 5. Moreover, it will enable us to use different types of web browsers and Android devices in Docker containers out of the box. All in all, Selenium-Jupiter can be used to carry out different types of tests for web and mobile applications, including functional, compatibility (using test templates to define different devices or browsers -types and versions-) or performance (using a significant number of browsers through Kubernetes) tests.

Moreover, Selenium-Jupiter provides advance capabilities for diagnostics, i.e., finding the root cause of a failed Selenium test. The first one is screenshotting. When a failure is detected, the state of the browser just before the failure happens can provide the clue to determine the nature of the problem. If the screenshot is not enough, the next step is to watch the browser during the test to observe the evolution of the web under test. This process is straightforward when running a test with the local browser, but it can be tricky in the usual case of running a test in a headless environment, such as Jenkins or Travis to name a few. In Selenium-Jupiter, this is done very smoothly thanks to the use of web browsers and Android mobile devices in Docker containers, in which the GUI session is recorded using VNC and FFmpeg. The last mechanism proposed is the access to browser logs to debug the JavaScript traces of our applications. Unfortunately, the Selenium API only allows us to do it when Chrome is used as a browser. The solution proposed is to use monkey patching in the console object at JavaScript level in a cross-platform browser extension, which is used to instrumentalize the browser (e.g., Firefox, Opera) controlled with Selenium WebDriver. To ease the access to such information, Selenium-Jupiter provides seamless integration with Jenkins and Slack, publishing the resulting files (PNG screenshots, MP4 recordings, and TXT logs) in the Jenkins GUI or Slack channel when a test fails.

 
 

Outline/Structure of the Workshop

Table of Contents

  • JUnit 5, the next level of JUnit (15 minutes)
    • Motivation and architecture
    • Jupiter: Programming and extension model in JUnit 5
  • Selenium-Jupiter, JUnit 5 extension for Selenium (50 minutes)
    • Using local browser
    • Using remote browsers
    • Using browsers and Android devices in Docker and Kubernetes
    • Test templates
    • Integration with Jenkins
  • Diagnostic techniques with Selenium-Jupiter (15 minutes)
    • Screenshotting
    • Session recording
    • Log gathering
  • Selenium-Jupiter beyond Java (10 minutes)
    • Command Line Interface (CLI)
    • Selenium Server

Examples

  • Hello World: JUnit 4 + Selenium WebDriver vs. JUnit5 + Selenium-Jupiter
  • Test using local browsers (e.g. Chrome, Firefox, Opera)
  • Test using Selenide
  • Test using options (e.g. headless Chrome)
  • Reusing the same browsers by different tests
  • Using remote browsers in SauceLabs
  • Test using Appium
  • Test using browsers (e.g. Chrome, Firefox, Opera) in Docker containers
  • Test using Window browsers (e.g. Edge, Internet Explorer) in Docker containers
  • Test using Android devices (e.g. Samsung Galaxy S6, Nexus 4, Nexus 5, Nexus One, Nexus S, Nexus 7) in Docker containers
  • Recordings and remote access (with VNC) when using Docker
  • Performance tests using Docker and Kubernetes
  • Test template example
  • Integration with Jenkins (access to screenshots, recordings, and browser logs available using the Jenkins GUI)
  • Examples using Selenium-Jupiter as a CLI tool
  • Examples using Selenium-Jupiter as a Server (Node.js test as example)
  • Diagnostic examples (screenshots, recordings, logs)

Learning Outcome

  • Main features of the programming and extension model of JUnit 5 (called Jupiter)
  • How to use Selenium from JUnit 5 tests for web and mobile (Android) testing
  • How Docker and Kubernetes can be used to ease functional, compatibility and performance tests
  • Integration with existing technologies, such as Selenide, Saucelabs, Appium, or Jenkins
  • How to use Selenium-Jupiter as CLI tool and a Selenium server
  • Learn diagnostic techniques for Selenium tests (screenshoting, session recording, log gathering)

Target Audience

Developers/testers/CTOs

Prerequisites for Attendees

In this workshop, we will execute different examples to show the features of Selenium-Jupiter. These examples are located in the GitHub repository: https://github.com/bonigarcia/selenium-jupiter-examples.

It is highly recommended to run these examples to follow this workshop. To do that, attendees should bring a laptop with the following requirements in term of software:

  • Required:
    • Java 8+ (JDK)
    • Some IDE (Eclipse, IntellijIDEA, Visual Studio or NetBeans) or alternatively Maven or Gradle
  • Recommended:
    • Docker engine (in Linux, Mac, or Windows)
  • Optional:
    • Linux (required only to execute Android in Docker containers)
    • Jenkins (required only to execute the integration with Jenkins example)
    • Appium server (required only to execute the integration with Appium example)
    • Kubernetes (required only for running Kubernetes examples)
    • Node.js (required only to execute the example using Selenium-Jupiter as a Server)
schedule Submitted 9 months ago

Public Feedback

comment Suggest improvements to the Speaker
  • Pallavi ...
    By Pallavi ...  ~  5 months ago
    reply Reply

    Hi Boni

    Great workshop topic. A small suggestion, do you think in 90 minutes mobile concepts can also be covered, appium - android .

    i see a lot of examples, and information in the workshop which would be shared with audience, web functional testing, container, CI integration, mobile testing. 

    if you intend to cover all these, can i request you to please update your pre-requiste section, and make this workshop for advance users. i feel it may not be suitable for a beginner level. what do you think about it.

    • Boni García
      By Boni García  ~  5 months ago
      reply Reply

      Hi Pallavi,

      Thanks for your comment. I have just updated my proposal according to your suggestions.

      Regarding the content and the time, IMHO it can be done in 90 minutes. I gave related talks in the past about Selenium-Jupiter, also with plenty of examples. Please take a look at my presentation in QA Fest 2019 (this talk was around 50 minutes). I included the video and slides in the proposal for SeleniumConf.

      I have also updated the pre-requisites and changed the level to intermediate. It's true it's a bunch of technologies here, but I feel most of the examples can be easily understood by the attendees. Besides, all of them are available in a separate GitHub repository (https://github.com/bonigarcia/selenium-jupiter-examples), which in my opinion it is a good way to ease the adoption of the presented approach.

      Kind regards,
      Boni

      • Pallavi ...
        By Pallavi ...  ~  5 months ago
        reply Reply

        Thanks Boni. Appreciate.

  • Anand Bagmar
    By Anand Bagmar  ~  6 months ago
    reply Reply

    Hi Boni,

    While this is a 90-min workshop, I am not seeing what parts of the content the participants will be doing under your instruction. Can you please clarify?

    Also, what are the pre-requisites for this workshop? (in terms of logistics, as well as pre-setup required for the attendees)?

    Regards, Anand

    • Boni García
      By Boni García  ~  6 months ago
      reply Reply

      Hi Anand,

      I have just updated my proposal, including the timing for each section and also a list of the examples I plan to show. Moreover, I have updated the section of prerequisites for attendees. Finally, I have included the integration with Kubernetes in the table of contents.

      Kind regards,
      Boni

  • Robin Gupta
    By Robin Gupta  ~  6 months ago
    reply Reply

    I believe the tool at hand is very specific to JAVA/Junit audience, leaving other users from TestNG/Ruby/Python in the cold.

    • Boni García
      By Boni García  ~  6 months ago
      reply Reply

      You are right. The workshop I am proposing is mainly focused on Java since Selenium-Jupiter is a JUnit extension. Nevertheless, Selenium-Jupiter can be used in different uses cases rather than JUnit test.

      First, it can be used as a Command Line Interface (CLI) tool to run different types of browsers (Chrome, Firefox, Opera, Explorer, Edge, and Android devices) and versions (stable, beta, dev) in Docker containers, which are accessed with noVNC (i.e. with another browser). This feature can be used for manual testing using different browsers without the need of installing the browser itself.

      Second, Selenium-Jupiter can be executed as a Server which speaks the JSON wire protocol/W3C WebDriver, becoming into a Selenium Server. In this case, Selenium-Jupiter acts as a hub (in the Selenium Grid jargon) which can be used from different language bindings than Java (e.g. JavaScript, Python, etc.).

      Just in case, I have updated the proposal TOC to include these features.


  • Liked Boni García
    keyboard_arrow_down

    Boni García - Diagnostic techniques for Selenium tests

    45 Mins
    Talk
    Beginner

    One of the main benefits of end-to-end tests with Selenium is the simulation of real user scenarios in an automated fashion. Nevertheless, end-to-end tests are often criticized because these kinds of tests do not isolate failures, and therefore troubleshooting failures can be difficult and costly for testers. This talk pretenses different techniques aimed to solve this problem. The objective is to provide useful assets to find the root cause of a failure in a Selenium test. The first one is screenshotting. Once a failure is detected, the state of the browser just before the failure happens can provide the clue to determine the nature of the problem. If the screenshot is not enough, the next step is to watch the browser during the test to observe the evolution of the web under test. This process is straightforward when running a test with the local browser, but it can be tricky in the usual case of running a test in a headless environment, such as Jenkins or Travis to name a few. The solution proposed is twofold. On the one hand, a Selenium session can be recorded using Virtual Frame Buffer (xvfb) and FFmpeg. On the other hand, we can use web browsers and mobile devices in Docker containers, in which the GUI session can be easily recorded using VNC and FFmpeg. The last mechanism proposed is the access to browser logs to debug the JavaScript traces of our applications. Unfortunately, the Selenium API only allows us to do it when Chrome is used as a browser. The solution proposed is to use monkey patching in the console object at JavaScript level in a cross-platform browser extension which is used to instrumentalize the browser (e.g., Firefox, Opera) controlled with Selenium WebDriver. All these mechanisms have been implemented in Selenium-Jupiter, a JUnit 5 extension for Selenium providing seamless integration with Docker. To ease the access to such information, Selenium-Jupiter provides seamless integration with Jenkins and Slack, publishing the resulting files (PNG screenshots, MP4 recordings, and TXT logs) in the Jenkins GUI or Slack channel when a test fails.

  • Liked Boni García
    keyboard_arrow_down

    Boni García - Selenium tests with Docker and Kubernetes: to infinity and beyond

    45 Mins
    Talk
    Intermediate

    Docker is a container technology that has become pervasive in current software architecture and infrastructure nowadays. Kubernetes is a container orchestration platform, allowing large numbers of containers to work together. This presentation presents how Docker and Kubernetes can be used to create advance Selenium tests. The use of containers allows using different types of web browsers and Android devices in Docker containers out of the box, allowing to create functional or compatibility (using different devices and/or browsers types and versions) in an effortless manner. The use of a Kubernetes cluster allows to scale up the approach, allowing to a use huge number of browsers to develop performance or load tests with Selenium. All these mechanisms have been implemented in Selenium-Jupiter, a JUnit 5 extension for Selenium which provides seamless integration with Docker and Kubernetes.

    Moreover, Selenium-Jupiter can be executed as a Server which speaks the JSON wire protocol/W3C WebDriver, becoming into a Selenium Server. In this case, Selenium-Jupiter acts as a scalable hub (using the Selenium Grid jargon), providing browsers for any kind of Selenium script (Java, JavaScript, Python, etc.) using Docker and Kubernetes internally to allocate the requested browsers types (Chrome, Firefox, Opera, Explorer, Edge, and Android devices) and versions (stable, beta, dev).