Webdriver connector for Botium - Tool for testing Conversational UI

location_city Virtual Platform schedule Sep 12th 10:30 - 11:15 AM place Online Meeting 2 people 35 Interested

Last year was dominated by the smart devices & voice based home assistants. These use the conversational interfaces unlike other applications to interact, built using advanced algorithms, ranging from natural language processing to AI/ML techniques. They are constantly learning by themselves improving the interactions with the user bringing up the challenge of non-deterministic output. To such interfaces, natural language is the input & we humans love having alternatives & love our synonyms, express using emojis gifs & pictures. Testing in this context moves to clouds of probabilities.

Unfortunately Selenium cannot be used to automate such systems and hence Botium was designed.
In this session I will cover the Selenium driver for Botium to automate E2E tests on Web UI and mobile along with testing strategy, testing NLP models & automating these tests to the CI/CD build pipelines with a DialogFlow based 'Coffee-Shop bot' as an example during my demo.

 
 

Outline/Structure of the Demonstration

  • What, Why and How of a Conversational AI?
  • Challenges
  • Test Approaches for a Conversational AI
  • Test types
  • Use Case – A Coffee Shop Bot
  • Automation with Botium
  • E2E tests on Web and mobile
  • Measuring the Quality

Learning Outcome

Key learnings:
How What & why of a conversational interface?
Testing approach for such an interface
What from my current tool set can I use for this new context?
How to automate & add it for my CI/CD pipeline for instant feedback?
How do I measure the quality?

Target Audience

Anyone interested to learn and understand how to test bots

schedule Submitted 10 months ago

Public Feedback


    • Simon Stewart
      keyboard_arrow_down

      Simon Stewart - Selenium: State of the Union

      Simon Stewart
      Simon Stewart
      Project Lead
      The Selenium Project
      schedule 9 months ago
      Sold Out!
      45 Mins
      Keynote
      Intermediate

      What's the latest in the world of Selenium since the last SeleniumConf? In this talk, we'll cover Selenium 4, how the project is run, how you can participate, reminiscing on the Old Days, and reflect on something that at first doesn't appear to be related to Selenium, yet is intimately related to it.

    • Rajdeep varma
      keyboard_arrow_down

      Rajdeep varma - The Joy Of Green Builds - Running Tests Smartly

      Rajdeep varma
      Rajdeep varma
      Automation Lead
      Bumble
      schedule 10 months ago
      Sold Out!
      45 Mins
      Talk
      Intermediate

      So you have got a few UI tests and they are running in parallel, great! However, life will not be so sweet once these 'a few' turns into 'a lot'. We grew from a few to 1500 UI tests (although not particularly proud of this number, there are situations and reasons)

      We started with a simple parallel distribution of tests 3 years ago. As test count increased failure count run time increased along with increased flaky tests. Mobile tests had their own challenges (eg. device dropping-off, random wi-fi issues, etc) To keep up with this, we created a queue and workers based solution which could distribute the tests more efficiently (https://github.com/badoo/parallel_cucumber). Over time, we made more improvements, in particular:

      • Segregation of failures based on infrastructure issues and re-queue the tests
      • If a device/emulator malfunction, rescue the tests to another device
      • Repeating a single test on 100s of the worker in parallel to detect flakiness
      • Repeat a test if a known network issue
      • Terminating the build early if more than a certain number of tests have failed
      • Health check of each device, before each test to ensure reliability
      • Muting a test if failure is known, and highlight outdated mutes if the related task is fixed

      In this talk, I will talk about the initial challenges with running UI tests in parallel (Selenium and Appium), how we approached the queue based solution and continuous improvement of this solution; finally, how attendees can use it at their workplace or create their own solution based on our learnings.

    • Christian Bromann
      keyboard_arrow_down

      Christian Bromann - The Nuts and Bolts of WebdriverIO

      Christian Bromann
      Christian Bromann
      Software Engineer
      Sauce Labs
      schedule 10 months ago
      Sold Out!
      90 Mins
      Workshop
      Beginner

      There are thousands ways if not more to setup your automation testing environment. It is often crucial when it comes to stability and maintainability. While you can build a custom framework that fits your needs there are often already sophisticated setups or frameworks available that get you up and running very quickly.

      WebdriverIO is one of these frameworks that is written in Node.js. It gets you up and running within minutes and allows you to scale up your test suite while maintaining your execution time by running tests in parallel. With its huge project community it is an ideal choice for small as well as big projects that use modern frameworks such as React, Angular or Vue.js.

      In this workshop you will learn everything you need to know to run a successful, stable and maintainable WebdriverIO framework. It starts with an introduction to the project and the basic concepts and will end with a sophisticated framework that includes testing strategies like Frontend Performance Testing as well as complex browser interaction with Puppeteer.

    • SWAPNA Narayanan
      keyboard_arrow_down

      SWAPNA Narayanan - How to build a simple automation tool for your organization

      SWAPNA Narayanan
      SWAPNA Narayanan
      Qe lead
      Estuate
      schedule 10 months ago
      Sold Out!
      45 Mins
      Case Study
      Intermediate

      Ever wondered how to develop a simple automation tool using selenium ?

      Create a simple automation tool for your organization in no time. This presentation demonstrates multiple ideas to build a script less automation tool from scratch in most simple ways through multiple architectural designs and diagrams.

      Why we need a tool on top of selenium ?

      Selenium is a powerful automation tool to automate any application. It can be clubbed with Appium, Jmeter etc for additional coverage.

      Functional , Web ,Mobile and Performance testing coverage can be managed this way.

      Problem statement:

      We need people with some coding skills to write the test cases and execute them in selenium / Appium. When it comes to building scalable and reusable frameworks, we need automation experts.

      I have seen a lot of teams from my e-commerce, Insurance and banking domain backgrounds with great domain expertise and no or minimal coding exposure .They are keen to explore and utilize the power of automation for the redundant regression tasks.

      Solution :

      A script-less automation tool will fill the gap in the above scenario.

      For any off the shelf tool there is a high cost involved.

      If the tool is built internally , comparatively very less cost is involved.

      More over, many additional functionalities like meaningful reports, AI metrics, NLP capabilities, integration with Test management tools etc can be integrated according to the requirements.

      The scope of this session starts here. This session will inspire you and gives you many ideas on creating tools for your need to get maximum ROI and build the gap between core domain experts and automation testing.

      Beneficiaries :

      Product owners , business analysts , functional experts who can start writing test cases on a daily basis , leveraging all the opportunities that selenium offers to the fullest, without actually writing a single line of code.

      What to expect from this session ?

      I will walk you through my experiences and the knowledge we gained as a team while we worked on building multiple automation tools.

      Different architectural diagrams that can be handy when you actually start to think and implement an automation tool project will be discussed.

      Ideas on different implementation capabilities using different Front end technologies and database designs will be discussed.

      Demo of an internal tool developed will complete the session

    • Irfan Ahmad
      keyboard_arrow_down

      Irfan Ahmad - From Test Pyramid to Test Cup : Evolution of Modern Testing Stack

      Irfan Ahmad
      Irfan Ahmad
      Release Manager
      upGrad
      schedule 10 months ago
      Sold Out!
      45 Mins
      Case Study
      Beginner

      We all have read or heard about test automation pyramid .But is it really supposed to be a pyramid ? or it's just a metaphor? lets find out

      At this talk we will share our learnings from implementing test automation pyramid for years and its variations finally arriving to test automation cup modal focusing on

      1.Need for growing up from automation testing pyramid

      2.How the modern testing stack has evolved

      3.How to choose and design your automation testing stack

      4.Real World examples from testing to validate our test cup model

      5.How to move from Pyramid to test cup and Our lessons learned

      6.Sharing Best practices ,limitations and patterns for success

    • Khyati Sehgal
      keyboard_arrow_down

      Khyati Sehgal / Shubhechchha Rai - Expanding the aura of automation, when it comes to mobile.

      45 Mins
      Demonstration
      Intermediate

      This talk will answer the questions "Will you be able to run your automation scripts without connecting the USB cable to your device in a limited range?"

      If we have to ask ourselves where is the mobile and when is it required? So the answer would be its 'Everywhere'!!

      It's next to possible to imagine a day without looking at mobile or using apps for that matter- Google, Facebook, Whatsapp, etc.

      When it comes to using it so testing comes hand-in-hand with it. As we develop applications over a mobile device, be it any platform ios, Andriod and windows we shall also think about usability and end-to-end testing.

      In this talk, we will discuss how Appium worked as a bridge between devices and controllers.

    • Shama Ugale
      keyboard_arrow_down

      Shama Ugale - Refactoring - a key ingredient of evolution of your automation framework

      Shama Ugale
      Shama Ugale
      Sr. QA Consultant
      ThoughtWorks
      schedule 10 months ago
      Sold Out!
      45 Mins
      Case Study
      Beginner

      Ever happened to you that your UI gets migrated to the angular stack while you have all the automated tests and infra in place in selenium Webdriver and Java framework. When you run your tests they are mostly flaky and and drain your efforts in fixing/false negatives?

      Is the solution to this problem finding a new tool that fits well for angular UI such as protractor? Or to spend some time refactoring your existing framework to support angular UI and leverage the existing tests and infra in place?

      In this talk we shall discuss the later solution and the implementation.

    • Shama Ugale
      keyboard_arrow_down

      Shama Ugale - On-demand Distributed test infra with Selenium Grid and Terraform

      Shama Ugale
      Shama Ugale
      Sr. QA Consultant
      ThoughtWorks
      schedule 10 months ago
      Sold Out!
      45 Mins
      Demonstration
      Beginner

      With the fast paced world it’s very important to make the decisions faster and that can be possible only if you have a good test coverage and a faster feedback.
      One can use selenium grid on the local systems and achieve the cross browser and device testing which may have constraints with respect to scalability and maintainability, or use SaaS alternatives such as BrowserStack/Saucelabs which fit well for lesser no of tests (gets a bit expensive as we get into concurrent sessions and increase the browser/device combinations) or moreover containerize the automation having the limitations with coverage of IE. How to design a solution to get the a scalable, on-demand, manageable and yet cost effective infra? Selenium Grid along with Terraform and AWS can be leveraged to write the whole infrastructure as code to prevent any human errors and manual intervention. In this talk, I shall take you through the journey of reducing the test execution pain-points in terms of time, resource availability, conflict handling, scalability and state management with a demonstration.