Webdriver connector for Botium - Tool for testing Conversational UI

Last year was dominated by the smart devices & voice based home assistants. These use the conversational interfaces unlike other applications to interact, built using advanced algorithms, ranging from natural language processing to AI/ML techniques. They are constantly learning by themselves improving the interactions with the user bringing up the challenge of non-deterministic output. To such interfaces, natural language is the input & we humans love having alternatives & love our synonyms, express using emojis gifs & pictures. Testing in this context moves to clouds of probabilities.

Unfortunately Selenium cannot be used to automate such systems and hence Botium was designed.
In this session I will cover the Selenium driver for Botium to automate E2E tests on Web UI and mobile along with testing strategy, testing NLP models & automating these tests to the CI/CD build pipelines with a DialogFlow based 'Coffee-Shop bot' as an example during my demo.

 
 

Outline/Structure of the Demonstration

  • What, Why and How of a Conversational AI?
  • Challenges
  • Test Approaches for a Conversational AI
  • Test types
  • Use Case – A Coffee Shop Bot
  • Automation with Botium
  • E2E tests on Web and mobile
  • Measuring the Quality

Learning Outcome

Key learnings:
How What & why of a conversational interface?
Testing approach for such an interface
What from my current tool set can I use for this new context?
How to automate & add it for my CI/CD pipeline for instant feedback?
How do I measure the quality?

Target Audience

Anyone interested to learn and understand how to test bots

schedule Submitted 1 month ago

Public Feedback

comment Suggest improvements to the Speaker
  • Deepti Tomar
    By Deepti Tomar  ~  4 weeks ago
    reply Reply

    Hello Shama,

    Thanks for your proposal!

    Could you please update the Outline/Structure section of your proposal with a time-wise breakup of how you plan to use 45 mins for the topics you've highlighted?

    Thanks!

    • Shama Ugale
      By Shama Ugale  ~  3 weeks ago
      reply Reply

      Hi Deepti,

      Here is how i plan approximately.

      Key learnings:
      - How What & why of a conversational interface? (3-5 mins)
      - Testing approach for such an interface (10 mins)
      - What from my current tool set can I use for this new context? (5 mins)
      - How to automate & add it for my CI/CD pipeline for instant feedback? (20 mins)
      - How do I measure the quality? (3-5mins)

      Thanks,

      Shama Ugale

  • Anand Bagmar
    By Anand Bagmar  ~  4 weeks ago
    reply Reply

    Hi Shama, how is this proposal different from your Appium Conf submission?

    • Shama Ugale
      By Shama Ugale  ~  4 weeks ago
      reply Reply

      Hi Anand,

      I had concentrated on running the bot for chat platforms on mobile devices for Appium Conf along with the test strategy and Botium box(UI component of Botium), in this talk I will concentrate on the Botium integration with Selenium Wedbriver and the grid setup with Botium with sample code.

       

      Thanks,

      Shama Ugale


  • Liked Simon Stewart
    keyboard_arrow_down

    Simon Stewart - Selenium: State of the Union

    45 Mins
    Keynote
    Intermediate

    What's the latest in the world of Selenium since the last SeleniumConf? In this talk, we'll cover Selenium 4, how the project is run, how you can participate, reminiscing on the Old Days, and reflect on something that at first doesn't appear to be related to Selenium, yet is intimately related to it.

  • Liked Christian Bromann
    keyboard_arrow_down

    Christian Bromann - The Nuts and Bolts of WebdriverIO

    Christian Bromann
    Christian Bromann
    Software Engineer
    Sauce Labs
    schedule 1 month ago
    Sold Out!
    90 Mins
    Workshop
    Beginner

    There are thousands ways if not more to setup your automation testing environment. It is often crucial when it comes to stability and maintainability. While you can build a custom framework that fits your needs there are often already sophisticated setups or frameworks available that get you up and running very quickly.

    WebdriverIO is one of these frameworks that is written in Node.js. It gets you up and running within minutes and allows you to scale up your test suite while maintaining your execution time by running tests in parallel. With its huge project community it is an ideal choice for small as well as big projects that use modern frameworks such as React, Angular or Vue.js.

    In this workshop you will learn everything you need to know to run a successful, stable and maintainable WebdriverIO framework. It starts with an introduction to the project and the basic concepts and will end with a sophisticated framework that includes testing strategies like Frontend Performance Testing as well as complex browser interaction with Puppeteer.

  • Liked swapna narayanan
    keyboard_arrow_down

    swapna narayanan - How to build a simple automation tool for your organization

    45 Mins
    Case Study
    Intermediate

    Ever wondered how to develop a simple automation tool using selenium ?

    Create a simple automation tool for your organization in no time. This presentation demonstrates multiple ideas to build a script less automation tool from scratch in most simple ways through multiple architectural designs and diagrams.

    Why we need a tool on top of selenium ?

    Selenium is a powerful automation tool to automate any application. It can be clubbed with Appium, Jmeter etc for additional coverage.

    Functional , Web ,Mobile and Performance testing coverage can be managed this way.

    Problem statement:

    We need people with some coding skills to write the test cases and execute them in selenium / Appium. When it comes to building scalable and reusable frameworks, we need automation experts.

    I have seen a lot of teams from my e-commerce, Insurance and banking domain backgrounds with great domain expertise and no or minimal coding exposure .They are keen to explore and utilize the power of automation for the redundant regression tasks.

    Solution :

    A script-less automation tool will fill the gap in the above scenario.

    For any off the shelf tool there is a high cost involved.

    If the tool is built internally , comparatively very less cost is involved.

    More over, many additional functionalities like meaningful reports, AI metrics, NLP capabilities, integration with Test management tools etc can be integrated according to the requirements.

    The scope of this session starts here. This session will inspire you and gives you many ideas on creating tools for your need to get maximum ROI and build the gap between core domain experts and automation testing.

    Beneficiaries :

    Product owners , business analysts , functional experts who can start writing test cases on a daily basis , leveraging all the opportunities that selenium offers to the fullest, without actually writing a single line of code.

    What to expect from this session ?

    I will walk you through my experiences and the knowledge we gained as a team while we worked on building multiple automation tools.

    Different architectural diagrams that can be handy when you actually start to think and implement an automation tool project will be discussed.

    Ideas on different implementation capabilities using different Front end technologies and database designs will be discussed.

    Demo of an internal tool developed will complete the session

  • Liked Irfan Ahmad
    keyboard_arrow_down

    Irfan Ahmad - From Test Pyramid to Test Cup : Evolution of Modern Testing Stack

    Irfan Ahmad
    Irfan Ahmad
    Release Manager
    upGrad
    schedule 1 month ago
    Sold Out!
    45 Mins
    Case Study
    Beginner

    We all have read or heard about test automation pyramid .But is it really supposed to be a pyramid ? or it's just a metaphor? lets find out

    At this talk we will share our learnings from implementing test automation pyramid for years and its variations finally arriving to test automation cup modal focusing on

    1.Need for growing up from automation testing pyramid

    2.How the modern testing stack has evolved

    3.How to choose and design your automation testing stack

    4.Real World examples from testing to validate our test cup model

    5.How to move from Pyramid to test cup and Our lessons learned

    6.Sharing Best practices ,limitations and patterns for success

  • Liked Khyati Sehgal
    keyboard_arrow_down

    Khyati Sehgal / Shubhechchha Rai - Expanding the aura of automation, when it comes to mobile.

    45 Mins
    Demonstration
    Intermediate

    This talk will answer the questions "Will you be able to run your automation scripts without connecting the USB cable to your device in a limited range?"

    If we have to ask ourselves where is the mobile and when is it required? So the answer would be its 'Everywhere'!!

    It's next to possible to imagine a day without looking at mobile or using apps for that matter- Google, Facebook, Whatsapp, etc.

    When it comes to using it so testing comes hand-in-hand with it. As we develop applications over a mobile device, be it any platform ios, Andriod and windows we shall also think about usability and end-to-end testing.

    In this talk, we will discuss how Appium worked as a bridge between devices and controllers.

  • Liked Shama Ugale
    keyboard_arrow_down

    Shama Ugale - Refactoring - a key ingredient of evolution of your automation framework

    Shama Ugale
    Shama Ugale
    Sr. QA Consultant
    ThoughtWorks
    schedule 1 month ago
    Sold Out!
    45 Mins
    Case Study
    Beginner

    Ever happened to you that your UI gets migrated to the angular stack while you have all the automated tests and infra in place in selenium Webdriver and Java framework. When you run your tests they are mostly flaky and and drain your efforts in fixing/false negatives?

    Is the solution to this problem finding a new tool that fits well for angular UI such as protractor? Or to spend some time refactoring your existing framework to support angular UI and leverage the existing tests and infra in place?

    In this talk we shall discuss the later solution and the implementation.

  • Liked Shama Ugale
    keyboard_arrow_down

    Shama Ugale - On-demand Distributed test infra with Selenium Grid and Terraform

    Shama Ugale
    Shama Ugale
    Sr. QA Consultant
    ThoughtWorks
    schedule 1 month ago
    Sold Out!
    45 Mins
    Demonstration
    Beginner

    With the fast paced world it’s very important to make the decisions faster and that can be possible only if you have a good test coverage and a faster feedback.
    One can use selenium grid on the local systems and achieve the cross browser and device testing which may have constraints with respect to scalability and maintainability, or use SaaS alternatives such as BrowserStack/Saucelabs which fit well for lesser no of tests (gets a bit expensive as we get into concurrent sessions and increase the browser/device combinations) or moreover containerize the automation having the limitations with coverage of IE. How to design a solution to get the a scalable, on-demand, manageable and yet cost effective infra? Selenium Grid along with Terraform and AWS can be leveraged to write the whole infrastructure as code to prevent any human errors and manual intervention. In this talk, I shall take you through the journey of reducing the test execution pain-points in terms of time, resource availability, conflict handling, scalability and state management with a demonstration.