Best Practices to implement the test automation framework starting from Design - To -> Infrastructure - To -> Execution.

Best Practices - How to get the best 'Return ON Investment' (ROI) from your Test Automation.

This has been observed that, most of the test frameworks wont be able to survive due to lack of expertise, no maintenance, no best practices being followed, and finally your test automation will be dead after few months, and there is no "Return ON Investment" from this. This is the most common problem, most of the companies are struggling and finally back to square to the Manual testing.

My proposal : HOW to leverage your test automation in terms of best practices, best ROI, and how to adopt best automation culture in your organisation.
I strongly propose some of the important points/suggestions to achieve this in your Organisation/Team.
1. Test Automation Practices:
  • Design Patterns (Web/Mobile/API)
  • What to Automate/Not to Automate
2. Common Automation Frameworks at Org Level:
  • How to design Generic Utilities, Libraries and different Components, which can be suitable for all the teams in the same Org.
  • Best practices to design your Tests (Automation).
    • Common Design Patterns
    • Common application level and Page libraries
    • Best Practices to use Assertions in your Tests (How and What to write for assertions). Most of the people don't write proper assertions and this is making your test unreliable and no defects found during execution.
3. Inclusion of API/Backend libraries in your UI test automation as an external Maven/Gradle Dependencies to avoid un-necessary tasks, some of the important points to be considered here:
  • User Creation from APIs (No need to automate user creation from web/app for all the test cases)
  • API tests are stabled most of the time
  • API calls takes lesser time as compared to web, hence include API calls in your UI/App framework to save time.
  • Less flaky test

4. Best Code Review Process (Do not merge your code into Master without proper Code Review)

  • Implement PR (PULL Request) Process
  • Static Code Analysis using SonarQube, Cobertura, JACOCO etc..
  • Get the benefits of Best Test Automation Quality Matrices
  • Sometimes, Manual (Functional Tester) should review your code (Assertion, test steps and use cases) to get the best coverage
5. Quality is A Team responsibility:
  • Developers, POs, Manual QEs and Automation engineers should be included to get an overview of test automation coverage.
6. Maintenance of the Frameworks
  • After couple of months, it should not make your life miserable if you don't maintain your libraries and framework properly.
  • Do not use Hard Coded values, make it simple and Generic.
7. Infrastructure Setup for Test Design and Test Execution:
  • Proper Browser - OS lab setup
  • Proper Mobile Labs setup with different Devices - IOT, iOS, Android, iPad, Tablets
  • Proper CI - CD common configuration using Jenkins, Dev Ops, AWS, Docker and Cloud setup
  • Handling multiple Docker nodes using Kubernates (use of Selenoid, GRID on Cloud)
 
 

Outline/Structure of the Talk

I'll be presenting some slides along with some sample codes and architectural diagrams.

  • How to leverage your Test Automation, get the best 'Return Of Investment' from your Test Automation:

My proposal : HOW leverage your test automation in terms of best practices, best ROI, best practices and how to adopt best automation culture in your organisation.

Outline/Structure of the presentation:

1. Introduction about me

2. Agenda

3. Problem Statement with some real time problems

4. Best Test Automation Practices - Code Snippets

5. Common Automation Frameworks with Design and Infrastructure setup diagrams

6. Inclusion of API/Backend libraries in your UI test automation as an external Maven/Gradle Dependencies to avoid un-necessary tasks, some of the important points to be considered here:

7. Best Code Review (PR)Process (Do not merge your code into Master without proper Code Review)

8. Maintenance of the Frameworks

9. Infrastructure Setup for Test Design and Test Execution

10. Q&A

Learning Outcome

I'll try my best to cover by showing some coding snippets, design diagrams, infrastructure setup with some real time problems that we face in automation world.

You will learn following important things from this presentation/talk:

The main focus of this talk to adopt best practices in terms of designing, coding, implementation, what to automate and not to automate, learning for manual testers, and the most importantly Automation Suite is finding the bugs during regression cycle.

This will justify the importance of automation in your organization and the best return on investment from test automation.

Selenium is an amazing tool, which purely justifies the concept of WEB Automation. If mix the flavour of API (Services), this combination can make a big noise and management is getting the confidence on the same to invest more n more on the same.

You will also learn basic cloud infrastructure setup for test automation like : Selenoid, GRID, AWS, Google Cloud etc..

Target Audience

1. Who wants to switch their career from Manual to Automation 2. Automation QA engineers 3. SDETs, 4. Developers

Prerequisites for Attendees

  1. You should have a basic knowledge on Selenium or any other automation tools
  2. You should know at least how Selenium works. basics of APIs
  3. Just attend this session with a pen and papers to write down some important points on best automation practices.
schedule Submitted 1 month ago

Public Feedback

comment Suggest improvements to the Speaker
  • Anupam Das
    By Anupam Das  ~  1 month ago
    reply Reply

    UI, API , Mobile lab set up for IOS and Android  & Proper CI - CD common configuration with Jenkins and Docker , these are most important functional test automation which you are going to cover.

    It would be a wonderful session Naveen .

    • Naveen Khunteta
      By Naveen Khunteta  ~  3 weeks ago
      reply Reply

      Thanks Anupam,

      will surely try to deliver as much as possible in terms of infra setup with Docker- Jenkins AWS/Azure cloud. 

      really appreciate your input. 

      thx!

  • Harish Bhat
    By Harish Bhat  ~  3 weeks ago
    reply Reply

    I like these aspects of the submission, and they should be retained.

  • Zainab Igwe
    By Zainab Igwe  ~  1 month ago
    reply Reply

    I like these aspects of the submission, and they should be retained:

    • All topics mentioned by him is quite important for every QA to be aware of.
    • Naveen Khunteta
      By Naveen Khunteta  ~  3 weeks ago
      reply Reply

      Hi Zainab,

      Thanks for considering this topic. Let;s see if its shortlisted :).

  • Akhilesh Kumar
    By Akhilesh Kumar  ~  1 month ago
    reply Reply

    I like these aspects of the submission, and they should be retained:

    • All the 8 aspects. All are essential tenets for making automation hugely successful..
    • Naveen Khunteta
      By Naveen Khunteta  ~  3 weeks ago
      reply Reply

      Thanks Akhilesh for considering this topic. Let;s see if its shortlisted :).

  • shantanu jain
    By shantanu jain  ~  1 month ago
    reply Reply

    I like these aspects of the submission, and they should be retained:

    • All the points are very well planned and are very useful for automation folks. 
    • Given the background of "NaveenAutomationLabs", this is definitely gonna be a "Best and most useful talk" of the conference.

    All the best Naveen. See you at the conference!!!

    • Naveen Khunteta
      By Naveen Khunteta  ~  3 weeks ago
      reply Reply

      Thanks Shantanu for considering this topic. Let;s see if its shortlisted :).

  • Pallavi R Sharma
    By Pallavi R Sharma  ~  1 month ago
    reply Reply

    Hi Naveen

    I could see the modifications, and refinements in the proposal. I can see there is a lot you want to talk about. The word "BEST" in here is something which is a bit well bothering. When we say something is Best, do we mean to say it is proven best, and the only approach to framework design over Selenium to provide best ROI? 

    I believe the implementation of this framework approach is with Java as a lanuguage, is that right? So the design patterns and practices taught will they be valid for any programming language used by audience and they would get "BEST ROI" for their selenium automation implementation? 

    and this "Selenium is an amazing tool, which purely justifies the concept of WEB Automation. If mix the flavour of API (Services), this combination can make a big noise and management is getting the confidence on the same to invest more n more on the same.

    You will also learn basic cloud infrastructure setup for test automation like : Selenoid, GRID, AWS, Google Cloud etc"

    so we want to convey that reduce UI tests, cover them at service layer, and have a grid implemented to run your tests on cloud rather than maintaining your own test lab to reduce cost? something like that? 

    - i am just wondering how does your apporach is the "BEST" out there considering so many implementations of selenium are available each claiming they are the best. 

    -can you please help us understand this Naveen. And back this up with help of data points. 

    Thanks

    Pallavi. 

    • Naveen Khunteta
      By Naveen Khunteta  ~  1 month ago
      reply Reply

      Hi Pallavi,

      Thanks for valuable comments once again. Please find my inline comments. I'll try my best to justify through my comments. :)

       

      I could see the modifications, and refinements in the proposal. I can see there is a lot you want to talk about. The word "BEST" in here is something which is a bit well bothering. When we say something is Best, do we mean to say it is proven best, and the only approach to framework design over Selenium to provide best ROI? 

      [Naveen]: The word "Best" is not for the proposal submitted by me. I am using word "Best practices" or "Standard practices" for the design, coding, test-assertions and infrastructure setup, which can surely give you the best ROI on automation. It's purely based out of using the right thing. 

      I believe the implementation of this framework approach is with Java as a lanuguage, is that right? So the design patterns and practices taught will they be valid for any programming language used by audience and they would get "BEST ROI" for their selenium automation implementation? 

      [Naveen]: You got it right, but just to mention these practices are applicable for all different types of languages (not only for Java). In every prog. language, you need to write the right code, right assertion, similar kind of setup for execution. GRID - Selenoid (Cloud setup) will remain same. So here focus is not for Java, but to give an idea how to implement the right things. I have seen with many companies and teams throughout my career, automation is dead after 2-3 months and we are back to square to manual testing again. But if we design the right things with Selenium with the right implementation of other tools, it will give you correct results with BEST ROI.

      and this "Selenium is an amazing tool, which purely justifies the concept of WEB Automation. If mix the flavour of API (Services), this combination can make a big noise and management is getting the confidence on the same to invest more n more on the same.

      You will also learn basic cloud infrastructure setup for test automation like : Selenoid, GRID, AWS, Google Cloud etc"

      so we want to convey that reduce UI tests, cover them at service layer, and have a grid implemented to run your tests on cloud rather than maintaining your own test lab to reduce cost? something like that? 

      - i am just wondering how does your apporach is the "BEST" out there considering so many implementations of selenium are available each claiming they are the best. 

      [Naveen]: Once again just to correct here, I am not saying that this is the best approach. I'm educating people to use the right things to get the BEST ROI. In my proposal, I have not mentioned at all -  about using my approach is the best one. 

      -can you please help us understand this Naveen. And back this up with help of data points. 

      [Naveen]: I am really glad to see experts comments here. Would love to share if you need more information on the same. 

       

      Thanks

      Naveen

  • Pallavi R Sharma
    By Pallavi R Sharma  ~  1 month ago
    reply Reply

    Hello Naveen

    An excellent detailed proposal, i have a few suggestions for improvement-

    a. Rephrase your title, as you i believe are talking in general about test automation ROI, where you are covering all aspects from service layer to ui to ci/cd so it would present a more clearer picture.

    b. i feel 45 minute session will not do justice to your talk. You should look for an increased time slot, because i believe it would attract- a huge crowd - and a lot of queries from audience. 

    c. The Learning Outcome doesn't matches with the topic, abstract of the proposal and rest of it. I would request you to redefine it please. 

    d. Lets refine it more, and structure it to make it more precise and true to its heading. I look forward to see more on it. 

    Thanks. 

    • Pooja Shah
      By Pooja Shah  ~  1 month ago
      reply Reply

      Hi Naveen, 

      I sense a very good blend of takeaways in this, thanks for proposing. However

      1. Could you please let us know when the nice improvements suggested by Pallavi are made. Short and crisp is the key to get readers and quick essence and self push to attend, specially the title.

      2. Do you also plan to have it very technical by showing the real world problem and code snippets to get most out of it?

      • Naveen Khunteta
        By Naveen Khunteta  ~  1 month ago
        reply Reply

        Hi @Pooja Shah, 

        Thanks for highlighting this, as this is my first proposal for Selenium Conf. 

        As per suggestions given by you, Pallavi and other folks, I have modified the title, learning outcome and have given the some important bullet points for each section in my proposal.

        The main idea behind this talk/presentation to educate other automation folks to implement the right things, and avoid the bad practices.
        I have already started collecting data and preparing the content for this talk, in terms of some common real time challenges, real time automation issues, basic automation lab setup for execution, how to implement real time PR (Code Review) processes with the less budget (cost) of infrastructure setup for execution and reporting.

        Also, I would present actual coding snippets, design diagrams (easy to understand), infrastructure setup, less theory but more practical real time exposure to the audience. 

        I hope this will surely help audience to invest on AUTOMATION. Kindly let me know if you need more information to add.

        Best Regards

        Naveen Khunteta

    • Amit Jain
      By Amit Jain  ~  1 month ago
      reply Reply

      I also agree with Pallavi's comments. Proposal should be rephrased as said. Its almost impossible to deliver such huge e2e test cycle including ROI analysis in just 45 minutes of time.
      Also, learning outcome can be more appealing and matching with topics to be covered during presentation. Thanks

      • Naveen Khunteta
        By Naveen Khunteta  ~  1 month ago
        reply Reply

        Thanks Amit, I have modified the title and made it more crisp and clear as per your suggestions. 

        Regards

        Naveen

    • Naveen Khunteta
      By Naveen Khunteta  ~  1 month ago
      reply Reply

      Hi Pallavi,

      Thanks for your valuable suggestions. Really appreciate that.

      I have refined this proposal as per points given by you. Modified the title, and learning outcome.

      I'll try to cover as much as possible in 40-45 mins, as I have already prepared the content on the same. I would definitely try to resolve the queries from audience as well as per my knowledge.

      Once again thanks! See you in conf in Bangalore.

      Best Regards

      Naveen

  • Prateek Saxena
    By Prateek Saxena  ~  1 month ago
    reply Reply

    I like these aspects of the submission, and they should be retained:

    • Everything mentioned above as this provides clarity and clear cut details.

    I think the submission could be improved by:

    • CICD is bit of more detail as there is a saying "if your automation is not running on CI you are not doing it correctly"
    • Naveen Khunteta
      By Naveen Khunteta  ~  1 month ago
      reply Reply

      Hi Prateek,

      Yes, this point is the important part of the proposal. I'll surely cover best practices to use CI CD setup.

  • Sandesh Savratkar
    By Sandesh Savratkar  ~  1 month ago
    reply Reply

    I like these aspects of the submission, and they should be retained:

    • You have covered all the small aspects while writing and I think that is most important and challenging criteria. I would surely recommend this topic.

  • Shital Pashankar
    By Shital Pashankar  ~  1 month ago
    reply Reply

    I like these aspects of the submission, and they should be retained:

    • As every organization should follow this type of approach

    I think the submission could be improved by:

    • ...
  • SATYA DUSANAPUDI
    By SATYA DUSANAPUDI  ~  1 month ago
    reply Reply

    I strongly recommend your idea Naveen.

    Sharing best practices will definitely improve organization's productivity and performance...

  • Curiosity Dose
    By Curiosity Dose  ~  1 month ago
    reply Reply

    This is very important topic. Naveen has great contribution for QA community. 

  • Anuradha Sharma
    By Anuradha Sharma  ~  1 month ago
    reply Reply

    I like these aspects of the submission, and they should be retained:

    • Waiting for this topic to be delivered by Naveen. He is quite experienced and having a big name in QA automation community. i would surely recommend this topic.

    I think the submission could be improved by:

    • you have almost covered all the points in your proposal. 
    • Naveen Khunteta
      By Naveen Khunteta  ~  1 month ago
      reply Reply

      Thanks Anuradha, I'll try to give my best for the QA community. 

      Thanks for considering this topic.

      ~Regards

      Naveen

      • Saurav Purohit
        By Saurav Purohit  ~  1 month ago
        reply Reply

        This is the most important challenge industry is facing today i.e how to make the stable tests and get ROI .This would be one of the important discussion . looking forward to you session

  • Shubham Gupta
    By Shubham Gupta  ~  1 month ago
    reply Reply

    Great work Naveen !!. Wish this could be held in Delhi

  • Santosh Maharana
    By Santosh Maharana  ~  1 month ago
    reply Reply

    Thanks Naveen for your proposal. Every QA must know these. 

  • Pavan Kumar K J
    By Pavan Kumar K J  ~  1 month ago
    reply Reply

    I really like these aspects of the submission, and they should be retained:

    • I strongly feel, this is the amazing  proposal for the conference with very good topics that are currently we are looking to improve on. 
    • All topics mentioned by Naveen will really really help Test Automation Engineers to enhance their Skills as well as what can be improved by us in the form of enhancing frameworks, writing quality code, creating utilities/functions, learning new tools and technologies that are currently trending in the Market etc etc. 
    • I am very sure #Naveen will give very good outcome on mentioned topics in conference.  

    I Strongly recommend Naveen to present all these topics in Conference. 

    I wish you all the very very best #Naveen

    #NaveenAutomationLabs 

  • Nikhil Pawar
    By Nikhil Pawar  ~  1 month ago
    reply Reply

    I like these aspects of the submission, and they should be retained:

    all of the above topics are a must for an any software Organization.

  • Asha Raj
    By Asha Raj  ~  1 month ago
    reply Reply

    looking forward for the suggested topics.

  • Swetha Bhimireddy
    By Swetha Bhimireddy  ~  1 month ago
    reply Reply

    Must needed topic for any project in quality perspective and I am sure Naveen can do it with ease.

  • Shiv Narayan Singh
    By Shiv Narayan Singh  ~  1 month ago
    reply Reply

    This is going to be the best conference. I am looking forward to it.

  • Shobhit ...
    By Shobhit ...  ~  1 month ago
    reply Reply

    This is going to be the best session on automation ever done and I can bet for that.

    Already I can see one point in the description like using rest API for login apart from using selenium, this will flaky tests and increase speed of automation......

    • Naveen Khunteta
      By Naveen Khunteta  ~  1 month ago
      reply Reply

      Thanks for showing your interest. Really appreciate. 

      Regards

      Naveen

    • negogo G
      By negogo G  ~  1 month ago
      reply Reply

      I really liked the topics chosen, and i am looking forward to it.

      Naveen you are doing a brilliant job for the testing society.

      All the best!

    • Ruchit Sharma
      By Ruchit Sharma  ~  1 month ago
      reply Reply

      I like these aspects of the submission, and they should be retained:

      • Really need this presentation for the whole QA community. And I think only Naveen can deliver this topic in the best way. Team, please consider this topic for Selenium Conf 2020.

      I think the submission could be improved by:

      • ...
      • Naveen Khunteta
        By Naveen Khunteta  ~  1 month ago
        reply Reply

        Thanks Ruchit for considering my name for this topic.

        ~Regards

        Naveen

        • Nazneen Shaikh
          By Nazneen Shaikh  ~  1 month ago
          reply Reply

          Something I really appreciate about Naveen is his aptitude for problem solving in a proactive way... 

          I really liked Naveen's  approach. Very supportive and relaxed.  His approach benefited me in terms of confidence. A relaxed and empowering course – very encouraging with a lot of practical work.

           

    • Hari prasad Komeravelli
      By Hari prasad Komeravelli  ~  1 month ago
      reply Reply

      I like these aspects of the submission, and they should be retained:

      • ...

      I think the submission could be improved by:

      • ...
    • Bishnu Das
      By Bishnu Das  ~  1 month ago
      reply Reply

      I like these aspects of the submission, and they should be implemented as soon as possible. Good work brother. Cheers

    • Hardik Antala
      By Hardik Antala  ~  1 month ago
      reply Reply

      Leveraging Test Automation is really important to transform the small industries.

      This submission could be really helpful.

    • Rahul  Chetry
      By Rahul Chetry  ~  1 month ago
      reply Reply

      This is the best proposal for the conference.lots of QA people will help from this conference .Great work Naveen Sir

    • Pankaja Lokesh
      By Pankaja Lokesh  ~  1 month ago
      reply Reply

      I like these aspects of the submission, and they should be retained:

      This article is helpful for QA Community both fresher experienced folks to understand and automate  real time projects  ( Web/ Mobile / API ) .

      • It will be great to recommend to present this topic in the conference.
    • Richard Raphael Michael
      By Richard Raphael Michael  ~  1 month ago
      reply Reply

      I like these aspects of the submission, and they should be retained:

      • All of the above topics are a must for an Organization. 

      I think the submission could be improved by:

      • How to balance between Manual and Automation?
      • If the project is still following water fall model how to automate modules in less time span before releases?
      • more insights on CD CI
      • Naveen Khunteta
        By Naveen Khunteta  ~  1 month ago
        reply Reply

        Thanks for your comment, Richard.

        I'll surely try to add more insights on points given for improvements.

         

        Regards

        Naveen

    • Zohair Khan
      By Zohair Khan  ~  1 month ago
      reply Reply

      I like these aspects of the submission, and they should be retained:

    • Sumit Chawla
      By Sumit Chawla  ~  1 month ago
      reply Reply

      I like these aspects of the submission, and they should be retained:

      • I believe this is going to be very informative session for QA community. Thanks Naveen for taking this initiative.
    • SHARDHA SHARMA
      By SHARDHA SHARMA  ~  1 month ago
      reply Reply

      I like these aspects of the submission, and they should be retained:

      • I would like to join this session , so that I will get much more clarity on selenium framework and I can implement in my office...

      I think the submission could be improved by:

      • ...
    • Priti Raz
      By Priti Raz  ~  1 month ago
      reply Reply

      I would love to watch his awesome videos which will clarify your doubts regarding Selenium.. So i hope and wish that he must get this opportunity 

    • Avinash Reddy
      By Avinash Reddy  ~  1 month ago
      reply Reply

      I like these aspects of the submission, and they should be retained.

    • Ramanathan Nagarajan
      By Ramanathan Nagarajan  ~  1 month ago
      reply Reply

      I like these aspects of the submission, and they should be retained:

      • ...

      I think the submission could be improved by:

      • ...reusability in long run as compared to manual testing shud be focal... 
    • Subhashree Mohapatra
      By Subhashree Mohapatra  ~  1 month ago
      reply Reply

      I like these aspects of the submission, and they should be retained:

      • ...This is very informative for the QA community ,I think this should be considered for selenium conf 2020.

      I think the submission could be improved by:

      • ...
      • Naveen Khunteta
        By Naveen Khunteta  ~  1 month ago
        reply Reply

        Thanks Subhashree, thanks for your recommendation and support.

        Regards

        Naveen

    • Ankit Choudhary
      By Ankit Choudhary  ~  1 month ago
      reply Reply

      I like these aspects of the submission, and they should be retained:

      • ...

      I think the submission could be improved by:

      • ...
    • Arif Shaikh
      By Arif Shaikh  ~  1 month ago
      reply Reply

      I like these aspects of the submission, and they should be retained:

      • ...

      I think the submission could be improved by:

      • ...
    • Aditya R Javali
      By Aditya R Javali  ~  1 month ago
      reply Reply

      This is the best proposal for the conference .These days most of the people don't understand the automation process.It will be helpful for the freshers and the experienced folks.It will be great to recommend to present this topic in the conference 

      kudos naveen 

      • Naveen Khunteta
        By Naveen Khunteta  ~  1 month ago
        reply Reply

        Thanks @Aditya, for understanding the value of this topic. Really appreciate.

        Regards

        Naveen

    • Vital Kumar
      By Vital Kumar  ~  1 month ago
      reply Reply

      I like these aspects of the submission, 

      I'm sure that, it would be a great learning. I'm really much curious.

      Thanks for come up with this topic.

      Thanks,

      VitalKumar U

    • Gaurav Burhade
      By Gaurav Burhade  ~  1 month ago
      reply Reply

      I like these aspects of the submission, and they should be retained:

      • ...

      I think the submission could be improved by:

      • ...
      • Naveen Khunteta
        By Naveen Khunteta  ~  1 month ago
        reply Reply

        Thanks Gaurav, for your recommendation for this topic. I appreciate that.

        Regards

        Naveen

    • Nidhi Choudhary
      By Nidhi Choudhary  ~  1 month ago
      reply Reply

      I like these aspects of the submission, and they should be retained.

      • Naveen Khunteta
        By Naveen Khunteta  ~  1 month ago
        reply Reply

        Thanks @Nidhi, for your recommendation and support. 

        Regards

        Naveen

    • Arun Chettur
      By Arun Chettur  ~  1 month ago
      reply Reply

      Very useful article. 

      • Naveen Khunteta
        By Naveen Khunteta  ~  1 month ago
        reply Reply

        Thanks for your recommendation, Arun!

        Regards

        Naveen


    • Liked Shweta Sharma
      keyboard_arrow_down

      Shweta Sharma / Nikita Jain - Accessibility testing 101

      45 Mins
      Talk
      Beginner

      "This world is such a beautiful place to live in." If you can read the first sentence without any screen readers or assistance, you're privileged. As technologists, shouldn’t we be more empathetic towards differently-abled people and make all parts of our website accessible to them? In my humble opinion, the true power of technology can be identified when it reaches out to people of all kinds having different physical or psychological challenges. We not only legally bound to provide Accessibility but also it should be considered as our moral responsibility.

      As testers, we have a wonderful opportunity to contribute to Accessibility by ensuring that the site is accessible in many different ways. Although it is impossible to identify all the issues that exist around Accessibility in the world, we are lucky enough to still understand a majority of them. With this understanding, there have been many measures taken in order to make your site accessible. But, don’t forget - we are QA engineers. We got to ensure that the site is accessible as per the standards set by WCAG 2.0 (AA) by testing for accessibility using various tools and techniques.

    • Liked Krishnan Mahadevan
      keyboard_arrow_down

      Krishnan Mahadevan - My experiments with Grid

      45 Mins
      Tutorial
      Intermediate

      Everyone starts off with a simple grid setup which involves a hub and one or more nodes.

      This traditional setup is a good start but the moment one starts to get serious with the selenium grid and decide to house their own selenium grid for their local executions, that is when issues start.

      My experiences with the Selenium grid in the past couple of years has led me to get introduced some of the most prevalent problems with maintaining an in-house selenium grid.

      • Nodes get unhooked randomly due to network glitches.
      • Nodes introduce false failures due to memory leaks.
      • Selenium Grid running out of capacity.
      • Nodes require OS upgrades/patches etc.
      • Needing to deal with auto upgrades by browsers (especially chrome and firefox)

      Some of these issues I managed to fix by building a "Self Healing" Grid wherein the nodes automatically get restarted after they have serviced "n" tests. But that still didn’t solve many of these other problems.

      That was when I felt, what if there was an on-demand selenium grid.

      What if the Grid could do the following ?

      • The Grid auto scales itself in terms of the nodes based on the current load.
      • The Grid does not require a lot of infrastructure to support it.
      • The Grid can plug itself into some of the cloud providers or leverage a solution such as Docker so that the nodes can be spun and shutdown at will.

      That was how the idea of "Just Ask" an on-demand grid was born.

      Just-Ask is an on-demand grid. It has no nodes attached to it.

      It’s designed to spin off nodes on demand, run test against the newly spun off node and after test runs to completion, clean-up the node as well. The node can be backed by anything. It could be Docker (or) it could be a VM running on any of the popular clouds.

      The session aspires to walk the audience through with my experiments with the selenium grid, my learnings on the selenium grid internals and how I used all of that knowledge to build my own On Demand Selenium Grid. What better avenue to share these learnings than a Selenium Conference.

      The session will introduce the audience to the grid internals and their concepts such as

      • What is a Selenium Remote Proxy ? What is it used for? What can you do with it?
      • What is a Hub (or) Node level Servlet ? When would you need one ?
      • All of this followed by a quick demo on "Just Ask", the on-demand grid that I have built and open sourced here: https://github.com/rationaleEmotions/just-ask

    • 20 Mins
      Talk
      Beginner

      In this talk, I will speak about the process and strategies on how can one analyze and select the right tools for test automation. We will look at some tools and processes that help to select the tools, frameworks, libraries, etc.

      I will demonstrate some helpful extensions and tools that will be very handy and useful during the process

      This process can be carried out before every test automation project or engagement. The selection of the right tools will help throughout the process of automation and will help to avoid delays, changes, re-work, incomplete automation, etc. and will save a lot of time, effort and resources for the organization.

      The session is designed for anyone in the Software Testing industry and the knowledge received from this session can be implemented right away in any of the upcoming or existing test automation projects.

    • Liked Vaibhav Singhal
      keyboard_arrow_down

      Vaibhav Singhal - Helping Test & Test Automation with AI-ML

      Vaibhav Singhal
      Vaibhav Singhal
      Test Lead
      ToTheNew
      schedule 1 month ago
      Sold Out!
      45 Mins
      Case Study
      Beginner

      "To implement or use AI (mostly creating a solution to help in testing) you don't need to be an expert or some certified data scientist etc. My talk revolves around how being a normal automation tester, we see some challenges and with limited knowledge, we start leveraging AI to help in our test. Talk create a mindset and case study how AI may help you in some of your day to day challenges in testing."

      The impact of AI has penetrated our lives and increasing daily. Related to AI-ML existence in Testing, two scenarios are:

      - Testing AI programs

      - AI helping Testing

      This talk is related to later aspects of "AI Helping Testing". There are a number of possible ways how it is impacting and many companies are already working on developing tools around same and many test solutions are already available in market.

      Whenever AI keywords are intercepted, the common image is of "autonomous car, robots" and then the question, how these cars and robots will help us in testing, rather the thought should be these the outputs of AI.

      AI/ML can be leveraged in the number of areas and scenarios to solve and help in our day to day testing activities. This talk would discuss about AI/ML, its impact, some existing solutions available, doing brainstorming ideas, so you can identify in your project. Also, USE CASE how we took AI benefit to solve our Automation problem.

      Use Case - Problem Statement

      1- Multiple automation suites running daily and sharing reports. Each report is having some failures. To do defect triaging for multiple failures is difficult.

      Solution: Displaying consolidated reports of actual and new failures suggested by the prediction model (qa analysis on failures was reduced by 80%) based on classification & Deep learning..

      2- Auto analysis while bug reporting directly to bug management tool.

      Solution: Model predicting the defect is already raised in bug management tool, as per the score it would take appropriate action to create new, update, no action, only notification to team etc.

      Talk includes below takeaways:

      1- Understanding how is AI/ML/Deep learning specifically in software testing.

      2- Brainstorming how to leverage AI to help in your tests.

      3- Initial steps to start for any model.

      4- Tools to leverage

    • Liked Diego Molina
      keyboard_arrow_down

      Diego Molina - Test Automation is not just coding

      Diego Molina
      Diego Molina
      Sr. Software Engineer
      Sauce Labs
      schedule 1 month ago
      Sold Out!
      45 Mins
      Talk
      Intermediate

      We learn more from our failures than our successes. I want to share one of my failure stories, where I learned that coding is not the most important task in Test Automation.

      A failure taught me that coding is not the most important task in Test Automation. I fixed a bug, added tests, and shipped it to production. After that, a bug report came in showing that the fix created a new bug. I felt bad, I did not understand how that happened. A hotfix solved the issue, but the root cause was not addressed: what did I miss?

      By taking a step back, I understood the situation, it all happened by overseeing basic concepts in testing (like understanding how the system works). The problem was that coding had more priority than creating a test plan. After this, I prioritized tasks better to avoid this situation to happen again.

      This talk shows that testing concepts are more important than ever, in a time where tools promise to do everything, we focus less on what and how to test, and more in using tools to test. It outlines how a test strategy can leverage a continuous testing setup. Finally, it shows that failing is ok, but failing again for the same reasons is not.

    • Liked Pallavi R Sharma
      keyboard_arrow_down

      Pallavi R Sharma - Learn Selenium with JavaScript

      Pallavi R Sharma
      Pallavi R Sharma
      Founder
      5 Elements Learning
      schedule 1 month ago
      Sold Out!
      480 Mins
      Workshop
      Beginner

      Selenium is a popular open source test automation tool, so popular it has its own conference event which run for 3 whole days across continents and is attended by hundreds of enthusiastic testers, and people wanting to know more about testing and automation. In my little experience of coaching for last 7 years, i feel what english language has done for common man[those from non english speaking countries] in earning bread and butter and giving them an elite status, selenium has done for manual testers.

      Anyone and everyone wants to learn Selenium. The magic of selenium or the flexibility is that it can be learnt with your choice of programming languages. I have taught people Selenium with Java, C#, Python and Ruby largely in past these years. I wish to delve into the world of Javascript now and see what this has to offer to us. So this workshop is for those open minded souls out there who wish to pick up another language and learn, unlearn, relearn selenium with it.

    • Liked Babu Narayanan Manickam
      keyboard_arrow_down

      Babu Narayanan Manickam - Deep Learning Based Selenium Test Failure-triage Classification Systems

      45 Mins
      Talk
      Intermediate

      Problem Statement:

      While running thousands of automated test scripts on every nightly test schedule, we see a mixed test result of pass and failures. The problem begins when there is a heap of failed tests, we caught in the test-automation trap: unable to complete the test-failure triage from a preceding automated test run before the next testable build was released.

      Deep Learning Model:

      The classification was achieved by introducing Machine Learning in the initial months and followed by Deep Learning algorithms into our Selenium, Appium automation tests. Our major classification was based on the failed test cases: Script, Data, Environment, and Application Under Test and that internally had hundreds of sub-classifications.

      To overcome this problem, we started to build and train an AI using Deep Learning, which simulates a human to categorize the test case result suite. Based on the test result failure for each test, the AI model would predict an outcome through API, categorizes and prioritize on the scale of 0 to 1. Based on the prediction model, the algorithm takes appropriate response actions on those test cases that are failed like re-run test again or run for different capabilities. We kick-started this by gathering the historical data set of 1.6 million records, which was collected over a 12 months period, data including the behavior of the test case execution and the resulting suite.

      This Deep Learning-based algorithm has been provided the quality to break down the new defects based on its category, and a classification score is given on a scale of 0-1. We’ve also established a cutoff threshold based on its accuracy of improving, and to group the failed test cases based on its similarity. Classification of the test cases is done in high granularity for sophisticated analysis, and our statistical report states that the classification of the defects has been increased with 87% accuracy over a year. The system has been built based on the feedback adapting models, where for each right classification it will be rewarded and for the wrong, a penalty is given. So whenever receiving a penalty the system will automatically enhance itself for the next execution.

      The algorithm has a powerful model for detecting false-positive test results calculated using the snapshot comparisons, test steps count, script execution time and the log messages. Also, the model has been built with other features like – duplicate failure detection, re-try algorithms and defect logging API, etc.

      The entire classification system has been packaged and deployed in the cloud where it can be used as a REST service. The application has been built with its own reinforcement learning where it uses the classification score to enhance itself and this is programmed to perform in an inconclusive range.

      In sum, this deep learning solution can assist all Selenium testers to classify their test results in no-time and can assist to take next steps automatically and allow team could focus its efforts on new test failures.

      Link: https://github.com/testleaf-software/reinforced-selenium-test-execution

    • Liked Sanjay Kumar
      keyboard_arrow_down

      Sanjay Kumar - ChroPath- Smarter way of writing automation script!!

      Sanjay Kumar
      Sanjay Kumar
      ChroPath Creator
      AutonomIQ
      schedule 3 months ago
      Sold Out!
      45 Mins
      Case Study
      Intermediate
      • Want to save 70-80% manual effort of automation script writing!!
      • Wasting time in verifying xpath one by one?
      • Want to complete automation script without wasting much time?
      • Are you still wasting time in writing english manual test cases?
    • Liked Srinivasu gangam
      keyboard_arrow_down

      Srinivasu gangam - Zero Touch Automation using NLP (Natural language processing) & AI

      Srinivasu gangam
      Srinivasu gangam
      Sr Manager
      Cotiviti India
      schedule 3 months ago
      Sold Out!
      45 Mins
      Demonstration
      Intermediate

      Problem Statement:

      As part of SDLC process:

      1. Is your product quality impacted due to a smaller number of QA resources available in the team?
      2. Are you waiting for QA resources to certify your code every time when you deploy? Is this impacting your product lead time (Speed to Market)?
      3. Is your Product delivery timelines are impacted due to last minute defects identified?
      4. Do you have your QA resources only in one location, but you want to “follow-the-sun” approach for Software delivery across multiple locations?
      5. Do you have manual testers who are not skilled in programming, but you want them to execute automated test scripts w/o any training efforts and automation setup?
      6. Would you like your team more agile and cross functional with Delivery?
      7. Would you like to increase your QA team’s productivity while they invest more time in script development rather than script execution?

      If answer is ‘Yes’ for above questions, "Zero touch automation" is the solution for above challenges that we have been facing part of SDLC.

      Solution: Zero touch automation with cutting-edge technologies

      In this session, I will cover how we solved this problem using innovative solutions, Cutting-edge technologies like NLP (Natural language processing), AI & Cloud solutions.

      You will learn how AI, NLP integrated with core automation components to achieve Zero touch automation.

      This solution is not just revolutionary, it is paradigm shift in test automation to get results to your email with detailed analysis of failure categorization with recommended actions to users.

      I will also cover how E2E automation will be driven with decisions taken by machines based on what user is looking for . There is no manual intervention in this process. NLP and AI play key role to help machines to take decisions.

      We will also cover how we empowered developer/release manager/any team member/Manager to trigger the scripts from their cell phone and get the detailed execution report without having any automation software installed in their computer or Phone.

      We will be demonstrating how the request will be initiated from User, understand the need from user using NLP & AI , Fetching the code from bitbucket to select appropriate automation scripts , running them on Selenoid/docker server , storing results to MongoDB , receiving email with test results and Failure analysis.

      What is the value of zero touch automation?

      1. Enable speed to market: Now that Developers does not need to wait for QA resource, Changes can be certified quickly and ready to push to production. Lead time will be significantly reduced.
      2. Increase quality: Now that test automation is easy and it can run multiple times in each environment, most of the defects will be uncovered and addressed before code goes to production.
      3. Ease of test execution: Test execution will be very easy, no automation or framework setup required from user side. Test execution can be done 24*7.
      4. Productivity: Increase QA team’s Productivity to focus more on script development rather than focusing on script execution and failure analysis .
    • Liked Tomasz Wojciechowski
      keyboard_arrow_down

      Tomasz Wojciechowski / Tomasz Wojtuń - How to run 2500 selenium tests in an hour.

      45 Mins
      Case Study
      Advanced

      If you are tired of waiting for your selenium tests status and old fasioned ways of reporting tests status, you are in a good place.

      We will show you our test environment setup and how we run over 2500 tests in about an hour. Not cool enough? Take a look how we make reports via slack.

      Do you want to check if we can be even more faster?

    • Liked Sameer Arora
      keyboard_arrow_down

      Sameer Arora - Triggering alerts on Slack along with screenshots in case of test case failure

      45 Mins
      Demonstration
      Beginner

      One of the biggest limitations with most of the frameworks in selenium is that we need to wait till the end to get the final test case report. Only then we can raise the alert that a particular functionality is broken. By then, it may be too late! In addition to this, in most of the frameworks, the HTML report generated is hosted on our company's private network and we need to connect to the VPN if we are outside the office to open the report and check the related screenshots to see which test case has failed and why.

      So, why not raise an alert as soon as the test case fails? And why not attach a screenshot along with that alert so that all the stakeholders can actually see what has failed in the application?

      In this session, we will learn a simple yet a really useful way of sending an alert along with the failed test case screenshot by integrating slack with selenium which will help us alert all the stakeholders as soon as an automated test case fails so that everyone can react to it immediately and the damage will be minimised.

    • Liked Varadharajan Srinivasan
      keyboard_arrow_down

      Varadharajan Srinivasan - How do we test billion data set in 1hour

      45 Mins
      Talk
      Intermediate

      We are now living in the age of Big Data. What makes Big Data so special is its ability to help the common man process petabytes of data to build innovative solutions. Cloud has played a pivotal role in building this data economy by making infrastructure more cost effective and enabling easy scalability.

      Today, the abundance of data helps business make intelligent decisions through business process automation. Data drives all our digital interactions, starting from products that we see on our Amazon home page, to our Netflix recommendations, to the advertisement on Google search results, to even the Friends that we get connected to on Facebook. All these systems are powered by data that is collected from the various system, cleaned, transformed and stored into Big Data solutions. Testing of these data systems have become very complex and requires a new approach. This paper highlights various aspects of Big Data Testing including:

      • Complexity in testing Big Data systems

      • Cloud based tools for testing Big Data cross various stages

      • Simplification of test processes and visualization through dashboards

      • Testing Big Data in the Agile/ DevOps

    • Liked Gaurav Singh
      keyboard_arrow_down

      Gaurav Singh - How to build an automation framework with selenium : patterns and practices

      Gaurav Singh
      Gaurav Singh
      Product Engineer
      Go-Jek
      schedule 3 months ago
      Sold Out!
      45 Mins
      Talk
      Beginner

      With an ever increasing no of businesses being conducted on web the testing need to write automated tests for the app's UI is something that can never be ignored. As you all know Selenium provides an API that enables us to do this quite effectively.

      However, when tasked with setting up the automation framework, there are a lot of questions that arise in the minds of aspiring test developers regardless of what level they are in their career.


      Some of such questions are:

      1. How does one actually go about the business of building a robust and effective automation framework on top of selenium?
      2. What are the elementary building blocks to include in the framework that an aspiring automation developer should know of?
      3. How should we model our tests? XUnit style vs BDD?
      4. Are there good practices, sensible design patterns and abstractions that we can follow in our code?
      5. What are some of the anti patterns/common mistakes we should avoid

      A lot of literature, documentation and blogs exists on these topics on the web already.

      However In this talk,

      I would combine this existing knowledge and my years of experience in building automation frameworks and breakdown these elements and walk you through exactly the sort of decisions/considerations and practices that you can take while starting to implement or improve the UI automation for your team.

      Hope to see you there!

    • Liked Khanh Do
      keyboard_arrow_down

      Khanh Do - Leveraging Artificial Intelligence to create self-healing tests

      Khanh Do
      Khanh Do
      QA Architect
      Kobiton
      schedule 3 months ago
      Sold Out!
      45 Mins
      Tutorial
      Intermediate

      A key requirement for successful test automation is to get past the brittle or fragile nature of test scripts. Any Selenium (or Appium) developer has encountered the dreaded "NoSuchElement Exception". A locator that worked yesterday may fail today. What's a test engineer to do?

      Fortunately the field of AI provides promising solutions and allows for the creation of self-healing tests. Tests that can find elements across all environments. Tests that can learn from "human-in-the-loop" intervention and work perfectly thereafter. Imagine automated tests that "just work"!

      This session will look at how to apply the latest in AI and Machine Learning technologies to improve your test scripts. With the plethora of new open source AI libraries made available by companies such as Google, the ability to leverage AI in your applications is more accessible than ever.

      This session will be a primer on AI technologies and how they can be utilized for perfect test automation.

    • Liked Srinivasu gangam
      keyboard_arrow_down

      Srinivasu gangam - Test case Generator with Optimized coverage - Orthogonal Array Testing Strategy (OATS)

      Srinivasu gangam
      Srinivasu gangam
      Sr Manager
      Cotiviti India
      schedule 2 months ago
      Sold Out!
      45 Mins
      Case Study
      Beginner
      1. Are you Confident about your test coverage?
      2. Do you have millions of permutations and combinations to be covered in testing?
      3. is automation the solution to cover million combinations?
      4. why can not my system decide what test cases to be executed to give complete test coverage?
      5. As part of CI/CD , instead of executing same test cases irrespective changes going in, would you like to execute more appropriate test cases ?

      I will be presenting quick demo on statistical approach and strategy which can help us to generate appropriate number of test cases automatically based on the impacted areas and business metrics . Let me share an example below.

      Let’s say we are testing any eCommerce application Flipcart or amazon. We will have to cover multiple combinations like different categories of items, different payment types, shipping types, Promotions, user types etc.. . I will use little bit of sets theory here.

      Let us take below 5 sets as an example. Assume that there is new payment type is getting introduced as “Paypal” as enhancement. If we need to make sure this new payment type is working with all products, all shipments types , all user types etc… we will have to cover all permutations and combinations which is nothing but Cartesian product of all these sets. The Cartesian product of below 5 sets is - 6*3*2*4*3 =432 test cases.

      Do we really need to execute 432 test cases just for one new payment type ? we just need to execute 24 test case to give confident test coverage .This can be done with OATS tool which is developed with intelligence with Business user Metrics + Orthogonal array testing approach ( OATS) approach . OATS tool will generate required number of test cases with simple input and one click .

      There are 2 key areas of input that we consider before we generate optimized tests.

      1. Business user metrics - Example : if you look at the below payment set , we would like to know what % of customers using each payment type in overall successful transactions. let us assume 60% Credit card payments , 20 % paytm and 20% rest of payment types . These metrics will help tool to give more weightage to Credit card test cases over other payment types. This make sense because we need to focus more on the areas our end users are using.

      2. Impacted areas: let us take an example that there is new payment type (AmazonPay) is getting introduced as en enhancement. We should give more weightage to payment set and make sure all existing payment types are tested with different combinations.

      Set1 - PAYMENT = { Cash, Credit card , Paytm , AmazonPay, Debit card}- # elements - 6

      Set2 – SHIPPING – {Shiptoday, Prime Shipping, ship Tomorrow}-# elements - 3

      Set3 – USER TYPE - { Guest User , Logged in user }-# elements - 2

      Set4 – PRODUCT TYPE – {Electronics, Groceries, Books , Stationary}- # elements - 4

      Set5 – PROMO TYPE – { Promo1 , Promo2 , Promo3}-# elements – 3

      I will be covering how this can be done in detail during presentation along with Tool Demo which was built in with an intelligence using sets theory . you will get an answer on why only 24 test cases are good enough instead of 432 test cases for the change new payment type got introduced.

      Advantages :

      1. Optimized coverage with less test cases

      2. Tester will be confident about test coverage

      3. This tool output can be integrated with Selenium key word driven framework . All this happens during run time w/o manual intervention.

      4. This strategy is good fit for automation testing .

      5. if any combinations are invalid to test, those combinations will be excluded while generating tests.

      6. There are tools in the market to provide statistics about untested code (Ex: JaCoCO ). These statistics can be used as input for OATS tool. We have not done this yet.

      Limitations:

      1. We have seen great value (reduced bug escape) applying this strategy to automated tests. However there is limitation in applying the same strategy to manual exploratory , Adhoc testing.

      2. We have not tried this strategy for performance testing yet.

      3. There are corner scenarios where human can only think will not be covered by this tool . These scenarios to be appended to generated test cases .