Not Only Cars: “AI, Please Test My App”
Autonomous cars were a Scifi dream not 10 years ago. A computer driving a car? No way. But it did happen, and is happening. And if scientists do it for a complicated task such as driving, can they do it for automated regression testing? In this talk we explore what is being done in the field today, but also speculate about the future: we introduce the 6 levels of autonomous testing (that correspond to the 5 levels of autonomous driving), and try and figure out what kind of help current AI techniques can bring to automated testing.
Outline/Structure of the Talk
- How autonomous driving works
- Machine Learning and Deep Learning
- How AI works today to assist you in testing your app
- Where can AI take us in the near term
- The 6 levels of AI in automated testing
- Dreaming about the long-term
Learning Outcome
- How autonomous driving works (unrelated, but fun to know!)
- How AI works today to assist you in testing your app
- Where can AI take us in the near term
- Don't worry about AI - it is a tool, not something that will take away your job
Target Audience
Test Automation Developers
Links
TestCon Vilnius 2017 - Say No(de) to the Browser: Testing your Webapp Frontend Code Using Just Node: https://www.youtube.com/watch?v=H_2cMSuNdS8
ReactNext 2016 - An Ode to Node: Testing Frontend the Backend Way: https://www.youtube.com/watch?v=hRVD78I3Fo0
CodeMotion Amsterdam - How to Build a Microservices Infrastructure in 7 Days: https://www.youtube.com/watch?v=tHeqy3Ej990
schedule Submitted 3 years ago
People who liked this proposal, also liked:
-
keyboard_arrow_down
Srinivasan Sekar / Sai Krishna - Advanced Appium
Srinivasan SekarLead ConsultantThoughtWorksSai KrishnaLead ConsultantThoughtWorksschedule 3 years ago
480 Mins
Workshop
Intermediate
At this workshop, you'll learn about advanced concepts in Appium. We will learn how to write a single script for Android, iOS and mobile web apps. Once our tests are established, we will work on framework design and report. We will also have a detailed look at how can we automate various mobile gestures using TouchActions and mobile endpoints in Appium and running distributed tests in parallel on various emulators and simulators.
-
keyboard_arrow_down
Simon Stewart - Fix a Bug, Become a Committer
480 Mins
Workshop
Beginner
Have you ever wondered how Selenium works under the covers? Do you get frustrated with locators not locating, pages not loading, or browsers behaving inconsistently from one run to the next? Selenium is an attempt to unify thousands of disparate elements across a wide spectrum of challenges into a single, common interface that works seamlessly with all the major browsers - and yet only a handful of volunteers work to maintain this gigantic effort. If you would like to enhance your own Selenium experience while contributing back to the software that has defined so many of our careers, come to this workshop. In it we'll dissect the different elements of Selenium, dive into its internals, learn how it was built and how to make changes to it, and even write a unit test you can contribute on the same day!
-
keyboard_arrow_down
Srinivasan Sekar / Sai Krishna - Code Once Test Anywhere: On Demand Private Appium Device Cloud using ATD
Srinivasan SekarLead ConsultantThoughtWorksSai KrishnaLead ConsultantThoughtWorksschedule 2 years ago
45 Mins
Case Study
Beginner
Mobile Test Automation is increasingly becoming very important. Almost all web applications are responsive these days and it's very important to test how the application works across devices. The same is true with the native application as well. At the same time, the number of devices and the custom OS versions on devices are also vast. This means that it's harder for a tester to manually run the automated tests over a list of devices to get device coverage and quicker results over every feature development.
We came up with a solution of executing tests in distributed or parallel fashion across remote devices from anywhere in the network using Appium Test Distribution. Same framework is officially used by Appium members for Beta testing of Appium.
USP of ATD over other Market Solutions:
- Device Cloud:
- Setup Devices anywhere within a network, ATD executes remotely without Grid
- Never worry about device location in network.
- Plug and Play:
- Connect your android/iOS devices or Emulators/Simulators and just execute tests.
- Multiple TestRunner:
- TestNG and Cucumber
- Parallel Test Execution:
- Runs across all connected iOS, Android real devices and Simulators/Emulators
- Test Coverage:
- Parallel(Run entire suite across all devices, which gives device coverage)
- Distribute(Run tests across devices to get faster feedback).
- Device Management:
- Manage devices remotely using Device Manager.
- Reporting:
- Covers detailed crashes logs from Android and iOS.
- Appium Servers logs
- Screenshots on Failures and On Demand Video logs
- Reporting Trends for multiple builds
- Manual Access to Remote Devices - OpenSTF support
Who loves/uses ATD?
ThoughtWorks, CeX, Jio, TravelStart, M800, Reward Gateway and lot more.
- Device Cloud:
-
keyboard_arrow_down
Rajdeep varma - Android Application Backdoor via Appium
45 Mins
Demonstration
Advanced
Application Backdoor via Appium
There's a shift towards open-source mobile test automation tools happening today among developers and QAs. Whether it be Appium, Calabash or anything else: all are good, with some major limitations.
While a chosen tool may work well when you first start using it, things can quickly get out of hand with changing business requirements. We started using Calabash at Badoo when there was no Appium. Given the capability of Appium to drive the whole device, we started automation of new apps with Appium. However, we realized a powerful feature was missing in Appium for Android! : The ability to call Application code from automation code like Calabash Backdoors.
As Appium UiAutomator server is based on instrumentation, we modified it such that we could instrument our app under test. This gave us the power to access context of Application under test and invoke public methods of Activity using reflection APIs. We use these methods to setup app state, seed DB OR even enable/disable some client-side A/B tests. This makes our application more testable and our tests more predictable.
This talk is going to be about how I achieved the above solution and benefits of backdoors.
There will be a small demo and code!
-
keyboard_arrow_down
Kushma Thapa - Running Automation tests twice as faster: How we got rid of Selenium Grid dependency.
45 Mins
Talk
Intermediate
With the rapid scaling of our health care application under consideration, in terms of number of users and the functionality involved as well as migration from a legacy system, the need to increase the coverage of automated tests for both systems seemed vital. Also, running a large no of test suites on daily basis against multiple browsers was critical.
As a part of this transition, the existing popular concept of using a continuous integration server with a grid machine and individual node machines poised issues in terms of cost as well as performance. Increase of automated test led to the increase in demand of node machines distributed across different machines on the same/different network. While the network latency for a limited number of node machines could be ignored, it eventually caused a significant reduction in performance.
Being part of a large enterprise, changing the architecture was not a feasible solution. In this talk we will talk about how selenium grid was removed from the equation to achieve automated tests that ran twice as faster and how it can be integrated on enterprise level requirement.
-
keyboard_arrow_down
Anton Angelov - Infinite Improbability Testing- Execute All Tests in Parallel
45 Mins
Talk
Intermediate
100000 tests executed under half an hour - sounds like a myth? Well, we made it, and I'm going to tell you a story how we got there. Through surveys, we discovered that many companies desire to utilize the benefits from parallel tests execution to optimize their software development process. However, they struggle with the process. Lack of available tooling, documentation, tests data arrangement/deletion, handling E2E tests specifics like browsers, emulators, etc. We were one of this companies. Because of that, we created an open-source tool for the job. Throughout the presentation, you will find statistics where, depending on the type of the tests, the tests execution can speed up from 4- 40 times which makes the run of the 100000 tests possible for 27 minutes.
It will be shown how to utilize the tool, its various features and where/when it is appropriate to use it. Also, you will find example solutions to most of the common challenges in executing tests in parallel. We believe that in the near future the parallel tests execution will be a necessity, much like unit tests or continuous integration now. This will be one of the pillars for the companies to improve their competitiveness and effectiveness.
-
keyboard_arrow_down
Nalilnikanth Meesala / Srinivasan Sekar - New hope in Selenium Docker
Nalilnikanth MeesalaSr. Quality AnalystThoughtWorksSrinivasan SekarLead ConsultantThoughtWorksschedule 3 years ago
45 Mins
Demonstration
Advanced
Idea behind this talk is to describe how to create a short lived containers for each test and scale to large Selenium Cluster using Selenoid. What is standard Selenium architecture and why it is not suitable for big clusters, Also states issues in maintaining a large Docker Selenium Grid / Selenium Clusters. In this session we will also be talking about the challenges we faced in using official selenium docker images and how we scaled up regression execution time from 3:3 (3 hours to 3 mins). Followed by a demo of how Selenoid server solves the problem with docker. How it works, how to use inside big Selenium cluster, where else it could be used.
-
keyboard_arrow_down
Syam Sasi / Jerry Zhao - How to build a device lab in your office in 48 hours!
45 Mins
Talk
Intermediate
Continuous testing is an integral part of continuous delivery pipeline. When it comes to mobile application, the testing become increasingly complex.
As part of our internal quarter hackathon at Carousell, we developed an automation testing framework and device lab in 48 hours which has both parallel and distributed mode of running.
We will share about how you can set up a similar device lab in your organization and the best practices to be followed.
-
keyboard_arrow_down
Raj Thapa - 100% automated, customized and continuously integrated performance test with result analysis and reporting
45 Mins
Talk
Intermediate
While performance tests are common among web applications to test their speed, scalability and reliability, a systematic approach to conduct dynamic, automated and customizable automated tests along with fully automated results reporting and display can be regarded as a major challenge.
In this session, discussions on an approach to conduct about the aforementioned with zero manual intervention will be carried out. The result being a fully automated test strategy that can easily be outlined to Product owners and higher-level management.
The discussion will be focused on the overall process that initiates with the execution of performance tests using Continuous Integration Server whose results are extracted, logged and used for performance analysis, monitoring and baselining.
The results are reported in a custom dashboard comprising of features that compare every build with historical data, compare deviations and provide a concrete idea of standard performances regarding the response time of the server over a period. A separate aggregated high-level report is also sent automatically as an email to respective stakeholders.
-
keyboard_arrow_down
Haritha Hari - Docker in Testing
45 Mins
Demonstration
Intermediate
Docker can be used to containerize Selenium tests and can be made to run against applications in another container. These tests can be further run on Selenium grid without the need of physical machines or VMs using Docker Selenium. Multiple and different versions of browsers, headless mode etc can be run inside containers and can be used to run tests in parallel. All these can be achieved with minimal changes to the existing Selenium tests.
-
keyboard_arrow_down
Shivaling Sannalli - Unify service and UI layer Automation - BDD way
90 Mins
Workshop
Intermediate
Software delivery is switching towards business driven development. This needs capturing requirements as presented by clients and translating these requirements into development tasks and tests.
Clients want visibility on the quality of the requirements that are captured. Quality in the delivery of the project measured using the test pyramid — heavy unit tests, more integration and service layer tests, and fewer UI tests. Test pyramid is the ideal quality structure for Agile projects.
In this presentation/session we will focus on how we can combine both service layer tests and UI layer tests in a BDD(Behavior Driven Development) way of Automation. That will bring test coverage visibility to business. Clients themselves can run this suite without having to understand technicalities of the implementation. Test scenarios will exactly be the requirements captured by business as behaviors.
To accomplish BDD way of automation for both service layer automation and UI layer we will use following tools/libraries:
BDD Tool: Gauge or Cucumber (both are open source)
Service layer Automation : Unirest for JAVA
UI layer Automation : Selenium WebDriver for JAVA
-
keyboard_arrow_down
Siddharth Kulkarni - Dr Culture Shock - Or How I stopped worrying and embraced Org Culture
45 Mins
Talk
Intermediate
The org culture mantra is usually considered a silicon valley mumbo jumbo. Although many companies and teams rant about culture, very few in my opinion take it seriously. In this talk I would like to discuss the importance of Org culture and how it makes or breaks businesses and teams. I will lay out some key pointers that will help the influencers in the audience make decisions about their org or team culture. The talk will be in the context of culture in tech companies.
-
keyboard_arrow_down
Ruchika Rawat - Lean Test Approach - in Agile
20 Mins
Case Study
Beginner
Talk Description Automation provides an immense amount of value in preventing regressions and helping to deliver quality software. As an organization’s automation grows and grows, it requires continuous maintenance so that tests remain fast, reliable, and valuable. If not scaled efficiently, an organization’s automation suite will turn into a messy, uncontrollable beast. Having a lean test suite will help to combat this. In this session will present methods to keep automated test suites lean and mean, so they always provide quick and accurate feedback to the software delivery team. Using a few examples, she will discuss a wide range of ideas including evaluating a test's value, parallelizing tests, and producing consistent results. Session attendees will walk away with strategies and practices to scale their test automation over time in a highly efficient and maintainable way.
-
keyboard_arrow_down
Nalilnikanth Meesala / Prasad Mudedla - Continuous Security with Selenium tests
Nalilnikanth MeesalaSr. Quality AnalystThoughtWorksPrasad MudedlaQuality ConsultantThoughtWorksschedule 3 years ago
45 Mins
Demonstration
Intermediate
We have come to a place where we run our selenium tests on pipelines every day.
Now are these tests confined to test only the functionality of the application?
No, we can use this tests to do much more. Let's test security with those same tests.Turn your selenium tests in to Security tests, get security testing in to your CI pipelines just by tweaking a bit of the selenium integration suite. Find and report security issues that exists which can be caught way before paying off for it.
In this Demo we will introduce a framework that will help your selenium tests turn into security tests and run on pipelines.
With ZAP API + selenium and CI pipelines, find security issues on the application see a red or green security pipeline. Not just that fetch the reports and take necessary actions on every build.
The talk covers different ways that ZAP tests the web application for security(automated with selenium tests). We will see what all of them can be integrated with selenium and can make most out the suite.
As an add we will see how your API can be tested for security using ZAP API :)
Tools used:
1. Selenium Web driver framework.
2. ZAP framework integrated with selenium framework.
3. GO (for CI/CD)
4. Maven for building and fetching reports. -
keyboard_arrow_down
Martin Schneider / Syam Sasi - One framework to rule them all - a holistic approach to Web and Mobile testing
Martin SchneiderService Software Engineer 2CarousellSyam SasiSenior Software EngineerCarousellschedule 2 years ago
45 Mins
Demonstration
Intermediate
Targeting both mobile devices and desktop browsers has become a necessity for successful tech applications. Whether an existing web application triggers the development of native apps or a desktop version is introduced to supplement the smartphone experience, supporting both worlds is a reality for a majority of businesses.
To maximise user experience, an application should behave as identical as possible across all platforms. However, more often than not versions for different platforms are built by different teams with different designers, possibly across multiple sites (even countries, continents, timezones) each with their own tech-stack and development process. In such a set-up, using different tools for automated testing could come naturally and add to the challenges of building a single product.
We argue that a common approach to test automation is critical. Not only will it save time and development costs, it can also increase communication and improve collaboration between teams and be a great facilitator for setting the focus on the product as a whole rather than separate deliverables for each platform. In this sense, it can serve as an umbrella across all involved teams.
We will demonstrate a simple JAVA-based solution using Selenium, Appium and Cucumber to combine testing for different platforms (namely Android, iOS and Web) within the same framework. We will talk about best practices in designing an extendable and easy-to-use test framework and how we can facilitate cloud-based services to integrate our tests into a CI pipeline. -
keyboard_arrow_down
Jim Holmes - Five Key Tips To Write Better WebDriver Scripts
90 Mins
Workshop
Beginner
Anyone can learn to write WebDriver scripts to help with automation and testing. Even team members with no coding experience can learn to help contribute value to their teams, and learn in fairly short order!
This session is targeted to help team members write WebDriver code that's accurate, understandable, and maintainable. Learning these tips will help you and your teams write stable WebDriver scripts that won't require constant tweaking and maintenance.
Attendees will see practical demonstrations of concepts including the Page Object Pattern, good Locator strategy, dealing with asynchronous actions, and writing custom APIs (also called "backing APIs") to help facilitate better testing.
Additionally, we'll discuss why it's so important to team testers up with software developers in order to get the best out of testable systems.
Examples will be in C#, but Ruby and Java will also be discussed—or any other platforms the attendees ask about.
-
keyboard_arrow_down
Gil Tayar - Not Your Grandparent's JavaScript: Using ES.Next To Write Clean WebDriver Tests
45 Mins
Demonstration
Intermediate
The goto tool for functional testing has till now been grownup languages like Java or C#. But a funny thing happened in the last few years - JavaScript has grown up. The advent of ES6 (and now ES7 and 8) has turned JavaScript from a language that was not taken seriously into a beautiful and powerful language.
JavaScript should have been the goto language for testing web applications, given that it is the lingua franca of today’s web developers, but that has not happened. This talk aims to correct that, and show how the features in modern JavaScript affect automated tests written for Selenium WebDriver.
-
keyboard_arrow_down
Gil Tayar - Frontend Testing: Stepping in and Collaborating with Developers
45 Mins
Talk
Intermediate
Testing is shifting left, moving closer to testing the code itself, and not the full product. But while managers dictate a shift to the left, the developers and testers themselves are confused as to how exactly to test the code. While the backend world already has established methodologies for testing, frontend developers and testers are still trying to figure out how to test their code.
Testers need to step in and work with the frontend developers. Testers need to understand the framework by which frontend code is tested. They need to understand the various kinds of testing that can be performed on frontend code, and which tools can be used for this.
I will give a recipe that you can follow to ease your fear of the unknown —writing tests for developer code. I will discuss the various kinds of testing and how they fit together in a coherent way, but most importantly - I will include sample code that you can use as a template in your own project, and if time permits—a bit of live coding will be done.
-
keyboard_arrow_down
Manjyot Singh - QAOPS - QA Testing in a DevOps World
45 Mins
Demonstration
Intermediate
QAOps is Continuous Testing Strategies when
frequent software delivery matters.
It is no surprise that automation and orchestration
make life very easy, be it a small organization or
a large-scale industry that houses hundreds of
servers. IT automation is essentially the ability to
orchestrate and integrate tools,
people, and processes through a certain workflow. I will be using Ansible and Docker to showcase the same and writing ansible tests on top of that.Ansible - Ansible is an IT automation tool which can configure systems, deploy software, and orchestrate more advanced IT tasks such as continuous deployments. Ansible is actually designed to be a “fail-fast” and ordered system, therefore it makes it easy to embed testing directly in Ansible playbooks.
Docker - A docker container is a lightweight, stand-alone, executable package of a piece of software that includes everything needed to run it: code, runtime, system tools, system libraries, settings. Containers isolate software from its surroundings, for example differences between development and staging environments and help reduce conflicts between teams running different software on the same infrastructure.
Public Feedback