Selenium DeTox for Achieving the Right Testing Pyramid
Our project was a classic example of Selenium gone wild! As our team embraced the test automation journey, we went crazy and implemented tons of Selenium tests, one for every permutation possible. Soon we realized our feedback cycles were delayed. Our builds were taking hours instead of minutes. And we had a set of complex, fragile tests, which resulted in a lot of false-negative scenarios and finger pointing.
At this point, our team had realized that this is not the path forward. We decided to seriously look at our Selenium tests. We pretty much moved 80% of our Se tests to lower-layers (non-GUI based) tests. And now we have the right testing pyramid on our project.
Join us, as we explain our journey (strategy, techniques, tools, mindset-change and approaches we took) through this transition.
Outline/Structure of the Case Study
- 5 Mins - Quick Context Setting
- 10 Mins - State of the Project with too many Se Test
- 10 Mins - Testing Pyramid
- 15 Mins - Our Transition
- 5 Mins - Q & A
Learning Outcome
Key take away from this session, includes:
- Problems with relying too much on Selenium (or any GUI driven) tests
- Importance of having the right Testing Pyramid
- Useful techniques for achieving the right test-pyramid balance
Target Audience
Teaming implementing (or planning to implement) UI based Test Automation
Video
schedule Submitted 9 years ago
People who liked this proposal, also liked:
-
keyboard_arrow_down
Dave Haeffner - How To Find Information On Your Own
30 Mins
Talk
Intermediate
Whether you're just starting out, or are well on your way, there are more resources to dig through than you have time or know what to do with. With blogs, videos, documentation, forums, meetups, conferences, books, and mailing lists, the signal to noise ratio is all out of whack. In 30 minutes, I'll be your tour guide as I walk you through the information landscape of Selenium and show you which resources are worthwhile for your context and how to find them.
-
keyboard_arrow_down
Ankita Gupta / Anamika - Hacker-proof your app using Functional Tests
30 Mins
Talk
Intermediate
Many Functional Testing/QA Engineers don't have insights into Security vulnerabilities. Usually an enterprise has a separate security testing team solely for that task and functional testers have to rely on them for the security audit.Security is an important part of Testing but not every build of the application is tested for security issues. All Functional testing teams have a load of automated test cases which are run on every build of the application but they don't check for security flaws.In this talk we would be showing how you can use your existing test cases and automatically perform security testing on your web application. This is made possible using IronWASP, an open source security scanner and its companion libraries.If you are a software tester or developer even without any security expertise this talk will help you secure your web application better using your existing functional test cases. -
keyboard_arrow_down
Anand Bagmar - Perils of Page-Object Pattern
45 Mins
Experience Report
Advanced
Page-Object pattern is very commonly used when implementing Automation frameworks. However, as the scale of the framework grows, there is a limitation on how much reusability really happens. It inherently becomes very difficult to separate the test intent from the business domain.
I want to talk about this problem, and the solution I have been using - Business Layer - Page - Object pattern, which has helped me keep my code DRY.
-
keyboard_arrow_down
Artem - Allure framework - crystal clear reports for your selenium tests [in any language]
30 Mins
Talk
Beginner
It's pretty cool to have selenium test results clear to everyone on the team. There are lots of frameworks in every language that aim to simplify test writing. But only few can provide sharp presentation of test execution output. Yandex team is working on Allure (https://github.com/allure
-framework/allure-core/wi ki) - an open-source framework designed to create crystal clear reports. Because of module structure it integrates easily with almost any testing tool, no matter which language you use. In my talk i will describe the basic principles and show how to integrate Allure reporting in your existing projects. -
keyboard_arrow_down
- - Testing the Web Platform with WebDriver
45 Mins
Talk
Intermediate
As the WWW morphed from a system of interlinked hypertext documents to an advanced application delivery platform, the requirements on the underlining technologies and the browsers implementing them changed dramatically.
To ensure correctness in behaviour and that application code works across multiple browsers, standards come equipped with test suites. But interoperability can only be guaranteed by a more thorough testing effort. The W3C has launched a new project called Test the Web Forward to bridge this gap, and WebDriver sits at its centrepiece.
The talk will focus on the infrastructure that is being created to run these tests, and exemplify how the WebDriver library can be utilized in a complex and untraditional test running framework. It will also give a glance into the ongoing W3C WebDriver specification work, and where we are headed. -
keyboard_arrow_down
derrick - Design Patterns beyond the Page Object: An investigation into the design patterns used while building page objects.
60 Mins
Demonstration
Intermediate
In an age where the Page Object Pattern and Page Factory Pattern dominate web testing conversations, there is still a need to understand and apply the design patterns of yesteryear. Ideas from the Facade Pattern, Factory Method Pattern, the Iterator Pattern, the Object Pool Pattern, and the Decorator Pattern all find their way into the Page Objects we build to represent the increasingly complex widgets found on today's websites.
In this presentation, we take it back to the old school, looking at novel ways to apply classic design patterns, like those developed by the Gang of Four and Code Complete, to new screen scraping problems. We will investigate three common scenarios where using the typical approach to page objects can be inefficient or difficult, including filling in a web form, iterating over data in a list, and traversing iframes to communicate with widgets. We will explore how to improve upon the naive approach to building these page objects through the use of classic design patterns. Finally, we will formalize our findings into new patterns which can be applied to more general scenarios.
Code examples will be presented in Python and based off of my work building automation tools for hubzero.org, a platform for scientific collaboration.
-
keyboard_arrow_down
Igor Khrol - Increase Selenium tests stability via JavaScript
30 Mins
Talk
Intermediate
UI-tests are not really stable. Some sync point might be missed and tests will be red from time to time without any obvious reason. Or accidentally some focus might go away and button will not be clicked. These and other cases make automated testing results unpredictable and these results are not trusted.
In my speech I want to share the experience how to reach reliable and reproducible results with Selenium tests. In order to reach it we should sacrifice to 100% end user emulation. The presentation is based on the real project where this idea was successful. Also more common recommendations will be given publicity.
-
keyboard_arrow_down
Dhimil Gosalia - Running Selenium tests on a cloud of real mobile devices
60 Mins
Demonstration
Intermediate
Many challenging aspects need to be considered when running your Selenium test suite on real mobile browsers: choosing the right Selenium library, emulators vs. real devices, 24x7 local device infrastructure availability, unsupported features, pop-ups, etc. Do the benefits outweigh the effort?
We first talk about how to test effectively on mobile browsers using Selenium. The problems we have faced with mobile emulators, and why we are scrapping them in favour of a real mobile devices cloud of Selenium nodes. Then this is followed by a demo on how to execute your tests on iOS and Android devices. Learn how to set up a real mobile devices cloud from scratch; when we will also discuss how to run health checks for your cloud, and troubleshoot properly.
Our expertise comes from building a real iOS and Android device cloud for our customers, looking to get accurate results from testing - something to be had only from actual devices and not just emulators.
-
keyboard_arrow_down
Mayur - Clojure for functional testing of Mobile and Web apps
30 Mins
Demonstration
Intermediate
Writing tests for modern websites is quite tricky. And systems that have device-browser interaction, challenge test engineers even more.
In this talk I'll show how we use Clojure to solve problems of functional and integration testing for mobile and web applications, at Helpshift.
We use wrappers and libraries written in Clojure to leverage well known tools such as Calabash for Android, Appium for iOS, and Selenium WebDriver for the web.
Clojure is a powerful modern Lisp. In this talk, I hope to show how we benefit from many of its features, including fast, interactive REPL oriented development, rich set of data structures, and the power of macros.
-
keyboard_arrow_down
dima kovalenko - Scaling and managing Selenium Grid
45 Mins
Talk
Intermediate
Managing the Selenium Grid can be very difficult, especially as you scale it up. This session will demonstrate how we were able to scale the Selenium Grid with multiple operating systems, in multiple data centers across many continents. With the use of Selenium Grid Extras, and open source project, we were able to get much better control of individual nodes, manage WebDriver versions and much more.
-
keyboard_arrow_down
Oren Rubin - Page Objects Done Right
60 Mins
Talk
Intermediate
Slides: http://www.slideshare.net/orenrubin/page-objects-presentation-selenium-conference-2014-38767492
In this talk I will walk the audience step by step at building tests using the Page Object Design Pattern, making several attempts until we reach the current recommendation. We'll see the dos, don'ts, and common pitfalls.
In this presentation I'll also cover the Page-Factory Design Pattern, and best practices for dealing with asynchronously and how to remove the deadly "random sleeps".
-
keyboard_arrow_down
Tarun Lalwani - Case Study - QTP/UFT to Selenium Migration - 80% reduced execution time
45 Mins
Case Study
Beginner
QTP/UFT has been one of the leading Test Automation tool in the market. QTP supports a wide variety of technologies and with the recent article from Telerik - "5 hidden costs of Selenium". Is it really worth migrating to Selenium? If you think No, then think again. We recently migrated a client from QTP to Selenium, and the results was a 80% savings in execution time using one single machine. This case study will share the challenges we faced initially and how we managed a framework with high re-usability and execution
-
keyboard_arrow_down
Syed Khaja Habeebuddin - There is more treasure in Selenium nodes than in all the pirate's loot on Treasure Island.
30 Mins
Talk
Intermediate
There is so much valuable data that Selenium Nodes writes into log file(s) when tests get executed on them. So much of this data is not mined for actionable intelligence.
In this session, we will show you how operating system level metrics can be combined with data from Selenium nodes to result in rich actionable intelligence that will tremendously aid in the analysis of test failures, react to test execution flakiness, and to improve your automated test resiliency.
-
keyboard_arrow_down
Vinay Voruganti - There are three things that matter in cross-browser test automation- locator, locator, locator – But how do you find the right one?
30 Mins
Talk
Intermediate
Many a time the same locator in your automated script does not work well at all in all browsers/versions and operating systems or works but with flaky results in some browsers/versions. And then numerous hours are spent using a trial and error approach to either continue the hunt for a different locator that works across the board or build “if-browser/version-use-this-locator” kind of logic in your test scripts.
In this session, we will show you an approach that will help you eliminate so much pain and cost from your cross browser test automation.
-
keyboard_arrow_down
Prasanna Kanagasabai / Ketan Soni - Testing "Injection" Attacks with Selenium
Prasanna KanagasabaiThat Security Guy ....ThoughtWorksKetan SoniLead ConsultantThoguhtWorksschedule 8 years ago
60 Mins
Demonstration
Intermediate
Business applications are growing at a break neck speed to cater to ever increasing business need. The dream of ever-connected systems and information at fingertips is quickly becoming a fact. This dream has brought out an evolution of online-real time applications with multiple requirements and functionalities. The down side to this security is being forced to take a back seat. Add to this the sheer quantum of code to cover is overwhelming to a manual security tester.
One of the most common attacks against web applications is injection attack; injection flaw allows a malicious user to send malicious input to an application. The consequences of having injection flaw in your application can range from a user be able to steal all the data from your database to extreme situation like he having a command access to your infrastructure. We in this session want to show the power of automation using selenium. We will demonstrate how we are writing some interesting scripts to automate the testing of injection attacks in web applications. The outcome of automation is that we have been able get a fair code coverage and gives the time to security tester to concentrate on more tests that need his manual expertise say business logic failure or a design failure.
Though selenium could be used to automate far larger scope but we choose Injections as a priority for these sessions as injections form a large part of the web application attack landscape. We intend to give you some of the learning’s we had in the past, and some pitfalls we noticed. One could take the same idea and extrapolate to other attacks too.Attacks we plan to cover
1. SQL Injection
2. Command Injection
3. XSS
-
keyboard_arrow_down
Puneet Kala - Selenium and Joomla Open Source
45 Mins
Experience Report
Beginner
In an era of highly interactive and responsive software processes where many organizations are using some form of methodology, test automation is frequently becoming a requirement for software projects.
Many organizations are using Selenium-IDE and RC for testing softwares, In this talk we are going to talk about How we can use Selenium-webdriver to setup a Testing Suite for an application.
We at Joomla!, have our testing suite built in Selenium-Webdriver along with PHPUnit and are using a Page-Object pattern to increase the code maintainability. We are going to talk about how one can setup a test suite and built our own reporting tool with Graphical representation of test results using the logs.
-
keyboard_arrow_down
Isaac Murchie - Selenium in the palm of your hand: Appium and automated mobile testing
45 Mins
Talk
Intermediate
The mobile world is growing, and it has never been easier to test applications—native, web, or hybrid—using Appium, a free, open-source implementation of the Selenium protocol for automating mobile devices. I will show how Appium, which is in active development and recently shipped version 1.0, allows developers and testers to harness all the power of the native testing frameworks provided by the mobile device manufacturers, but within the context familiar to Selenium users.
Further, Appium provides a very good opportunity for looking at the challenges for a technology designed to automate web browsers as it moves into a new realm. I will discuss how functionality was repurposed for this new context, as well as the ways in which the protocol was extended and made more flexible in light of the the expanded use case.
As a case study in extended functionality, I will discuss the gesture API, which allows a standardized way to automate the sorts of advanced mutli-finger gestures used on modern mobile devices.
-
keyboard_arrow_down
Sreedevi Vedula / Ramalingam S - WebDriver and Cucumber in the JavaScript Land!
Sreedevi VedulaQuality AnalystThoughtWorksRamalingam SQuality AnalystThoughtWorksschedule 8 years ago
90 Mins
Demonstration
Intermediate
WebDriverJS and Cucumber.js are new entrants in the WebDriver eco-system and are gaining popularity by the day! These JavaScript ports for WebDriver and Cucumber enable us to test the JavaScript UI apps built using frameworks like Angular JS, Ember.JS with great ease.
The UI tests can be written in JavaScript, thereby using the same technical stack of the application and the tests seamlessly integrate with the code for continuous integration and continuous delivery.
The session is a demonstration of test framework using Cucumber.js and WebDriverJS for testing an open-source Angular JS application.
-
keyboard_arrow_down
Naresha K . - Better Selenium Tests with Geb
60 Mins
Tutorial
Beginner
Selenium tests can quickly grow in size and could become difficult to maintain, unless adhered to DRY principles to the extreme. Geb is built on top of Selenium WebDriver, which brings in the expressive power of groovy to WebDriver API. Geb can be used for all browser automation tasks that can be achieved with WebDriver. This talk will show how Geb can help us to write concise tests that are highly readable.
Prior knowledge of Groovy is NOT essential to get started with Geb, however would be helpful in advanced use cases.
-
keyboard_arrow_down
Sushant Choudhary - How we trained our dragon built on enterprise practices to adopt and fail fast with OSS.
30 Mins
Talk
Intermediate
Traditionally companies built over enterprise infrastructure and practices have been hesistant to adopt new practices,framework or tools especially one which can change with a single commit on github. So was the case when mobile portfolio of one of the largest airline was developed and tested in an agile way and striving for continous delivery.
This is a story of remote-enablement and developing a sense of confidence in the drgaon to go open source and base its complete testing infrastructure on OSS.Today their Mobile testing with its own challenges posed by H/W,OS and Design guidelines is under the OSS umbrella with practices like BDD and compliance with test pyramid adopted with a scalable automation framework built on Ruby,Cucumber,Watir Webdriver and Calabash.