Q & A with the Selenium Committee

Selenium: State of the Union

This talk suggests a reframe in how we understand the business value of automated testing. One shift is to see automation as "checking" rather than "testing". Another is the shift from software delivery to service delivery, including fully embracing DevOps. The resulting approach could be called Checking as a Service or CheckOps, and forces us to rethink traditional automation priorities. In this talk, Bret will explain how change in approach has affected teams he's worked with and how you can use it to improve your ability to deliver valued services.

Have you ever wondered how Selenium works under the covers? Do you get frustrated with locators not locating, pages not loading, or browsers behaving inconsistently from one run to the next? Selenium is an attempt to unify thousands of disparate elements across a wide spectrum of challenges into a single, common interface that works seamlessly with all the major browsers - and yet only a handful of volunteers work to maintain this gigantic effort. If you would like to enhance your own Selenium experience while contributing back to the software that has defined so many of our careers, come to this workshop. In it we'll dissect the different elements of Selenium, dive into its internals, learn how it was built and how to make changes to it, and even write a unit test you can contribute on the same day!

Hike is used by 100 Million users and many of our users have cheap smart phone (~ $120 USD) that can install no more than 3 mobile apps.

So the questions is: Should testing of app be limited to its functionality? At Hike, we believe "Performance is Queen!" For our users, if we misuse the critical resources such as Battery, CPU, Network and Memory, its a deal-breaker. Hence pref-testing is very important.

During the initial days of Hike, we were very reactive and only did (manual) perf testing, when our users reported issues.

Now, every Sprint (2 weeks) and every public release (monthly), we run our automated perf tests. We measure our app's performance using several app specific use-cases on 4 key areas:

• CPU,
• Memory,
• Battery and
• Network (data consumption.)

We also benchmark the following scenarios for app latency:

• App launch time upon Force Stop
• App launch time upon Force Kill
• App's busiest screen openning time
• Scrolling latency in different parts of the app
• Contact loading time in Compose screen

We still have a long way to go in terms of our pref-testing journey at Hike. But we feel, we've some key learnings, which would be worth while to share with the community. Join us, for a fast paced perf-testing session.

This small talk is about the intricate fabric of discussion between an interviewer and an interviewee for the post of Automation Engineer.

The audience would be taken through a maze of silly, run of the mill, subjective, bizarre and occasionally brilliant questions/answers/discussions found in an average interview for Automation Engineer/Selenium Engineer. 

We'll try to look at the 4 quadrants of the Selenium Interview circle, covering it from the perspectives of :

1. The Interviewer
2. The Interviewee
3. Current Industry state
4. How can we improve?

This might seem like a soft, non-technical, not-so-important topic of discussion but we must remember that bad interviews lead to bad candidates lead to bad deliverables. As they say, For want of a nail the battle was lost. 

And in today's cut-throat world none of us can afford losing a good candidate to a bad interview.

Lets gather around a fire of questions, on the beach of answers to surgically comprehend the What, How and Why of Selenium Interviews.

Today, there are so many possibilities to automate our test, Can we use Selenium with others automation tools?
In this session ,I'll explain how we can use Selenium integrated with IBM Rational Functional Tester creating test scripts in a demo.

We see two upcoming trends in the world of software delivery.

1.Docker is becoming a standard for managing infrastructure using containers.

2.Testing code and its infrastructure starts to grow at scale with more complexity, dependencies and technology diversity.

A container is an entire portable runtime environment: an application, plus all its dependencies, libraries and other binaries, and configuration files needed to run it, bundled into one package. By containerizing the application platform and its dependencies ,all differences in OS distributions and underlying infrastructure are abstracted away which makes it easy to share and execute anywhere.

At this talk we will learn how to leverage the container technology to solve the challenges of growing testing infrastructure and continuous delivery with key focus on below items.

• Basics of the containers technology and specifically it’s application on the test automation. 
• How Docker can reduce the time of test execution, ease the setup of clean test environments and drastically reduce the differences between the development, acceptance and production environments leading to the higher quality of the released software.
• Examples to containerize entire testing stack together consisting of major automation tools (selenium, appium, phantomjs), performance tools (jmeter,gatling) with cucumber. 
• Integrating and managing testing container with other application containers to achieve easily manageable continuous delivery pipeline.
• Best practices and patterns for docker success.

These days we find most of the apps are being developed across different platform, iOS, android, windows and to keep the user base which uses web, mobile web and websites. 

When apps are being developed for cross domains, most of the functionality provided by the app is very similar, varying thing is PLATFORM. 

 In rapid development cycle, where there are tools which allows you to write once and reuse across multiple platforms, makes development very faster. 
But at the same time, if we have different automation suite for different platforms, it becomes very difficult to keep a pace with ongoing functionality. 
This is the exact problem we faced, and the solution we came up with is "One Page to test them all! -A cross platform mobile automation framework! "

Page Object Model

Well, Page Object Model was again a natural fit for this framework. Most implementations of POM recommend different POMs for each platform. But we wanted to have a single Page Object Model for all the 3 platforms to ensure maximum code reuse and reduce overall time spent in adding new automation.
 

Single Page Object Model across platforms

This was complicated because we had native screens as well as webview screens and so it was not possible to use the same Page Object. To solve this, we introduced abstractions for the elements on the screen and encapsulated the respective native driver implementations.

This also allowed us to implement common automation tasks in one place for e.g waiting for new pages to load, so that this code is not repeated across multiple step definitions and platforms. This helped us move to thinking in higher domain level concepts than in terms of low level UI interactions.

So, in summary, we write our tests for one platform and run them for all with an abstraction layer in place.

Problem Statement

Expected Result: Mobile is taking over the world and wow! my product works awesomely everywhere.

Actual Result:  OMG! it breaks on iOS 6 :-( 

Holy Jesus! did we also test on firefox version 30.0 on Windows machine ?? 

Having an application on all major platforms(Desktop Web, Mobile Web, Mobile Native apps etc.) brings a daunting requirement of verifying every single feature before giving a +1 for release and so it becomes essential for the QA folk to test and provide proper feedback as quickly as possible, which immediately takes the complete reliance only on manual testing out of the question and pushes for the need for automated testing with scalable automation framework embracing any product need in the future.

We surely have 5 points to be answered before we think about such solution :

1. Do we have a single test code which can test the product everywhere with a simple mechanism to trigger and manage them?
2. Where is the plan to reduce Time To market having so many tests running before each code push?
3. Do we have 1 click solution to monitor all the test results in one go to assert the state of ThumbsUp for release?
4. Is continuos integration in place?
5. How can I integrate all of the above 4 points using the same beautiful tool Selenium along with other aligned open-source projects like Appium, Shell and Jenkins?

In modern times we have many different cloud testing services to choose from. These cloud services are useful and help reduce the burden of building and maintaining your own Selenium Grid environment. However, there are many scenarios in which you need your tests running locally, such you work for a government (or agency) and cannot expose your data to the cloud, or the service costs are too expensive to run all tests on every commit.

This presentation will feature getting started with setting up your own mobile device grid. Running your tests in parallel and distributed, capturing critical report data (logs, screenshots, and video), and leveraging cloud test services (such as SauceLabs). I will also talk about the challenges and lessons I learned along the way.

Today over 100 million users share over 40 billion messages per month on Hike. Its not just simple 1:1 chat messages. Users can do a VoIP call or share rich multi-media content in 8 different languages in group chats with hundreds of members. User can transfer large (upto 100 MB) file using Wifi-Direct .i.e. device to device file transfer without using Internet. And many more features. How do you ensure that you can roll out a release every month without breaking any of these features?

With such a large user based, which is very sensitive to frequent upgrades due to data consumption cost, rigorously testing the app becomes extremely critical.

When we started our automation journey in 2014, we were looking for a device lab which can simplify our testing effort. However we gave up and ended up building our own setup. The reason being, we require multiple devices that can communicate with each other for a single test. And we have 6000+ such tests, which we want to run in parallel. While many device labs allow you to run tests in parallel, they don't allow the devices to communicate with each other. Also its not possible to run the same test across multiple devices. Imagine testing a group-chat flow with photo sharing or imagine the device to device file transfer using hotspot. How would you test these features?

If this interests you, join us and we'll share our learning trying to achieve this at Hike.

Analytics are an increasingly important capability of any large web site or application. When a user selects an option or clicks a button, dozens—if not hundreds—of behavior-defining “beacons” fire off into a black box of “big data” to be correlated with the usage patterns of thousands of other users. In the end, all these little data points form a constellation of information your organization will use to determine its course. But what if it doesn’t work? A misconfigured site option or an errant variable might seem insignificant, but if 10,000 users are firing 10,000 incorrect values concerning their click patterns, it suddenly becomes a problem for the QA department―a department which is often left out of conversations involving analytics.

Join Marcus Merrell to learn how analytics work, how to get involved early, and how to integrate analytics testing into the normal QA process, using Selenium and other open source tools, to prevent those misfires from slipping through.

Marathon is a open source test automation suite for Java Swing and Java/FX applications. Marathon provides Selenium/WebDriver bindings for executing test scripts agains Java application.

In this workshop we explore steps by which you can set up an environment for testing a Java/Swing application.

Whether it is Dallin Oaks quote that reads ““Desires dictate our priorities, priorities shape our choices, and choices determine our actions.” Or Mahatma Gandhi’s quote that goes “Action expresses priorities”, the message I want to bring to the audience is as simple as “It is all about prioritization at the end of the day”. While this applies to all of us in all disciplines, it is even more relevant for us as testers in today’s world of increasing task scope and complexity. Whether it be manual or automated testing, what test matrices to use, what test data to incorporate, what defects we want to pick to defend, what tests to automate, what tool to leverage for automation, the overall success of a quality effort, is based on smart choices that a tester makes. Smart choices are not possible without educated prioritization strategies. For example, if I were an attendee in the Selenium conference even deciding which track session to attend amongst the varied options I have, is a choice driven by an underlying prioritization technique. This proposal is aimed at discussing and presenting core prioritization strategies and mapping them with very specific software testing and test automation examples that the attendees can take back and apply in their day to day practices with immediate effect. One would be amazed to see the varied styles of prioritization that are available – being privy into them and understanding them with examples, will help the attendee choose which style of prioritization works best with their personality and leverage them right away.

In this talk we will delve into one the biggest challenges that Test Automation developers face, finding elements i.e. a robust test means finding the same element with high fidelity while the Application Under Test keeps changing.

We will categorise the methods, and show where developer fail, where machines (Record/Playback) fail, and suggest a new way for locating elements, and analyze the skill-set required to overcome those difficulties.

I know what you’re thinking: the creators of WinRunner, QTP/UFT are now embracing Selenium?

Ten years after Selenium came into existence as an open source alternative to Mercury Interactive, the perception and relationship between QTP and Selenium has morphed from competition to collaboration with complementary test automation frameworks.

Join Ori Bendet, HPE Inbound Product Manager for Functional Testing to discuss how HPE’s R&D uses Selenium and other open source tools. Understand the new roles and responsibilities of dev/test @HPE and how they fit into current team structure. Discover their lessons learned about how Selenium and open source has contributed to the success and maturity of HPE's own quality assurance and testing tools across the entire portfolio.

Push Notifications are the latest way of sending updates to our users. More and more Organisations are implementing Web Push Notifications along with emails and other notification systems.

So the Big Question that arises is "How do we automate them?"

We have come up with a library in various languages which can be integrated with your Automation suite and provide you everything you need about the notification triggered.

You can easily then trigger and verify the push notification sent and ship out the product without worries :)

One of the first things that you learn when you enter the quality assurance space is the famous triangle braked down to Unit test at the lower, on top of it Acceptance Tests based on API (in some places this layer is integration tests but the idea is the same) and at the top of the Pyramid we have the User Interface (UI) Tests. This Pyramid, in the last two decades was the main principle on how to approach testing activities (mainly automation).

In the last couple of year we are all taking part in the digital transformation that is taking place all over. Mobile Native applications as well as web applications take part in almost any activity that we are doing during the day, business are building their strategy on this channel and shifting resources and budgets to deliver applications maintained and expend their market share.

So what changed?

The users are no longer static, they are interacting with those apps while they are on the train on their way to the office, while waiting for a flight at the airport or drinking coffee while waiting for their next meeting – those “interactions” are done most of the time while they are on the go working with mobile device. In addition the application take advantage of the sensors that those devices provide in order to provide better user experience. The environment where our end users use our application has significant impact on the functionality and performance of our application which at the end of the day we call quality.

During the last year we developed a new approach for digital application testing – The “Hourglass” – This new approach expend the known Pyramid and update it to the digital era – The client side is richer and contains many components that impact the quality of application. It redefine the definition of coverage. At the top of the pyramid we add 2 additional triangles (the gives the hourglass shape) – The first one is devices – what devices should we test, how we should approach the changes that happens in the devise market. The second is the environment, the places that our end users will use and interact with the application. We leverage the automation investment and get the real digital coverage which will help to reach high quality applications.   

Automated visual testing is a major emerging trend in the dev / test community. In this talk you will learn what visual testing is and why it should be automated. We will take a deep dive into some of the technological challenges involved with visual test automation and show how modern tools address them. We will review available Selenium-based open-source and commercial visual testing tools, demo cutting edge technologies that enable running cross browser and cross device visual tests at large scale, and show how visual test automation fits in the development / deployment lifecycle.

If you don’t know what visual testing is, if you think that Sikuli is a visual test automation tool, if you are already automating your visual tests and want to learn more on what else is out there, if you are on your way to implement Continuous Deployment or just interested in seeing how cool image processing algorithms can be, this talk is for you!