Visualizing Real User Experience Using Integrated Open Source Stack (Selenium + Jmeter + Appium + Visualization tools)

schedule Sep 10th 01:30 PM - 02:15 PM place Pavilion East/West

Traditional approach in performance testing does not include client side processing time (i.e. DOM Content Load, Page Render, JavaScript Execution, etc.) as part of response times, performance tests has always been conducted to stress the server so tools like Jmeter have been very popular to execute tests. With increasing complexity of architectures (Web, Browser, Mobile) on the client side it has been important to understand the real user experience.   Commercial tools have started to provide features that can provide insights into real user experience after the bytes are transferred to the client end.  With the ability to call Selenium scripts via Jmeter the ability to conduct real user experience tests using open source stack has opened up new avenues to comment on real user experience.   This enables us to comment on

  • Provides Page load times similar to On Load time of real browsers
  • Generates HAR file with following statistics
  • Details of summary of request times and content types
  • Waterfall chart with page download time breakdown statistics such as  DNS resolution time, Connection time, SSL handshaking time, Request send time, wait time and receive time.

By integrating the open source stack tools it enables us to provide the same insights which a commercial of the shelf tools would offer.   At Gallop we have implemented this at multiple clients providing them insights into various bottlenecks at the client side which helped us to provide greater value proposition

 
50 favorite thumb_down thumb_up 2 comments visibility_off  Remove from Watchlist visibility  Add to Watchlist
 

Outline/structure of the Session

1. Why Real user experience is important?

2. Client Side architecture: Deep Dive

3. Integrated open source test framework (Selenium, Jmeter, Appium+ Visualization)

4. How to analyze tests

5. Challenges and Learning from our Experience

Learning Outcome

1. Appreciate the need for calibrating real user experience

2. Tools that are available to paint the real user experience

3. Assembling an open source  integrated framework that can address across the technology stack of Web, Mobile app(Native) as well Mobile web sites 

4. Analyze test results

5. List of frequently encountered bottlenecks

Target Audience

Test automation Engineers, Architects, Performance testing engineers and architects

schedule Submitted 2 years ago

Comments Subscribe to Comments

comment Comment on this Proposal
  • Anand Bagmar
    By Anand Bagmar  ~  2 years ago
    reply Reply

    I am glad you are talking about client-side performance testing!

    That said, are you sure this is Real User Monitoring? That would be on real users devices, including the last-mile network, right? If I understand correctly, what you are talking about is capturing client-side performance leveraging Selenium scripts in real browsers, but not necessarily how the end-user would see it - which means this is synthetic monitoring / testing (https://en.wikipedia.org/wiki/Synthetic_monitoring), and not really real-user monitoring (https://en.wikipedia.org/wiki/Real_user_monitoring). 

    Lastly, how does a load-testing tool (jmeter) fit into this?

    • rajesh sarangapani
      By rajesh sarangapani  ~  2 years ago
      reply Reply

      Hi Anand, Over the last year or so at Gallop we have seen increasing instances of customers who want to understand the last mile performance testing under load (for various reasons such as e.g. real time network conditions 2G/3G/4G, signal strength).   If you look at a year or two ago we were just doing traditional jmeter type of tests ...just load the server with the required TPS and concurrency.   Then we moved on to understand what does it look like at the client side (probably using Selenium or appium).   Now we are running performance tests that is  a combination of load on the server (with jmeter) + client side tests which will you on load how the client side performance metrics are (this is the real user experience tests eventually).   In production scenario (real user monitoring or synthetic monitoring) we don't need jmeter because the server is getting loaded any which case by actual users.   But in the case of test bed we would need jmeter to load and when we run the test we would like to get an integrated view to the performance tests to make the analysis and deep dive into actual issues (via the waterfall chart)....we have utilised jmeter integrations to orchestrate the tests to achieve this...hope this clarifies the overaching thought process and the real life need of such a solution


  • Anand Bagmar
    Anand Bagmar
    Director - Quality
    Vuclip Inc.
    schedule 2 years ago
    Sold Out!
    45 mins
    Demonstration
    Intermediate

    Typically in organizations, there are multiple projects / products. These products may be of implemented using tech-stacks over many years. Yet - they interact with each other in some way. To manage the complexity around Test Automation, many organizations prefer to have a common Test Automation solution across these products in an effort to build, standardize and maintain the framework.

    However, this is not a good idea! With this approach one potentially ends up having to compromise on the quality of automation that can be done for each product, limited by the toolset.

    The better approach would be to use the tools and technologies that are "right" for each product. This does have other disadvantages, but you would ensure each product is well tested! The only missing piece which remains is that these different products talk with each other. You need to test the integration between them in an automated way to verify all is well.

    "TaaS" is an open-source product solution that allows you do achieve the "correct" way of doing integration testing across a variety of products via Test Automation.

    Example: 

    For one set of products, Selenium-based toolset may be the right choice, where as for legacy reasons, QTP may be used for some other product. With TaaS - you will be able to automate the Integration Testing between these products, by re-using the tests already implemented in the individual product suites.

     

  • moiz
    moiz
    Software Engineer
    Saucelabs
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Beginner

    Appium, often dubbed "Selenium for mobile", at heart its a web server written in NodeJs. Its architecture is modular, which means that it is composed of many small, independently maintained and tested modules. Testing Appium is challenging, but clearly very important, since thousands of users depend on it for their testing. Appium also has all the usual challenges of a large open source project, for example, ensuring consistency of JavaScript code style across hundreds of contributors. It's important to have high-quality and readable code.

     
    I will be discussing approaches to and strategies for testing these kinds of large, modular applications. On the Appium team, we use a combination of unit, functional, and integration tests. Modern services like GitHub, Travis CI, and Sauce Labs make it possible for large open source projects to be tested thoroughly, keeping the code and the app at high quality. I will also discuss the use of tools like JSLint and Gulp, which help prevent code style issues.
     
    Testing the tool which is used for testing is clearly very important. This talk aims to showcase how testing should be approached for large, modular projects which has many collaborators.
  • Liked Rémi
    keyboard_arrow_down

    Rémi - Mobile end to end testing at scale: stable, useful, easy. Pick three.

    Rémi
    Rémi
    Software engineer
    Facebook
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Beginner

    This talk is about how Facebook turned a great idea with a terrible track record into a great tool for thousands of developers.

    The promise of E2E testing — complex, real-world test scenarios from the point of view of and end user — is appealing.
    Many attempts have been made over the years at automating large parts of companies' and developers' testing and release processes, yet most of these efforts ended up in bitter and hard learned lessons about the inherent challenges of the whole approach.

    My work at Facebook over the last two years has been making mobile end to end testing at scale a reality.
    When others said it couldn't be done, or fell by the wayside, we relentlessly pushed forward, solving problems deemed intractable, and finding new, untold vistas of horror before us

    We've come a long way: E2E testing is now an integral part of Facebook's mobile development and release process.
    We'll cover what challenges we faced, and how we chose to solve or make them irrelevant.

  • Liked Jonathan Lipps
    keyboard_arrow_down

    Jonathan Lipps - The Mobile JSON Wire Protocol

    45 mins
    Talk
    Intermediate

    The JSON Wire Protocol (JSONWP) is the version of the WebDriver spec currently implemented by all the Selenium clients. It defines an HTTP API that models the basic objects of web automation---sessions, elements, etc... The JSON Wire Protocol is the magic that powers Selenium's client/server architecture, enables services like Selenium Grid or Sauce Labs to work, and gives you the ability to write your test scripts in any language.

    The JSONWP has served Selenium faithfully for a number of years, but the future of automated testing lies beyond the borders of the web browser. Mobile automation is an essential ingredient in any build, and tools like Appium or Selendroid have made it possible to run tests against mobile apps using the JSONWP. The JSONWP's current incarnation isn't enough to automate all the new behaviors that mobile apps support, however. Complex gestures, multiple device orientations, airplane mode, and the ability to use both native and web contexts, for example, are all essential to mobile automation.

    For this reason the leaders of the Selenium project, in concert with other Selenium-based projects like Appium and Selendroid, met to discuss the future of the JSONWP. We've been working on its next version, called the "Mobile JSON Wire Protocol" (MJSONWP). Appium and Selendroid already implement much of the MJSONWP spec. In this talk I'll dive into the specifics of the MJSONWP extensions, how they relate to the original JSONWP, and how the Selenium clients have begun to implement them.

    Finally, I will talk about the future of the MJSONWP and how it's related to the current and future versions of the WebDriver spec. I'll share how you can get help with the creation of the MJSONWP, and discuss issues with the authors of the new spec before the API is set in stone. We need the help of everyone who's involved in mobile automation to come up with the best and most future-proof version of the MJSONWP. Ultimately, your understanding of how Selenium works will be improved, and you'll have a much better handle on how projects like Appium and Selenium work together to make sure you have the best automation methods available.

  • 45 mins
    Demonstration
    Intermediate

    How many times do we test the same things at multiple layers, multiple levels, adding time to the build process and testing cycle, delaying the feedback?

    We know what to test and how to test, but what is the right place to test it?

    In this workshop, we will demonstrate how as QA’s we can identify what tests can be classified as unit tests, integration test and functional test. Using a case study, we will see how each component can be tested as part of unit testing; the integration of different parts and the functioning of a software system as a whole and how functional tests fit into this big picture. We will then bring all these tests together to understand and build the testing pyramid and how it enables us to build the right testing framework with fewer selenium i.e functional tests.

  • Liked Andrei Solntsev
    keyboard_arrow_down

    Andrei Solntsev - Selenide: Concise UI Tests in Java

    45 mins
    Demonstration
    Advanced

    Selenium WebDriver is a great tool, but it's not a testing library. It's a browser manipulation tool. There is a gap. 

    Selenide adds a possibility for easy and stable testing.

     

    Why yet another Selenium wrapper?

    There are several testing libraries around Selenium webdriver. But it seems that they do not resolve the main problems of UI tests. Namely, instability of tests caused by dynamic content, JavaScript, Ajax, timeouts etc. Selenide was created to resolve these problems. With Selenide, you can forget all these common timing issues and concentrate on business logic.

     
  • Liked James Farrier
    keyboard_arrow_down

    James Farrier / Xiaoxing Hu - Making Your Results Visible - A Test Result Dashboard and Comparison Tool

    45 mins
    Demonstration
    Intermediate

    If a test fails in the woods and no one is there to see it does anyone care, does anyone even notice. What happens when failing tests become the norm and you can't see the wood from the trees? 

     

    After watching last years Allure Report presentation I was inspired.  Selenium tests (and automation tests in general) are often poorly understood by the team as a whole.  Reports/emails go unread with tests failing becoming an expected outcome rather than a glaring red flag.  We looked at what Allure brought to the table and from that base created a dashboard which was designed to:

    • Display the results of test runs in a way that was useful to managers, testers and the rest of the development team.  Including tools to filter out specific test runs and view the overall trend of the test run results.
    • Make debugging tests easier by grouping errors, displaying history of test results, filtering tests and offering visual comparison of test runs.
    • Help mitigate the problems flaky tests cause with test run result reporting (say that three times fast).
    • Help with our mobile device certification process, by easily providing a view to compare test runs across devices.

    Since it's creation the dashboard has been used and praised by managers through to developers.  With our full suite of tests from unit to integration to selenium and appium being stored on the dashboard.  We've managed to:

    • Decrease the time taken to debug test cases.
    • Increase the visibility of all our test suites, with managers having a better idea of how our selenium test suite is progressing and testers better understanding the coverage of unit tests.
    • Focus the organization on quality.

    We are working with legal at present to have this project open sourced and available to all prior to Selenium Conf 2015.

  • Liked Ragavan Ambighananthan
    keyboard_arrow_down

    Ragavan Ambighananthan - Distributed Automation Using Selenium Grid / AWS / Autoscaling

    45 mins
    Talk
    Advanced

    Speed of UI automation has always been an issue when it comes to Continuous Integration / Continuous Delivery. If UI automation suite takes 3 hours to complete, then any commit happens during this time will not be visible in test environment, because the next deployment will happen only after 3 hours. 

    With 2000+ developers and average 250+ checkins per day, the above issues is replicated 250+ times every day. This is not productive and feedback cycle is super slow!

    Another issue is , with 35+ different project teams using 10 or more different jenkins jobs to run their UI automation. So many jobs means (350+), individual teams need to go through the pain of managing their own jenkins job, its a duplicate effort and waste of time. Automation teams need to spend time on writing reliable automation and not managing jenkins jobs.

    Solution is to reduce the UI automation run time from hours to minutes and also use only handful of jobs to run the Distributed Automation!

    Goal: To run all UI automation scenarios within the time take by the longest test case

  • Liked Sveta Kostinsky
    keyboard_arrow_down

    Sveta Kostinsky - Selenium Today vs. Selenium Tomorrow: Digital as the Convergence of Mobile & Web Programs

    45 mins
    Talk
    Beginner

    Today, mobile is increasingly trumping web as the most important brand engagement point; enterprises are moving away from mobile and web projects independent of each other. The rapid adoption of responsive web encourages teams to discover one approach to measuring software quality regardless of form factors.

     

    Selenium is current market leading solution for web testing, but how does it stand with mobile? The truth is that working with Selenium presents a few challenges, including:

    • Building and maintaining an internal structure to support it
    • Bridging an architectural gap
    • Requirements demand support for unattended test execution
    • Lack of real network conditions for mobile testing

     

    There is a solution to address these challenges!

    Let’s work through a demo and show how to test mobile & web in parallel with Selenium

  • Liked giri
    keyboard_arrow_down

    giri - Big Data - Verify Data Completeness and Data Correctness using Selenium/JAVA

    giri
    giri
    Director
    Gallop Solutions
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Beginner

    Using Hadoop MapReduce, Java programs are written to process large amounts of Data. Testing has to be performed to check the accurate functioning of these applications. Testing process includes manually verifying business logic on each node for MapReduce process accuracy, Data aggregation/Segregation rules and Generation of Key Value pairs. At the same time, Output data files are also verified for Transformation Rules, Successful Load, Data Integrity and Data Accuracy.

    Due to the enormous amount of data and various business rules, manual testing process is time consuming and may lead to slippage of validations. Implementing the automation testing process using Selenium & Java Adapters will make sure the data is complied with all the business/transformation rules and checks the data integrity.

  • Liked Russell Rutledge
    keyboard_arrow_down

    Russell Rutledge - Blazing Fast UI Validation - 5000 Reliable Tests in 10 Minutes on One Machine

    Russell Rutledge
    Russell Rutledge
    Senior Technical Lead
    Nike
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Advanced

    A big blocker for putting a website on truly continuous production delivery is the amount of time it take to validate that the site works correctly.  Tests themselves take time to run, and test results are unreliable to the point where it takes a human to investigate and interpret them.  When counting the time that it takes to both run and interpret results, test runs for an enterprise web site can take an entire day from inception to useful result.

    This session describes common points of failure in test execution that add both latency and unreliability and what can be done to overcome them while still preserving the value of UI validation.  We'll discuss why, after addressing these concerns, UI can be unblocked to reliably field thousands of validation scenarios on a local machine in a matter of minutes. 

  • Liked David Giffin
    keyboard_arrow_down

    David Giffin - A Large-scale, Data-driven Company's Journey of Going From Manual to Automated Testing In 6 Months

    David Giffin
    David Giffin
    Technologist
    TrueCar
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Beginner

    Manual Testing.  Depending on how you've been influenced by those two simple words, reactions vary from slight disgust to full-on depression.  Of course, the solution is clear: automate, but how do you get there when your company is continually pushing out the next big feature?  As the set of features to cover increases, the lack of scalability of manual testing becomes more apparent.

     

    This is a problem that we struggled with at our company.  Automation tactics were explored and implemented, but problems persisted as proposed solutions did not cater to the demands of the manual testers.

     

    After years of failure and disappointment, our latest stint resulted in success.  Not only do we have hundreds of automated tests across various platforms (mobile and web) and products, but manual testing has been eliminated with zero casualties.  As we move forward towards Continuous Delivery and improved automation performance, we wanted to take this moment to look back and share stories of failure and success.

  • Liked Trinath Babu
    keyboard_arrow_down

    Trinath Babu - Visual Test Automation using Selenium

    Trinath Babu
    Trinath Babu
    Sr. Manager
    Gallop Solutions
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    Visual Test Automation using Selenium

    Visual Testing is the method of verifying that the application’s GUI appears correctly to its users. Most of the people say visual testing is hard to automate. Given the number of web browsers, operating systems, screen resolutions, responsive design, internationalization, etc.) the nature of visual testing can be complex. But with existing open source and commercial solutions, this complexity is manageable, making it easier to automate than it once was, since verification with traditional automated functional testing tools can be very challenging.

    It can be easily achieved by integrating Selenium with Applitools. This talk mainly focuses on verifying the application’s graphical user interfaces (GUI) and finding the visual bugs using Applitools. It is very helpful for all sites having graphical functionalities like (charts, graph, dashboards etc).  Verify that the GUI appears correctly across all devices & browsers. The nature of visual testing can be complex. But with existing open source and commercial solutions, this complexity is manageable, making it easier to automate than it once was. And the payoff is well worth the effort.

    Take pressure off manual QA: increase coverage, test faster & more accurately.  Reduce maintenance efforts: automatically propagate changes across execution environments. Release faster, with confidence & flawless.

    Applitools Eyes Express captures the screen you want to test, and compares it to a baseline image – instantly, with a single click. No extra testing code necessary, no boring error logs.

    For example, a single automated visual test will look at a page and assert that every element on it has rendered correctly. Effectively checking hundreds of things and telling you if any of them are out of place. This will occur every time the test is run, and it can be scaled to each browser, operating system, and screen resolution you care about.

    Put another way, one automated visual test is worth hundreds of assertions. And if done in the service of an iterative development workflow, then you’re one giant leap closer.

     

    Each of these tools follows some variation of the following work flow:

    1. Drive the application under test (AUT) and take a screenshot
    2. Compare the screenshot with an initial “baseline” image
    3. Report the differences
    4. Update the baseline as needed
  • Liked Oren Rubin
    keyboard_arrow_down

    Oren Rubin - Selenium Wat!

    Oren Rubin
    Oren Rubin
    Founder and CEO of Testim
    Testim.io
    schedule 2 years ago
    Sold Out!
    45 mins
    Case Study
    Intermediate

    Every language and framework which lives as long as Selenium has its fuckups.. and we're here to embrance them and joke about them.

    E.g. JS Wat https://www.youtube.com/watch?v=FqhZZNUyVFM

  • Liked Tanay Nagjee
    keyboard_arrow_down

    Tanay Nagjee - Run your Selenium tests in a fraction of the time

    45 mins
    Talk
    Intermediate

    Comprehensive functional testing is a widely accepted best practice and Selenium is the prevalent tool of choice. But long-running Selenium test suites cannot be fit into short continuous integration (CI) cycles and are often run once a night or less. Running functional tests less frequently means bugs are discovered later than they should. Tanay Nagjee will discuss how Selenium test suites can be parallelized at a very fine granularity and included in CI builds. By leveraging a cluster of compute horsepower (on-premise and/or cloud), large Selenium suites can execute in a fraction of the time by smartly parallelizing the individual tests and running them on individual *cores* (not hosts). Tanay will outline the approach and tools to achieve these results with Selenium, and will present a live demonstration.


  • Liked Dilip S
    keyboard_arrow_down

    Dilip S - TestComplete supports Selenium – Other Commercial tools will follow soon?

    Dilip S
    Dilip S
    Associate Architect
    Gallop
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Beginner

    Oliver Wendell Holmes once said: I would not give a fig for the simplicity this side of complexity, but I would give my life for the simplicity on the other side of complexity.

    Tool evangelists around the world have been using this phrase for selling their products. They make it a point to look into customer’s eyes and say “Scalability, you know, that’s the main problem with Selenium. What about 3 years down the line, when you have multiple applications in your landscape and Selenium does not support it?”

    But they simply ignore the fact, knowingly off course, that this ever belting technology world is going where Selenium is right now. Applications are shrinking into browser windows and changing tracks to align itself to this mobile era.

    Even commercial tool like TestComplete from SmartBear has started supporting Selenium and many will follow soon. Reason for this change is not only that most of the organizations are preferring open source tools like Selenium for starting point of their automation activities but also the fact that Selenium by far has proven itself to be one of the best automation tools when it comes to mobile or browser based desktop automation.

    Here our aim is to display how seamlessly Selenium integrates with TestComplete and QAComplete and it is for us to understand that it is not Selenium which needs other tools to extend it but it is the other way round.

  • Liked Vishal Aggarwal
    keyboard_arrow_down

    Vishal Aggarwal - Selenium Next Generation Framework

    Vishal Aggarwal
    Vishal Aggarwal
    Test Architect
    Gallop Solutions
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Beginner

    This talk focuses on the technical side of automated acceptance tests for web applications. There are a lot of high-level frameworks that allow definition of acceptance tests in natural language (Robot, JBehave, Cucumber etc). But when it comes to the technical implementation of the test cases, you are often forced to use the rather low-level WebDriver API directly.

    GEB addresses exactly this problem. It is an abstraction of the WebDriver API and combines the expressive and concise Groovy language with a jQuery-like selection and traversal API. This makes the test implementation easier and the code more readable. On top of that, we get support for the page object pattern, asynchronous content lookup and a very good integration in existing test frameworks which makes it simply next generation of automation framework.

  • Liked vishnu nallani chekravarthula
    keyboard_arrow_down

    vishnu nallani chekravarthula - Extending Selenium Element Locator Strategies – Element Filtering

    45 mins
    Talk
    Intermediate

    Element Locator strategies for Selenium WebDriver are highly flexible, and have been later inherited by many commercial tools. Although the locator strategies are flexible, they are also limited in a sense that, Selenium WebDriver does not currently allow its users to identify/filter UI elements with multiple locator strategies(at a time), as many commercial tools do.

    The solution discussed in this article describes a library that allows Selenium WebDriver users to extend the Selenium element locator strategies for Element Filtering and few use cases for the library.

    The solution approach allows users to continue to use the existing UI Element definitions in their tests, and extend them, using the By reference. The library will replace the existing Selenium WebDriver “By” reference.

    Filtering based on multiple locator strategies

    There are various scenarios where to uniquely identify an UI element, a complex XPath has to be written. However, the element can be identified uniquely using multiple locator strategies for the UI Element. The UI Elements can also be filtered, when there are multiple matches in a page. This is the UI Element recognition mechanism used in many commercial test automation tools.

    The algorithm for filtering UI Elements based on multiple locator strategies is based on priority of locator strategies. The priority of locator strategies when filtering is:

    1. ID
    2. Name
    3. TagName
    4. ClassName
    5. XPATH
    6. LinkText and PartialLinkText
    7. CSS

    The By.elementFilter method takes multiple locator strategies, and searches the page for elements matching a particular locator strategy/property, and checks if it is a unique match on the page, if not then it uses the next locator strategy passed to it and so on.

    This method is also very helpful when the application undergoes constant changes and UI Elements might have either of XPATH, ID , NAME, TagName, ClassName etc still unchanged. That way, it helps reduce a lot of maintenance effort in Selenium WebDriver implementations which is due to UI element changes.

    Filtering based on Index

    When there are multiple similar UI Elements in a page, such as cells in a grid/table, it makes sense to identify objects based on their Index based on their appearance on the web page.

    The By.indexFilter method allows users to define an UI Element based on its Index of occurrence of the UI Element. The Index starts from 1.

    Filtering based on relative element

    When a UI element cannot be identified uniquely and reliably by any of its properties, but has some elements in its hierarchy or relative to a particular element, this method can be used to identify the element

    The By.relationFilter method allows users to define an UI Element in relation to another element. The relation can be defined as “Left”, ”Right”, ”Top”, ”Bottom”, ”Child”, ”Parent”

    Filtering for Tables

    When dealing specifically with Tables, which have the

    html tag, the By.tableFilter method allows the user to quickly identify specific cells in the table, without having to write complex XPaths or logics to achieve the same.

    The By.tableFilter method allows users to define a cell in the table with Row,Column numbers. This allows users to directly use the UI Element in their code instead of writing their logic each time. This also increases efficiency and readability of the code.

  • Liked Surendran Ethiraj
    keyboard_arrow_down

    Surendran Ethiraj - VAS (Value Added Services) using Selenium Webdriver

    45 mins
    Demonstration
    Intermediate

    The evolution of Test Automation started with automation tools that had record and playback features.  This allowed Automation Testers to record and structure the script in such a way that it could be reused. Tools like Selenium, which provided APIs, could interact with different browsers. The Automation Testers could use these APIs to interact with web applications. Additionally, it was possible to develop frameworks for reusing each of the components of the framework. Currently, the focus has shifted more and more towards the designing of frameworks rather than just the tools, so that the testing framework could be integrated with test management applications and continuous integration tools to aid test-driven development.

    With that background, we have come up with certain Value Added Services (VAS), a step ahead of developing functional automation scripts. Imagine creating an Automation Framework which will not just check for the functionality of the application, but also check for security, page performance, page layout, accessibility and have an output that can be a trigger to other aspects of testing.

    This paper presents three of the Value Added Services that we offer. We are working on creating many more such services on top of the Automation Framework.