schedule Sep 10th 04:00 PM - 04:45 PM place Pavilion East/West

Typically in organizations, there are multiple projects / products. These products may be of implemented using tech-stacks over many years. Yet - they interact with each other in some way. To manage the complexity around Test Automation, many organizations prefer to have a common Test Automation solution across these products in an effort to build, standardize and maintain the framework.

However, this is not a good idea! With this approach one potentially ends up having to compromise on the quality of automation that can be done for each product, limited by the toolset.

The better approach would be to use the tools and technologies that are "right" for each product. This does have other disadvantages, but you would ensure each product is well tested! The only missing piece which remains is that these different products talk with each other. You need to test the integration between them in an automated way to verify all is well.

"TaaS" is an open-source product solution that allows you do achieve the "correct" way of doing integration testing across a variety of products via Test Automation.

Example: 

For one set of products, Selenium-based toolset may be the right choice, where as for legacy reasons, QTP may be used for some other product. With TaaS - you will be able to automate the Integration Testing between these products, by re-using the tests already implemented in the individual product suites.

 

 
10 favorite thumb_down thumb_up 7 comments visibility_off  Remove from Watchlist visibility  Add to Watchlist
 

Outline/structure of the Session

  • Describe case study why Common Test Automation Framework is not a good idea
  • Describe TaaS - why it is good, how it works, its limitations
  • Demo of TaaS integration
  • How can teams use TaaS
  • Next steps for TaaS and how the community can help

Learning Outcome

  • In large organisation, having multiple products of different tech-stacks, why it is not a good idea to have a common test automation framework
  • A way to automate the integration testing between different products, which use different tools and technologies as development and testing stack.
  • A demo of how TaaS can be used

Target Audience

All roles involved in planning, managing, implementing and maintaining Test Automation

schedule Submitted 2 years ago

Comments Subscribe to Comments

comment Comment on this Proposal
  • Sherwin Angelo
    By Sherwin Angelo  ~  1 year ago
    reply Reply

    Hi Anand,

    Taas looks very nice and i really would like to do a POC and see whether it can help at my current client. I do wonder whether TAAS can be integraded with Jenkins?

     

    Thanks,

     

    Regards,

     

    Sherwin

     

    • Anand Bagmar
      By Anand Bagmar  ~  1 year ago
      reply Reply

      Hi Sherwin,

      Thank you for your kind words about TaaS!

      Here is what you would need to do to use Taas:

      1. Since TaaS is an integration enabler, and uses the service-based-approach, you would need to setup your "service-providing" test framework on a dedicated machine, and setup TaaS server on that.
      2. All tests, needing to consume the services of this other framework, would use TaaS Client to invoke such available services. These tests can be triggered either manually, or by CI (as feasible by your test framework).

      Hope this clears it up. Happy to have a skype / google hangout call with you to walk through this in more details.

      Regards,

      Anand

      • Sherwin Angelo
        By Sherwin Angelo  ~  1 year ago
        reply Reply

        Hi Anand,

        Thanks for the quick reply. In Theory it seems clear. I do need to understand how i can setup a service providing framework on a dedicated machine and setup frameworks as clients to connect to it. Are there any example on how to set this up?

        Thanks,

        Kind regards,

         

        Sherwin

        • Anand Bagmar
          By Anand Bagmar  ~  1 year ago
          reply Reply

          Hi Sherwin,

          Using these slides (http://www.slideshare.net/abagmar/integration-testing-in-enterprises-using-taas) for reference, here are the steps you need to follow to setup the TaaS Server:

          1. I am guessing you now understand the high-level picture of how TaaS solves a specific problem (slide #24) and you have created specific contracts-based tests in your test framework that will be served as "services" via TaaS server to the TaaS clients (slides #25-34)
          2. To use this, you need to get a dedicated machine, where your test framework is available. This is the machine which will run the specific tests invoked by TaaS Server - on request from TaaS Client. Once the test framework is setup, you now need to setup the TaaS server here, and start it. When all this is done, now your dedicated machine is available for servicing requests. See slide #35.

          There are some tests available here (https://github.com/anandbagmar/taas/tree/master/test) for the TaaS server and Client which you can see for reference.

          • Sherwin Angelo
            By Sherwin Angelo  ~  1 year ago
            reply Reply

            Thanks a lot.. i will check it out and try to set it up

  • Dave Haeffner Test
    By Dave Haeffner Test  ~  2 years ago
    reply Reply

    Can you provide a link to your open-source solution?

    • Anand Bagmar
      By Anand Bagmar  ~  2 years ago
      reply Reply

      The project is available here on github: github.com/anandbagmar/taas

      The gem is available here on rubygems: https://rubygems.org/gems/taas

      All my blogs related to taas are available here: http://essenceoftesting.blogspot.in/search/label/taas


  • Liked Anand Bagmar
    keyboard_arrow_down

    Anand Bagmar - To Deploy or Not-to-Deploy - decide using TTA's Trend & Failure Analysis

    45 Mins
    Demonstration
    Intermediate

    The key objectives of organizations is to provide / derive value from the products / services they offer. To achieve this, they need to be able to deliver their offerings in the quickest time possible, and of good quality!

    In order for these organizations to to understand the quality / health of their products at a quick glance, typically a team of people scramble to collate and collect the information manually needed to get a sense of quality about the products they support. All this is done manually.

    So in the fast moving environment, where CI (Continuous Integration) and CD (Continuous Delivery) are now a necessity and not a luxury, how can teams take decisions if the product is ready to be deployed to the next environment or not?

    Test Automation across all layers of the Test Pyramid (be it Selenium-based UI tests, or, xUnit based unit tests, or, Performance Tests, etc.) is one of the first building blocks to ensure the team gets quick feedback into the health of the product-under-test. 

    The next set of questions are:
        •    How can you collate this information in a meaningful fashion to determine - yes, my code is ready to be promoted from one environment to the next?
        •    How can you know if the product is ready to go 'live'?
        •    What is the health of you product portfolio at any point in time?
        •    Can you identify patterns and do quick analysis of the test results to help in root-cause-analysis for issues that have happened over a period of time in making better decisions to better the quality of your product(s)?

    The current set of tools are limited and fail to give the holistic picture of quality and health, across the life-cycle of the products.

    The solution - TTA - Test Trend Analyzer

    TTA is an open source product that becomes the source of information to give you real-time and visual insights into the health of the product portfolio using the Test Automation results, in form of Trends, Comparative Analysis, Failure Analysis and Functional Performance Benchmarking. This allows teams to take decisions on the product deployment to the next level using actual data points, instead of 'gut-feel' based decisions.

  • 45 Mins
    Demonstration
    Intermediate

    How many times do we test the same things at multiple layers, multiple levels, adding time to the build process and testing cycle, delaying the feedback?

    We know what to test and how to test, but what is the right place to test it?

    In this workshop, we will demonstrate how as QA’s we can identify what tests can be classified as unit tests, integration test and functional test. Using a case study, we will see how each component can be tested as part of unit testing; the integration of different parts and the functioning of a software system as a whole and how functional tests fit into this big picture. We will then bring all these tests together to understand and build the testing pyramid and how it enables us to build the right testing framework with fewer selenium i.e functional tests.

  • Liked Justin Woolley
    keyboard_arrow_down

    Justin Woolley / David Anderson / Marvin Ojwang - Automating for the Second Screen with WebdriverJS

    45 Mins
    Talk
    Intermediate

    There has been a recent explosion in second-screen technologies such as Chromecast, but designing test automation for second-screen applications is far from straightforward. This new paradigm lacks major automated tool support, and coordinating test execution across multiple devices is tricky and error-prone.

    Our automation solution uses WebdriverJS and WebSockets to perform end-to-end test automation that covers our web player controller and second screen application.

    Learn about our approach to second-screen automation which we’ve used to build a reactive, responsive test suite. We’ll describe our solutions to synchronizing test flow between the controller and target device, validation on the device, targeting different integration components, and device management.

  • Liked James Farrier
    keyboard_arrow_down

    James Farrier / Xiaoxing Hu - Making Your Results Visible - A Test Result Dashboard and Comparison Tool

    45 Mins
    Demonstration
    Intermediate

    If a test fails in the woods and no one is there to see it does anyone care, does anyone even notice. What happens when failing tests become the norm and you can't see the wood from the trees? 

     

    After watching last years Allure Report presentation I was inspired.  Selenium tests (and automation tests in general) are often poorly understood by the team as a whole.  Reports/emails go unread with tests failing becoming an expected outcome rather than a glaring red flag.  We looked at what Allure brought to the table and from that base created a dashboard which was designed to:

    • Display the results of test runs in a way that was useful to managers, testers and the rest of the development team.  Including tools to filter out specific test runs and view the overall trend of the test run results.
    • Make debugging tests easier by grouping errors, displaying history of test results, filtering tests and offering visual comparison of test runs.
    • Help mitigate the problems flaky tests cause with test run result reporting (say that three times fast).
    • Help with our mobile device certification process, by easily providing a view to compare test runs across devices.

    Since it's creation the dashboard has been used and praised by managers through to developers.  With our full suite of tests from unit to integration to selenium and appium being stored on the dashboard.  We've managed to:

    • Decrease the time taken to debug test cases.
    • Increase the visibility of all our test suites, with managers having a better idea of how our selenium test suite is progressing and testers better understanding the coverage of unit tests.
    • Focus the organization on quality.

    We are working with legal at present to have this project open sourced and available to all prior to Selenium Conf 2015.

  • Liked Sveta Kostinsky
    keyboard_arrow_down

    Sveta Kostinsky - Selenium Today vs. Selenium Tomorrow: Digital as the Convergence of Mobile & Web Programs

    45 Mins
    Talk
    Beginner

    Today, mobile is increasingly trumping web as the most important brand engagement point; enterprises are moving away from mobile and web projects independent of each other. The rapid adoption of responsive web encourages teams to discover one approach to measuring software quality regardless of form factors.

     

    Selenium is current market leading solution for web testing, but how does it stand with mobile? The truth is that working with Selenium presents a few challenges, including:

    • Building and maintaining an internal structure to support it
    • Bridging an architectural gap
    • Requirements demand support for unattended test execution
    • Lack of real network conditions for mobile testing

     

    There is a solution to address these challenges!

    Let’s work through a demo and show how to test mobile & web in parallel with Selenium

  • Liked rajesh sarangapani
    keyboard_arrow_down

    rajesh sarangapani / Prabhu Epuri - Visualizing Real User Experience Using Integrated Open Source Stack (Selenium + Jmeter + Appium + Visualization tools)

    45 Mins
    Demonstration
    Advanced

    Traditional approach in performance testing does not include client side processing time (i.e. DOM Content Load, Page Render, JavaScript Execution, etc.) as part of response times, performance tests has always been conducted to stress the server so tools like Jmeter have been very popular to execute tests. With increasing complexity of architectures (Web, Browser, Mobile) on the client side it has been important to understand the real user experience.   Commercial tools have started to provide features that can provide insights into real user experience after the bytes are transferred to the client end.  With the ability to call Selenium scripts via Jmeter the ability to conduct real user experience tests using open source stack has opened up new avenues to comment on real user experience.   This enables us to comment on

    • Provides Page load times similar to On Load time of real browsers
    • Generates HAR file with following statistics
    • Details of summary of request times and content types
    • Waterfall chart with page download time breakdown statistics such as  DNS resolution time, Connection time, SSL handshaking time, Request send time, wait time and receive time.

    By integrating the open source stack tools it enables us to provide the same insights which a commercial of the shelf tools would offer.   At Gallop we have implemented this at multiple clients providing them insights into various bottlenecks at the client side which helped us to provide greater value proposition

  • Liked Bhumika S
    keyboard_arrow_down

    Bhumika S - The 10 Myths of Testing that we need to Break

    45 Mins
    Talk
    Beginner

    This talk aims to break 10 (amongst many) Myths that desperately need to be broken around Software testing and Testers. These so-called myths, have been around for so long that they are almost the norm. Breaking these myths is necessary to help budding as well as experienced Quality Analysts and teams, roll out better quality software products and build better team practices as well.

  • Liked Anand Bagmar
    keyboard_arrow_down

    Anand Bagmar - Create your Future!

    45 Mins
    Talk
    Beginner

    With all the focus on Test practices and tools getting better for the present, it is easy to forget about the Future of Testing, especially in terms of Tools and Infrastructure? A question that always comes up in my mind - “Are we so caught up in the past and present, that we will not be as effective in the future?”

    In this talk, we will go on a journey to figure out what new challenges are coming up in the future, and more importantly, what do we need to do next to prepare for it. Also, just preparing for the future is not sufficient. We have an opportunity to stretch beyond our current set of skills, capabilities and boundaries to influence out future! The question is - will we make use of this opportunity? Will Selenium help us take the step in the right direction? Or, will it hold us back?

  • Liked Jason Watt
    keyboard_arrow_down

    Jason Watt - Challenges of the Mobile Cloud

    Jason Watt
    Jason Watt
    Senior Software Engineer
    Salesforce
    schedule 2 years ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    Creating a mobile app is now the new cross platform problem. The major mobile platforms tend to gear their development tool chain towards individuals and their workstations.  But what if you want to introduce a CI solution to this environment? What if your app is launching on more than one platform and there's a team of 20+ developers working on it? What if your tests are more than just Selenium based?

    This is normally where you can look to the cloud for scale but mobile has a ton of challenges to do so.  Come and learn from some of the challenges and pitfalls I've encountered while working towards this goal.

  • Liked Sarah Thompson
    keyboard_arrow_down

    Sarah Thompson - DevOps meets QA - Using Puppet to set up and manage your Selenium Grid

    Sarah Thompson
    Sarah Thompson
    QE UI Toolsmith
    Puppet Labs
    schedule 2 years ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    For testers, setting up and maintaining a Selenium Grid infrastructure can be timely and costly. A lot of the time, we are asked to do this as part of our day to day job when we really want to focus on testing the product!

    There are some great cloud based alternatives out there that allow you to easily run your tests on a wide range of Operating Systems and against multiple browser types (at a price).

    But what if you already have plenty of devices available within your own company (be it physical machines, virtual machines or cloud based resources) and you want to be able to setup and manage your own grid infrastructure:

    • to allow you greater control over the configuration (perhaps you want to have a headless browser like PhantomJS available on your grid)
    • to run your tests faster (the cloud based alternatives are a good bit slower for obvious reasons)
    • or to simply save money.
  • Liked Satyajit Malugu
    keyboard_arrow_down

    Satyajit Malugu - Adapting testing and automation for Mobile First

    Satyajit Malugu
    Satyajit Malugu
    Senior mobile tester
    Godaddy
    schedule 2 years ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    Mobile-first is a paradigm shift in developing software where products are created with an emphasis on mobile experiences, rather than desktop. Mobile sites, native apps become the new centerfold for the company around which various other initiatives and goals are driven. This paradigm has been in adoption by major industry players including Google, facebook, Airbnb, ebay, shopify etc.

     

    From a development standpoint it is not really different from traditional websites after you get UX but testing is quite different. You have to consider various devices, screen sizes, performance and visual testing. Automation should focus on tackling the most time consuming manual aspects which include functionality, visual parity and device compatibility.

     

    Our $5B company has been in a mobile-first transformation this year, spanning many legacy products and teams. As a senior SDET leading the testing efforts I have been involved in various phases of this transformation. I believe this talk would give practical insights in mobile web development and testing.

  • Liked Manoj Pahuja
    keyboard_arrow_down

    Manoj Pahuja - Measuring the metrics to make them reliable, fast and valuable- What, why and How?

    Manoj Pahuja
    Manoj Pahuja
    Test Architect
    Godaddy
    schedule 2 years ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    UI Tests are fragile by nature. The reliability and execution times are common challenges. If you add scaling UI tests for multiple runs, multiple times a day and environmental instabilities, it becomes very difficult to find the right information related to these tests. Running through and Understanding test results across different builds takes a lot of time. You get too many false alarms and now you have reached a point where nothing is an alarm. Get to know some practical approaches and success stories on how you can distribute all tests across multiple machines, VMs, cloud services and still be able to combine all the results in a easy to read and understandable report. Fix reliability and improve speed of your tests.

  • Liked Dmytro Zharii
    keyboard_arrow_down

    Dmytro Zharii - Smoke Tests and Page Objects: Let's do the right thing fast!

    45 Mins
    Talk
    Intermediate

    "Write the Smoke Tests" -- this is the ultimate answer for the questions:
    - How do I start a new test automation project?
    - What is the way to test that my PageObjects are still working correctly after the "nightly" product changes?
    - What is the fastest way for my Test Automation Framework to bring the real value for my teammates and managers?

    Can you imagine you can generate the Page Objects code in a fast and easy way with SWD Page Recorder?
    Can you imagine you can generate the Smoke Tests for your PageObjects and the Application Under Test by pressing one button?

    Let me show how you can increase your productivity in average 45% by using good tooling, patterns and practices during your daily work.

  • Liked Aaron Evans
    keyboard_arrow_down

    Aaron Evans - Looking ahead: testing responsive, mobile, and native apps with Selenium

    Aaron Evans
    Aaron Evans
    Test Consultant
    One Shore
    schedule 2 years ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    Selenium made test automation easier and affordable for many software development teams, but it had many limitations.  It was limited to DOM manipulation in the browser, it depended on explicit waits.  

    Webdriver helped overcome some of these deficiencies and took Selenium to the next level.  Other extensions like Appium have enabled us to use the familiar Selenium API for testing mobile apps.  Proprietary frameworks allow you to integrate Selenium with native extensions and ALM tools.  But a new category of apps is coming with responsive UIs, rich client side Javascript frameworks, touch screens (with pinch/zoom, swipe, rotation, etc) and interact with native device features (such as GPS, accelerometer, local storage) and apps are becoming a collection of interactive services.  

    Is Selenium becoming outdated?  What can we do to keep up with these new interfaces and architectures?

    In this talk, we'll discuss some of the challenges and limitations facing testers using Selenium with this new generation of apps.  We'll cover some of the solutions people are using today, and propose a new way to address these issues and others going forward.

  • Liked Anthony Browness
    keyboard_arrow_down

    Anthony Browness - Dependency Injection in the Page Object Model

    45 Mins
    Talk
    Advanced

    The Page Object Model is awesome, but you can make it even better with Dependency Injection.  I'll show you how to get started and some benefits of using Spring IOC in your Page Objects.  Though the concepts of this talk can be extended to any language, the examples will be shown in Java.

  • 45 Mins
    Case Study
    Advanced

    Page-Object pattern is very commonly used when implementing Automation frameworks. However, as the scale of the framework grows, there is a limitation on how much reusability really happens. It inherently becomes very difficult to separate the test intent from the business domain.

    Learn more about this problem, and the solution (by means of real-project examples) - Business Layer - Page - Object pattern, which has helped me keep my code DRY.