We are now living in the age of Big Data. What makes Big Data so special is its ability to help the common man process petabytes of data to build innovative solutions. Cloud has played a pivotal role in building this data economy by making infrastructure more cost effective and enabling easy scalability.

Today, the abundance of data helps business make intelligent decisions through business process automation. Data drives all our digital interactions, starting from products that we see on our Amazon home page, to our Netflix recommendations, to the advertisement on Google search results, to even the Friends that we get connected to on Facebook. All these systems are powered by data that is collected from the various system, cleaned, transformed and stored into Big Data solutions. Testing of these data systems have become very complex and requires a new approach. This paper highlights various aspects of Big Data Testing including:

• Complexity in testing Big Data systems

• Cloud based tools for testing Big Data cross various stages

• Simplification of test processes and visualization through dashboards

• Testing Big Data in the Agile/ DevOps


Outline/Structure of the Talk

1. Business use case

2. Challenges in big data migration

3. Possible solutions

4. Pros and cons of various solutions

5. Technical overview of the solution

6. Business benefit from implementation of rh solution

Learning Outcome

1. Complexity in testing with large volume of data in TB's and XB's

2. Various options available

3. Various data formats for big data processing

4. How to accelerate testing for big data Programs

Target Audience

Developers, Testers, Data engineers

Prerequisites for Attendees

No Prequisite

schedule Submitted 1 year ago

Public Feedback

    • Gaurav Singh

      Gaurav Singh - How to build an automation framework with Selenium : Patterns and practices

      Gaurav Singh
      Gaurav Singh
      Test automation Lead
      schedule 1 year ago
      Sold Out!
      45 Mins

      With an ever increasing no of businesses being conducted on web the testing need to write automated tests for the app's UI is something that can never be ignored. As you all know Selenium provides an API that enables us to do this quite effectively.

      However, when tasked with setting up the automation framework, there are a lot of questions that arise in the minds of aspiring test developers regardless of what level they are in their career.

      Some of such questions are:

      1. How does one actually go about the business of building a robust and effective automation framework on top of selenium?
      2. What are the elementary building blocks to include in the framework that an aspiring automation developer should know of?
      3. How should we model our tests? XUnit style vs BDD?
      4. Are there good practices, sensible design patterns and abstractions that we can follow in our code?
      5. What are some of the anti patterns/common mistakes we should avoid

      A lot of literature, documentation and blogs exists on these topics on the web already.

      However In this talk,

      I would combine this existing knowledge and my years of experience in building automation frameworks and breakdown these elements and walk you through exactly the sort of decisions/considerations and practices that you can take while starting to implement or improve the UI automation for your team.

      Hope to see you there!

    • Diego Molina

      Diego Molina - Test Automation is not just coding

      Diego Molina
      Diego Molina
      Sr. Software Engineer
      Sauce Labs
      schedule 1 year ago
      Sold Out!
      45 Mins

      We learn more from our failures than our successes. I want to share one of my failure stories, where I learned that coding is not the most important task in Test Automation.

      A failure taught me that coding is not the most important task in Test Automation. I fixed a bug, added tests, and shipped it to production. After that, a bug report came in showing that the fix created a new bug. I felt bad, I did not understand how that happened. A hotfix solved the issue, but the root cause was not addressed: what did I miss?

      By taking a step back, I understood the situation, it all happened by overseeing basic concepts in testing (like understanding how the system works). The problem was that coding had more priority than creating a test plan. After this, I prioritized tasks better to avoid this situation to happen again.

      This talk shows that testing concepts are more important than ever, in a time where tools promise to do everything, we focus less on what and how to test, and more in using tools to test. It outlines how a test strategy can leverage a continuous testing setup. Finally, it shows that failing is ok, but failing again for the same reasons is not.

    • Naveen Khunteta

      Naveen Khunteta - Best Practices to implement the test automation framework starting from Design - To -> Infrastructure - To -> Execution.

      45 Mins

      Best Practices - How to get the best 'Return ON Investment' (ROI) from your Test Automation.

      This has been observed that, most of the test frameworks wont be able to survive due to lack of expertise, no maintenance, no best practices being followed, and finally your test automation will be dead after few months, and there is no "Return ON Investment" from this. This is the most common problem, most of the companies are struggling and finally back to square to the Manual testing.

      My proposal : HOW to leverage your test automation in terms of best practices, best ROI, and how to adopt best automation culture in your organisation.
      I strongly propose some of the important points/suggestions to achieve this in your Organisation/Team.
      1. Test Automation Practices:
      • Design Patterns (Web/Mobile/API)
      • What to Automate/Not to Automate
      2. Common Automation Frameworks at Org Level:
      • How to design Generic Utilities, Libraries and different Components, which can be suitable for all the teams in the same Org.
      • Best practices to design your Tests (Automation).
        • Common Design Patterns
        • Common application level and Page libraries
        • Best Practices to use Assertions in your Tests (How and What to write for assertions). Most of the people don't write proper assertions and this is making your test unreliable and no defects found during execution.
      3. Inclusion of API/Backend libraries in your UI test automation as an external Maven/Gradle Dependencies to avoid un-necessary tasks, some of the important points to be considered here:
      • User Creation from APIs (No need to automate user creation from web/app for all the test cases)
      • API tests are stabled most of the time
      • API calls takes lesser time as compared to web, hence include API calls in your UI/App framework to save time.
      • Less flaky test

      4. Best Code Review Process (Do not merge your code into Master without proper Code Review)

      • Implement PR (PULL Request) Process
      • Static Code Analysis using SonarQube, Cobertura, JACOCO etc..
      • Get the benefits of Best Test Automation Quality Matrices
      • Sometimes, Manual (Functional Tester) should review your code (Assertion, test steps and use cases) to get the best coverage
      5. Quality is A Team responsibility:
      • Developers, POs, Manual QEs and Automation engineers should be included to get an overview of test automation coverage.
      6. Maintenance of the Frameworks
      • After couple of months, it should not make your life miserable if you don't maintain your libraries and framework properly.
      • Do not use Hard Coded values, make it simple and Generic.
      7. Infrastructure Setup for Test Design and Test Execution:
      • Proper Browser - OS lab setup
      • Proper Mobile Labs setup with different Devices - IOT, iOS, Android, iPad, Tablets
      • Proper CI - CD common configuration using Jenkins, Dev Ops, AWS, Docker and Cloud setup
      • Handling multiple Docker nodes using Kubernates (use of Selenoid, GRID on Cloud)
    • Khanh Do

      Khanh Do - Leveraging Artificial Intelligence to create self-healing tests

      Khanh Do
      Khanh Do
      QA Architect
      schedule 1 year ago
      Sold Out!
      45 Mins

      A key requirement for successful test automation is to get past the brittle or fragile nature of test scripts. Any Selenium (or Appium) developer has encountered the dreaded "NoSuchElement Exception". A locator that worked yesterday may fail today. What's a test engineer to do?

      Fortunately the field of AI provides promising solutions and allows for the creation of self-healing tests. Tests that can find elements across all environments. Tests that can learn from "human-in-the-loop" intervention and work perfectly thereafter. Imagine automated tests that "just work"!

      This session will look at how to apply the latest in AI and Machine Learning technologies to improve your test scripts. With the plethora of new open source AI libraries made available by companies such as Google, the ability to leverage AI in your applications is more accessible than ever.

      This session will be a primer on AI technologies and how they can be utilized for perfect test automation.