Exploratory Testing explored using Risk Analysis for Mobile Applications

Exploratory testing is an approach that can be described as simultaneous learning, test design and test execution. It is testing parallel with learning. We all consciously or un-consciously do exploratory testing. We use the concept of “Think Different” every time we do exploratory testing. Creative exploratory tests help to break the monotonous scripted testing and bring out newer test case each time.

This paper brings out the approach of exploratory testing which is an emerging topic and clubbed with risk based analysis brings a powerful approach to testing. Today’s fast growing technology and rapid changes in projects, makes exploratory testing an important approach for every tester. Scripted testing alone often fails to find hidden defects. Exploratory testing, however, can expose these risks because it incorporates exploration and experience into the testing process.

 

One common misconception about exploratory testing is, it is unstructured, and this paper tries to showcase the concept of doing exploratory testing with risk analysis and gives it a more structured approach.

Also this paper highlights exploratory testing in mobile applications, using risk analysis. Mobile application testing embeds a lot of features, navigation flows, OS, device dependency. Mobile testing always requires an investigative approach apart from the fundamental test approaches. Exploratory testing with risk analysis makes a mobile testing much structured and easier to delve deep into the application and find bugs.

 

 
2 favorite thumb_down thumb_up 1 comment visibility_off  Remove from Watchlist visibility  Add to Watchlist
 

Outline/structure of the Session

Introduction

Software development lifecycle faces challenges like rapid changes in requirements, changes in design and frequent new change requests. There has been a constant shift from traditional testing practices to newer ways of testing.

The term exploratory testing was coined by Cem Kaner. It is testing parallel with learning. Author James Bach, described exploratory testing as “scientific thinking in real-time.” It gives us numerous number of combinations and choices. It also broadens the way we think and do testing.

Every testing project is different and each project must be approached differently by individual and organization, Exploratory Testing enhances a product’s test coverage by helping the team better understand the system and find issues when time is of the essence.

Risk-based testing (RBT) is a type of software testing that is used to prioritize the tests of features and functions in software, based on the risk of failure, the function of their importance and likelihood or impact of failure. Risk mitigation/avoidance activities avoid risks or minimise their impact.

Blend of Exploratory Testing along with Risk based Analysis

Risk based exploratory testing gives a proper structure to exploratory testing. Exploratory testing is sometimes confused with "ad hoc" testing. Ad hoc testing normally refers to a process of improvised, impromptu bug searching. By definition, anyone can do ad hoc testing. The term "exploratory testing"--coined by Cem Kaner, in Testing Computer Software-- refers to a sophisticated, thoughtful approach to ad hoc testing. 

Exploratory testing with risk analysis demands skilled and experienced tester, who can explore the application and identity the risks and execute the test accordingly. This way tester can identify the risks and create and execute tests accordingly.

Need for Risk based Exploratory Testing in Mobile applications.             

Mobile systems involve a lot of learning from system and Customers, lot of changes are made based on what the customer wants, the typical errors in mobile systems, device dependencies make mobile testing more complex.

In mobile testing, One step leads to another and new scenarios can be identified in each round of testing, which basically adds to knowledge of the tester. In such cases, Exploratory Testing in Mobile Applications using Risk analysis comes a very powerful way to find defects.

Testing on wide range of mobile devices makes it impossible to test all test cases, Risk Analysis in such cases becomes essential for testing in such cases. Exploratory testing with risk analysis makes mobile tester’s life easier and handy.

 

 

 

Approach to Exploratory Testing in any Application using Risk analysis

  1. Risk Identification and Analysis for the application.
  2. Carry out Exploratory Tests on the identified areas of Risk
  3. Maintain a Test Charter to document for the test scenarios and outputs
  4. Perform Gap analysis to check if any test scenarios have been missed.

 

Step 1:   Risk Identification and Analysis for the application.

 

Risk identification and Analysis is the 1st step in his process. Identify the risks yourself or if you are working with a team, brainstorm the application existing issues and assess the risk areas. Review any previous reports. Categorize the risk areas and prioritize the tests.

 

Step 2: Carry out Exploratory Tests on the identified areas of Risk

   Test with high risks can be prioritized 1st for exploratory tests. This approach makes sure that all high risk areas of the project are covered. Next we can carry out exploratory tests on the identified areas of risk.

     Step 3: Maintain a Test Charter to document for the test scenarios and outputs

Exploratory test without documentation can lead to chaos and redundant work. So it’s a good practice to maintain a test charter to document test scenarios or keep adding new scenarios as per tester’s exploration. This can later be assimilated with other testers work.

Step 4: Perform Gap analysis to check if any test scenarios have been missed.

Last step is to brainstorm and do gap analysis, to identify missed out scenarios. This helps us to achieve a test thoroughness. This also makes sure that test coverage is achieved.

How is Exploratory testing useful in Mobile applications?

Exploratory testing is useful in case of complex projects, where there is very little known about the project. This type of testing can be implemented in Acceptance phases (UAT) where end user can be asked to do a round of exploratory testing.

Mobile Testing has a very wide scope, because of the number of platforms, Device range and network scenarios. Mobile testing requires rapid feedback and is mostly done with agile technology. Mobile screens and navigations keeps changing as per design and OS, so the traditional test scripts become obsolete very soon. In this case, exploratory risk testing comes handy.

The tests can be run based on prioritization and defects can be reported. During exploration testers come across newer flows and hence chances to uncover newer defects.

 

 

 

 

Pros of exploratory testing

  • Less preparation is needed, important bugs are found quickly, and the approach tends to be more intellectually stimulating to execute than scripted tests.
  • Testers can derive conduct tests based on previous results to guide their future testing on-the-fly. They do not have to complete a current series of scripted tests before focusing in on or moving on to exploring a more target rich environment.

 

Challenges of exploratory testing

  • Exploratory testing requires highly skilled testers who are able to produce results quickly and consistently
  • Defining Exit criteria for exploratory testing becomes difficult.
  • If not documented properly, exploratory test can lead to clutter and make Test reporting difficult.

 

Conclusion

Main objective of testing is to achieve quality while working with deadlines. Exploratory testing can be combined with several other testing techniques and contribute to achieving the goal. The best results can be achieved using this technique.

This paper only covers a few ways that adding exploratory testing to your testing regimen can improve your test coverage and help reduce risk. It can take the testing to a newer level .There are many, many other advantages, Pros and Cons not covered here in this paper. So keep exploring!

 

 

 

Learning Outcome

Main objective of testing is to achieve quality while working with deadlines. Exploratory testing can be combined with several other testing techniques and contribute to achieving the goal. The best results can be achieved using this technique.

This paper only covers a few ways that adding exploratory testing to your testing regimen can improve your test coverage and help reduce risk. It can take the testing to a newer level .There are many, many other advantages, Pros and Cons not covered here in this paper. So keep exploring!

 

 

 

Target Audience

Testers,PM

schedule Submitted 1 year ago

Comments Subscribe to Comments

comment Comment on this Proposal
  • Aditya Garg
    By Aditya Garg  ~  1 year ago
    reply Reply

    Do you think you would be able to complete this presentation in 20 minutes. 

    Are you going to give a practical explanation or cover the theory part.


  • Liked Archana Sridhar
    keyboard_arrow_down

    Archana Sridhar - Automation Testing with Devops in Agile methodolgy

    20 Mins
    Talk
    Beginner

    Abstract

    What is DevOps?

    DevOps – a portmanteau of Development & Operations – is a software development methodology which looks to integrate all the software development functions from development to operations within the same cycle.

    This calls for higher level of coordination within the various stakeholders in the software development process (namely Development, QA & Operations).

     

    Devops is not a tool, it’s not a methodology, but it’s a process of integrating software development and quality. Devops is not release management system, it integrates with release management system by standardizing the process.

     

    Devops Trends

    Traditional Development teams and Operations departments normally have completely different motivations and deliverables. Development keeps changing things based on user or market demands for new features (and fixes of existing features). Operations try to keep everything as constant as possible in order to deliver maximum availability and stability of software services. These two contradicting goals create a bottleneck when it comes to delivery of business value.

     

    Why DevOps: Automated Deployment

    Most companies are adopting this Devops model in order to facilitate a practice of automated software deployment. DevOps provides the structure that enables teams to push software out as a service on a weekly, or daily, or even hourly basis. The traditional concept of a “software releases” transforms into a continuous cycle of service improvement.

    DevOps is a logical evolution in removing all the obstacles to getting high-quality software in the hands of customers. It smoothens the process for agile development and continuous integration, automating the process makes it easier, is more transparent and leads to higher quality.

     

    Beyond Functional Testing: Automation for Load Testing, Stress Testing, and Performance Testing

    As processes are maturing and Devops is moving at high speed, you may want to integrate performance testing, stress tests and load tests along with it. It’s about creating a continuous process to both check and monitor the software services.

    Once the processes are mature and stable enough to be moved to Devops, functional testing can be integrated with load, stress and performance, to continuously integrate with the process to aid more control and structured way of operation. Performance tests can be integrated to check the performance over a period of time on real time data rather than idealistic data. Similarly load and stress can be tested over a period of time.

     

    Devops in Real world

    DevOps practices promise bridge all communication gaps that usually exist between software engineers, IT, QA and other parts of the business. DevOps should create complete transparency within the business to facilitate agile planning and prompt decision-making. So, as the organization integrates DevOps, it makes services continuous.

    DevOps projects must identify and align with business needs, it always ask why you're taking up a DevOps approach and what success will look like. DevOps specific goal will be to shorten the cycle to fix bugs or grapple with fewer bugs in each new release, improving customer retention.

     

    Devops and Cloud computing

    DevOps developers successfully use private or public cloud to create more complex environments that simulate production use for version testing. The scalable, self-service nature of cloud computing allows developers to provision and migrate new versions for testing with little -- if any -- reliance on IT administrators. This makes testing faster and simpler than traditional provisioning requests to IT.

    Public cloud is usually preferable because it offers high scalability and self-service without any danger of exhausting local IT resources. When testing is complete, the public cloud resources can be released to save money until the next testing cycle starts.

    Approach for Integrating Devops and Automated Testing in cloud

    Step 1: Get proper Test automation in place

    Since a key aspect of DevOps is to ensure that deployments are made continuously, as well as effectively, the need to for having tests in place is pretty obvious – untested and faulty code has the potential to mitigate the benefits of DevOps.

    Step 2: Choosing the right CI Integration and integrating it

    Integrating the automation suite with a build deployment tool to enable centralized execution and reporting.

    Step 3: Making customizations as requirements for organization and cloud integration

    The customizations made can be pertaining to code and testing framework, to check mode of reporting, environment availability. The integrations related to cloud must be made.

    Step 4: Monitoring Results

    We can also use a small subset of existing high priority test cases to be executed periodically on production to actively monitor the environment.

    Challenges in implementing Devops

    1. The organization processes and practices must be matured enough for Devops.
    2. Exploratory or ad-hoc testing may at first appear to have no room in DevOps.
    3. DevOps is still a relatively young method, partly experimental, so support is solely dependent on internet. 

     

    References & Appendix

     

    Author Biography

    Archana Krushnan is currently working as Test Analyst at Nihilent Technologies Pvt Ltd. She has been in the field of Software Testing for 5.5 years and worked on several domains such as Telecom, Ecommerce and Healthcare. Her interests include exploring new things in testing and sharing on her blog. She has previous experiences of working with companies like Reliance Communications Ltd and Fab.com. She has good knowledge in Selenium, Jmeter and Soap UI. She is ISTQB Foundation Level Certified.

  • Liked madhavi
    keyboard_arrow_down

    madhavi / YogeSh Karanje - Understanding Agile Iteration

    20 Mins
    Talk
    Beginner

    The heartbeat of Agile development is the iteration – the ability of the team to create working, tested, value delivered code in a short time box – with the goal of producing an increment of potentially shippable code at the end of each iteration. This is a significant challenge for the team, and mastering this skill takes guidance and practice. This white paper describes the basic iteration pattern and the activities that a team engages in to meet this key challenge.

  • Liked YogeSh Karanje
    keyboard_arrow_down

    YogeSh Karanje / Nagesh Deshpande - How IoT is changing our Lives

    20 Mins
    Talk
    Beginner

    This paper provides an overview of the Internet of Things (IoT). It gives information how IoT is changing our lives in different fields. It gives brief information for applications in media, environmental monitoring, tag devices, energy management, medical and healthcare fields.

    How IoT is improving our life and adding the accuracy and flexibility. This paper highlights the major areas where we are directly or indirectly using IoT enabled applications. How current revolution in Internet, mobile, and machine-to-machine (M2M) technologies can be seen as the first phase of the IoT.This paper also highlights the Enabling Technologies for the IoT and given the information about the technologies widely used in today’s mobile phones

  • Liked Archana Sridhar
    keyboard_arrow_down

    Archana Sridhar - Continuous Integration with BDD for faster and quality release in Agile

    20 Mins
    Talk
    Intermediate

    This paper is to elaborate the concept of Behaviour Driven Development (BDD) with the help of Cucumber approach and how it will be useful in testing. This testing technique collaborates with both Testers and Business objectives. It can be applied to all testing levels and can be executed by all users. Main objective of this approach is to make tests easier to read and execute. Tests if easily readable can be execute by business users and they can get user friendly reports without much hassle of understanding and executing code.

    BDD approach basically focuses on the user and how system responds. BDD integrated with CI(Continuous Integration tool) can be very beneficial to fast paced projects, very common these days.BDD and CI approach together will bring development practices and performance, alignment with the business, measurement, effectiveness, efficiency, and non-learning solutions all together this solution if implemented can work and deliver exceptional results.

    BDD uses examples in conversation to illustrate behaviour and is therefore very useful in helping clarify requirements. Good BDD frameworks we can reduce regression test cycles, saving you time and money. Successful BDD means there is the potential to deliver defect-free software at point of release which operates as expected with reduced complexity