With the seismic shift in industry and development of new technologies emerging, QA’s testing approaches are also changing, we must know the right strategies and algorithms to test. One of the latest technology emerging is Artificial Intelligence and Machine Learning. And its applications like Self driving cars, Virtual Assistants are everywhere. They have great impact in our life and most of our decisions, behaviour & destinations depend on them.

So in this presentation/Workshop i would like to present all the ways/strategies/ challenges faced while testing AI/ML applications. Join me in creating a Machine Learning application from scratch and then take it to testing stage, creating edge case scenarios and validations.

Time Management: To make sure that all people are upto date with with setup for hands-on, i will be sharing this document with the participants 12 days before in a temp slack channel, where they can share the progress and ask queries to resolve them quickly.
*No internet is required for participants if they follow the setup doc.

 
 

Outline/Structure of the Workshop

Presentation contains following three modules for attendee of different experience types:

Initiation with brief talk on AI/ML concepts [10 minutes]

  • Creating a Beer Wine Classifier [Total -> 30 minutes]
    • Understanding the problem, and coming out with human based solution of same [5 min]
    • Choosing the right machine learning algorithm to test [5 min]
    • Brainstorming possible test cases [5 min]
    • Creating a model [5 min]
    • Executing test case and validating with happy path [5 min]
    • Testing with edge case scenarios [5 min]

  • Image Classifier(CLI Version) [Total -> 30 minutes]
    • Understanding the problem, and coming out with human based solution of same [5 min]
    • Choosing the right tensorflow library to test [5 min]
    • Creating data sets [2 min]
    • Identifying multiple scenarios [3 min]
    • Creating a model [5 min]
    • Executing test case and validating with happy path [5 min]
    • Testing with edge case scenarios [5 min]
  • Android Real time Image Classifier [Total -> 10 minutes]
    • Exporting our model created in second activity to mobile application [5 min]
    • Validating the model with real time images by hovering camera openly at objects [5 min]
QA [10 min]
SUMMARY:
I will be initiating with theory and explain modular topics via slides in presentation attached.
And parallelly doing the workshop with attendees.

I will be sharing about the model creation steps, strategies and challenges that one may require/face while testing AI/ML apps

Learning Outcome

  • What is AI/ML
  • How technology is shifting towards AI, ML
  • Where does a QA step in
  • Writing test cases for happy paths and edgy scenarios
  • Challenges while testing AI,ML application
  • Maintaining test suite and updating with new upcoming data

Target Audience

This workshop is for audience of all levels (beginner, intermediate, Expert)

Prerequisites for Attendees

Understanding of STLC and attentiveness in Machine Learning and Artificial Intelligence.

schedule Submitted 7 months ago

Public Feedback

comment Suggest improvements to the Speaker
  • Anand Bagmar
    By Anand Bagmar  ~  6 months ago
    reply Reply

    Hi Tarun,

    I like this proposal, but have a few concerns related to time management. Can you please take a look and clarify the below questions?

    • You have mentioned that this workshop is for audience of all levels (beginner, intermediate, Expert). How do you plan to ensure everyone will be at the same pace (rather, no one lagging behind) as you go through the workshop?
    • Also, have you conducted this workshop before in 90-min?
    • Apart from having docker installed, is there no other pre-setup required? In your outline structure, there is no reference and time allocation for the setup required to be done.
    • Is Internet a requirement for your workshop? That does not seem to be mentioned in the requirements
    • What aspects of the workshop will the attendees also be doing? How do you intend to support them in the process, and also manage time?

    Regards, Anand

    • Tarun Maini
      By Tarun Maini  ~  6 months ago
      reply Reply

      Hi Anand, Thanks for kind words. 
      Please find my replies below:
      1. "This workshop is for all audience levels" corresponds to the 3 activity levels:
         a. Beginners: Basic Beer-Wine Classifier
         b. Intermediate: Image Classification(CLI Based, Non Dynamic)
         c. Advanced: Image Classification( Dynamic/Real time via mobile android device)
       I hope if everyone follows above all activities sequentially, they can achieve same pace/level.

      2. Yes i have performed same activity in 90 minutes, in which first two activities were parallely performed by attendees with me and third one was demoed by me only as it requires Android SDK setup, and Android Device.

      3. Above two activities can be performed by attendees by two ways:
         a. Either git cloning mentioned repos  and installing individual tools/utils like tensorflow, etc
        OR
         b. Just installing DOCKER and pulling images with required tools/utils already burned to perform activity

       *Latter one is simple and quick
       * I share  this document a week before for setup, and create a temp slack channel for resolving attendees queries quickly, if any.

      4. Internet is not required while performing activity once setup is complete.

      5. I will be writing training cases at white board while interacting with audience actively, after explaining the concepts
         a. Then they will be creating model at their system with me demoing same, with training data developed above+some already present in code( They are free to modify, add of their own)
         b. They will be executing test cases against test cases present in code files already + some edge scenarios getting developed by discussing with audience interactively.

      Regarding Support, i will be sharing all the steps, commands in slack channel(or any medium) while workshop or in worst case people can pair, if something is not working at someone's system. That's also the best way of absorbing things.

      All the points written above are based on my previous experiences, i am very happy to alter ways and smoothen the journey.

      Regards,
      Tarun

  • Deepti Tomar
    By Deepti Tomar  ~  6 months ago
    reply Reply

    Hello Tarun,

    Thanks for your proposal!
    To help the program committee understand your presentation style, can you provide a link to your past recording or record a small 1-2 mins trailer of your talk and share the link to the same?

    Also, please update the proposal with the time-wise break up as mentioned in response to Manoj's comment.

    Thanks!

    • Tarun Maini
      By Tarun Maini  ~  6 months ago
      reply Reply

      Hello Deepti, Thanks for reviewing the proposal!
      Please find the link of drive containing some video samples of my last talk/workshops.
      And i have  updated the proposal time-wise break up above in abstract also.
      Let me know if more information is required.

  • Manoj Kumar
    By Manoj Kumar  ~  6 months ago
    reply Reply

    Hi Tarun,

    Thanks for submitting this proposal.  Is it possible for you to share a rundown of the 90 min version? 
    I'm keen to understand, How well the attendees will understand all the aspects in a 90 min version and got to see where exactly you will trim down!

    Thanks

    • Tarun Maini
      By Tarun Maini  ~  6 months ago
      reply Reply

      Hi Manoj, Please find rundown as below:
      Initiation with brief talk on AI/ML concepts  [10 minutes]

      • Creating a Beer Wine Classifier [Total -> 30 minutes]
        • Understanding the problem, and coming out with human based solution of same [5 min]
        • Choosing the right machine learning algorithm to test [5 min]
        • Brainstorming possible test cases [5 min]
        • Creating a model [5 min]
        • Executing test case and validating with happy path [5 min]
        • Testing with edge case scenarios [5 min]

       

      • Image Classifier(CLI Version) [Total -> 30 minutes]
        • Understanding the problem, and coming out with human based solution of same [5 min]
        • Choosing the right tensorflow library to test [5 min]
        • Creating data sets [2 min]
        • Identifying multiple scenarios [3 min]
        • Creating a model [5 min]
        • Executing test case and validating with happy path [5 min]
        • Testing with edge case scenarios [5 min]
      • Android Real time Image Classifier [Total -> 10 minutes]
        • Exporting our model created in second activity to mobile application [5 min]
        • Validating the model with real time images by hovering camera openly at objects [5 min]
      QA [10 min]
       
      SUMMARY:
      I will be initiating with theory and explain modular topics via slides in presentation attached.
      And parallelly doing the workshop with attendees.
      • Manoj Kumar
        By Manoj Kumar  ~  6 months ago
        reply Reply

        Thank you!

  • Maaret Pyhajarvi
    By Maaret Pyhajarvi  ~  6 months ago
    reply Reply

    Hi Tarun,

    do you have a 90 minute version of your workshop on experiencing AI + testing available?

    • Tarun Maini
      By Tarun Maini  ~  6 months ago
      reply Reply

      Hello Maaret, 
      Yes i do have 90 minute version of same workshop.


  • Liked Srinivasan Sekar
    keyboard_arrow_down

    Srinivasan Sekar / Sai Krishna - Testing And Observability in an Integrated Microservices environment

    45 Mins
    Case Study
    Intermediate

    Leading-edge applications are dynamic and adaptive in capabilities that require people to use increasingly dexterous tools and supporting infrastructure, including microservices. All of these applications leverage data in new ways. Decoration and tagging of data with intelligent meta-data have become more important than data itself. To keep up with evolving needs, enterprise devs across industries are shifting from traditional app architectures in favor of more fluid architecture for building data-centric applications.

    Microservices break traditionally structured applications into manageable pieces that can be developed and maintained independently. microservices are often decoupled, allowing for updates with little to no downtime, as the other components can continue running.

    Moving to distributed Microservices ecosystem brings its own challenges; Among them is the loss of visibility into the system, and the complex interactions now occurring between services. Monitoring these applications only reports the health of it but Observability provides granular insights about the behavior of the system along with rich content. In this talk, we will cover the difference of Monitoring and Observability, data path engineering challenges, pillars of observability, distributed tracing of various microservices, testing in distributed microservices ecosystem, automated observability, etc.

  • Liked Tarun Maini
    keyboard_arrow_down

    Tarun Maini - Infrastructure security from the eyes of QA in Devops team

    Tarun Maini
    Tarun Maini
    Senior Quality Analyst
    ThoughtWorks
    schedule 7 months ago
    Sold Out!
    45 Mins
    Tutorial
    Beginner

    Being a functional QA before, Tarun got the chance to grew up as an Infrastructure QA in Devops team for worlds’s first enterprise level Blockchain project where whole infrastructure is over cloud platform.

    Ensuring that all resources are spinning up properly was the main thing in blockchain because breakdown of any block/peers/node in blockchain can affect whole application.

    For the private network to work via quorum client all nodes should need to be in sync with each other to provide consensus for incoming transactions.

    In this talk Tarun will share his experiences and demo via a quick tutorial as how he wrote 1500+ test cases for INFRASTRUCTURE only that run completely under 60 seconds and at the same time ensure the security of infrastructure system at cloud resource e.g. AWS.

    And some incidents like:

    • How some private confidential data files were saved from being public and getting visible to world.
    • How access to some endpoints, ports visible to world were detected with tests & blocked.
    • How it was ensured that all application pods running continously on ec2 containers are healthy.

      both for infra as well as blockchain testing to make things run smoothly.