Testing Conversational AI - Stragegy to Automation

Last year was dominated by the smart devices and voice based home assistants. These use the conversational interfaces unlike other application to interact with. They are built using advanced algorithms, ranging from pattern and expression matching engines to natural language processing and AI/Machine learning techniques. These systems are constantly learning by themselves improving the interactions with the user bringing up the challenge in the testing world of non-deterministic output. To such interfaces, natural language is the input and we humans really love having alternatives and love our synonyms and our expressions using emojis gifs and pictures. Testing in this context moves to clouds of probabilities.

In this session I will cover the strategy for testing such interfaces, testing the NLP models and sharing experience on how to automate these tests and add it to the CI/CD build pipelines.


Key learning:
* How What and why of a conversational interface?
* How can I build my testing approach for such an interface?
* What from my current tool set can I use for this new context?
* How do I automated and add it for my CI/CD pipeline for instant feedback?
* How do I measure the quality?

 
 

Outline/Structure of the Talk

  • How Conversational Interfaces are different from the other Applications?
  • What are the challenges in testing the Conversational Interfaces?
  • How do you test the Conversational Interfaces?
  • Testing Strategies for Conversational Interfaces
  • How do we automate the Conversational Interfaces
  • How do we monitor Conversational Interfaces in production

Learning Outcome

  • Understand the technologies used to build Conversational Interfaces and how they make it different than other applications for testing
  • Begin with Conversational Interfaces testing
  • Build test cases for Conversational Interfaces
  • Automate Conversational Interfaces

Target Audience

QA, Project Managers, Product Owners, BA, UI/UX

schedule Submitted 3 months ago

Public Feedback

comment Suggest improvements to the Speaker