Testing your Bot!

Chatbots are one of the most widely adopted AI/ML implementations in the business sector. A chatbot is an intelligent machine used to imitate human conversation through text and voice commands. Today bots are widely used as a personal assistant, customer service, HR, sales and marketing to name a few. In short, bots are everywhere and we rely on them to a certain extent, this makes it extremely important to assure the quality of the chatbots and test them thoroughly. They are built using NLU/NLP-Services (Natural language understanding and processing) and are subjected to constant training and improvement which has direct impact on tests. Voice based bots like Siri and Alexa depend on speech recognition technologies. As the chatbots user do not have any barriers and due to the unpredictable user’s behavior it becomes utmost difficult to verify the correctness on the output. In this talk, we will discuss how the chatbots are different as compared to the other applications and the challenges they bring onto the table while verifying their behavior, and focus on the testing strategies and automation testing of the bots.

 
 

Outline/Structure of the Talk

  • How ChatBots are different from the other Application?
  • What are the challenges in testing the ChatBots?
  • How do you test the ChatBots?
  • Testing Strategies for ChatBots
  • Tools for ChatBot Testing

Learning Outcome

  • Understand the technologies used to build Chatbots and how they make it different than other application for testing
  • Begin with ChatBot testing
  • Test Strategy for chatbots
  • Tools for testing Chatbots

Target Audience

QA, Dev, QA Managers, Product Managers/Owner

schedule Submitted 1 year ago

Public Feedback

comment Suggest improvements to the Speaker

  • Liked David Bernstein
    keyboard_arrow_down

    David Bernstein - Five Development Practices for Agile DevOps

    David Bernstein
    David Bernstein
    To Be Agile
    schedule 1 year ago
    Sold Out!
    60 Mins
    Keynote
    Beginner

    These five development practices are core to Extreme Programming (XP) and essential for DevOps—automating the build for continuously integrating software as it is written, collaborating with team members through pair programming, practicing Agile design skills that enable testability, using test-first development to drive design, and refactoring code to reduce technical debt. Together, these five technical practices are proving to be essential for sustained success with Agile DevOps. However, many teams haven’t been exposed to the benefits of these practices or understand how to use them effectively. David Bernstein explores these engineering practices and their use in reducing risk and building quality in at every level of the development process. He makes the business case for these practices by showing how they address the inherent risks and challenges of building software. David then examines how these XP practices help us address the core issues of software development.

  • Liked Serhat Can
    keyboard_arrow_down

    Serhat Can - Managing incidents in microservices

    Serhat Can
    Serhat Can
    Technical Evangelist
    Atlassian
    schedule 11 months ago
    Sold Out!
    60 Mins
    Keynote
    Beginner

    Microservices require a shift in mindset and tooling. As monitoring is changing with more focus on distributed tracing because of the changing needs, the incident response must also adopt. In this presentation, I’ll answer “why” and “how” of on-call and incident response in microservices.

  • Liked Hikita Keiichi
    keyboard_arrow_down

    Hikita Keiichi - Terraform で自社サービスを便利に! Custom Provider開発におけるDevOpsへの取り組みのご紹介 〜かゆいところに手を届かせたい人へ〜

    45 Mins
    Talk
    Intermediate

    NTTコミュニケーションズでは自社クラウドとして "Enterprise Cloud 2.0(以下ECL2.0)" というサービスを提供している。

    私達のチームでは、現在上記サービス向けのオーケストレーション機能の開発を行っており、具体的には "Terraform" に対するプラグイン(Terraformの用語でCustom Providerと呼ぶ)の開発によりこれを実現しようとしている。

    Terraform自体はHashiCorp社により開発されたDevOpsツールであり、クラウド上のリソースを定義ファイルの状態に合致するように生成・操作が可能である上、APIを保有するシステム(ECL2.0も同様)であればCustom Providerを開発することでこのツールに組み込むことができ、利便性・拡張性が非常に高い。

    一方で、Custom Provider開発を通じたCI、特にリグレッションの観点で見ると、いくつか考え無くてはならない課題があるとも感じており、私達のチームではそれに対して各種の工夫を交えながら開発を進めてきた。

    本プレゼンテーションでは、Terraformのご紹介、Custom Provider開発におけるDevOpsの取り組みに関するご説明、Custom Provider開発の過程を通じて感じた課題や具体的な改善点やTips(実は意外とかゆいところに手が届く使い方がちゃんとできる!)等について、ツール/コードの紹介やデモを交えてご紹介させて頂く。

  • Liked Mitsuyuki Shiiba
    keyboard_arrow_down

    Mitsuyuki Shiiba - Service Operation Centered Development - サービス運用をまんなかにおいた開発

    20 Mins
    Talk
    Beginner

    サービス運用をまんなかにおいた開発についてお話します。

    僕は2010年から楽天の大阪支社でウェブアプリケーションエンジニアとして仕事をしています。僕のいる部署は中規模から小規模のたくさんのサービスを担当していて、1つのチームまたはグループでサービスの開発と運用の両方を担当しています。

    サービスの開発と運用の両方を担当しているため、僕らはサービスの運用のことを常に考えながら開発に取り組んでいます。運用のことを考えずに開発をすると全てが自分たちに跳ね返ってくるからです。

    このセッションでは、サービスの運用をまんなかに置いて開発をするときに、どのようなことを考えるか、また、どのように他のチームや組織と向き合うか、について自分の経験を元にお話したいと思います。

    資料は英語ですが、セッションは日本語です。

  • Liked Masato Ishigaki
    keyboard_arrow_down

    Masato Ishigaki - Data-Driven x DevOps

    Masato Ishigaki
    Masato Ishigaki
    Product Owner
    DMM.com
    schedule 1 year ago
    Sold Out!
    20 Mins
    Talk
    Beginner

    DevOpsの領域だけでは対応しきれない、プロダクトのグロースの部分をData-Drivenの考え方でどういうふうに開発プロセスを補完していくかについて述べていきます。

    LeanStartupのBMLループを開発プロセスとして各ループ遷移(Learn→Idea,etc...)の中でどういったことを意識しながらData-DrivenとDevOpsの考え方を使ってプロセス設計していくかについて、KGI,CSF,KPIの設定からビッグデータ基盤・データ分析基盤、優れた指標などDMM.comの事例を用いりながら説明していくます。

  • Liked Arata Fujimura
    keyboard_arrow_down

    Arata Fujimura - DevOps導入支援、始めました

    Arata Fujimura
    Arata Fujimura
    Manager
    Classmethod, Inc.
    schedule 10 months ago
    Sold Out!
    20 Mins
    Talk
    Beginner
    クラスメソッド社では2018年7月から、DevOps導入支援サービスを始めました。
    一般的にDevOpsという言葉の定義は明確ではありません。そこで、支援サービスを始める上でまずは我々なりのDevOpsの定義を行ない、リーンキャンバスなどを使って仮説を立て、仮説の検証を行ないながら支援サービスの方向性を模索してきました。
    当セッションでは、DevOps導入支援サービスの立ち上げから今に至るまでにやってきたこと、その結果からわかったこと、次にやろうとしていることについて、ざっくばらんにお話しさせて頂きます。
  • Liked Hideya Ikeda
    keyboard_arrow_down

    Hideya Ikeda - 富士通のSIプロジェクトがどのようにDevOpsに取り組んでいるか

    Hideya Ikeda
    Hideya Ikeda
    Manager
    FUJITSU LIMITED
    schedule 11 months ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    SIビジネスはなくなると言われ始めて数年、富士通のビジネスの中心はまだまだSIです。

    • ウォーターフォール、多層請負契約が中心のSIプロジェクトがDevOpsというテーマにどう取り組んでいるのか?
    • 富士通のSIプロジェクトは年間数千件。どのように標準化、適用推進活動をしているのか?
    • これらのSIプロジェクトでは、どんなツールを使って、どんなプラクティスを実践しているのか?
    • DevとOpsを隔てる壁へのアプローチは?
    • そして、今後SIビジネスがなくなっていくことにどう備えているか?

    こういった内容を泥臭い実践事例や苦労話を交えてお話ししたいと思います。

  • Liked Kohsuke Kawaguchi
    keyboard_arrow_down

    Kohsuke Kawaguchi - Wasted Gold Mine & What Data Can Do To DevOps

    Kohsuke Kawaguchi
    Kohsuke Kawaguchi
    CTO
    CloudBees, Inc.
    schedule 10 months ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    As CTO of CloudBees and the creator of Jenkins, I get to see lots of real-world software development. Our automation in software development is sufficiently broad that it is producing lots of data, but by and large most of those are simply thrown away. Yet at the same time, the management is feeling like they are flying blind because they have little insight! In this talk, Kohsuke will discuss how we collectively seem to miss the golden opportunity to improve the software development process itself, based on data. IOW, learning is lacking at the organizational level, let alone "machine" learning!

  • Liked Seiji Kawakami
    keyboard_arrow_down

    Seiji Kawakami - DevOpsに関わるPOとして大切なこと

    Seiji Kawakami
    Seiji Kawakami
    Scrum Master
    KDDI
    schedule 10 months ago
    Sold Out!
    20 Mins
    Talk
    Intermediate

    DevOpsを上手に回すために、開発チームとビジネスが一体となって動く必要があります。

    一言でビジネスと言っても多種多様なセクションを含んでおり、なかなか一人で全てを回すことは現実的に難しい状況だと思います。

    上手くDevOpsを回すための作戦としPOチームを組閣、社内の業務改善プロジェクトがどのように動いていったのか?

    実際にPOチームに編入されたメンバーと、開発チームの双方の立場からプロジェクトを振り返ります。

  • Liked Atsushi Fukui
    keyboard_arrow_down

    Atsushi Fukui / Yukitaka Ohmura - DevOps with Database on AWS

    20 Mins
    Talk
    Beginner

    DevOps のアプローチでシステム開発に取り組む上で、データベースに対する変更をどのように扱うかは難しい問題です。このセッションでは、ライフサイクルが異なるアプリケーションとデータベースに対する変更を適切に管理することを目的として、代表的なデータベーススキーマ変更の手法を紹介し、それを CI/CD パイプラインに組み込む方法について考えます。

  • Liked Julian Totzek-Hallhuber
    keyboard_arrow_down

    Julian Totzek-Hallhuber - There is no such thing as DevSecOps!

    20 Mins
    Talk
    Intermediate

    DevOps/DevSceOps is a journey and requires a lot of changes in your organization and the affected teams. I would like to take you on a journey from waterfall to DevSecOps. Looking at the traditional approach and the issues you will face or you are actually facing while implementing DevOps. Moving on with what will happen if you set up DevOps without security and finishing with best practices approaches and tips how to get security included in your DevOps process to finally implement a full, secure DevSecOps process.

  • Liked Lim Yang Wei
    keyboard_arrow_down

    Lim Yang Wei - To DevOps, or not, it's all about Trust, Freedom and Responsibility

    20 Mins
    Talk
    Beginner

    In our team, the thing that we valued most is how our codes are making impact to business. It can be deploying feature that create new profit, fixing bug, increasing deployment time and etc.

    Making impact means in the end codes have to be deployed to production. We never set rules related to deployment. Anyone, anytime, anywhere. We even do have so-called "First day deployment" operation which make sure new member make their first deployment to production within 24 hours since they join our team. How do we make sure that works ? We trust. In our team, everyone is AWS admin from the first day. We want make sure everyone has the essential tools to setting up infrastructure, coding, releasing, monitoring. Human being, when given trust they take full responsibilities. Responsibility to complete work and hence making impact to business.

    It sounds like psychology, right ? Or I will just say culture.

    In the end, it's all about trust, freedom and responsibility. Let me show you how and why we inject (or being injected) these cultures into our team, how we improve and make sure it's getting better.

  • Liked Michael Migliacio
    keyboard_arrow_down

    Michael Migliacio / David Nguyen - What is DevOps Coaching? The Art, Science, and Culture Of Engineering Enablement (DevOpsコーチングは何ですか?技術、文化、科学)

    45 Mins
    Talk
    Beginner

    井の中の蛙大海を知らず - "A frog in a well does not know the great sea."

    Software engineering may be difficult, but fostering a working environment that enables skilled engineers to perform their best can sometimes seem downright impossible. Every day, many engineering teams are battling a messy whirlwind of forces like unmovable deadlines, impostor syndrome, psychological safety issues, personnel/leadership conflicts, fierce technological preferences, and more. With teams more distributed all over the world than ever before, cultural differences can exacerbate many of these difficulties.

    As a software engineering coach, my job is to not only introduce new technology to software teams currently looking to transition to DevOps, but to strengthen their working relationships within their organization. Coaches aren’t simply technical instructors. Rather, they are change agents that guide a team towards better outcomes for their project as well as their interactions with one another.

    In this presentation, I will discuss tips, tricks, and techniques that technical leaders and managers alike can utilize to better coach engineering teams, including concepts like the definition of empathy (and, more importantly, what doesn't count), the trust-influence relationship model, introducing new technologies in a meaningful and consumable way, and a 5-step process to provide teams confidence to own their new DevOps solutions moving forward.

    スピーカーは日本語が分かるから、セッションコンテンツの日本語版もあります。

  • Liked Matty Stratton
    keyboard_arrow_down

    Matty Stratton - The Psychology of Chaos Engineering

    45 Mins
    Talk
    Intermediate

    Chaos Engineering, failure injection, and similar practices have verified benefits to the resilience of systems and infrastructure. But can they provide similar resilience to teams and people? What are the effects and impacts on the humans involved in the systems? This talk will delve into both positive and negative outcomes to all the groups of people involved - including users, engineers, product, and business owners.

    Using case studies from organizations where chaos engineering has been implemented, we will explore the changes in attitude that these practices create. This talk will include a brief overview of chaos engineering practices for unfamiliar members of the audience, but the main focus will be on human elements. I will discuss successful implementations, as well as challenges faced in teams where chaos was a “success” from a technical perspective, but contained negative impact for the people involved.

    After seeing this talk, attendees will have a better understanding of the human factors involved in chaos engineering, good practices to care for the people and teams working with chaos, and be even more excited about this practice.

  • Liked ARUN SINGH
    keyboard_arrow_down

    ARUN SINGH - A Journey towards self healing infrastructure

    ARUN SINGH
    ARUN SINGH
    Site Reliability Engineer
    Adobe Systems
    schedule 11 months ago
    Sold Out!
    20 Mins
    Talk
    Beginner

    In this talk I will be sharing my Journey towards self healing infrastructure, which started with implementing self healing via SaltStack then Moving towards Rundeck and now finalised on StackStrom and How this self healing is helping us to resolve thousands of alerts without SRE intervention? This will include the pros-cons and high-lows we encountered in our journey while looking for a suitable workflow engine. How we added new things in existing technologies to make it fit for our environment and how user can learn from our mistakes and setup in your environment within hours.

    I will be talking about different workflow engines and a brief comparison between them. What to be used when? and why not? etc

  • Liked David Nguyen
    keyboard_arrow_down

    David Nguyen / Michael Migliacio - Modern Web Testing with Cypress

    120 Mins
    Workshop
    Advanced

    In today’s ever-changing and competitive landscape of online retailers, companies invested heavily into end to end testing practice to ensure the best quality for their product. The seamless shopping experience between mobile and physical store is the universal high bar for success.

    Retailers that embraced omni-channel mobile first strategy often build their system top a multi-tenant architecture. In order to make all tenants work in harmony as one unit, across all engineering teams, Test Driven Development (TDD) must be one of the Objective Key Result (OKR). The center piece of that effort is the investment in automated end-to-end testing. During this session, come join David for a in depth look of how Cypress.io enabled dev team incorporate automated testing for both internal web applications, and its public facing e-commerce site.