location_city Tokyo schedule Apr 21st 01:00 - 01:45 PM JST place Hall A people 10 Interested

Do bugs speak?

Yes, they do. People speak different languages like English, German, French, Chinese etc. But is communication to bugs possible? It is important to understand them, because they really tell us something. There is valuable information underlying the defects of a software, and information mining from defects promises for improvements in terms of quality, time, effort and cost.

Problem Definition

A comprehensive analysis on all created defects can provide precious insights about the product. For instance; if we notice that a bunch of defects heap together on a feature, we can conclude that the feature should be investigated and cured. Or we can make some observations about the severity or assignee of similar defects. Therefore, there are some potential patterns to be discovered under defects.

Wrap-up

Defect analysis is very important for QA people, and especially for QA managers. We utilize lots of aspects to get an idea about the product itself or our procedures. For instance while monitoring defect distribution across testing types, we will discuss how to get an idea about the quality of our testing approach. I.e whether we are applying all types in a balanced way. (functional, performance, documentation, etc.) Or over another graph, in which we track the gap between open defects and resolved defects, we will discuss what action items we can take when the gap widens. Finally, with ML assistance, we will see how we can reduce manual effort and cost.

Results & Conclusion

In this session, we discuss data mining from bugs and usage of ML in defect management. Objective of the study is:

  • To present in which ways defects can be analyzed
  • To present how ML can be used to make observations over defects
  • To provide empirical information supporting (b)

 

 
 

Outline/Structure of the Talk

* How to learn from bugs

* Metrics to learn from bugs

* Ways to collect metrics

 

Learning Outcome

Lessons Learned

After all the experience I have had to build a successful bug management process, I had insights about the most critical parts of building such as lifecycle:

  • What kind of environment we should build to be able to extract hidden patterns and valuable information from defects.
  • What kind of monitoring ways can be used to make the status visible in the most possible way. (Various pie charts, bar graphs and tables will be shown to demonstrate distribution of defects across different aspects. )
  • What can be the valuable information in each monitoring activity.
  • How can we classify/cluster defects using NLP techniques over various algorithms (including SVM, Decision Trees, Ensemble Methods) with benchmarking and results (accuracy rates).

The presentation aims to help also to attendees in terms of:

  • Mine valuable data from defects
  • Get insights about test cycles
  • Reduce defect assignment errors
  • Perform correct defect triage

 

Target Audience

Testers, PO, Developers

schedule Submitted 11 months ago
help