Network Anomaly Detection and Root Cause Analysis

location_city Bengaluru schedule Sep 1st 03:30 - 04:15 PM place Grand Ball Room 2 people 88 Interested

Modern telecommunication networks are complex, consist of several components, generate massive amounts of data in the form of logs (volume, velocity, variety), and are designed for high reliability as there is a customer expectation of always on network access. It can be difficult to detect network failures with typical KPIs as the problems may be subtle with mild symptoms (small degradation in performance). In this workshop on network anomaly detection we will present the application of multivariate unsupervised learning techniques for anomaly detection, and root cause analysis using finite state machines. Once anomalies are detected, the message patterns in the logs of the anomaly data are compared to those of the normal data to determine where the problems are occurring. Additionally, the error codes in the anomaly data are analyzed to better understand the underlying problems. The data preprocessing methodology and feature selection methods will also be presented to determine the minimum set of features that can provide information on the network state. The algorithms are developed and tested with data from a 4G network. The impact of applying such methods is the proactive detection and root cause analysis of network anomalies thereby improving network reliability and availability.

 
 

Outline/Structure of the Talk

Introduction

Anomaly Detection Overview

Methodology

Experimental Results

Summary

Learning Outcome

Participants will learn about data pre-processing for data sets with many variables (numeric and categorical), feature selection, application of principal component analysis, and application of finite state machine methods.

Target Audience

People interested in learning about the application of machine learning techniques to real problems.

Prerequisites for Attendees

Knowledge of basic machine learning techniques and statistics.

schedule Submitted 2 years ago

Public Feedback


    • Liked Dr. Dakshinamurthy V Kolluru
      keyboard_arrow_down

      Dr. Dakshinamurthy V Kolluru - ML and DL in Production: Differences and Similarities

      45 Mins
      Talk
      Beginner

      While architecting a data-based solution, one needs to approach the problem differently depending on the specific strategy being adopted. In traditional machine learning, the focus is mostly on feature engineering. In DL, the emphasis is shifting to tagging larger volumes of data with less focus on feature development. Similarly, synthetic data is a lot more useful in DL than ML. So, the data strategies can be significantly different. Both approaches require very similar approaches to the analysis of errors. But, in most development processes, those approaches are not followed leading to substantial delay in production times. Hyper parameter tuning for performance improvement requires different strategies between ML and DL solutions due to the longer training times of DL systems. Transfer learning is a very important aspect to evaluate in building any state of the art system whether ML or DL. The last but not the least is understanding the biases that the system is learning. Deeply non-linear models require special attention in this aspect as they can learn highly undesirable features.

      In our presentation, we will focus on all the above aspects with suitable examples and provide a framework for practitioners for building ML/DL applications.

    • Liked Dr. Manish Gupta
      keyboard_arrow_down

      Dr. Manish Gupta / Radhakrishnan G - Driving Intelligence from Credit Card Spend Data using Deep Learning

      45 Mins
      Talk
      Beginner

      Recently, we have heard success stories on how deep learning technologies are revolutionizing many industries. Deep Learning has proven huge success in some of the problems in unstructured data domains like image recognition; speech recognitions and natural language processing. However, there are limited gain has been shown in traditional structured data domains like BFSI. This talk would cover American Express’ exciting journey to explore deep learning technique to generate next set of data innovations by deriving intelligence from the data within its global, integrated network. Learn how using credit card spend data has helped improve credit and fraud decisions elevate the payment experience of millions of Card Members across the globe.

    • Liked Srijak Bhaumik
      keyboard_arrow_down

      Srijak Bhaumik - Let the Machine THINK for You

      Srijak Bhaumik
      Srijak Bhaumik
      Sr. Staff Software Developer
      IBM
      schedule 2 years ago
      Sold Out!
      20 Mins
      Demonstration
      Beginner

      Every organization is now focused on the business or customer data and trying hard to get actionable insights out of it. Most of them are either hiring data scientists or up-skilling their existing developers. However, they do understand the domain or business, relevant data and the impact, but, not essentially excellent in data science programming or cognitive computing. To bridge this gap, IBM brings Watson Machine Learning (WML), which is a service for creating, deploying, scoring and managing machine learning models. WML’s machine learning model creation, deployment, and management capabilities are key components of cognitive applications. The essential feature is the “self-learning” capabilities, personalized and customized for specific persona - may it be the executive or business leader, project manager, financial expert or sales advisor. WML makes the need of cognitive prediction easy with model flow capabilities, where machine learning and prediction can be applied easily with just a few clicks, and to work seamlessly without bunch of coding - for different personas to mark boundaries between developers, data scientists or business analysts. In this session, WML's capabilities would be demonstrated by taking a specific case study to solve real world business problem, along with challenges faced. To align with the developers' community, the architecture of this smart platform would be highlighted to help aspiring developers be aware of the design of a large-scale product.

    • Liked Venkatraman J
      keyboard_arrow_down

      Venkatraman J - Detection and Classification of Fake news using Convolutional Neural networks

      20 Mins
      Talk
      Intermediate

      The proliferation of fake news or rumours in traditional news media sites, social media, feeds, and blogs have made it extremely difficult and challenging to trust any news in day to day life. There are wide implications of false information on both individuals and society. Even though humans can identify and classify fake news through heuristics, common sense and analysis there is a huge demand for an automated computational approach to achieve scalability and reliability. This talk explains how Neural probabilistic models using deep learning techniques are used to classify and detect fake news.

      This talk will start with an introduction to Deep learning, Tensor flow(Google's Deep learning framework), Dense vectors (word2vec model) feature extraction, data preprocessing techniques, feature selection, PCA and move on to explain how a scalable machine learning architecture for fake news detection can be built.

    • Liked Dr. Savita Angadi
      keyboard_arrow_down

      Dr. Savita Angadi - What Chaos and Fractals has to do with Machine Learning?

      Dr. Savita Angadi
      Dr. Savita Angadi
      Sr. Analytical Consultant
      SAS
      schedule 2 years ago
      Sold Out!
      45 Mins
      Talk
      Advanced

      The talk will cover how Chaos and Fractals are connected to machine learning. Artificial Intelligence is an attempt to model the characteristics of human brain. This has lead to model that can use connected elements essentially neurons. Most of the biological systems or simulation related developments in neural networks have practical results from computer science point of view. Chaos Theory has a good chance of being one of these developments. Brain itself is an good example of chaos system. Several attempts have been made to take an advantage of chaos in artificial neural systems to reproduce the benefits that have met quite a bit success.

    • Liked Venkatraman J
      keyboard_arrow_down

      Venkatraman J - Hands on Data Science. Get hands dirty with real code!!!

      45 Mins
      Workshop
      Intermediate

      Data science refers to the science of extracting useful information from data. Knowledge discovery in data bases, data mining, Information extraction also closely match with data science. Supervised learning,Semi supervised learning,Un supervised learning methodologies are out of Academia and penetrated deep into the industry leading to actionable insights, dashboard driven development, data driven reasoning and so on. Data science has been the buzzword for last few years in industry with only a handful of data scientists around the world. The industry needs more and more data scientists in future to solve problems using statistical techniques. The exponential availability of unstructured data from the web has thrown huge challenges to data scientists to exploit them before driving conclusions.

      Now that's overload of information and buzzwords. It all has to start somewhere? Where and how to start? How to get hands dirty rather than just reading books and blogs. Is it really science or just code?. Let's get into code to talk data science.

      In this workshop i will show the tools required to do real data science rather than just reading by building real models using Deep neural networks and show live demo of the same. Also share some of the key data science techniques every aspiring data scientist should have to thrive in the industry.