Network Anomaly Detection and Root Cause Analysis
Modern telecommunication networks are complex, consist of several components, generate massive amounts of data in the form of logs (volume, velocity, variety), and are designed for high reliability as there is a customer expectation of always on network access. It can be difficult to detect network failures with typical KPIs as the problems may be subtle with mild symptoms (small degradation in performance). In this workshop on network anomaly detection we will present the application of multivariate unsupervised learning techniques for anomaly detection, and root cause analysis using finite state machines. Once anomalies are detected, the message patterns in the logs of the anomaly data are compared to those of the normal data to determine where the problems are occurring. Additionally, the error codes in the anomaly data are analyzed to better understand the underlying problems. The data preprocessing methodology and feature selection methods will also be presented to determine the minimum set of features that can provide information on the network state. The algorithms are developed and tested with data from a 4G network. The impact of applying such methods is the proactive detection and root cause analysis of network anomalies thereby improving network reliability and availability.
Outline/Structure of the Talk
Introduction
Anomaly Detection Overview
Methodology
Experimental Results
Summary
Learning Outcome
Participants will learn about data pre-processing for data sets with many variables (numeric and categorical), feature selection, application of principal component analysis, and application of finite state machine methods.
Target Audience
People interested in learning about the application of machine learning techniques to real problems.
Prerequisites for Attendees
Knowledge of basic machine learning techniques and statistics.
Video
Links
schedule Submitted 5 years ago
People who liked this proposal, also liked:
-
keyboard_arrow_down
Dr. Dakshinamurthy V Kolluru - ML and DL in Production: Differences and Similarities
45 Mins
Talk
Beginner
While architecting a data-based solution, one needs to approach the problem differently depending on the specific strategy being adopted. In traditional machine learning, the focus is mostly on feature engineering. In DL, the emphasis is shifting to tagging larger volumes of data with less focus on feature development. Similarly, synthetic data is a lot more useful in DL than ML. So, the data strategies can be significantly different. Both approaches require very similar approaches to the analysis of errors. But, in most development processes, those approaches are not followed leading to substantial delay in production times. Hyper parameter tuning for performance improvement requires different strategies between ML and DL solutions due to the longer training times of DL systems. Transfer learning is a very important aspect to evaluate in building any state of the art system whether ML or DL. The last but not the least is understanding the biases that the system is learning. Deeply non-linear models require special attention in this aspect as they can learn highly undesirable features.
In our presentation, we will focus on all the above aspects with suitable examples and provide a framework for practitioners for building ML/DL applications.
-
keyboard_arrow_down
Dr. Manish Gupta / Radhakrishnan G - Driving Intelligence from Credit Card Spend Data using Deep Learning
Dr. Manish GuptaDirector - Machine Learning & Data ScienceAmerican ExpressRadhakrishnan GVice President, Commercial Decision Science and Machine Learning ResearchAmerican Expressschedule 5 years ago
45 Mins
Talk
Beginner
Recently, we have heard success stories on how deep learning technologies are revolutionizing many industries. Deep Learning has proven huge success in some of the problems in unstructured data domains like image recognition; speech recognitions and natural language processing. However, there are limited gain has been shown in traditional structured data domains like BFSI. This talk would cover American Express’ exciting journey to explore deep learning technique to generate next set of data innovations by deriving intelligence from the data within its global, integrated network. Learn how using credit card spend data has helped improve credit and fraud decisions elevate the payment experience of millions of Card Members across the globe.
-
keyboard_arrow_down
Srijak Bhaumik - Let the Machine THINK for You
20 Mins
Demonstration
Beginner
Every organization is now focused on the business or customer data and trying hard to get actionable insights out of it. Most of them are either hiring data scientists or up-skilling their existing developers. However, they do understand the domain or business, relevant data and the impact, but, not essentially excellent in data science programming or cognitive computing. To bridge this gap, IBM brings Watson Machine Learning (WML), which is a service for creating, deploying, scoring and managing machine learning models. WML’s machine learning model creation, deployment, and management capabilities are key components of cognitive applications. The essential feature is the “self-learning” capabilities, personalized and customized for specific persona - may it be the executive or business leader, project manager, financial expert or sales advisor. WML makes the need of cognitive prediction easy with model flow capabilities, where machine learning and prediction can be applied easily with just a few clicks, and to work seamlessly without bunch of coding - for different personas to mark boundaries between developers, data scientists or business analysts. In this session, WML's capabilities would be demonstrated by taking a specific case study to solve real world business problem, along with challenges faced. To align with the developers' community, the architecture of this smart platform would be highlighted to help aspiring developers be aware of the design of a large-scale product.
-
keyboard_arrow_down
Venkatraman J - Detection and Classification of Fake news using Convolutional Neural networks
20 Mins
Talk
Intermediate
The proliferation of fake news or rumours in traditional news media sites, social media, feeds, and blogs have made it extremely difficult and challenging to trust any news in day to day life. There are wide implications of false information on both individuals and society. Even though humans can identify and classify fake news through heuristics, common sense and analysis there is a huge demand for an automated computational approach to achieve scalability and reliability. This talk explains how Neural probabilistic models using deep learning techniques are used to classify and detect fake news.
This talk will start with an introduction to Deep learning, Tensor flow(Google's Deep learning framework), Dense vectors (word2vec model) feature extraction, data preprocessing techniques, feature selection, PCA and move on to explain how a scalable machine learning architecture for fake news detection can be built.
-
keyboard_arrow_down
Dr. Savita Angadi - What Chaos and Fractals has to do with Machine Learning?
45 Mins
Talk
Advanced
The talk will cover how Chaos and Fractals are connected to machine learning. Artificial Intelligence is an attempt to model the characteristics of human brain. This has lead to model that can use connected elements essentially neurons. Most of the biological systems or simulation related developments in neural networks have practical results from computer science point of view. Chaos Theory has a good chance of being one of these developments. Brain itself is an good example of chaos system. Several attempts have been made to take an advantage of chaos in artificial neural systems to reproduce the benefits that have met quite a bit success.
-
keyboard_arrow_down
Venkatraman J - Hands on Data Science. Get hands dirty with real code!!!
45 Mins
Workshop
Intermediate
Data science refers to the science of extracting useful information from data. Knowledge discovery in data bases, data mining, Information extraction also closely match with data science. Supervised learning,Semi supervised learning,Un supervised learning methodologies are out of Academia and penetrated deep into the industry leading to actionable insights, dashboard driven development, data driven reasoning and so on. Data science has been the buzzword for last few years in industry with only a handful of data scientists around the world. The industry needs more and more data scientists in future to solve problems using statistical techniques. The exponential availability of unstructured data from the web has thrown huge challenges to data scientists to exploit them before driving conclusions.
Now that's overload of information and buzzwords. It all has to start somewhere? Where and how to start? How to get hands dirty rather than just reading books and blogs. Is it really science or just code?. Let's get into code to talk data science.
In this workshop i will show the tools required to do real data science rather than just reading by building real models using Deep neural networks and show live demo of the same. Also share some of the key data science techniques every aspiring data scientist should have to thrive in the industry.