Deep Neural networks are both computationally intensive and memory intensive, making them difficult to deploy on mobile phones and embedded systems with limited hardware resources and taking more time for Inference and Training. For many mobile-first companies such as Baidu and Facebook, various apps are updated via different app stores, and they are very sensitive to the size of the binary files. For example, App Store has the restriction “apps above 100 MB will not download until you connect to Wi-Fi”. As a result, a feature that increases the binary size by 100MB will receive much more scrutiny than one that increases it by 10MB. It is challenging to run computation-intensive DNN-based tasks on mobile devices due to the limited computation resources.

This talk introduces the Algorithms and Hardware that can be used to accelerate the Inferencing or reduce the latency of deep learning workloads. We will discuss how to compress the Deep Neural Networks and techniques like Graph Fusion, Kernel Auto-Tuning for accelerating inference, as well as Data and model parallelization, automatic mixed precision, and other techniques for accelerating training. We will also discuss specialized hardware for deep learning such as GPUs, FPGAs, and ASICs, including the Tensor Cores in NVIDIA’s Volta GPUs as well as Google’s Tensor Processing Units (TPUs). We will also discuss the Deployment of the Large Size Deep Learning Models on the Edge devices.

Keywords: Graph Optimization, Tensor Fusion, Kernel Auto Tuning, Pruning, Weight sharing, quantization, low-rank approximations, binary networks, ternary networks, Winograd transformations, data parallelism, model parallelism, mixed precision, FP16, FP32, model distillation, Dense-Sparse-Dense training, NVIDIA Volta, Tensor Core, Google TPU.

As we know, a basic GAN has two neural Networks – the Discriminator (D) and the Generator (G). The Generator attempts to generate images that look like real images. The Discriminator tries to distinguish the generated images from real images. By combined loss minimization of these two neural networks, the entire model trains and eventually reach a state of equilibrium, where the Discriminator no longer can distinguish the fake images, generated by the Generator, from real images.

In this talk, I'll be discussing how GANs can be used to achieve Super Resolution. Super Resolution is the process of upscaling and or improving the details within an image. Often a Low Resolution image is taken as an input and the same image is upscaled to a higher resolution, which is the output. The details in the High Resolution output are filled in where the details are essentially unknown.

The traditional way of upscaling an image was to perform interpolation over the pixels such as Bicubic Interpolation. Drawbacks of this method are images that get smoothened or details and definitions are lost.

The need for high resolution is extensive right from a portrait we capture from our phone to image recognition, forensic science, etc.

There are numerous documents and research papers being published for COVID-19 and doctors are not able to absorb the content of all the literature. It has become a real challenge to extract relevant information in a short span of time.

Knowledge Graph along with SQUAD application can help process multiple documents and extract precise information from a set of documents quickly. This will be a very handy application for healthcare professional to extract relevant information without going in detail with each application.

The session will demonstrate the following:

a) Text Processing of COVID-19 literature

b) Named Entity Extraction from the documents using BERT/Spacy

c) Building a Knowledge Graph of the documents

d) Building question-answer application

I will discuss how I started CellStrat AI Lab and what goes to make it successful as one of the leading AI research groups in India.

I will cover facets such as origination, growth, researcher motivation, content portfolio, project activity, research areas, marketing techniques and talent development pipeline.

This presentation will give an idea to corporate and academic institutions as to how they can create and nurture a world-class AI Lab within their organizations.

Artificial Intelligence fulfillment is a hot topic these days across continents and boardrooms of all small to mid-range to large companies alike. A common question for everybody is to understand as to how they can utilise the technology, deploy it in their business and what will it take to achive this business objective. “How to Fulfill Management’s Enterprise AI Initiative” topic addresses most of such queries and attempts to answer them taking a well-rounded perspective in the Indian Enterprise context.

There has been much research in efficient and scalable model approaches required for network design and object detection. Feature Pyramid Networks (FPN) enabled feature fusion at different scales. Path Aggregation Networks (PANet) conduct bottom-up path augmentation in an FPN, which shortens information transfer from bottom layers to topmost features. A more recent scheme, NAS-FPN (Neural Architecture Search-Feature Pyramid Network), combines RL-based NAS with an FPN to enable combination of top-down and bottom-up connections to fuse features across scales.

A recent state-of-the art model EfficientDet uses a weighted bi-directional BiFPN with multi-scale feature fusion across layers to account for learnable feature importance while applying bottom-up and top-down feature fusion. EfficientDet combines BiFPN with efficient model scaling techniques proposed in EfficientNet model, such that the baseline network, the feature network and the bounding box/class prediction networks are all scaled uniformly and efficiently using compound scaling technique across resolution/width/depth dimensions. The EfficientNet-D6 achieves top of the line accuracy (mAP) on COCO dataset with 4x lesser parameters and 13x fewer FLOPS then the recent comparable NAS-FPN model.

As the COVID-19 pandemic rages on across the world as one of the most catastrophic healthcare crises in recent memory, the medical world is grappling with several challenges at once to cope with this crisis in the best possible manner. Following problem areas need urgent attention:

• Timely screening and identification of infected Covid-19 patients to help in triaging of scarce healthcare resources
• Monitoring the response to clinical interventions to decide best treatment possible.
• Rapid scale-up of capacities in both testing of new patients and monitoring treatment response

The current gold standard for confirmatory diagnosis of SARS-CoV2 infection is the RT-PCR lab test which is an expensive and time-consuming test relying on identification of viral genetic material – it is not possible to quickly scale up testing capacity using this technique in countries like India due to lack of the required equipment. In this scenario, we need to come up with an accurate and rapidly scalable method as an alternative testing strategy.

The Chest X-Ray is a basic test that is widely available in healthcare facilities across India and the developing world and can be done and analyzed within a short time frame.

Deep Convolutional Neural Networks offer great promise in automating the classification of Chest X-ray images to accurately identify COVID-19 patients from patients with normal lungs and those with other lung pathology related conditions like viral and bacterial pneumonias. This technique can be used as a reliable screening methodology to identify COVID-19 patients and recommend the appropriate therapy before it is too late.

Various CNN architectures have been explored recently to perform efficient image classification and segmentation on lung radiology images originating from X-Ray and CT Scans. These Deep Learning based techniques leverage the power of CNNs to extract hidden features from radiology images that can further be exploited for accurate identification of the disease process affecting the patient.

This technique promises to be a game changer in COVID-19 screening that can both be scaled up rapidly and can also be used as a Clinical Decision Support Tool by radiologists inside hospitals, critical care units and isolation wards. It also has the potential to reduce the time required by experienced radiologists to screen through huge volumes of Chest X-Ray images, while maintaining optimal levels of accuracy in diagnosis.

A QA system finds the answers out of the help with the information retrieval, computational linguistics and knowledge representation of the data for finding those answers for the questions given. I have used the #google #BERT pretrained model for building a QA system. The QA system is extracting the data from Web/Database using IR and feeding the paragraph as input to the system which retrieves the answer based on the answer given as input to the system. BERT is one such pre-trained model developed by Google which can be fine-tuned on new data which can be used to create NLP systems like question answering, text generation, text classification, text summarization and sentiment analysis.

In this talk, I will be discussing the end to end architecture of modern Question Answering Systems and how they are different from chat-bots using SQUAD 2.0. How QA Based systems can help modernize multiple industries such as pharmaceutical, Automobile and Retail etc,domains in solving various complex challenges. I will be discussing in detail about masked language models and sentence prediction and how that's helping BERT/DistilBERT to be top production-ready algorithms.

We will see how various BERT variants like SciBERT and BioBERT are helping in focusing on a specific domain or area. A clear understanding of Token Embedding - Segmanet Embedding - Position Embedding as the primary base behind these algorithms.

Reinforcement Learning broadly involves Value-based methods and Policy-based Methods. Actor-based methods are suitable for discrete action spaces, whereas Policy-based methods are suitable for continuous action spaces.

A simple Policy Gradient algorithm (a policy-based method) predicts the probability of Actions of an Agent. Since, the Policy is updated using Monte Carlo approach (i.e. taking random samples), the variance of log probabilities produced by the PG algorithm is very high, causing unstable learning. This variance can be reduced by using a Baseline to remove outliers. A good choice to be used as a baseline is the Q-Value of the State, Action pair.

In this session, we will talk about Actor Critic algorithms, which uses two networks - an Actor and a Critic. The actor outputs the Policy, while the Critic gives out the Q-value which can be used to evaluate the Policy. In this zero-sum game, an equilibrium is reached when an ideal policy is discovered.

We will demonstrate with a Code demo, how this algorithm can be used to predict Actions in a Stock market scenario.

In Deep Reinforcement Learning, we know that policy Gradient methods work efficiently in predicting the probability of the Actions, in case of discrete Action Spaces and the exact action itself incase of continous Actions. The Predicted Action is evaluated by another Neural Network. We have used this to train a Robotic Arm to perform specific Tasks.

But what if we have more One Agent in an environment to be trained?.You can have an autonomous group of agents sharing a common environment. These Agents can work Collaboratively towards maximizing their total goal, or they could be interacting Competitively against each other to get the Reward, or they can be interacting in a mixed mode.

In this session, I will talk about Multi Agent Reinforcement Algorithms (MARL) which can be used in either collaborative mode, or competitive mode or a mixed mode between Agents. I will present a brief Code walk through of using the above algorithm in Open AI's Multi Agent Particle Environment.

The recent advances in AI over the last decade has been heavily powered by Deep Learning with a new SOTA algorithm being released every other week by developers and researchers all across the globe. The concept of neural networks came into existence in the 1960s but made a come-back when fueled by the massive datasets and GPU farms. But the field of AI is much richer than just this one type of algorithm. Symbolic reasoning based algorithms also pioneered in the 1960s and were more practical for the time as they were less computationally intensive.

Deep Learning is great at understanding and finding patterns in massive streams of data but fails when it comes to using logic and reasoning to understand a concept with minimal data. Symbolic AI, on the other hand, is good at reasoning and creating abstractions from pre-defined concepts but fails to scale to real-world problems. However, there is something interesting here, the weaknesses of neural nets are actually the strengths of symbolic AI (and vice-versa), for the most part. So why not, Hybridize AI with Neural Nets and Symbolic AI to pave out a path for human-level AGI?

In this session, we are going to learn how this radical approach to AI can help solve real-world problems by demonstrating my proof-of-concept which can navigate a simple environment using natural language commands in a few shots of training. This demo will also allow the attendees to understand the usage of pretrained models for generalizing to new tasks using a multi-modal approach. Finally, we will look at the industry use cases of Neuro-Symbolic AI and the potential advantages of this approach over vanilla deep learning in that domain.

Quantum computing has surged in the news and the new breakthroughs put it in a spotlight. Quantum computing can greatly enhance the ML/DL landscape. The new innovation quantum neural networks can greatly accelerate the training and inference of neural networks.

In this presentation we will see different architectures of QNN. QNN can be build using variational circuits.

The quantum variational circuits can be used as neural networks. The different architectures in quantum neural network are explored in this presentation. What are the advantages of QNN over traditional ANN. A simple image classification problem using QNN is demonstrated.

Deep Neural Networks are being used in production scale systems for computer vision, audio/video classification and processing, and in many other rich-data centric situations. However, at their core, neural networks are found to be susceptible to noisy, so-called adversarial input, which can potentially enable them to be hacked, or otherwise attacked. How do we understand and quantify the robustness of neural networks to such input? Are there best practices and methods that enable us to understand and accommodate these kinds of attacks on neural networks?

This talk addresses this specific topic, starting from some of the fundamentals, while exploring adversarial examples, their generation and how they can impact neural network efficacy. We then see how real world attack patterns are related to these adversarial examples, and evaluate some of the scenarios that may arise, specifically in a telecommunications network context. We then discuss some general ideas around defence against such adversarial examples. Additionally, we may also discuss elements of explainability and interpretability as relevant to adversarial attacks on neural networks, and practical considerations with respect to implementation of these techniques and best practices. Relevant research or resources on adversarial attacks and how they can be addressed for deep learning models may also be discussed if time permits.

OpenAI made a breakthrough in Deep Reinforcement Learning when they created OpenAI five,a team of 5 agents and beat some of the best Dota2 players in the world.

Underlying algorithm that helped them in achieving this is Proximal Policy Optimization.OpenAI told that PPO has become its default reinforcement algorithm because of its ease and good performance.It uses novel policy gradient techniques that was more efficient and less costly than the traditional reinforcement learning methods.Key contribution of PPO is ensuring that a new update of policy does not change it too much from the previous policy.This leads to less variance at the cost of some bias but ensures smoother training.It also makes sure that the agent does not go down the unrecoverable path of taking senseless actions.

In this talk,I will be discussing the architecture of PPO,its major changes from Policy Gradient methods and how it can be implemented through code

Generative adversarial networks (GANs) are algorithmic architectures that use two neural networks, pitting one against the other (thus the “adversarial”) in order to generate new, synthetic instances of data that can pass for real data. They are used widely in image generation, video generation and voice generation

While GAN images become more realistic over time,one of the main challenges is controlling the output i.e:changing specific features such as pose,face shape and hair style. NVIDIA proposed a novel method which generated images starting from very low resolution and continuing to high resolution.By modifying the input at each level,it controls the visual features in that level from coarse features(pose,face shape) to find details(hair color without affecting other levels)

In this talk,I will be discussing how the lack of control affected the generation of images in previous GAN models and the changes introduced by StyleGAN that help in controlling the style and generating impressive images of synthetic human faces

With the cases of COVID-19 exponentially rising, it is essential to identify the disease at an early stage. Also, given the volume of the patients, it is impossible to do testing on each and every person who feels sick. Also, when testing kits are scarce, we need to find an alternative method to classify between the COVID-19 and other cases. To solve this problem we can use the chest X-ray images of the patients and identify the patients with COVID-19 patients as the first level of screening.

This talk focuses on creating the dataset of normal Xrays and COVID-19 patient Xrays. What are the hurdles in using various pre-trained models? Identifying the correct pre-trained model for this application. This talk further focuses on training and creating a viable model to identify COVID-19 patients.

Furthermore, it talks about how to deploy this model quickly on the production server to make it available to the healthcare facilities so that they can do the pre-screening of the patients.