Role of Hessian and its Eigen pair in Deep Learning

While we speak of optimization methods in Machine Learning/Deep Learning, stochastic gradient descent and its variants are quite popular. Choosing an optimal learning rate is crucial towards reaching minima. While some are chosen with theoretical reasons, some are empirically driven. Hessian play an important in understanding and driving the learning rates towards achieving convergence at a faster rate. This topic is getting increasingly popular among deep learning researchers with a quest to accelerate training time with effective convergence properties.

In this talk, we will analyze the role of Hessian in driving the error function curvature and also understand the implications of its eigenvalues. Understanding the eigenvalue spectrum can reveal a lot about the behavior of the optimization algorithm.


Outline/Structure of the Talk

The general form of cost function used in traditional machine learning/ deep learning algorithms will be presented. The influence of Hessian ( the second derivative of the cost function) in choosing the learning rate will be demonstrated.

Learning Outcome

Geometrical understanding of cost functions, learning rate, the importance of convergence will be understood. The highlight of the talk will be that some of the core mathematical concepts will be presented in an elegant manner.

Target Audience

Machine learning/ deep learning practitioners, enthusiasts

Prerequisites for Attendees

Understand the basics of machine learning optimization methods


schedule Submitted 3 years ago

  • Kanimozhi Uma

    Kanimozhi Uma - Semantic Web Technologies for Business and Industry Challenges

    Kanimozhi Uma
    Kanimozhi Uma
    Senior Data Scientist
    Crayon Data
    schedule 3 years ago
    Sold Out!
    20 Mins

    The term semantic technologies represents a fairly diverse family of technologies that have been in existence for a long time and seek to help derive meaning from information. Some examples of semantic technologies include natural language processing (NLP), data mining, artificial intelligence (AI), category tagging, and semantic search. Semantic technology reads and tries to understand language and words in its context. Technically, this approach is based on different levels of analysis: morphological and grammatical analysis; logical, sentence and lexical analysis, in other words: natural language analysis. The major focus of this talk will be an overview of Semantic Technologies; what differentiates Semantic Technologies and its organisational needs. The most relevant Semantic Web concepts and methodologies will be discussed that helps us to develop an understanding for which use cases this technology approach provides substantial advantages.

    Semantic Web standards are used to describe metadata but also have great potential as a general data format for data communication and data integration. Machine learning solutions have been developed to support the management of ontologies, for the semi-automatic annotation of unstructured data, and to integrate semantic information into web mining. Machine learning can be employed to analyze distributed data sources described in Semantic Web formats and to support approximate Semantic Web reasoning and querying. From this talk you will get acquainted with specialised terminology, which enables us to dive deeper into the semantic technologies field. A general view of tools and methods to develop semantic applications with getting to know in which business domains it can be useful to embrace semantic solutions.

    Learn different knowledge modelling approaches to understand which applications require taxonomies or ontologies as underlying knowledge graph. Also, will discuss on the value of consistent metadata of different types and how they add up to a semantic layer around our digital assets. We will see how metadata schemes and their respective values can be managed with controlled vocabularies and taxonomies. Linked Data structure and its serialisation formats such as Resource Description Framework (RDF) and its subject-predicate-object expressions with the fundamental learning of graph based data.

    The most important linguistic concepts that get applied for text mining operations will be discussed along with different word forms, homographs, lemmatisation, ambiguity and more. We will also see how linguistic concepts in combination with knowledge modelling are used for semantic text mining. Get an overview on how Semantic Data Integration provides the conversion of data into RDF with a step-by step process on how different data types can be transformed into RDF. SPARQL: The query language of the Semantic Web will be demonstrated with a concrete example on how to make use of multi-facetted data. Semantic Web Architecture for organisations gives an overview on how semantic technology components play together to solve a business use case.

    Semantic technologies are algorithms and solutions that bring structure and meaning to information. Semantic Web technologies specifically are those that adhere to a specific set of W3C open technology standards that are designed to simplify the implementation of not only semantic technology solutions, but other kinds of solutions as well.