Practical Learning To Learn
Gradient descent continues to be our main work horse for training neural networks. One recurring problem though is the large amount of data required. Meta learning frames the problem not as learning from a single large dataset, but learning how to learn from multiple related smaller datasets. In this talk we'll first discuss some key concepts around gradient descent; fine-tuning, transfer learning, joint training and catastrophic forgetting and compare them to how simple meta learning techniques can make optimisation feasible for much smaller datasets.
Outline/Structure of the Talk
- High level review of gradient descent and it's general behaviours in differing cases of datasets.
- Discussion of two methods for meta learning; MAML and Reptile; with a description of how they are related.
Learning Outcome
- A practical understanding of approaches to first order meta learning.
Target Audience
Applied machine learning researchers and engineers.
Prerequisites for Attendees
- Some background / experience in training models using gradient descent.
Video
Links
- Personal Blog
- List of publications while at Google Research
- "Deep Reinforcement Learning for Robotics (youtube)" (Melbourne ML/AI meetup talk)
- "The building blocks of neural networks (slides)" (Thoughtworks Evolution)
schedule Submitted 2 years ago
People who liked this proposal, also liked:
-
keyboard_arrow_downJuliet Hougland - How to Experiment Quickly
45 Mins
Invited Talk
Intermediate
The ‘science’ in data science refers to the underlying philosophy that you don’t know what works for your business until you make changes and rigorously measure impact. Rapid experimentation is a fundamental characteristic of high functioning data science teams. They experiment with models, business processes, user interfaces, marketing strategies, and anything else they can get their hands on. In this talk I will discuss what data platform tooling and organizational designs support rapid experimentation in data science teams.
-
-
keyboard_arrow_downMel Flanagan - Transparent Government: The Stories we can Tell with Data
30 Mins
Talk
Intermediate
There is an increasing and powerful global push to open up the trove of information governments generate, collect, and manage. There is a vast array of data ranging from; open data, big data from multiple sources, sensitive information about citizens, and complex information about businesses interactions with government, such as contracts and procurement, taxes and royalties.
This creates many opportunities to use this data to tell stories. To help citizens and the private sector understand what is happening across governments, means not just access to this data, but tools that allow people to dig through, analyse it, use diagrams and maps to make sense of it and to connect information in a way that is understandable, engaging and useful to the general public.
This talk shows some of the possibilities available today and looks at what may be possible as governments around the world open up. It shows some of the ground-breaking work done in Australia. The talk touches on issues around transparency, accountability, systems of protection, data ethics frameworks, and ultimately how to build trust.
Throughout, we'll see some of the work by Nook Studios building systems for government, including the ground-breaking Common Ground mining title information system, as well as tools that help link and connect information in a meaningful way using data pathways.
-
