Following the success of the AutoDL 2019-2020 challenge series, including an official competition of NeurIPS'19, and the first Meta-Learning challenge at AAAI'21, we organized MetaDL, a few-shot-learning competition challenging DL methods for meta-learning at NeurIPS'21 on December 8th, 2021.
MetaDelta++: Improve Generalization of Few-shot System Through Multi-Scale Pretrained Models and Improved Training Strategies
Meta-learning aims at learning quickly on novel tasks with limited data by transferring generic experience learned from previous tasks. Naturally, few-shot learning has been one of the most popular applications for meta-learning. Recently, an ensembled few-shot system MetaDelta is proposed to boost the performance, which won first place in the AAAI 2021 MetaDL challenge with leading performance. However, the generalization ability of MetaDelta is still limited by the homogeneous model setting and weak pretraining and fine-tuning strategies, hindering MetaDelta from being applied to more diverse scenarios and problems. We further boost the performance and generalization ability of MetaDelta by leveraging pre-trained models at multi-scale and improved training strategies, including semi-weakly supervised pretraining, data augmentation, separated learning rate at each layer, lazier BN statistics update, and better decoder design. Our system MetaDelta++ substantially boosts the performance and generalization abilities by a large margin and stands the 1st place in phase 1 of the NeurIPS 2021 MetaDL system with a large margin compared to MetaDelta and other teams.
We are grateful to Microsoft and Google for generous cloud unit donations and to 4Paradigm for donating prizes. This project is also supported by ChaLearn and HUMANIA chair of AI grant ANR-19-CHIA-00222 of the Agence Nationale de la Recherche (France). Researchers and students from Université Paris-Saclay, Universiteit Leiden, TU Eindhoven, and 4Paradigm have contributed. The challenge is hosted by Codalab (Université Paris-Saclay).
Meta-learning competition at NeurIPS 2021
$1000 of prizes in each track!
Aug 2, 2021: Track 1 opens, few-shot-learning from 128x128 color images from 5 different domains.
Aug 20, 2021: Challenge bootcamp where newcomers to the field can learn how to get started, with expert coaching.
Sep 2,2021: Track 2 opens, few-shot-learning from tabular data, from 5 different domains. (exact timeline TBD)
Oct 2, 2021: End of challenge track 1.
Dec 6-14: NeurIPS conference.
The top ranking participants will be invited to co-author a group paper on the results and analysis of the challenge, to be published in PMLR.
New Benchmark for Meta-Learning
Bootcamp and learn about Few Shot Learning
Speakers: Mike Huisman, Felix Mohr, Jan N. van Rijn
Date: August 21st, 8am - 12am (Bogota Timezone)
About Meta Learning
For a comprehensive overview of Meta-learning, we refer to the following resources:
Metalearning: Applications to Automated Machine Learning and Data Mining (2nd edition, Open Access, by Brazdil et al. (2022))
Meta-learning: A survey (Book chapter, by Vanschoren (2019))
Meta-learning in neural networks: A survey (Arxiv, Hospedales et al. (2020))
A Survey of Deep Meta-Learning (Artificial Intelligence Review, Huisman et al. (2021))
This challenge would not have been possible without the help of many people.
Adrian El Baz (Université Paris-Saclay, France)
Isabelle Guyon (Université Paris-Saclay; INRIA, France and ChaLearn, USA)
Jennifer He (4Paradigm, China)
Mike Huisman (Leiden University, the Netherlands)
Zhengying Liu (Université Paris-Saclay, France)
Yui Man Lui (University of Central Missouri)
Felix Mohr (Universidad de la Sabana)
Jan N. van Rijn (Leiden University, Netherlands)
Sebastien Treguer (Université Paris-Saclay, France)
Wei-Wei Tu (4Paradigm, China)
Jun Wan (Chinese Academy of Sciences, China)
Lisheng Sun (Université Paris-Saclay, France)
Haozhe Sun (Université Paris-Saclay; LISN, France)
Phan Anh Vu (Université Paris-Saclay; LISN, France)
Ihsan Ullah (Université Paris-Saclay; LISN, France)
Joaquin Vanschoren (Eindhoven University, the Netherlands)
Jun Wan (Chinese Academy of Science, China)
Benjia Zhou (Chinese Academy of Sciences, China)
Tyler Thomas (CKCollab, USA)
Loic Sarrazin (Artelys, France)
Anne-Catherine Letournel (UPSaclay, France)
Adrien Pavao (UPSaclay, France)
ChaLearn is the challenge organization coordinator. Microsoft and Google are the primary sponsors of the challenge. 4Paradigm donated prizes, datasets, and contributed to the protocol, baselines methods and beta-testing. Other institutions of the co-organizers provided in-kind contributions, including datasets, data formatting, baseline methods, and beta-testing.
Leiden University (the Netherlands)
Universidad de la Sabana (Colombia)