MetaLearn Challenges


MetaDL @ NeurIPS'21

We are grateful to Microsoft and Google for generous cloud unit donations and to 4Paradigm for donating prizes. This project is also supported by ChaLearn and HUMANIA chair of AI grant ANR-19-CHIA-00222 of the Agence Nationale de la Recherche (France). Researchers and students from Université Paris-Saclay, Universiteit Leiden, TU Eindhoven, and 4Paradigm have contributed. The challenge is hosted by Codalab (Université Paris-Saclay).

Following the success of the AutoDL 2019-2020 challenge series, including an official competition of NeurIPS'19, and the first Meta-Learning challenge at AAAI'21, we are organizing MetaDL, a few-shot-learning competition challenging DL methods for meta-learning at NeurIPS'21.

Meta-learning competition at NeurIPS 2021

$1000 of prizes in each track!

Provisional schedule:

    • Aug 2, 2021: Track 1 opens, few-shot-learning from 128x128 color images from 5 different domains.

    • Aug 20, 2021: Challenge bootcamp where newcomers to the field can learn how to get started, with expert coaching.

    • Sep 2, 2021: Track 2 opens, few-shot-learning from tabular data, from 5 different domains. (exact timeline TBD)

    • Oct 2, 2021: End of challenge track 1.

    • Dec 6-14: NeurIPS conference.

The top ranking participants will be invited to co-author a group paper on the results and analysis of the challenge, to be published in PMLR.

Join our bootcamp and learn about Few Shot Learning

Speakers: Mike Huisman, Felix Mohr, Jan N. van Rijn

Date: August 21st, 8am - 12am (Bogota Timezone)


New Benchmark for Meta-Learning

We have worked hard to come up with several new meta-learning 128x128 image datasets. We feel that the meta-learning community can greatly benefit from a new benchmark, and plan to publish these datasets open source after the challenge. It is our goal to establish this as a new benchmark for the meta-learning community. Participating in this challenge will give you a head start at getting to work with these datasets.

The challenge is with code submission and will be run on the Codalab platform with generous donations of cloud units from Microsoft and Google. The challenge winners will receive prizes donated by ChaLearn, if they agree to open-source their code. However, there is no such requirement to enter the challenge. The top ranking participants will be invited to co-author a paper on the challenge results, planned to be published in the PMLR, the proceedings track of the Journal of Machine Learning Research.

The competition has a focus on fast solutions. Per submission, the user will get 2 hours of calculations on each meta-dataset. Presumably, this competition favors first-order solutions (such as FO-MAML, LSTM meta-learner) over second-order solutions (such as MAML, TURTLE).


Congratulations to the AAAI 2021 MetaDL winners

About Meta Learning

For a comprehensive overview of Meta-learning, we refer to the following resources:

ChaLearn (USA)

University Paris-Saclay (France)

Codalab, UPSaclay (France)

Leiden University (the Netherlands)

Universidad de la Sabana (Colombia)

4Paradigm (China)

Organization

This challenge would not have been possible without the help of many people.

  • Adrian El Baz (Université Paris-Saclay, France)

  • Isabelle Guyon (Université Paris-Saclay; INRIA, France and ChaLearn, USA)

  • Jennifer He (4Paradigm, China)

  • Mike Huisman (Leiden University, the Netherlands)

  • Zhengying Liu (Université Paris-Saclay, France)

  • Yui Man Lui (University of Central Missouri)

  • Felix Mohr (Universidad de la Sabana)

  • Jan N. van Rijn (Leiden University, Netherlands)

  • Sebastien Treguer (Université Paris-Saclay, France)

  • Wei-Wei Tu (4Paradigm, China)

  • Jun Wan (Chinese Academy of Sciences, China)

  • Lisheng Sun (Université Paris-Saclay, France)

  • Haozhe Sun (Université Paris-Saclay; LISN, France)

  • Phan Anh Vu (Université Paris-Saclay; LISN, France)

  • Ihsan Ullah (Université Paris-Saclay; LISN, France)

  • Joaquin Vanschoren (Eindhoven University, the Netherlands)

  • Jun Wan (Chinese Academy of Science, China)

  • Benjia Zhou (Chinese Academy of Sciences, China)

The challenge is running on the Codalab platform, administered by Université Paris-Saclay and maintained by CKCollab LLC, with primary developers:

  • Tyler Thomas (CKCollab, USA)

  • Loic Sarrazin (Artelys, France)

  • Anne-Catherine Letournel (UPSaclay, France)

  • Adrien Pavao (UPSaclay, France)

ChaLearn is the challenge organization coordinator. Microsoft and Google are the primary sponsors of the challenge. 4Paradigm donated prizes, datasets, and contributed to the protocol, baselines methods and beta-testing. Other institutions of the co-organizers provided in-kind contributions, including datasets, data formatting, baseline methods, and beta-testing.

Contact the organizers.