The official website for the open domain question answering challenge at NeurIPS 2020.
HomeThis leaderboard displays all results for submissions to the EfficientQA unrestricted track. It includes all systems that were submitted to the official leaderboard, used for each of the EfficientQA restricted tracks, as well as systems that were run outside of the leaderboard system.
| Model | Participant | Affiliation | System size * = self-reported |
Accuracy |
|---|---|---|---|---|
| UnitedQA | Microsoft Research & Dynamics 365 AI | Microsoft Research & Dynamics 365 AI | -- | 54.00 |
| Hybrid | Facebook AI | QA@FBAI | -- | 53.89 |
| Model 2 | FAIR-Paris&London | 5.85 GiB | 53.33 | |
| Model 1 | FAIR-Paris&London | 5.66 GiB | 52.78 | |
| soseki | ousia_tohoku | Studio Ousia & Tohoku University | -- | 52.44 |
| R2D2_v2_full | KNOT@FIT | Brno University of Technology | *66 GiB | 50.33 |
| Soseki | ousia_tohoku | Studio Ousia & Tohoku University | 5.63 GiB | 50.17 |
| Soseki | ousia_tohoku | Studio Ousia & Tohoku University | 4.86 GiB | 50.11 |
| Soseki | ousia_tohoku | Studio Ousia & Tohoku University | 5.37 GiB | 49.39 |
| Noah_NLP | Huawei Noah’s Ark Lab | -- | 48.06 | |
| Noah_NLP | Huawei Noah’s Ark Lab | -- | 47.61 | |
| R2D2_v2 | KNOT@FIT | Brno University of Technology | 5.95 GiB | 47.28 |
| Salesforce | -- | 46.83 | ||
| R2D2 | KNOT@FIT | Brno University of Technology | 5.38 GiB | 46.06 |
| Salesforce | *5.7 GiB | 45.83 | ||
| CDPR-FID | BearQA | UC Berkley | -- | 40.83 |
| R1D1 | KNOT@FIT | Brno University of Technology | 5.29 GiB | 40.56 |
| uclnlp-fair-efficientqa-xlg | uclnlp-efficientqa | University College London and Facebook AI Research | *1.16GiB | 39.11 |
| DPR model serving using Vespa.ai | Vespa Team | Vespa.ai | -- | 35.00 |
| REALM | Orgainizers | EfficientQA | 27.19 GiB | 34.89 |
| uclnlp-fair-efficientqa-lg | uclnlp-efficientqa | University College London and Facebook AI Research | 336.23 MiB | 33.44 |
| RDR-500MB | naver-clova-parsing | Naver Clova | 484.69 MiB | 32.06 |
| RDR-6GB | naver-clova-parsing | Naver Clova | 5.78 GiB | 32.06 |
| EFF-Large | Beauty-P | PA Tech | -- | 31.11 |
| ba45-micro | BA45 | BA45 | 5.57 GiB | 31.11 |
| ba45-micro | BA45 | BA45 | 5.53 GiB | 29.97 |
| uclnlp-fair-efficientqa-lg | uclnlp-efficientqa | University College London and Facebook AI Research | 226.09 MiB | 29.44 |
| ba45-micro | BA45 | BA45 | 5.33 GiB | 28.83 |
| RDR-500MB | naver-clova-parsing | Naver Clova | 496.99 MiB | 28.22 |
| T5-1.1-XL + SSM | Organizers | EfficientQA | 5.65 GiB | 27.89 |
| EFF-Small | Beauty-P | PA Tech | -- | 27.89 |
| NTUSTTTTT | Jia-Zhi_Guo | National Taiwan University of Science and Technology | 5.65 GiB | 27.17 |
| uclnlp-fair-efficientqa | uclnlp-efficientqa | University College London and Facebook AI Research | 29.43 MiB | 26.78 |
| uclnlp-fair-efficientqa | uclnlp-efficientqa | University College London and Facebook AI Research | 157.26 MiB | 25.78 |
| uclnlp-fair-efficientqa | uclnlp-efficientqa | University College London and Facebook AI Research | 5.87 GiB | 25.78 |
| hsjdpr | LSLL | LSLL | 13.97 GiB | 25.61 |
| aqa-udm | AQA-UDM | University of Montreal | 14.17 GiB | 25.22 |
| uclnlp-fair-efficientqa | uclnlp-efficientqa | University College London and Facebook AI Research | 28.61 MiB | 24.83 |
| ssm_nq_torch | machnev | Higher School of Economics | 3.02 GiB | 20.11 |
| FRY-T | FRY | FRY | 423.29 MiB | 19.39 |
| NTUSttttt | Chia-Chih_Kuo | National Taiwan University of Science and Technology | 488.96 MiB | 18.39 |
| T5-1.1-small + SSM | Organizers | EfficientQA | 486.61 MiB | 17.61 |