🚨 [AI Research] "Hype, Sustainability, and the Price of the Bigger-is-Better Paradigm in AI" by @GaelVaroquaux, @SashaMTL & @mer__edith is a MUST-READ for everyone in AI. Selected quotes: "The bigger-is-better norm is also self-reinforcing, shaping the AI research field by informing what kinds of research is incentivized, which questions are asked (or remain unasked), as well as the relationship between industrial and academic actors. This is important because science, both in AI and in other disciplines, does not happen in a vacuum – it is built upon relationships and a shared pool of knowledge, in which prior work is studied, incorporated, and extended." (page 2) - "If we consider health applications, where we would like machine learning to positively contribute to society, AI models are often built in data-scarce contexts on which large models can more easily overfit (Varoquaux & Cheplygina, 2022). For example, to predict epilepsy, Eriksson et al. (2023) find no prediction-performance difference between logistic regression, boosted trees, or deep learning, even as these approaches differ dramatically in terms of resources required. The largest source of medical data are probably electronic health records – Rajkomar et al. (2018) reported performance of deep learning, though their appendices show that logistic regression achieves the same performance. For education, one promise of machine learning is that it could produce customized pedagogy, shaped to the student based on which topics or skills they have learned, and which they are striving the understand. For this purpose, a random embedding of the student’s learning sequence outperforms more resource intensive optimized deep or Bayesian models (Ding & Larson, 2019)." (page 4) - "As members of the AI community, we find that, in recent years, AI research has acquired an unhealthy taste for scale. This comes with dire consequences– economic inequalities and environmental (un)sustainability, datasets that erode privacy and emphasize corrosive social elements, a narrowing of the field, and a structural exclusion of small actors such as most academic labs and many startups. This fixation on scale has emerged via norms that shape how the scientific community acts. We believe that scientific understanding and meaningful social benefits of AI will come from de-emphasizing scale as a blanket solution for all problems, instead focusing on models that can be run on widely-available hardware, at moderate costs. This will enable more actors to shape how AI systems are created and used, providing more immediate value in applications ranging from health to business, as well as enabling a more democratic practice of AI." (page 10) 🔎 Read the full paper below. 🔥 Interested in AI policy, compliance & regulation? Join 35,200+ people who subscribe to my weekly newsletter (link below).