ColBERT-Zero: To Pre-train Or Not To Pre-train ColBERT models

A Chaffain, LA, A Chatelain, F Krzakala

Blog post 🤗 Model Paper

Asymptotics of SGD in Sequence-Single Index Models and Single-Layer Attention Networks

LA, B Loureiro, L Stephan, F Krzakala, L Pesce, L Zdeborová

PDF GitHub

Repetita Iuvant: Data Repetition Allows SGD to Learn High-Dimensional Multi-Index Functions

LA, Y Dandi, F Krzakala, L Pesce, L Stephan

PDF GitHub

Online Learning and Information Exponents: On The Importance of Batch size, and Time/Complexity Tradeoffs

LA, Y Dandi, F Krzakala, B Loureiro, L Pesce, L Stephan

PDF GitHub Poster

The Benefits of Reusing Batch for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponents

Y Dandi, E Troiani, LA, L Pesce, L Zdeborová, F Krzakala

PDF GitHub

Escaping mediocrity: how two-layer networks learn hard generalized linear models

LA, F Krzakala, B Loureiro, L Stephan

PDF GitHub Poster

From high-dimensional & mean-field dynamics to dimensionless ODEs: A unifying approach to SGD in two-layers networks

LA, L Stephan, F Krzakala, B Loureiro

PDF GitHub Poster

A benchmark of expert-level academic questions to assess AI capabilities

Center for AI Safety, Scale AI & HLE Contributors Consortium

Benchmark Website Paper