BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//CERN//INDICO//EN
BEGIN:VEVENT
SUMMARY:How attention learns structure from data? [Oberseminar Mathematics
  of Artificial Intelligence]
DTSTART:20261123T113000Z
DTEND:20261123T140000Z
DTSTAMP:20261117T191400Z
UID:indico-event-1803@math-events.uni-bonn.de
DESCRIPTION:Speakers: Claire Boyer (Université Paris-Saclay)\n\nTransform
 er architectures have demonstrated remarkable empirical success\, yet thei
 r ability to extract statistical structure from data remains only partiall
 y understood. In this talk\, I will present recent work shedding light on 
 attention mechanisms through a statistical and operator-theoretic lens.Bas
 ed on large-prompt asymptotics\, one can show that softmax attention conve
 rges to a linear operator acting on the input-token distribution under Gau
 ssian assumptions. This regime enables a precise analysis of both the outp
 uts and the training dynamics using concentration arguments\, revealing a 
 surprising bridge between nonlinear softmax attention and tractable linear
  models.These results suggest that attention can be understood as a flexib
 le statistical operator that adapts to the underlying data distribution\, 
 providing a unifying framework to study its role in representation learnin
 g and in-context inference. I will conclude by discussing the connections 
 between transformer architectures and principal component analysis\, using
  this setting to illustrate the power of concentration methods in high-dim
 ensional statistics.\n\nhttps://math-events.uni-bonn.de/event/1803/
LOCATION:Endenicher Allee 60/1-016 - Lipschitzsaal (Mathezentrum)
URL:https://math-events.uni-bonn.de/event/1803/
END:VEVENT
END:VCALENDAR
