Ali Backour
MEng student at MIT · Machine Learning Researcher
I am an MEng student at MIT studying generative models, where I previously earned a B.S. in Mathematics and B.S. in Computer Science. I am co-advised by Tommi Jaakkola and Vincent Sitzmann. I am particularly interested in understanding the mathematical structure underlying the empirical behavior of generative models and other modern machine learning systems. I am also a TA for Reinforcement Learning (6.7920).
Outside of research, I enjoy running, biking, and playing ice hockey.
Research Interests
- Diffusion models
- Geometry of data manifolds and representation spaces
- Information theory for machine learning
- Understanding and evaluating large language models
- Cryptography, privacy, and ML security
Current Research
A Geometric View of Why Guidance Improves Diffusion Samples
Research project · Generative modeling, geometry, diffusion models
Developing a geometric explanation for why classifier-free guidance improves diffusion samples. The central idea is that unguided diffusion may favor smoother, easier-to-model regions of the data distribution, while underrepresenting high-curvature or boundary-like regions where the score field varies more rapidly. I study this effect on controlled manifolds such as the catenoid and test whether guided image samples move toward higher-curvature regions in stable and DINO representation spaces.
Blog post / Paper in progress
Selected Publications
The Dynamics of Continuous Mixture Collapse in Language Models
NeurIPS 2026 Workshop on Interpretability as a Science · Language models, latent reasoning, dynamical systems
Latent-state reasoning methods replace discrete intermediate tokens with continuous states, such as mixtures of token embeddings, but pretrained language models often fail to preserve these mixtures. I identify three distinct sources of failure. Transformer architectures distort mixture geometry, and training amplifies this. Even with perfectly linear transport, the softmax readout and autoregressive feedback form a dynamical system that either amplifies small differences until one component dominates or contracts mixtures until they become indistinguishable; the observed transition occurs near the predicted threshold, and pretrained-model rollouts lie mostly on the amplifying side. Finally, exact preservation of many-component mixtures generally requires a context-dependent correction whose dimensionality can grow with the number of components.
Encrypt What Matters: When Selective Homomorphic Inference Is Efficient
NeurIPS 2026 Workshop on Beyond Private Training: The New Landscape of AI Privacy · Homomorphic encryption, privacy-preserving ML
Studied selective homomorphic inference, where only a sensitive region of interest is encrypted under FHE and computations independent of it run in plaintext. This produces the same output as full FHE on the same model, without retraining, and its efficiency depends on how quickly encrypted dependencies spread through the network. For small encrypted regions, locality-preserving architectures achieve order-of-magnitude speedups, while architectures with early global mixing gain essentially nothing.
Implicit Augmentation from Distributional Symmetry in Turbulence Super-Resolution
NeurIPS 2025 Workshop on Machine Learning and the Physical Sciences · ML for physics, turbulence, equivariance
Studied whether convolutional neural networks can acquire rotational equivariance implicitly from turbulence data, without explicit rotation augmentation or specialized equivariant architectures. We found that statistical isotropy in turbulent flows acts as a natural source of augmentation: models trained on more isotropic mid-plane data exhibit lower equivariance error than those trained near anisotropic boundary layers, and increasing temporal or spatial sampling further improves equivariance.
Contact
The best way to reach me is by email at abackour@mit.edu.