Ali Backour
Math and Computer Science student at MIT · Machine Learning Researcher
I am a student at MIT studying mathematics and computer science. I am broadly interested in understanding intelligence through mathematical, computational, and philosophical lenses. My research interests include generative models, representation learning, geometry for machine learning, information theory, and the behavior and limitations of large language models.
I am especially drawn to questions where empirical behavior in modern ML systems points toward a deeper mathematical structure: Why does guidance improve diffusion models? What does the geometry of an image or representation space mean? How should we evaluate whether language models are genuinely reasoning, exploring, or merely following familiar patterns? I like projects that begin with a concrete anomaly or surprising experiment and develop into a more principled explanation.
Outside of research, I enjoy running, biking and playing ice hockey.
Research Interests
- Generative models and diffusion models
- Geometry of data manifolds and representation spaces
- Information theory for machine learning
- Understanding and evaluating large language models
- Cryptography, privacy, and ML security
Selected Research
A Geometric View of Why Guidance Improves Diffusion Samples
Research project · Generative modeling, geometry, diffusion models
Developing a geometric explanation for why classifier-free guidance improves diffusion samples. The central idea is that unguided diffusion may favor smoother, easier-to-model regions of the data distribution, while underrepresenting high-curvature or boundary-like regions where the score field varies more rapidly. I study this effect on controlled manifolds such as the catenoid and test whether guided image samples move toward higher-curvature regions in stable and DINO representation spaces.
Blog post / Paper in progress
Implicit Augmentation from Distributional Symmetry in Turbulence Super-Resolution
NeurIPS Workshop paper · ML for physics, turbulence, equivariance, super-resolution
Studied whether convolutional neural networks can acquire rotational equivariance implicitly from turbulence data, without explicit rotation augmentation or specialized equivariant architectures. We found that statistical isotropy in turbulent flows acts as a natural source of augmentation: models trained on more isotropic mid-plane data exhibit lower equivariance error than those trained near anisotropic boundary layers, and increasing temporal or spatial sampling further improves equivariance.
Research Notes / Blog Posts
Mixing Is Not Meaning: Evaluating Soft-Token Reasoning in LLMs
Research project · Language models, representation learning, continuous reasoning
Studied whether language models can meaningfully interpret soft tokens formed as mixtures of token embeddings. We found that pretrained models often collapse these mixtures to their dominant discrete component, suggesting that naive embedding interpolation does not preserve distributional meaning. We introduced diagnostic probes for soft-token understanding and explored preliminary soft-token fine-tuning objectives.
Actions That Do Nothing: A Small Pitfall in Reinforcement Learning
Blog post · Reinforcement learning, action masking, 2048
A short case study from training reinforcement learning agents for 2048. The main lesson is that actions that leave the state unchanged can create surprisingly bad learning dynamics, especially for value-based methods. I discuss why simple reward penalties may not be enough, how action masking helps, and how this connects to invalid-action masking and action-elimination ideas in reinforcement learning.
Course Projects
Encrypt What Matters: ROI-Guided FHE for CNN Inference
Research project · FHE, privacy-preserving ML, efficient encrypted inference
Course project for MIT 6.5610 on privacy-preserving ML inference. We explored a region-of-interest approach to fully homomorphic encrypted CNN inference, encrypting only sensitive input regions and using CNN locality to reduce encrypted computation.
Writing
Coming soon
I am currently preparing blog posts on geometry for ML, information theory for ML, and the role of divergences as learning objectives.
I will be a teaching assistant for Reinforcement Learning (6.7920), and I am writing course notes to make the material more structured and accessible.
Contact
The best way to reach me is by email at abackour@mit.edu.