Research
Towards an Effective Theory of LLMs
I take an effective theory perspective on large language models: I look for a compact, macro-level description of their computation that abstracts away from microscopic detail, evolves under approximately closed dynamics (its macro-variables can be propagated forward without tracking the underlying activations), and is useful enough to explain, predict, and steer behavior.
One instantiation is Representational Effective Theory (RET), where we learn these macro-variables representationally and entirely unsupervised, with no task labels. RET yields interpretable "mental state" trajectories, supports early prediction of behaviors such as sycophancy, and provides causal handles for steering. See the paper and project page.
A closely related work learns the macro-variables we call the "plan", studying how language models plan their outputs from an information-theoretic perspective: how far ahead they look (horizon), whether they keep alternatives open (branching), and how much they reuse earlier computation (history). See the paper (ICLR 2026).
Making LLMs Safer
A second, more applied line of my work aims at making LLMs safer. Here I study emergent misalignment, where fine-tuning on a narrow objective spills over into broadly harmful behavior. In BLOCK-EM, we trace this behavior to a small set of internal features and, during fine-tuning, block the model from strengthening them, cutting emergent misalignment by up to 95% with no loss in task performance. See the paper (ICML 2026).
We also study why misalignment arises in the first place, framing both emergent misalignment and its subtler, subliminal form as a shared, data-mediated phenomenon, and showing how dataset structure, task difficulty, and model capability together decide whether it emerges. See the paper (under review).
Theory-Driven Architectures
This third line is more mathematical, aimed first at a rigorous understanding of attention. Through an interacting-entities perspective, we show that self-attention can represent any interaction among entities and can learn it: gradient flow provably converges to such a solution, which generalizes both in-distribution and out-of-distribution (for instance, to longer sequences). The same perspective points to new mechanisms, HyperFeatureAttention for feature coupling and HyperAttention for higher-order interactions, validated in preliminary experiments. See the paper (ICML 2025).
I am now developing these designs further, ablating Hyper-Feature Attention variants (mixed-order heads, optional hyper-value branches, skip-residual stabilization) and building a Chinchilla-style, compute-matched framework to chart their scaling laws against standard Transformers.
Other Work
Alongside these threads, I work on a few other problems, including scalable multi-agent reinforcement learning over networked systems, where Transformers capture the long-range, long-horizon interactions between agents (best student paper at the European Control Conference, ECC 2026; see the paper), and federated reinforcement learning that coordinates training across separate data sources. For these and other papers, see my Google Scholar.
Get in Touch
I'm always happy to hear from anyone interested in collaborating. Just send a short email about the topics you're most interested in (general or specific is fine).