I’m a final year PhD student at the Center for Language and Speech Processing at Johns Hopkins University, advised by Kevin Duh and Kenton Murray.

I have previously interned with Nikhil Mehta at Google/Google DeepMind and with Maha Elbayad at Meta FAIR. Before JHU, I was at Yale University studying math and computer science, and working with Dragomir Radev at the Yale LILY Lab.

My current research interests are in making language models more efficient to produce and deploy by exploiting structure in their weights: using redundancy and symmetries to compress models or merge them through direct parameter interventions, without retraining. More specifically, these are the areas I am working on:

• Redundancy and compression: Our recent work on feed-forward parameter sharing and optimal-transport based model width reduction demonstrate compression via targeting redundant structures in LLMs and other pretrained transformers.
• Merging for seamless post-training: We recently proposed ORBIT, which reduces catastrophic forgetting of core LLM abilities via a novel model merging technique while performing SFT on specialized domain data.
• Model merging and weight symmetries: Our paper on merging text transformers explores the connection between different SGD solutions via our proposed model merging technique that exploits numerous transformer symmetries.

I have also worked extensively in multilingual NLP and machine translation (MT). See my publications page for more details!

If you would like to get in contact with me, please reach me at nverma7 (at) jhu (dot) edu.