I am a Computer Science Ph.D. Student at the Center for Language and Speech Processing at Johns Hopkins University, supervised by Kevin Duh and Kenton Murray.

I have previously interned with Nikhil Mehta at Google/Google DeepMind and with Maha Elbayad at Meta FAIR. Before JHU, I was at Yale University studying math and computer science, and working with Dragomir Radev at the Yale LILY Lab.

My current research interests are in making language models more efficient to produce and deploy by exploiting structure in their weights: using redundancy and symmetry to compress models and merge them through direct parameter interventions, without retraining. More specifically, these are the areas I am working on:

• Redundancy and compression:: Our recent work on feed-forward parameter sharing and optimal-transport based model width reduction demonstrate compression via targeting redundant structures in LLMs and other pretrained transformers.

• Merging to enable seamless post-training: We recently proposed ORBIT, which enables post-training fine-tuning of specific tasks while minimizing forgetting of foundational LLM abilites, via a novel model merging scheduling technique.

• Weight symmetries and the geometry of merging: Our paper on merging text transformers demonstrates lowered loss barriers between different BERT solutions via our proposed model merging technique that exploits numerous transformer symmetries.

I have also worked extensively in multilingual NLP and machine translation (MT). See my publications page for more details!

If you would like to get in contact with me, please reach me at nverma7 (at) jhu (dot) edu.