Hello! I’m Nandan Thakur (नंदन ठाकुर / নন্দন ঠাকুর). I’m a (third) fourth-year PhD student in Computer Science at the University of Waterloo advised by Prof. Jimmy Lin. My PhD is supported by the David R. Cheriton Graduate Scholarship. During my PhD, I have interned at Google, Vectara and Databricks. I’ve also collaborated with industry partners including Snowflake, Micrsoft and Huawei.

📄 View my CV

Research

I focus on three core aspects focused around evaluation, data in information retrieval:

• Constructing challenging and realistic benchmarks with high-quality, human-curated evaluation samples
• Designing efficient retrieval systems that balance quality and cost, generalize well to challenging domains
• Standardizing RAG evaluation to build a more principled foundation within the IR & NLP community

In my research, I’ve developed widely used retrieval benchmarks such as BEIR or MIRACL, and trained efficient retrieval models like GPL or SWIM-IR. This advances accelerate RAG systems–such as TREC-RAG–to produce better language model answers by (i) leveraging cleaner training data (e.g., RLHN) (ii) reducing hallucinations across domains and languages (e.g., NoMIRACL, MIRAGE-Bench), and (iii) enabling evaluation on realistic benchmarks and metrics (e.g., FreshStack).

Past

Prior to my PhD, I was a NLP research assistant at the UKP Lab in TU Darmstadt advised by Prof. Iryna Gurevych and Nils Reimers. I have prior industry experience as a Data Scientist working at KNOLSKAPE. I completed my undergraduate from BITS Pilani KK Birla Goa Campus.

2025 (Recent Updates)

• [Jun 2025] My invited guest lecture with over 400+ participants on Modern IR Evaluation in the RAG Era is available now! [YouTube]
• [Jun 2025] I was invited at Weaviate Podcast to talk about RAG benchmarks, Check out the video now! [YouTube]
• [Jun 2025] I have been invited to talk at Hamel’s mini RAG course on “Modern Information Retrieval Evaluation In The RAG Era”. Sign up here: [https://maven.com/p/fae749/modern-ir-evaluation-in-the-generative-rag-era].
• [May 2025] Our new work on nuggetizing search arena RAG answers is available as a preprint now!
• [Apr 2025] My exciting internship work at Databricks on FreshStack to create realistic RAG benchmarks is available on arXiv!
• [Apr 2025] Our TREC 2024 RAG support and Nuggets preprints have been accepted at SIGIR 2025!
• [Jan 2025] Gave a research talk on “Accelerating Multilingual RAG Systems” at Microsoft Research, Bangalore. [video].
• [Jan 2025] My work during my internship at Vectara on “MIRAGE-Bench: Automatic Multilingual Benchmark Arena for Retrieval-Augmented Generation Systems” is now accepted at NAACL 2025.
• [Jan 2025] Our contribution on including MIRACL in “MMTEB: Massive Multilingual Text Embedding Benchmark” is now accepted at ICLR 2025.

Selected Talks & Recordings

Selected Publications

Old Updates