Hello! My name is Nandan Thakur (नंदन ठाकुर / নন্দন ঠাকুর) [CV].

I’m currently a (second) third-year PhD student in the David R. Cheriton School of Computer Science at the University of Waterloo in Canada, advised by Prof. Jimmy Lin. My research area includes improving evaluation and benchmarking of information retrieval systems across diverse domains and languages. I am the author of the BEIR benchmark and also developed the MIRACL, SWIM-IR and similar datasets. More recently, I have been working on better evaluation of Retrieval-Augmented Generation (RAG) systems, where I am co-hosting the first ever RAG competition: TREC 2024 RAG.

During my PhD, I’ve been involved in research collaborations with Vectara (2024) and Huawei (2022). I’ve done a few internships during my PhD at Google Research (2023). Before my PhD, I was a research assistant at the UKP Lab (TU Darmstadt) in Germany, supervised by Prof. Iryna Gurevych and Nils Reimers (2019 - 2021). I received my undergraduate degree at Birla Institute of Technology and Science, Pilani (BITS Pilani) in 2018. I also have industrial experience, working as a Data Scientist in KNOLSKAPE (2018 - 2019) and undergraduate internships at EMBL Heidelberg (Summer 2018) and Belong.co (Fall & Winter 2017).

I maintain a list of my publications under the Research tab. Feel free to reach out to me via email: nandan.thakur@uwaterloo.ca.

I am actively looking for research internships in Fall 2024!

Recent News

• _{[May 2024] Collaboration with Snowflake AI towards building better BEIRv2 and TREC-RAG [blogpost].}
• _{[Apr 2024] I will be attending in-person NAACL 2024 in Mexico City, Mexico between 16-20 June 2024 and SIGIR in Washington DC, USA between 14-18 July 2024. If interested, do reach out!}
• _{[Apr 2024] Received a 3K USD grant from Google to attend the NAACL 2024 Conference in Mexico City, 2024.}
• _{[Apr 2024] My work on “Systematic Evaluation of Neural Retrieval Models on the Touch{'e}~2020 Argument Retrieval Subset of BEIR” has been accepted at SIGIR 2024 (Reproduction).}
• _{[Apr 2024] My work on “Resources for Brewing BEIR: Reproducible Reference Models and Statistical Analyses” has been accepted at SIGIR 2024 (Resource).}
• _{[Mar 2024] My Google internship work on “SWIM-IR: Leveraging LLMs for Synthesizing Training Data Across Many Languages in Multilingual Dense Retrieval” has been accepted at NAACL 2024.}
• _{[Feb 2024] Started part time research colaboration on improving multilingual RAG systems with Vectara.}
• _{[Jan 2024] Gave two research talks on “Hetergeneous Benchmarking of Information Retrieval” in IIT-D (Delhi) and IIIT-Delhi [presentation] [video].}
• _{[Nov 2023] TREC RAG 2024 has been accepted and will be conducted as a shared task in TREC 2024.}