Hello! My name is Nandan Thakur (नंदन ठाकुर / নন্দন ঠাকুর) [CV].

I’m currently a (second) third-year PhD student in the David R. Cheriton School of Computer Science at the University of Waterloo in Canada, advised by Prof. Jimmy Lin. My research is supported by the David R. Cheriton Graduate Scholarship (2024).

My research is on “Heterogeneous benchmarking of Retrieval and RAG systems across diverse domains and languages”. I developed the BEIR benchmark in 2021, which has become an industry standard for measuring model generalization. I’ve been fortunate to be a part of MIRACL, SWIM-IR and similar datasets. Recently, I have been working on better evaluation of Retrieval-Augmented Generation (RAG) systems, where I am co-hosting the first ever RAG competition in TREC 2024: TREC 2024 RAG. During my PhD, I’ve been involved in many research collaborations, including SnowFlake AI, Vectara and Huawei. I’ve also been fortunate to intern at Google Research.

Before my PhD, I was a research assistant at the UKP Lab (TU Darmstadt) in Germany, supervised by Prof. Iryna Gurevych and Nils Reimers (2019 - 2021). I received my undergraduate degree at Birla Institute of Technology and Science, Pilani (BITS Pilani) in 2018. I also have industrial experience, working as a Data Scientist in KNOLSKAPE (2018 - 2019) and undergraduate internships at EMBL Heidelberg (Summer 2018) and Belong.co (Fall & Winter 2017).

If you wish to learn more about my research, visit Research where I mention in depth and maintain a list of all my publications. For any questions, best way to reach out to me is via email: nandan.thakur@uwaterloo.ca, nandant@gmail.com or Twitter.

I am actively looking for research internships in Fall 2024!

Recent News

2024

• [May 2024] I have been awarded the David R. Cheriton Graduate Scholarship starting Fall 2024 for my scholastic excellence in my PhD! [Link]
• [May 2024] Collaboration with Snowflake AI towards building better BEIRv2 and TREC-RAG [blogpost].
• [Apr 2024] I will be attending in-person NAACL 2024 in Mexico City, Mexico between 16-20 June 2024 and SIGIR in Washington DC, USA between 14-18 July 2024. If interested, do reach out!
• [Apr 2024] Received a 3K USD grant from Google to attend the NAACL 2024 Conference in Mexico City, 2024.
• [Apr 2024] My work on “Systematic Evaluation of Neural Retrieval Models on the Touch{'e}~2020 Argument Retrieval Subset of BEIR” has been accepted at SIGIR 2024 (Reproduction).
• [Apr 2024] My work on “Resources for Brewing BEIR: Reproducible Reference Models and Statistical Analyses” has been accepted at SIGIR 2024 (Resource).
• [Mar 2024] My Google internship work on “SWIM-IR: Leveraging LLMs for Synthesizing Training Data Across Many Languages in Multilingual Dense Retrieval” has been accepted at NAACL 2024.
• [Feb 2024] Started part time research collaboration on improving multilingual RAG systems with Vectara.
• [Jan 2024] Gave two research talks on “Heterogeneous Benchmarking of Information Retrieval” in IIT-D (Delhi) and IIIT-Delhi [presentation] [video].