Welcome ...

... to the website of the Dialogue Modelling Group in Amsterdam!

We carry out research at the interface of computational linguistics, cognitive modelling and artificial intelligence. Our aim is to understand how we use language to communicate with each other in situated environments and how dialogue interaction shapes learning -- about the world and about language itself. These are some of the topics we work on: Computational semantics and pragmatics, LLM evaluation, visually grounded language, conversational agents and learning from interaction, uncertainty in language understanding and generation, language variation and change in communities of speakers, safety in conversational AI.

News

• 13/01/2025 Lorenzo Proietti from the Sapienza NLP group has joined our lab as visiting PhD student for 6 months. Welcome, Lorenzo!
• 02/01/2025 A few new pre-prints are out - check out the publications section.
• 01/12/2024 Alberto has moved on to a postdoc position at the Amsterdam University Medical Centres. Congratulations on the new job!
• 10/10/2024 Raquel gave a keynote talk at COLM.
• 30/09/2024 Papers accepted at EMNLP and Findings of EMNLP from our group!
• 01/09/2024 Esam has moved on to a new position at the MPI in Nijmegen. Congrats and good luck with your next step!
• 18/07/2024 Papers accepted to COLM and ICMI!
• 17/05/2024 Ece successfully defended her PhD thesis on Visual and Linguistic Processes in Deep Neural Networks: A Cognitive Perspective. Congratulations Dr. Takmaz!
• 15/05/2024 Several papers accepted at CogSci 2024 and ACL 2024! Check out the publications section.
• 18/01/2024 Several papers accepted at EACL 2024!
• 15/12/2023 Mario successfully defended his PhD thesis on Neural Models of Language Use. Congratulations Dr. Giulianelli!
• 10/12/2023 We have received an Outstanding Paper Award at EMNLP and a Best Data Award at the GenBench Workshop for our paper Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models led by Jirui Qi and Arianna Bisazza.

[news archive]

People

Affiliated Members

Alumni (incomplete list)

Alberto Testoni, postdoc 2023-2024, moved to postdoc in Responsible NLP and ML for Healthcare at the Amsterdam University Medical Centres.
Esam Ghaleb, postdoc 2022-2024, moved to postdoc at the Max Planck Institute for Psycholinguistics in Nijmegen.
Ece Takmaz, PhD candidate 2019-2024, moved to postdoc at Utrecht University.
Mario Giulianelli, PhD candidate 2019-2023, moved to postdoc at ETH Zurich.
Arabella J. Sinclair, postdoc 2019-2022, moved to lecturer (assistant professor) at University of Aberdeen.
Marco Del Tredici, PhD candidate 2017-2020, moved to Machine Learning Scientist at Amazon Berlin.
Elia Bruni, postdoc 2016-2018, moved to Marie Curie fellow at UPF Barcelona.
Ravi Shekhar, PhD candidate 2016-2019, based in Tento and co-supervised with Raffaella Bernardi; moved to postdoc at Queen Mary University of London.
Julian J. Schlöder, PhD candiate 2014-2018, moved to postdoc at the ILLC.
Janosch Haber, MSc student 2018-2019, moved to PhD candidate at Queen Mary University of London.
Laura Aina, MSc student 2015-2017, moved to PhD candidate at UPF Barcelona.
Bill Noble, MSc student 2014-2016, moved to PhD candidate at CLASP, University of Gothenburg.

Resources

Some datasets, code, and other publicly available resources developed by the Dialogue Modelling Group. See also the links provided with each publication.

Language and vision

• TRADE: TRuly ADversarial ad understanding Evaluation
  A new diagnostic test set consisting of 300 randomly selected image-based advertisements from the Pitt Ads dataset. Each ad is associated with 3 options: one valid explanation from the available annotations in the Pitt Ads dataset and two newly created adversarial negative explanations. .
• The BLA Benchmark: Investigating Basic Language Abilities of Multimodal Models
  BLA is a novel, automatically constructed benchmark to evaluate multimodal models on basic linguistic constructions—active-passive voice, coordination, and relative clauses—that even preschool children can typically master.
• Generating Image Descriptions via Sequential Cross-Modal Alignment Guided by Human Gaze
  Code repository for our work on leveraging eye-tracking data to model image description generation.
• MALeViC: Modeling Adjectives Leveraging Visual Contexts
  Dataset of synthetic images with pairs of statements and truth values related to size adjectives, as well as code repository for several experiments and analyses that leverage this dataset.
• The PhotoBook Dataset: Building Common Ground through Visually Grounded Dialogue
  A large-scale collection of 2,500 human-human, visually-grounded and goal-oriented English conversations between pairs of participants. Funded by ParlAI, Facebook AI Research. The website includes links to download several subsets of the data and to code repositories for several experiments and models.
• VISTA: Visually Grounded Talking Agents
  Code and overview of research on visually-grounded language in interactive settings presented in a series of papers led jointly with Raffaella Bernardi.

Linguistic variation and change

• DUPS: Diachronic Usage Pair Similarity
  Dataset of similarity judgements of English word usage pairs from different time periods, as described by Giulianelli et al. 2020
• Short-term Meaning Shift
  Dataset of words annotated by users of the subredit r/LiverpoolFC for meaning change. See Del Tredici et al. 2019.
• The Road to Success
  Dataset from 20 Reddit forums (subedits) and code to reproduce the findings by Del Tredici and Fernández 2018.

Other topics

• MBBQ: Multilingual Bias Benchmark for Question-answering
  A carefully curated version of the English BBQ dataset extended to Dutch, Spanish, and Turkish, designed to measure stereotypes in generative LLMs that are commonly held across these languages. See Neplenbroek et al. (2024)
• The LAMBADA dataset: Word prediction requiring a broad discourse context
  Dataset to evaluate the capabilities of computational models for text understanding by means of a word prediction task. See Paperno et al. 2016. Dataset available in HuggingFace here.
• Corrective Feedback in First Language Acquisition
  Dataset of corrective feedback moves in CHILDES corpora annotated according to the taxonomy proposed by Hiller and Fernández 2016.
• How to Reject What in Dialogue
  Dataset of rejection moves from several dialogue corpora annotated according to the rejection taxonomy proposed by Schlöder and Fernández 2019.
• Collective Annotation
  Datasets and resources related to the problem of aggregating the judgements of multiple individuals, regarding a linguistic annotation task or similar, into a single collective judgement that reflects the view of the community. Project jointly led with Ulle Endriss.
• Quantified Referring Expressions
  Data from an online elicitation experiment on the use of quantified referring expressions.

Publications

News archive

• 24/10/2023 Paper accepted at WACV 2024 on gesture detection, led by Esam.
• 07/10/2023 Several papers accepted at EMNLP 2023! Check out the publications section
• 01/09/2023 Vera Neplenbroek joins the group as PhD candidate in the LESSEN project. Welcome, Vera!
• 05/06/2023 Anna Bavaresco joins the group as PhD candidate, supervised by Raquel. Welcome, Anna!
• 02/05/2023 Several papers accepted at ACL and Findings of ACL from our group! Check out the publications section
• 06/02/2023 Our article on semantic adaptation via linguistic interaction has now been published open access in the Cognitive Science journal.
• 01/02/2023 Alberto Testoni joins the lab as postdoc. Welcome, Alberto!
• 23/01/2023 Papers by Sandro and Michael accepted at EACL!
• 08/09/2022 Michael Hanna starts his PhD, supervised by Sandro Pezzelle. Welcome, Michael!
• 07/09/2022 A blog post and Twitter thread about our work on Structural Persistence in Language Models.
• 01/09/2022 Esam Ghaleb has joined our group as postdoc to work on modelling face-to-face dialogue within the Language in Interaction consortium. Welcome, Esam!
• 26/05/2022 Ece Takmaz receives the Best Shared Task Paper Award at the CMCL workshop (ACL 2022) - congrats, Ece!
• 25/03/2022 Check out Joris' blogpost on Implementing a Transformer from Scratch.
• 20/03/2022 Arabella J. Sinclair moves out to take up a Lectureship at the University of Aberdeen - congrats, Janie!
• 14/03/2022 Mario presents his work on strategies of language production at the Language Technology Group Seminar of the Univerisity of Oslo.
• 01/10/2021 Joris Baan has joined the group as ELLIS PhD candidate, co-supervised by Barbara Plank. Welcome, Joris!
• 01/09/2021 Sandro Pezzelle is now Assistant Professor in Responsible AI at the ILLC - congrats, Sandro!
• New work by our group to appear at EMNLP, CoNLL, TACL, and SemDial; checkout our publications.
• 22/02/2021 Blog post by science journalist Iris Proff, featuring work led by Ece on using human gaze to guide image description generation. Ece will be discussing this work at the AI Meetings of Koç University on 9 March.
• 15/12/2020 Blog post by science journlist Iris Proff featuring Marco's work on fake news detection.
• 10/12/2020 Mario's MSc AI thesis on lexical semantic change has received the Amsterdam Data Science MSc Thesis Award!
• 06/11/2020 Congrats to Marco, who has successfuly defended his PhD thesis Linguistic Variation in Online Communities: A Computational Perspective
• New work by our group published at EMNLP (Generating Image Descriptions via Sequential Cross-Modal Alignment Guided by Human Gaze and Refer, Reuse, Reduce...), at Findings of EMNLP (Be Different to Be Better!), at COLING (Words are the Window to the Soul...) and at CLiC-It (Grounded and Ungrounded Referring Expressions in Human Dialogues).
• ACL 2020: Mario will be presenting the paper Analysing Lexical Semantic Change with Contextualised Word Representations, Lieke the paper Learning to Understand Child-directed and Adult-directed Speech and Ece will be a live microblogger in the Language Grounding track (@ecekt2).
• Spring 2020: Sandro and Raquel were awarded a grant by the Research Priority Area Human(e) AI to carry out an interdisciplinary project on adaptation of conversational agents to different age groups.
• Spring 2020: Ece has joined EACL's student board.