Exploring Age Adaptation of Conversational Systems
About
What if conversational agents would be able to adapt in the way they interact with humans? Unfortunately, state-of-the-art AI systems currently lack sophisticated adaptation capabilities. Our long-term goal is to develop artificial agents that can adapt to individuals/user groups at any level (age, expertise, language style, etc.) and that are perceived as trustable by users.
In the following project, we focused on users of different age groups, and asked:
- • Can some degree of adaptation be achieved by training a state-of-the-art system with data targeted to specific age groups?
- • Does this lead to differences in the perceived degree of anthropomorphism, social presence, trust, appreciation, and comprehensibility of the message?
Below you can find an open-source demo that showcases the results of the project regarding the first of these research questions.
This project is a collaboration between the Dialogue Modelling Group at the Institute for Logic, Language and Computation (ILLC) and the Persuasive Communication group at the Amsterdam School of Communication Research (ASCoR) at the University of Amsterdam.
System adaptation
Try it yourself. The state-of-the-art language model would first receive a prompt with corresponding criteria to generate an adapted output. Select a combinations from the options below to see what kind of adaptation different systems produced:
Prompt
↓
This option allows you to choose whether the resulting output will be adapted towards a particular age group or not.
In the non-adaptive setting, the model takes a prompt and generates an output by sampling from the distribution of words that are most likely to follow the prompt.
In the age adapted setting, the model is trying to generate text that is controlled for age.
Adaptation
↓
The Bag of Words method relies on lists of words that are representative of each age group’s language. The words are automatically extracted from the full version of the dataset via a frequency-based approach. In particular, for each age group, the words are ordered by frequency and the most frequent words are kept. The words that are in both age groups are removed. This leaves a list of words that are specific to each age group.
The discriminator model is a classifier that is trained to distinguish between two groups. In this case, the classifier is used to distinguish between two groups. The classifier is trained using a pre-trained language model. The discriminator parameters that are then used in this age-adaptive setting come from the training epoch with the highest test accuracy.
Model
↓
This option allows you to adapt the example model output towards a younger or an older age group. The age adaption is evaluated using a classifier model. You can see the evaluation of the classifier model along with the resulting output.
Style
↓
The model output is an example of what the selected model produced, given the particular criteria selection you have made. You can see how well our classifer evaluated the output below.
Model output
Prompt
model output
80% model probability toward ‘younger’;
20% model probability toward ‘older‘;
Test yourself
To test whether adaptations produced by the language models are detectable, we paired outputs from different models together and asked human evaluators to judge which one sounded more like coming from an older or a younger speaker. In total, we collected 9,000 judgments from 467 different participants.
How would you do yourself? Try to guess the age group of the following examples and see if your guesses agree with our human evaluators.
Which one of the two outputs sounds like a text that could be written by a younger speaker?
80% of our participands agrees with you
80% our automatic evaluation agrees with you
20% our automatic evaluation agrees with you
Find out more
The work received funding from the University of Amsterdam’s Research Priority Area Human(e) AI and from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 819455).