Contribution

Personalized Language Model Selection
through Gamified Elicitation of Contrastive Concept Preferences

Why?

Language models are widely used for different Natural Language Processing tasks while suffering from a lack of personalization. Personalization can be achieved by, e.g., fine-tuning the model on training data that is created by the user (e.g., social media posts). Previous work shows that the acquisition of such data can be challenging. Instead of adapting the model's parameters, we suggest selecting a model that matches the user's mental model of different thematic concepts in language.

How?

In this paper, we attempt to capture individual language understanding of users. In this process, two challenges have to be considered. First, we need to counteract disengagement since the task of communicating one's language understanding typically encompasses repetitive and time-consuming actions. Second, we need to enable users to externalize their mental models in different contexts, considering that language use changes depending on the environment. In this paper, we integrate methods of gamification into a visual analytics (VA) workflow to engage users in sharing their knowledge within various contexts. In particular, we contribute the design of a gameful VA playground called Concept Universe. During the four-phased game, the users build personalized concept descriptions by explaining given concept names through representative keywords. Based on their performance, the system reacts with constant visual, verbal, and auditory feedback.

Contribution

  • We combine multiple NLP methods with visualization techniques to produce an engaging, interactive environment for capturing the users’ mental models of particular language concepts.
  • Throughout the seven levels of the game, the users describe a given concept by representative keywords.
  • We integrate multiple game mechanics to both engage the users and simulate different contexts in which the language is used.
  • To support user engagement, the system stimulates the users to explore the language space, to overcome challenges, and to collaborate with or compete against a virtual player.
  • It further provides multi-channel (i.e., visual, verbal, and auditory) feedback on the users' successes at all levels. At the same time, each game mechanics presents a different context in which the language is applied, e.g., a setting with time pressure, or a reflective situation when interacting with a second virtual player.
  • With this paper, we aim at gaining first insights on what type of game elements can be successfully applied in gameful VA processes and how the users perceive this gameful design.


Design

  • We use the linguistic model for processing purposes to measure the user's performance and build virtual players.
  • The goal of the visual interface is three-fold: (1) to enable the users to describe concept names by entering representative keywords; (2) to measure users' input using multiple quality metrics and provide feedback on the entered keyword descriptiveness; (3) to display the entered keyword relatedness to the optimal linguistic model.
  • As the effectiveness of the applied game dynamics is user-dependent (i.e., people have different preferences), we integrated six game elements into the interface. With these elements, we aim to engage users while they perform the task. Furthermore, the exploration, challenge, competition, and collaboration dynamics are implemented as separate game levels that enables us to simulate and evaluate different contexts in which the language is used.



The linguistic model was designed in two iterations and was settled after the final implementation of the VA narrative. The VA narrative took five iterations; during the design process, we considered multiple visualization techniques for representing the language space, such as a graph, tree, as well as 2D projection layout. The game elements were reviewed according to the particular VA narrative. The checkmark-labeled game elements are implemented in the current design of the interface.

DEMO coming soon!