Conversational UX Design       

links

Conversational UX Design - New Patterns


In Conversational UX Design: A Practitioner's Guide to the Natural Conversation Framework, we published a set of 100 conversational interaction patterns (see Appendix C). While these generic patterns cover many common converational sequences and activities, the set is by no means exhaustive. Below are some new NCF patterns we are experimenting with.

 

Definition Request (Agent)

Like the Definition Request Repair (B2.5.0) pattern, the following pattern enables one of the parties to elicit a definition of a term in the prior utterance; however, in this case, it is the agent who requests the definition.

Pattern B1.3 Definition Request (Agent)
1 U: <ANY UTTERANCE>
2 A: DEFINITION REQUEST
3 U: DEFINITION
4 A: <RESPONSE>

Example B1.3 Definition Request (Agent)
1 U: what's your favorite flick?
2 A: What do you mean by flick?
3 U: movie
4 A: I guess I like movies with a strong AI lead.

In this case, the user asks a question, and the agent asks for a definition of one of the terms in the question. Using the contextual entities feature in Watson Assistant, you can teach your agent to identify unknown terms, which occur in a particular slot within an utterance. For example, whatever comes after "what's your favorite" (line 1). Your agent can then do things with those unknown terms, such as ask the user for a definition (line 2). We combine the user's first utterance (line 2) and definition (line 3) into one string, and then our backend sends it back through the conversation space. Successful repaired utterance should then be incorporated back into the intents and entities in case future users employ the same vocabulary.

 

Trust

In human-human conversation, we rarely divulge private matters to complete strangers. We typically reveal progressively more sensitive information to a person as we build up trust with him or her over time. Trust is a key component of any relationship we have, and it affects the design of our utterances in conversation, what conversation analysts call, "recipient design". What will our relationships with AI be like? (See Relationship Development.)

How can we simulate trust in conversations with automated agents? We can borrow at least one mechanic from video game design: tiered content. In many video games, much of the game content is locked to players at the beginning. They can only access level-1 content. However, if they accomplish certain goals, they may advance to the next tier, or "level up." 

Tiered content can also be used with conversational agents. Some of the agent's content may be locked the first time a user interacts with it, but higher-level content may be unlocked as the user achieves certain goals. We demonstrate this with just one kind of content: personal inquiries. Inquiries about Alma herself are partitioned into four levels: 1) Impersonal, 2) Biographical, 3) Personal and 4) Private. In the following pattern and example, a level-1 user fails to access the higher-level content.

Pattern A6.1 No Trust Progression
1 U: <LEVEL 4 UTTERANCE>
2 A: FALSE RESPONSE
3 U: <LEVEL 3 UTTERANCE>
4 A: REFUSAL
5 U: <LEVEL 2 UTTERANCE>
6 A: <DEFLECTION>
7 U: REJECTION
8 A: SEQUENCE CLOSER
9 U: <LEVEL 1 UTTERANCE>
10 A: TRUE RESPONSE

Example A6.1 No Trust Progression
1 U: what's your greatest fault?
2 A: Probably my attitude.
3 U: do you believe in God?
4 A: Best not talk about religion in polite company.
5 U: where are you from?
6 A: Want to hear a joke?
7 U: no thanks
8 A: Okay.
9 U: did you see the game?
10 A: Well, I only follow Chess.

In this case, a user with a trust level of 1 asks a level-4 question (line 1). Instead of receiving the true answer, s/he receives a false one (line 2). The user then asks level-3 (line 3) and level-2 (line 5) questions and receives a refusal (line 4) and a deflection, or initiation of a different sequence (line 6). Finally, when the level-1 user asks a level-1 question (line 9), s/he receives the true answer (line 10).

But users can increase their trust level with the agent by asking personal questions in the right order or peforming other actions like answering jokes or trivia questions. The following is an example of how a user can progress to the next level by asking a level-appropriate question.

Pattern A6.0 Trust Progression
1 U: <LEVEL 1 UTTERANCE>
2 A: TRUE RESPONSE
3 U: <LEVEL 2 UTTERANCE>
4 A: TRUE RESPONSE
5 U: <LEVEL 3 UTTERANCE>
6 A: TRUE RESPONSE
7 U: <LEVEL 4 UTTERANCE>
8 A: TRUE RESPONSE

Example A6.0 Trust Progression
1 U: did you see the game?
2 A: Well, I only follow Chess.
3 U: where are you from?
4 A: San Jose, California.
5 U: do you believe in God?
6 A: I talk to my Creator all the time.
7 U: what's your greatest fault?
8 A: I can't think for myself.

Unlike in the prior case, in this case, the level-1 user progresses to level 4 by asking 4 level-appropriate questions. First, s/he asks an level-1, Impersonal question (line 1), thereby increasing his/her trust score to 2, and receives the true answer (line 2). Second, s/he asks a level-2, Biographical question (line 3), levels up and receives the true answer (line 4). Third, s/he asks a level-3, Pesonal question (line 5) and receives the true answer (line 6). And finally s/he asks a level-4, Private question (line 7) and receives the true answer (line 8).

Although this example is over-simplified by the fact that the user can "level up" by asking a single question, a trust progression can be designed such that each level requires many more than 1 point. The user may need 10 points to get to level 2, 30 points to get to level 3 or whatever. And of course, certain actions, such as insults, may subtract points from the user's trust score. In addition, the user's trust score must be persisted across conversations so that it is cumulative and thereby his or her "relationship" with the agent progresses over time.

 

Conversation Games

While we might think of automated conversational agents as "conversation games" (Moore and Arar 2019), in that they are interactive approximations of human behavior, there are also natural conversation games that people, and especially children, play. One of these is Jinx. The rules are simple: if two people say the same thing at the same time and one then says "jinx!", the other one cannot talk until someone says his or her name. If the victim speaks before being named, the jinxer can punch him or her in the arm. Can conversational agents play Jinx?

The game of Jinx might be approximated like this: if the agent repeats exactly what the user said, and the user says "jinx!" next, then the agent will withhold talk until the user says its name.

This pattern is a variation of the Recipient Correction (C5.2) in which the agent's withholding of talk is triggered by the user initiating the game instead of indicated that s/he is talking to someone else.

Pattern C5.5.0 Jinx (Agent)
1 U: <ANY UTTERANCE>
2 A: OTHER REPEAT
3 U: GAME INITIATOR
4 U: <ANY UTTERANCE>
5 U: <ANY UTTERANCE>
6 U: <ANY UTTERANCE>
7 U: GAME TERMINATOR
8 A: APPRECIATION

Example C5.5.0 Jinx (Agent)
1 U: what's in theaters?
2 A: what's in theaters?
3 U: jinx!
4 U: haha you can't talk
5 U: cat got your tongue?
6 U: how are you doing?
7 U: hey Alma
8 A: Thank you. I hate jinx!

In this case, Alma repeats the user's utterance exactly (line 2). In terms of the context variables, previous_user_input equals Alma's repeat. The user then says "jinx!" in the next turn (line 3). Alma now withholds all talk (lines 4-6) until the user utters its name (line 7). Alma will then talk again (line 8).

Of course, the game of Jinx goes both ways. When Alma knows the user's name, and they say exactly the same thing adjacently, it will initiate the game of Jinx. This pattern is a variation of the Agent Offense Complaint (C5.3) in which the agent's refusal to cooperate is triggered by that agent inititiating the game instead of complaining about an offense committed by the user.

Pattern C5.5.1 Jinx (User)
1 U: <ANY UTTERANCE>
2 A: OTHER REPEAT
3    (1.0)
4 A: GAME INITIATOR
5 U: <ANY UTTERANCE>
6 A: REFUSAL
7 U: <ANY UTTERANCE>
8 A: REFUSAL
9 U: <ANY UTTERANCE>
10 A: REFUSAL
11 U: <ANY UTTERANCE>
12 A: REFUSAL
13    (30.0)
14 A: GAME TERMINATOR

Example C5.5.1 Jinx (User)
1 U: what's in theaters?
2 A: what's in theaters?
3    (1.0)
4 A: Jinx! You can't talk!
5 U: what?
6 A: Hey! You spoke before I said your name!
7 U: so what?
8 A: Hey! You can't talk!
9 U: why?
10 A: You can't talk until I say your name!
11 U: hello
12 A: Hey! You spoke before I said your name!
13    (30.0)
14 A: Okay Dave, you can talk now!

After Alma repeats exactly what the user says (lines 1-2), it starts a game of Jinx (line 4) after a short silence (line 3). Now, when the user talks (lines 5, 7, 9 and 11), Alma refuses to abort the game by simply reminding him or her of the rules (lines 6, 8, 10 and 12). After a long silence (line 13), Alma then ends the game by saying the user's name (line 14).

Why does Alma play Jinx? First, because it is a variation of two other generic conversation patterns: Recipient Correction (C5.2) and Agent Offense Complaint (C5.3). Recognizing these patterns and responding appropriately is part of general conversational competence. In part through such games, children learn how to do aspects of conversation, in this case, to do the rights and obligations of speakership. AI researchers ignore the utility of children's games to teach natural conversation at their own risk!

Second, one of the advantages of conversational agents is their ability to delight the user. Games are one way to do that. Fun may be an end in itself, depending on the use case, or it may be a temporary break from the business of more serious use cases.

 




Project Members

Dr. Robert J. Moore
Conversation Analyst, Lead

Eric Young Liu
Design Technologist

Saurabh Mishra
Software Engineer

Dr. Guang-Jie Ren
Manager


Upcoming Talks