Building My First VUI

Insights from a summer of learning about conversation design

UPDATE: Unfortunately, we didn’t place in the challenge but we are thankful for the opportunity and proud of what we created. Here are the results

This summer, Joseph Cochrane-Brown and I created Sow Study Aid for the Action On Google Challenge. Our goal was to use Quizlet’s API to create a captivating flashcard study experience through conversation. Before we started, we knew nothing about Bots, VUI’s, or API.AI. Here’s what we learned.

Design

We wanted our persona to mimic the experience of studying with a friend. So, Joseph and I took notes while helping each other study flashcards. A few characteristics stood out that helped us hone in on what makes for a friendly study session.

We discovered that the main challenge was to keep the user interested and motivated while being efficient during the study process. After scripting some demo conversations, we were able to prioritize the top 3 key elements of our design.

1. Personality

The difference between a fun study session and a boring one, is the personality of the person you’re studying with. Unfortunately, it’s not as easy as bottling up the personality of a charismatic study partner.

Some of the ways we crafted the persona of Sow Study Aid were by adding contextual gifs or soundbites based on whether you got the card right or wrong, placing random witty remarks in well traveled parts of the conversation, and by using casual or familiar language to help set a relaxed atmosphere.

Unfortunately, creating a likable personality for a bot is extremely subjective. Attempt at your own risk. But, the reward of nailing the persona of a friendly study partner was too great for us to pass up.

#meandjosephafterwefoundtheperfectgif

2. Conditional Responses

Technically, every point in a conversation for a bot is a conditional response. To us, a conditional response means thinking through the ways a possible change in context could alter the tone and wording of responses.

The simplest example is when you get 2 cards wrong in a row, Sow Study Aid occasionally adds to the response string,

“That’s 2 in a row. Let’s turn this around!”

That’s a very minute change that doesn’t seem to register the same way a ground breaking feature does. But, adding those conditional responses throughout the application will drastically improve the conversational experience.

3. Intuitive Intents

The notes that we took on our demo study sessions, laid out an obvious set of intuitive intents / features which created the foundation of our application. Some examples include reviewing the last 4 missed cards, navigating through the deck, and switching decks at any time.

When studying, there’s a few things that you naturally expect your partner to do. Those are the features that we focused on making available, and adding enough response variations for them to be invoked naturally.

Implementation

To keep from frying our brains, we went to the library and used wall-sized whiteboards to map out the conversation visually. Not having to keep track of a mental map of complex context and inputs, freed us up to be more creative with how we laid out the conversation.

Despite all that whiteboard real estate, we initially broke up the conversation into 3 simple sections: the introduction, topic selection, and study process. We simplified the structure of the conversation in a way that allowed us to break down the points of emphasis and most traveled paths of the conversation in ways that a script wouldn’t allow.

In each section, we subdivided further into intents and contexts. After intents and contexts were developed, we created variables that kept track of the things that a study partner would keep track of in the back of their head, like the cards that you’ve missed.

Lastly, we stored those variables in Firebase so the next time the app is invoked, all the information from the last study session is available to be picked up where the student left off.

#proofweactuallymadesomething

#myvoiceistooquietbutItried

Resources

Nothing would have been possible without the videos, articles, and example applications that we found while building Sow Study Aid. Here’s a curated list of anything and everything that was helpful for us.

Conversation Design: Speaking the Same Language_Six principles of human conversation poised to revolutionize Voice User Interface design The next generation of…_design.google

Deploy your Assistant app fulfillment webhook using Cloud Functions for Firebase._A quick analysis of why you should use Cloud Functions for Firebase for your Assistant apps and a walkthrough deploying…_medium.com