Hey Siri: Help me pick a lunar new year gift
At the end of 2020, I was lucky to be invited to a job interview related to the Siri communication design role. This was indeed an exciting moment for me to be able to take part in the research and design sessions of voice user interface (VUI) which I was completely a novice. My past research and work experience were sorely focused on a graphical user interface (GUI), and with limited knowledge in voice-based interaction design. Therefore, I would like to share some of my gains and reflections through this endeavour.
Background —
Creating engaging and personalized gift recommendations in Siri
With recent advances in conversational AI, many tasks have been automated even in some personalized works, for instance, wearing outfit suggestions. In this design, we bring a new Siri model to personalize gift recommendation request.
Challenges —
How do we normalize the gift recommendations process in the form of voice assistant?
The design starts with a triggering issue we may all face in daily life, that is how to pick the right gift for the right person. So how we select gifts for friends in daily life, and how do we project similar task in a conversational voice assistant become the key challenges.
Soluiton —
Keyword-based gift recommendation via conversational AI
We proposed a personalized gift recommendation system based on a personalized keywords system. To build the personalized model, two steps are required, a persona model to collect keywords regarding age, career, and hobits; and a gift category model to narrow down gift category. By using keywords from these two models, a finalized gift wish list would be generated and displayed to the end-user.
Design process —
01. Understand users diverse intention for holiday gifts
To start with the apps users groups and their needs, I first conducted semi-constructed interviews and literature research to understand users intentions to send gifts in lunar new years. Based on the analytics, I further summarised results in terms of relationship, intention, recommended gift category, and other factors for classification.
02. Understand gift recommendation flow in real life, and explore potential oppotunities for generalization
The design then goes with a contextual understanding of how people seek advice on gift recommendations in their daily life. By coding from 8 semi-structured interviews and cognitive walkthroughs, we summarised and nominated a potential gift advice flow and we further quantified persona and category keywords system that could be well manipulated as gift sorting parameters to be used for generalization of the gift recommendation system.
03. Initiate the designated dialogue flow to finalize users requests
We framed our gift communication into three sections by their intentions, greeting, gift sort keywords, and final recommendation stage. Below are example scripts we wrote for the system. We further classified different info requirement for persona model building.
04. Polish the dialogue for better quality gift recommendation model builds
Before diving into the design tool and building out the flows directly, I started by drafting the conversation flow and writing sample dialogues. It helped me to think about the structure and logic of my design. In addition, writing sample dialogues is an effective way to discover problems so that I can discuss and get clarifications from pilot tests and get improved in an early stage. Moreover, it in return encouraged me to write engaging yet short dialogues to achieve system intention.
I shared one dialogue scene in ask users to share three habits of their gift receivers. In version 1, this is phased as:
“Can you tell me some hobits the person have?”
Although straightforward, this question itself is too open-ended for our gift recommends model. This may result in two undesired situations, one as users simply say “I don't know” because the question itself is a bit vague and time-consuming to answer; the other is user spends a larger chunk of time sharing that info. Take in mind, in this step the system only called for some keywords but not the rationales.
So, here is version 2, which is phased as:
“Can you name three hobits the person have?“
We restricted the number of hobits and provide a clear goal for our customers. In return, both keyword extract accuracy and time improved. The following sections contain a final version of dialogues designed for the proposed interaction.
USER
Hey siri, I want to pick a lunar new year gift for my girlfriend, do you have any recommends? 嘿Siri，我想为我的女朋友挑选一个新年礼物，你有什么建议吗？👉🏻 She is 28 她28岁 👉🏻 ——— She worked as a administration staff 她是一位公司职员👉🏻 ——— Selection in progress… 用户选择中… 👉🏻 ——— Selection in progress… 用户选择中… 👉🏻
SYSTEM
——— We record your request, before that I need to ask some question to prepare a better gift for you. May I know Lily’s age? 好的没问题，在开始之前我需要问你一些问题，她的年纪是多大呢？👉🏻 Would you like to tell me about her career? 那她的职业是什么呢? ——— Great, I think I started to grab some idea now. Can you select some of the hobits she mght like? 棒极了，我开始有一些思路了。那你可以告诉我她的三个喜好吗👉🏻 ——— Fabulous, here is the one last step to sort our gift categeories briefly 👉🏻 请再接再厉，还有最后一步就要完成了 ——— Bravo, here are the gifts I preapred for you, you can refresh to view more… 噔噔噔，你的专属礼物清单已经生成
05. Mockups that well merge graphical and voice interface
Step 1
In the first interaction cycle, users initiate the dialogue to seek advice on gift recommendation in the lunar new year. The system will then ask several questions regarding the gift receivers to personalize the gift recommendation flows. All the interactions in these steps are verbal. Conversely, interactions in the following steps are all based on graphics.
Step 2
Two selection filters would be promoted to seek explicit gift categories and price range preference, those parameters greatly reduced recommendation failure rate.
Step 3
In the last steps, all sorted gifts that may well-satisfied gift parameters are promoted and displayed. Their snapshot, name, price, and source are clearly cited for reference. In addition, users can refresh the page to explore more gift oppotunities.
Takeaways —
A concise interaction flow is mandatory for multimodal interface
It is well discussed and examined that the conversational design interactions should be short, concise, and problem-oriented to facilitate solve of the issue. While jumping out of the unimodality sound-based interactions by combining the visual interface, it leaves the ultimate questions for product designers to answer, that is on what occasion we use the most suitable modality to finish the interactions?
For instance, multiple selection option shoule be promoted by visual🎨 other than voice becasue of the inefficiency to hold multple audio information by head, while questions regarding to personality building and exploring may be more proficient by verbal🔉appraoch as it provides a more intuitive and natural channel for user to phase their answer.
Iterative disambiguate process and adaptive function framework are key to usability success
Working on industry projects has boosted my proficiency in designing with adaptive thinking as well as my proficiency in refining requirements from ambiguous to concretely defined. Only by comprehensively viewing the corner cases and extreme conditions, can we make the interactions useful for all of its users.
The agile working environment fosters the needs for agile development, that requirements may face rapid changes as the post-release deploy and customer feedback surges. Therefore, learning how to create a holistic design framework to involve those changes is a necessitate for fresh graduate designers.
