School Case Study
AI design, Smart Home HCI
Advised by Julian Scaff
We conceptualized a way to break screen borders to use the system everywhere in the room by projection as an interface medium. Meanwhile, combining voice inputs, users is able to use smart home applications without physical contact to expand use cases and use everywhere.
In this project, we focused on designing conversational AI and the gesture-voice input methods. My main contributions included gesture research, prototyping animations, and the design of AI visualizations.
We considered the home ai is an extension of user's eyes and hands, where it sees and proceed decisions for the users. In this part, I contributed to the story script and ai dialogue writing.
Projection-based interfaces enable screen-free control, offering a more hygienic and versatile alternative to traditional touchscreens. Already emerging in home entertainment and automotive contexts, this technology unlocks new possibilities for smart home interactions.
Imagine answering a video call with a simple hand gesture while cooking, or securely sending a PIN code to a delivery person while in the shower—no physical contact required. With these scenarios in mind, we designed a seamless activation system that supports fluid, contactless interaction, allowing users to stay focused and uninterrupted in their daily routines.
Wake up interface by spreading the hand
Raising an open hand with fingers spread triggers the projection interface to appear on the wall.
Open room setting/app deck by moving up & down
We designed the interface with active feedbacks and intuitive interaction when the hand moves close and swipes.
Open an app
Our design of "using anywhere" experience allows the user to open an app at any area in the house, the inputs of gestures matches mental models, which are pinch, drag and release.
Move an app card
In the case of designing for home AI, I look to what users will be naturally drawn to: connection and interaction. When interacting with technology in private living space, especially a butler-like systems, the home AI is designed to be neutral and less human, to be less harmful and aggresive.
When interacting with technology, especially AI-driven systems, users often want a sense of familiarity and relatability. Thus the motion of AI should be intuitive and understandable. Made in After Effects.
Idle State
Speaking State
Generating State
Loading State