The project was included in the UNICEF AI for Children case study, and I co-authored the human-robot interaction part of it
I led the research and development of emotional interaction, successfully applying for a patent for the team
Team won the HUAWEI Human-Computer Interaction Innovation Contribution Award in 2022
User net promoter score is 35.3%, which is a relatively high level, and is rated by users as the most flexible robot in human-computer interaction at present.
XiaoYi Spirit is an intelligent companion robot powered by a dual-chip system with a computing power of 17 TOPS.Featuring 9 precision servos and an OLED display system, it can express over 2,000 facial expressions and natural body movements. Through its smart dual cameras and 5-microphone array, it accurately recognizes faces, movements, emotions, and precisely locates sound sources.Multiple sensor systems enable XiaoYi Spirit to perceive environmental changes in real-time, delivering natural and fluid human-robot interaction.
XiaoYi has its own special story and personality, and can show different levels of emotions in its expressions, making its interactions become more engaging and relatable. By recognizing children's emotions through speech, tone, and facial expressions, XiaoYi responds appropriately and guides children in understanding their feelings, providing timely and positive companionship.
We've developed a comprehensive suite of gamified interactive content, thoughtfully designed to cover everything from picture books and music appreciation to language learning and cognitive games. Acting as an intelligent guide, the robot encourages children to explore knowledge through observation, critical thinking, and inquiry, while developing problem-solving skills in an engaging, shared play environment.
The video demonstrates our robot's innovative English learning feature through seamless TV connectivity and interactive gameplay.
Smart TV Integration:
The robot synchronizes with TV displays to create an immersive large-screen learning environment.
Interactive Gaming Experience:
- Voice-guided gameplay instructions
- Natural conversation interactions
- Touch-based interactive games
- Real-time feedback and encouragement•
Professional Curriculum Design:
- Developed by certified language education experts
- Crafted by experienced game designers
- Age-appropriate content and difficulty progression• Daily English Conversation:
- Natural language processing for daily communications
- Contextual learning scenarios
- Adaptive conversation patterns
This comprehensive solution transforms traditional English learning into an engaging, gamified experience, creating an immersive environment that naturally motivates children to practice and improve their language skills.
Leveraging both voice timbre and facial recognition capabilities, the robot excels in managing multi-person conversations. We've also fine-tuned our speech recognition system specifically for children's speech patterns and linguistic habits, ensuring smoother and more intelligent interactions between the robot and childrens.
By combining sound source detection and facial recognition, the robot can precisely locate users in its surroundings, enabling more responsive and dynamic interactions. Thanks to its highly flexible servo motors, the robot can easily keep up with children's rapid movements and activities.
As a core figure in human-computer interaction design, I lead an innovative project that spans Human-Robot Interaction (HRI) and child cognitive psychology. The most exciting aspect of this project is its unprecedented deep integration of humanities and technology. Here, traditional single-domain knowledge is insufficient to support product design; we must rely on the collective wisdom of interdisciplinary teams to create truly valuable solutions.
Our project has achieved innovative breakthroughs in multiple areas:
What moves me most is seeing users establish genuine emotional connections with our robots. In a social media survey, I was delighted to discover bloggers spontaneously interacting daily with our robot "Xiao Yi," gaining a large following on video platforms. These authentic user feedbacks not only make me feel gratified but also deeply aware of the importance and responsibility of our work.
Preliminary research begins with understanding the initial product objectives, followed by in-depth user research to validate the feasibility of value propositions and define target user profiles and usage contexts (including When and Where). The insights gained from user research serve as key foundations for product design, ultimately guiding the development and refinement of product strategy.
Preliminary research begins with understanding the initial product objectives, followed by in-depth user research to validate the feasibility of value propositions and define target user profiles and usage contexts (including When and Where). The insights gained from user research serve as key foundations for product design, ultimately guiding the development and refinement of product strategy.
At the start, we identified the key questions that needed to be addressed and made assumptions based on communications with upstream and downstream stakeholders.
Centered around these key questions, we designed two user research themes for exploration.
Based on the research findings, we revised and supplemented our initial assumptions.
Based on the relevant conclusions from user research, we have identified key challenges:
Based on the relevant conclusions from user research, we have identified key challenges and developed a preliminary product strategy to address these challenges. This includes product goals and value propositions, product solutions, and initial market strategies. This will serve as the core guidance for our subsequent design work.
Based on the relevant conclusions from user research, we have identified key challenges:
"Experience Shapes Form, Form Determines Architecture."
To address the technical challenges in children's speech recognition (such as non-standard pronunciation, irregular grammar, and environmental noise), we innovatively developed lip-reading technology to effectively tackle real-world application scenarios.
In anthropomorphic design, I not only need to balance technical feasibility but also consider how to establish genuine emotional connections between robots and humans. This makes me feel like a director creating life, needing to imbue products with unique personalities and souls. Through carefully designed interaction modes, we enable robots to exhibit warmth, understanding, and care, making them trustworthy companions in users' lives.
Designing multimodal interaction systems is like orchestrating an intricate play, requiring perfect coordination of various interaction modes such as voice, visual, and tactile. In this process, I organize a cross-disciplinary team comprising AI experts, roboticists, psychologists, HRI designers, and artists (including scriptwriters, illustrators, animators, etc.). Each expert brings a unique perspective and expertise, collectively crafting smooth and natural human-machine interactions.
In content design, we are committed to creating interactive experiences that bring lasting joy to children and families. Each interaction scenario is thoughtfully conceived, considering both educational value and entertainment, making the robot an indispensable part of family life.
In developing the AI decision-making architecture, I work closely with the technical team, focusing on the robot's social decision-making capabilities. Our biggest challenge is to make robots interact with people "just right" - responding appropriately to the right person at the right time. This requires us to build a complex decision system that comprehensively considers:
Through the design and execution of the following release process, we ensured optimal feature presentation in a short timeframe while establishing a solid foundation for subsequent product iterations.
Through the design and execution of the following release process, we ensured optimal feature presentation in a short timeframe while establishing a solid foundation for subsequent product iterations.
Release Strategy Development
1. Develop product release strategy and robot demonstration plan.
2. Optimize feature presentation effectiveness within brief launch events.
Exhibition Hall Experience Design
1. Design robot interactions that autonomously attract attention and provide seamless guidance, considering exhibition hall noise levels and traffic flow.
2. Engage customer interest and guide them through core functionality experiences.
1. Design multiple channels for collecting user feedback during the exhibition period, including on-site questionnaires and online surveys.
2. Analyze feedback and public opinion to identify product highlights and areas for improvement, providing basis for future iterative optimization.
Based on the technical expertise and design experience accumulated during robot development, I assisted other departments with several specialized research initiatives.
1. How to resolve the contradiction between the serious image of system-level voice assistant and the needs of users across all age groups for smart speakers?
2. Will introducing the "Xiao Yi Sprite" as a child-specific voice assistant conflict with the existing system-level voice assistant?
3. How to address the user experience issues caused by the current limited AI capabilities conflicting with Xiao Yi's young female persona?
1. Research and investigate children's current in-vehicle behaviors and activities
2. Survey parents' expectations and needs for smart vehicle children-oriented functions
3. Analyze competitors' solutions and approaches to in-vehicle children's experience
4. Develop design strategies to enhance children's in-vehicle experience
As a core participant in this innovative project, I feel honored to be involved in shaping the future of human-computer interaction. This work is both challenging and creative, allowing me to fully utilize my professional expertise while constantly pushing boundaries and exploring new possibilities. In the future, I will continue to dedicate myself to promoting the deep integration of technology and humanities, creating more meaningful human-computer interaction experiences.
As technology advances, robots will play increasingly important roles in people's lives. They will not only be functional assistants but also emotional companions. This brings new challenges and considerations:
This project has given me not just professional skill enhancement but also a transformation in thinking: