Do you like him? - Designing a Multimodal Emotional Expression for a Mobile Robot.
ROLE
Project Management, Design Management, UX Design, Development
CUSTOMER/CLIENT
Technische Hochschule Ingolstadt, AININ gGmbH
TOOLS
Notion, Jira, FigJam, VS Code, Python, ROS
METHODS
Interdisciplinary agile PM, User Scenarios, WoZ, Focus Groups, Guerilla Testing
As part of the Master's course "Natural User Interfaces" at Technische Hochschule Ingolstadt, ENAMOUR — "Empathy with aN AutonoMOUs Robot" — was developed. The goal was to teach a quadruped robot (Unitree A1) to express convincing emotions: recognisable through body language, facial expressions on an LCD display, and fitting sound signals.
The project was interdisciplinary and brought together four study programmes: Mechanical Engineering (head mechanics), Computer Science (system architecture), Artificial Intelligence (speech recognition), and UX Design (emotion design and user studies). Our 13-person UX team was responsible for the emotional expressiveness of the robot from the first study through to the final implementation.
The result was a functional prototype named "Spike" that responds to voice commands, expresses emotions such as joy, sadness, anger, curiosity, and fear — and was presented to a broad audience at the THI On Campus Festival.
As one of the two Project Managers, I took on organisational responsibility for the UX team - alongside my active involvement in design and implementation.
Scrum-Adapted Framework
We worked within a Scrum-inspired, self-organised framework adapted to our needs. Each sprint lasted one week (two weeks in exceptional cases). Every sprint meeting consisted of a review, team discussion, and planning for the next cycle.
Tools
Cross-Team Communication
In addition to project management, two dedicated communication partners coordinated the weekly exchange with the Computer Science, AI, and Mechanical Engineering teams. This ensured that technical constraints were fed into our design decisions early on — for example, when the CS team had to switch to high-level movement commands and we adjusted our gesture concepts accordingly.
Quasi-experiment · n = 27 · Online · approx. 15 min
The first study examined how effectively Spike's body movements communicate emotions, and which design direction was preferred for the display face. Participants watched videos of the robot and rated gesture clarity, emotion recognisability, and design variables such as eyes, muzzle, and display style on a Likert scale.
Key findings: Joy and curiosity were well recognised; anger less so. The display was required to show eyes. Participants also requested additional emotional states — sleepiness and affection — which were incorporated in the second iteration of the gesture design.
.gif)
Quantitative · n = 43 · Online · max. 30 min
Building on Study 1, we tested four display design concepts: Alpha (cartoonish, colourful), Beta (realistic), Gamma (robotic, blue tones), and Delta (cartoonish, monochrome). Each group evaluated their variant using emotion videos, Likert scales, and open comments. The outcome was a combination of Alpha and Gamma: a robotic blue design with emotion-specific colours and cartoon-like symbols for added clarity. Colour gradients and iris details that brought the eyes to life were particularly well received.
Qualitative · n = 8 · In-person · Focus Group + UEQ
The third study evaluated the finished system. Eight participants assessed all eight emotions including sounds and transitions. In addition to a custom questionnaire, the standardised User Experience Questionnaire (UEQ) was used. Emotions were correctly recognised at an average of 4.14 out of 5. The visual display was more convincing than the sound signals. The display design achieved solid UEQ scores; participants were able to reliably identify emotions and follow the interaction concept.
Beyond my role as Project Manager, I was actively involved in the technical implementation of the UX component — specifically in system architecture and ROS development.
System Architecture
The overall system followed a modular pipeline: voice input → AI intent recognition → JSON transfer via ROS → UX emotion logic → output to display, sound, and movement. Our UX team was responsible for the emotion control layer.
ROS and Python 2.7
We worked with ROS Noetic in an Ubuntu VM, as ROS was barely functional on Windows or macOS. Communication with other teams was handled via ROS Publisher/Subscriber. Since Python 2.7 does not support common data science libraries, we built our lookup tables as nested dictionaries — an elegant solution with low overhead.
Emotion Logic (3 Sprints)
- Sprint 1: Technical MVP — receive intent, match emotion, output JSON to CS team
- Sprint 2: Dynamic output, intent list finalised, dictionary structure built
- Sprint 3: Synchronisation of motion, sound, and display; randomisation of sounds and movements
for more natural behaviour; emotion state machine for realistic transitions
We also developed a JSON-based intent list format that served as the central interface to the AI team and was refined iteratively throughout the project.
ENAMOUR was successfully demonstrated at the THI On Campus Festival (June 2022) in front of a broad audience — families, fellow students, and faculty. Spike moved, responded to commands, and displayed recognisable emotions on its screen.
Emotion Recognition Rate (Study 3): Ø 4.14 / 5 · Overall User Experience: Ø 3.77 / 5. The visual display was the strongest channel for conveying emotion; sound was perceived as complementary.
For me personally, this project was an intense experience in interdisciplinary collaboration: as Project Manager I maintained structure and communication, as UX Designer I shaped the interaction concept, and as a ROS developer I translated design decisions into running code — a combination I could not have experienced so directly in any other single role.
Links