ESR: Design Thinking for the Next Generation of Emotion-Sensing Interview Robots
Design Thinking for Developing a Case-based Reasoning Emotion-Sensing Robot for Interactive Interview
This paper presents a prototype for an Emotion-Sensing Robot (ESR) designed for simulated job interviews, utilizing Case-based Reasoning (CBR) and Design Thinking. The system integrates facial expression analysis, blink rate detection, and semantic sentiment analysis to evaluate interviewee nervousness and provide personalized feedback.
TL;DR
This study introduces a novel Emotion-Sensing Robot (ESR) tailored for the high-stakes environment of job interviews. By merging Design Thinking with Case-based Reasoning (CBR), researchers created a system that doesn't just "see" a user, but interprets their nervousness through facial expressions, blink rates, and semantic analysis to provide a human-centric feedback loop.
Background: Why Give Robots Emotional Intelligence?
While Human-Robot Interaction (HRI) has advanced technically, robots still struggle with "intuition." In a simulated interview, a robot that ignores a user's visible anxiety is less effective as a training tool. The authors argue that by introducing affective computing, robots can become active participants in mitigating user stress, thereby achieving more complex, goal-oriented tasks.
Methodology: The CBR and Design Thinking Approach
The core of this research lies in its interdisciplinary framework. Instead of a purely black-box machine learning model, the authors employed Case-based Reasoning (CBR). CBR mimics human problem-solving by comparing new "cases" (user interactions) to past experiences stored in a library.
The Multimodal Sensing Pipeline
The system tracks three key indicators:
- Facial Happiness Index: Using Google Vision API to detect confidence.
- Blink Rate: A critical physiological marker where a rate >20 per 30s signals nervousness.
- Semantic Analysis: Analyzing resume text and interview responses for positive/negative sentiment and keyword relevance.
The weight assignment for these factors is calculated using the Nearest Neighbor method, allowing the robot to adjust its evaluation based on the specific strengths or weaknesses of the interviewee.
Experimental Insights: Experience vs. Anxiety
The study evaluated 14 participants across different fields (IT, Mechatronics, Design). The findings provided fascinating insights into clinical "nervousness":
- The Recovery Gap: Both experienced and inexperienced users started the interview "nervous." However, participants with professional experience returned to a "normal" state almost immediately after self-introduction, whereas students remained in a high-anxiety state throughout.
- The Smile Factor: Only participants with work experience displayed "happy" facial markers, suggesting a higher degree of learned social confidence.
Critical Analysis & Conclusion
The true value of this work is not just in the hardware, but in the user-centered design. By using Design Thinking (User Journey Maps and Personas), the authors ensured the robot's logic aligned with real-world emotional triggers.
Limitations: The sample size (n=14) is small, and the "nervousness" threshold for blink rates (over 60 being "not sure") suggests some noise in the sensor data that needs further refinement.
Future Outlook: This methodology can be extended beyond interviews to medical triaging, customer service, or therapeutic robots, where understanding the "why" behind an emotion is just as important as the emotion itself.