HappyHour: Transforming Smartphones into Emotionally-Aware Therapists

Happy hour - improving mood with an emotionally aware application

2015-07-01
Pedro Carmona, David Nunes, Duarte M. G. Raposo, David Silva, Jorge Sá Silva, Carlos Herrera
Summary
Problem
Method
Results
Takeaways
Abstract

This paper introduces HappyHour, a Human-in-the-loop Cyber-Physical System (HiTLCPS) designed as a mobile application to improve user mood through walking exercises. It leverages an Artificial Neural Network (ANN) to infer emotions from smartphone sensors, ECG smartshirts, and weather APIs, achieving enhanced Quality of Experience (QoE) through emotionally-aware network management.

TL;DR

HappyHour is a pioneering Human-in-the-loop Cyber-Physical System (HiTLCPS) that doesn't just track your mood—it actively tries to fix it. By fusing data from your heart rate, your environment, and even the weather, the system uses an Artificial Neural Network (ANN) to detect negative emotions like anxiety or boredom. Once detected, it triggers a "Behavior Change Intervention," suggesting mood-boosting walks while optimizing your phone’s internet connection via Multi-Path TCP to prevent any connectivity-induced frustration.

Problem & Motivation: The Gap in Affective Computing

Most "affective" apps are passive; they act as digital diaries. However, the authors argue that for a system to be truly human-aware, the human must be an integral part of the control loop.

The challenge lies in two areas:

  1. Contextual Inference: Predicting mood is notoriously difficult due to the "physiological distinctions" between individuals.
  2. System Frustration: If a user is already anxious, a lagging app or a dropped WiFi connection acts as a "negative reinforcer."

HappyHour's insight is to treat the system's technical configuration (privacy and networking) as a therapeutic tool.

Methodology: The Core Architecture

The system follows a classic HiTL paradigm. It senses (Sensing), thinks (Emotion Inference), and acts (Actuation).

1. Multi-Modal Sensing

The input isn't just a survey. It includes:

  • Physiological: Continuous ECG/Heart-rate via a Bluetooth smartshirt.
  • Environmental: Accelerometer (movement) and Microphone (ambient noise/music identification).
  • Meteorological: Real-time weather data (temperature, cloudiness) which correlates with mood.

2. The Neural Network "Brain"

The authors compared various ML techniques. While Support Vector Machines (SVM) were faster, Artificial Neural Networks (ANN) provided the best balance of accuracy and CPU efficiency for mobile hardware.

Model Architecture Fig 1. The HappyHour control loop showing the feedback mechanism between human emotion and system response.

The team optimized the hidden layer structure, discovering that a two-layer configuration (3 nodes then 2 nodes) significantly outperformed a single-layer version in specificity.

3. Actuation: The "Happy Hour" Change

When the NN flags a negative state:

  • Walking Suggestions: The app suggests nearby Points of Interest (POIs).
  • Network Hedging: It enables MPTCP, using both 4G/LTE and WiFi simultaneously. If you walk out of WiFi range, your session doesn't drop, eliminating the frustration of modern connectivity "dead zones."

Experiments & Results

In testing, the two-hidden-layer ANN proved robust. Even though it required more training epochs (3000 vs 100), the increased sensitivity (0.720) was deemed critical for a therapeutic context where missing an "anxiety" state is more costly than a few extra seconds of background training.

Performance Table Table 1. Comparison of ANN configurations. The two-layer model provides superior specificity (0.830).

Furthermore, the system utilized crowd-sourced "Heatmaps" (Fig 2) to characterize POIs. This allows an anxious user to see if a nearby park is "agitated" (high movement) or "calm" before they even arrive.

Heatmaps Fig 2. Real-time heatmaps showing attendance and movement levels at various locations.

Critical Insight & Conclusion

HappyHour represents a shift from Human-Computer Interaction to Human-in-the-Loop Control. The true technical "gem" in this paper isn't just the emotion detection, but the use of networking protocols (MPTCP) as a psychological countermeasure. By proactively eliminating technical friction, the system respects the user's fragile emotional state.

Limitations: The reliance on a specialized "smartshirt" in 2015 was a barrier to entry. However, in today’s era of Apple Watches and high-end wearables, the groundwork laid by HappyHour is more relevant than ever. Future iterations could integrate "LLM-based coaching" to provide even more personalized feedback during those corrective walks.

Find Similar Papers

Try Our Examples

  • Search for recent papers that utilize Multi-Path TCP (MPTCP) or similar networking protocols specifically to improve Quality of Experience (QoE) in mobile health or affective computing applications.
  • Which study first introduced the "Emotion Grid" for mobile sensing, and how has the transition from manual self-reporting to automated ANN-based inference evolved since the EmotionSense framework?
  • Explore how contemporary wearable devices (like Apple Watch or Oura Ring) have improved the accuracy of emotion recognition in Human-in-the-loop systems compared to the smartshirt sensors used in this paper.
Contents
HappyHour: Transforming Smartphones into Emotionally-Aware Therapists
1. TL;DR
2. Problem & Motivation: The Gap in Affective Computing
3. Methodology: The Core Architecture
3.1. 1. Multi-Modal Sensing
3.2. 2. The Neural Network "Brain"
3.3. 3. Actuation: The "Happy Hour" Change
4. Experiments & Results
5. Critical Insight & Conclusion