Memento: Transforming Wearables into Emotional Biographers via EEG
Memento: An Emotion Driven Lifelogging System with Wearables
2017-07-01
Summary
Problem
Method
Results
Takeaways
Abstract
Memento is an emotion-driven lifelogging system that integrates EEG sensors with smart glasses to automatically capture memorial moments. By leveraging Brain-Computer Interface (BCI) techniques, it triggers video recording based on the user's emotional arousal and achieves SOTA-level emotion tagging on wearable devices.
## TL;DR
Memento is a pioneering lifelogging system that uses EEG (brainwave) sensors embedded in smart glasses to "sense" when you are having a memorable emotional experience. Instead of recording 24/7 or waiting for you to press a button, it automatically triggers a video log when it detects a significant change in your emotional state, effectively digitizing your life based on how you *feel* rather than just what you *see*.
## Background: The Limits of Passive Sensing
Lifelogging has long been caught between two extremes:
1. **Manual Logging**: High cognitive load; you often forget to record the most spontaneous moments.
2. **Continuous Passive Logging**: Massive data redundancy and rapid battery drain.
The authors of Memento realized that **memorable moments usually coincide with emotional changes**. By tapping into the user's internal mental state using Electroencephalography (EEG), they created a "proactive" system that logs only what matters.
## The Technical Challenge: Brainwaves in the Wild
Measuring EEG on a laboratory couch is easy; measuring it while a user is walking down a street wearing smart glasses is a nightmare. The authors identify two primary "noise" sources:
* **Physiological Noise**: Eye blinks and jaw clenching (EMG) overlap with brain signals.
* **Motion Artifacts**: As the user moves, the EEG electrodes drift on the scalp, causing signal spikes.
### Strategy 1: Robust Preprocessing
To combat this, Memento uses a specialized pipeline:
* **Band-pass Filtering**: Focusing on the 7-31 Hz range (Alpha and Beta bands) which are most indicative of mental states.
* **Kernel-based Correlation**: A time-domain approach to identify and "subtract" the specific electrical signature of eye blinks.
* **Sensor Fusion Quality Scoring**: It uses the built-in Accelerometer (IMU) to detect vigorous movement. If the signal is too noisy due to movement, the system intelligently drops the segment to save energy.

## Methodology: Two-Phase Recognition
The most brilliant aspect of Memento is its **Edge-Cloud Split**. Emotion recognition algorithms (like Support Vector Machines or Deep Neural Networks) are too heavy for a Google Glass battery.
1. **Phase 1 (On-Glass)**: The device calculates **Katz’s Fractal Dimension (FD)**. This is a lightweight mathematical measure of the "complexity" of the brainwave. A sudden spike in the FD trend signals an "Arousal" event, which triggers the camera.
2. **Phase 2 (Cloud)**: The raw features are uploaded to a private cloud. Here, a full classification is performed to tag the video with specific emotions (e.g., "Happy," "Surprised," "Sad").

## Performance and Insights
The system was validated using the **DEAP dataset** (a standard benchmark for emotion analysis).
* **Accuracy**: The average RMS error was approximately 2.76. While not perfect, it is highly effective at catching the *direction* of emotional change.
* **User Satisfaction**: In a 6-hour "wild" study, 83% of the moments users later identified as "memorable" were successfully captured by Memento's automatic trigger.
* **Energy Efficiency**: The sensing and processing overhead on Google Glass added less than 200mW, proving that "always-on" mental state monitoring is feasible on current-gen hardware.

## Critical Analysis & Conclusion
Memento successfully shifts the paradigm of lifelogging from "Environment-Aware" to "Self-Aware."
**Limitations**:
* **Hardware Integration**: Currently uses a separate Muse headband and Google Glass; true commercial success requires integrated electrodes within the glass frames.
* **Social Privacy**: Automatic recording still faces social hurdles, though the "emotional trigger" makes it more purposeful than "always-on" cameras.
**Future Outlook**:
As wearable SoCs (System-on-Chips) integrate more **Low Power Units (LPUs)**, we can expect Memento's logic to move entirely on-device, potentially enabling real-time emotional feedback for mental health or performance coaching.
