Earthsensum: Redefining Environmental Communication Through the Senses of Smell and Taste

TL;DR

Environmental health issues often feel abstract or "invisible." The Earthsensum project breaks this barrier by integrating the chemical senses—smell and taste—into a multisensory HCI framework. Using Mobile VR and AR, the researchers moved beyond basic visualization to create a symbolic digital language for chemical experiences, successfully increasing user awareness and driving pro-environmental motivation.

Background: Tuning into the Chemical Frequency

While the "Internet of Senses" is an emerging trend, atmospheric and environmental data are still largely presented through graphs and maps. However, humans encounter the environment most intimately through breathing and ingestion. Pollutants enter our bodies triggering health issues long before they are visible. The Earthsensum project posits that if we can digitally represent these "chemical sense" experiences, we can foster a deeper, more instinctive understanding of environmental health.

The Core Problem: The Representation Vacuum

Existing Geographic Information Systems (GIS) and dashboards are "chemically silent." The challenge is two-fold:

1. Technological: Digital systems lack standardized ways to "display" smells or tastes.
2. Cognitive: Sensory perception is subjective. How do you communicate the "smell of ammonia" to a user in a way that is scientifically relevant and emotionally resonant?

Methodology: Building a Symbolic Sensory Map

The researchers developed a two-phase design process to bridge the "subjectivity gap."

Phase 1: Cross-Sensory Analogies

Before coding the apps, the team conducted experiments to see if people shared a common "language" for smells like Solid Waste (Ammonia) or Pulp Industry emissions (Hydrogen Sulfide).

• Haptic Mapping: Participants associated stimuli with physical objects (e.g., sharp vs. round).
• Graphic Mapping: Participants chose colors and geometric shapes to represent their sensations.

Figure 1: The paper prototype used to map chemical stimuli to visual and haptic attributes.

Phase 2: Digital Prototypes (MVR & MAR)

• Mobile Virtual Reality (MVR): Created "Virtual Tours" where users "visited" remote pollution sites. While in the headset, they were prompted to smell/taste actual samples, then learned about the molecular impact of that location.
• Mobile Augmented Reality (MAR): Acted as a "social sensor." Users could label their current location with sensory symbols (e.g., a "rough, purple triangle" for a harsh chemical smell), effectively crowdsourcing a multisensory map of the city.

Figure 2: UI/UX design for the MVR application focusing on "Call for Action" and molecular signatures.

Experiments and Results

The study utilized a specific correspondence chart (Table 1) linking real-world locations (like waste treatment stations) to specific chemical stimuli.

Key Findings:

• Symbolic Consistency: Most participants associated Ammonia with a "Trigger Ball" (pointed shape) and "Rough" texture, while Dimethyl Sulfide (sea smell) was linked to "Round" shapes and "Soft" textures.
• Educational Impact: 100% of the taste group and 87.5% of the smell group recognized that the system effectively enabled them to express their sensory experiences.
• Behavioral Shift: 87.5% of total participants reported that the experience inspired them to take positive action toward environmental health.

Figure 3: Psychometric results showing valence and arousal responses to different stimuli.

Critical Insight: Why This Works

The project succeeds because it targets the limbic system. Smell and taste neural signals are sent to brain areas involved in emotions and memory. By linking a "bad" smell stimulus with a VR tour of a pulp factory, the information is no longer just "data"—it becomes a lived memory.

Future Outlook and Limitations

While the study provides a robust framework, the current prototypes are "Lo-Fi." The researchers note that true integration—where digital devices can actually synthesize these scents on demand—is the next frontier. Furthermore, testing in real-world "smellscapes" (rather than a lab) is essential to validate these findings against ambient noise and environmental variables.

Conclusion

Earthsensum demonstrates that the future of Sustainable HCI lies in Multisensory Literacy. By giving a digital "voice" to our noses and tongues, we can transform environmental health from a distant scientific topic into an immediate, personal reality.