Environmentally Embedded IoT: Tracking Student Engagement Without the Friction of Wearables

TL;DR

Researchers at Azusa Pacific University have developed a system that turns classroom furniture into a smart grid. By embedding sensors like IMUs, force resistors, and thermal cameras directly into desks and chairs, the system estimates student attention in real-time. This data is fed back to the instructor via a "traffic light" color-coded dashboard on tablets and haptic alerts on smartwatches, allowing for immediate pedagogical adjustments.

The Problem: The "Hawthorne Effect" of Wearables

In the quest to measure student engagement, many researchers have turned to wearables—smartwatches, EEG caps, or eye-tracking glasses. While accurate, these methods often fail in real-world educational settings.

• Obtrusiveness: Students feel "watched," which alters their natural behavior.
• Compliance: Ensuring every student wears and charges a device is a logistical nightmare.
• Privacy: Physical devices collecting biometric data often raise more red flags than environmental sensors.

The authors argue that the classroom itself should be the sensor, moving the technology into the background of the learning experience.

Methodology: The "Smart" Seat & Desk

The system architecture relies on the ESP32—a low-cost, dual-core WiFi module chosen for its rapid boot times and stable execution compared to complex Linux-based systems like the Raspberry Pi.

The Sensor Suite

1. Posture & Movement (IMU): Using the MPU-6050 to track the tilt and vibration of chairs.
2. Weight Distribution (Force Sensors): Strain gauges and FSRs detect if a student is leaning forward (engaged) or slouching (disengaged).
3. Physiological Proxies (Thermal Camera): The AMG8833 8x8 sensor tracks body temperature fluctuations. Interestingly, body temperature often rises during high engagement and drops during boredom.
4. Facial Analysis (ESP32-CAM): Capturing expressions and breath rates to provide high-fidelity engagement data.

Figure 1: Tablet interface displaying individual student attention levels via a color-coded grid.

Turning Data into Instruction: The Power Mean

A critical challenge in this research is how to present data to a busy teacher without overwhelming them. The authors utilize a Power Mean (Generalized Mean) formula to calculate aggregate class inattention:

$M_p={\left(\frac{1}{n}{\sum }_{i=1}^n{x}_i^p\right)}^{\frac{1}{p}}$

By adjusting the parameter $p$, the system can be "weighted" to be more sensitive to a few struggling students, ensuring the instructor is alerted even if only a minority of the class is significantly disengaged.

Feedback Loops

• The Tablet: Placed on the podium, it shows a "heat map" of the classroom.
• The Smartwatch: Provides a vibrating alert if the class inattention score crosses a programmable threshold (e.g., 70%), prompting the teacher to change the pace or introduce an activity.

Figure 2: The instructor's smartwatch view, showing the aggregate inattention value.

Experiments and Progress

While the hardware phase is complete, the project is currently in the Machine Learning training phase. The researchers are collecting physiological data and labeling it against synchronized video recordings to build a robust model that can accurately "predict" attention based purely on chair-tilt, weight shift, and thermal signatures.

Critical Insight: Balancing Privacy and Insight

The brilliance of this approach lies in its use of low-resolution sensors. By using an 8x8 thermal array (AMG8833), the system can detect "leaning" and "temperature" without ever capturing a high-resolution facial image that could be used for invasive surveillance. This "privacy-by-design" makes the system far more likely to be adopted in sensitive secondary education environments.

Conclusion

This work moves us closer to the "Sensitive Classroom"—an environment that listens to its occupants' physiological needs. While still in its labeling phase, the infrastructure provides a blueprint for how IoT can support instructors as a tool for assessment, rather than a distraction for students.

Future Work: The team plans to investigate how real-time weight distribution correlates with muscle fatigue and long-term student performance metrics.