Overview
Emotion Aura is a real-time computer vision app that detects your facial emotions and hand gestures through the webcam, then visualizes them as an interactive GPU particle system. MediaPipe analyzes the webcam feed to extract face landmarks and blendshapes, a FACS-based classifier maps them to 5 emotions, and a custom Three.js particle system with GLSL shaders renders a glowing aura around your face. Hand gestures create force fields — open palm pushes particles away, closed fist attracts them. Everything runs client-side, no backend required.
Why I Built This
Always wanted to try some computer vision stuff — emotion detection, hand movement tracking. This was a fun side project to explore those ideas.
Features
- Emotion Detection — 5 emotions classified from 52 facial blendshapes using FACS-based weighted rules, with EMA temporal smoothing for fluid transitions
- GPU Particle System — 5,000 particles with custom GLSL neon spark shaders, per-emotion color palettes and movement profiles, face-anchored spawning
- Dual-Hand Gesture Interaction — Simultaneous 2-hand tracking — open palm creates repulsive force field, closed fist attracts particles, with visual aura feedback
- Blink-Triggered Fireworks — Blink your left or right eye to launch a burst of particle fireworks on the corresponding side of the screen
- Staggered Inference — Face and hand detection alternate across frames — each ML model at 30fps while rendering stays at 60fps
- Zero Backend — Everything runs client-side in the browser — ML models loaded from CDN, no server needed