IoT-Enabled Adaptive Gesture Recognition with Dashboard Visualization for Smart Environment Control

DOI

https://www.doi.org/10.59256/ijrtmr.20260602017

Published In

Volume 6, Issue 2 (2026)

Published Online

March-April 2026

Pages

106-111

Article Type

Original Article

Check for updates

Touchscreens and physical remotes work fine—until they don't. In a hospital prep room, a wheelchair user's kitchen, or anywhere shared surfaces carry infection risk, they become an actual problem rather than a minor inconvenience. This work describes a gesture-controlled IoT system we put together using an ESP32-CAM as the capture device, a MediaPipe-plus-CNN-LSTM pipeline for recognition, and MQTT for getting commands to devices. The camera is cheap, wireless, and small enough to tuck into a corner without anyone noticing it. Recognised gestures reach lights, HVAC controls, or media systems in roughly 174 ms. A React.js dashboard on a local server keeps track of everything—what the camera currently sees, what commands went out, device states, usage patterns over time. There is also a native Android app built in Android Studio that shows the same information on a phone and sends Firebase push notifications when something worth knowing about happens. We ran the whole system continuously for thirty days in a test room fitted with real smart devices. Accuracy across ten gesture types came out at 94.7%. The two gesture classes that kept underperforming during the first week had, by day 30, closed most of the gap on their own—the background retraining loop gathered new samples for those classes only and quietly updated the weights, never touching the gestures that were already working well.