Interactive Web View for DEXTRA System System Showcase Active Python 3.10 React 18 MediaPipe 0.10

Control Without Contact
Touchless Computing Reimagined

Interactive Web Showcase Notice

This page is an interactive website showcasing the entire DEXTRA system architecture, 21-landmark poses, and voice command engine — it is a web-based presentation view, not the desktop backend production daemon.

GitHub Repository Pose Simulator 22 Gestures
22
Hand Gestures
40+
Voice Commands
30+
FPS Tracking Speed
<350ms
ASR Execution Latency

21-Landmark Hand Pose Simulator

Real-time MediaPipe skeletal joint visualization & PyAutoGUI gesture heuristic decoder

MediaPipe 21 Landmarks 30.2 FPS

Click a pose preset on the right to animate 21 joint nodes and view live telemetry math.

Select Hand Pose

Live Heuristic Telemetry

Active Pose: Index Move
Detected Action: Move Cursor (x: 480, y: 320)
Finger State [T, I, M, R, P]: [0, 1, 0, 0, 0]
Euclidean Distance (L8 to L4): 84.2 px
Click Threshold: 30.0 px
PyAutoGUI Signal: pyautogui.moveTo(1152, 720)

22 Hand Gesture Catalog

Comprehensive touchless input taxonomy for seamless desktop navigation

Voice Command Engine & Speech Simulator

Powered by local Hugging Face Transformer Whisper-tiny with 'DEXTRA' Wakeword Monitor

Speech Simulation Console

Click any command chip below to simulate background microphone voice activation and view decoded STT transcription.

Listening for 'DEXTRA'...

Popular Spoken Commands

ASR Decoder & Execution Output

[0.00s] Initializing Whisper-tiny STT Pipeline...
[0.12s] Microphone sounddevice calibrated (RMS Ambient: 0.042)
[0.45s] Wakeword Monitor Active. Spoken keyword trigger: "DEXTRA"
-- Ready for Spoken Input --

Multi-Threaded System Architecture

Concurrent daemon threads for high-frame-rate tracking and instantaneous voice execution

1

OpenCV Webcam Stream

Captures 30+ FPS BGR frames from internal camera index 0 or USB external mic/webcam.

python/camera.py
2

MediaPipe 21 Landmarks

Extracts 21 3D coordinates & calculates Euclidean distances for finger posture heuristics.

python/gesture_engine.py
3

EMA Pointer Smoothing

Applies exponential moving average buffer to eliminate jitter before PyAutoGUI dispatch.

python/gesture_mapping.py
4

Whisper STT Engine

Background sounddevice listener triggering local Hugging Face Whisper model upon 'DEXTRA' wakeword.

python/voice_engine.py

System Configuration & Pointer Physics

Live interactive parameter tuning mirroring dextra_settings.json schema

Threshold & Timing Sliders

Cursor Smoothing Buffer 5 frames
Click Distance Threshold 30 px
Drag Hold Duration 0.60 s
Double Click Time Window 0.35 s
Cooldown Period 0.30 s

Pointer Physics Simulation

Move your cursor inside the container below to experience how EMA smoothing softens jitter in real-time.

Hover & move mouse inside container 
{
  "camera_index": 0,
  "cursor_smoothing": 5,
  "click_threshold_px": 30,
  "drag_hold_time_sec": 0.6,
  "double_click_time_sec": 0.35,
  "cooldown_period_sec": 0.3,
  "wakeword": "DEXTRA"
}

Project Metrics & QA Bug Tracker

Derived live from DEXTRA_Project_Tracker.xlsx management sheet

Harihar R

@HARIHAR1406
System Ideator

Conceived initial system concept, architectural vision & gesture-driven paradigm.

View GitHub Profile →

Mohamed Salman P

@Salman4256
Core Gesture Lead

MediaPipe 21 Landmarks, OpenCV Camera Pipeline, PyAutoGUI Mouse Events.

View GitHub Profile →

Logesh M

@logeshM1
Voice Engine Lead

Sounddevice Mic Engine, Hugging Face Whisper ASR, Shortcut Key Mapper.

View GitHub Profile →

Muthamil V

@muthamil13
FastAPI Backend Lead

FastAPI REST Endpoints, WebSockets Image Streamer, Thread Safety Locks.

View GitHub Profile →

Godfrey T R

@TheOrionGD
React SPA Lead

React SPA Settings GUI, HTML5 Canvas Trainer Streamer, Design System.

View GitHub Profile →
Bug ID Target Module Severity Description Owner Status Resolution Details
B01 python/camera.py Medium VideoCapture camera locking on server restart causing frame capture freeze. Salman Closed Added try-finally cleanup blocks and explicit camera.release() calls.
B02 python/gui_server.py High FastAPI WebSocket stream disconnect error on /trainer/stream client closure. Muthamil Closed Handled WebSocketDisconnect gracefully & added client connection manager.
B03 python/gesture_engine.py Medium MediaPipe clearcut telemetry upload connection errors (Status_ConnectFailed: 12029). Salman Closed Disabled Clearcut telemetry background uploads via environment configuration.
B04 python/gesture_mapping.py High Mouse cursor jitter and accidental double-clicks during fast movement. Salman Closed Implemented EMA smoothing filter (alpha=0.35) and click cooldown timer.
B05 python/voice_wakeword.py Medium High CPU utilization during continuous audio stream buffer recording. Logesh Closed Optimized sounddevice blocksize to 1024 samples with ring buffer queue.
B06 Git Repository Low Default tracking branch misalignment on GitHub remote. Godfrey Closed Updated local remote HEAD to dextra branch and pruned obsolete remotes.