PROJECT: HOUSE Production Edge AI Platform

Autonomous Edge Face Recognition & Access Control System

Zero-latency biometric authentication, multi-zone spatial computer vision, and IoT hardware orchestration built with C++, CUDA, and distributed microservices.

Executive Summary

Designed and deployed a distributed edge AI security platform that automates biometric access control with sub-second response times. The system continuously ingests high-resolution RTSP camera feeds, performs real-time neural face detection and 512-D embedding extraction via GPU acceleration, verifies identities using temporal hysteresis filtering, triggers smart deadbolts via authenticated cloud APIs, and provides real-time natural language audio confirmation through reverse-engineered two-way camera speaker protocols.

Languages
C++20 Python 3.11 C# / ASP.NET Core SQL Bash
Vision & AI
OpenCV TensorRT NVIDIA CUDA InsightFace / ArcFace ONNX Runtime
Protocols & IPC
RTSP WebRTC GStreamer ZeroMQ WebSockets Baichuan TCP
Hardware & IoT
August Smart Lock API Reolink Dual-Lens IP Cameras NVIDIA GPUs Edge Compute Servers
Infrastructure & Storage
Docker Microservices MySQL 8.0 Google Drive Cloud Sync Systemd Daemons
Live System Telemetry & HUD Playback 1280×480 Panoramic HUD
REAL-TIME EDGE INFERENCE
Live Walkup Demonstration Sequence
Click any timestamp to jump video directly to that event

0:01 – Detection: Subject enters spatial zone; bounding boxes and spatial trackers (Human [Porch]) engage on the real-time telemetry HUD.

0:09 – Verification: Facial recognition confirms identity (confidence ≥ 68%) with 4-frame temporal streak verification.

0:11 – Hardware Actuation: System issues encrypted unlock payload to August deadbolt; verifies physical deadbolt retraction (LockStatus.UNLOCKED).

0:13 – Audio Feedback: Two-way talk channel activates; outdoor camera speaker announces: “Hello Michael. System unlocked.”

0:16 – Entry & Re-lock: Subject enters, door closes, and fallback 300-second auto-lock countdown begins.

Key Engineering Pillars

Core microservice subsystems powering autonomous edge security and biometric orchestration

PILLAR 1

Real-Time Computer Vision & Edge Acceleration

  • Zero-Copy Shared Memory (POSIX IPC): Ultra-low-latency inter-process frame sharing (/dev/shm) between RTSP frame capture, inference engines, and storage daemons without CPU copy bottlenecks.
  • Deep Neural Biometrics: Dual-stage face detection and high-dimensional (512-D) cosine similarity embedding matching running on NVIDIA TensorRT / CUDA.
  • Temporal Streak & Hysteresis Filtering: Multi-frame confirmation streak algorithms prevent false positives and spoofing while tolerating natural head angles and dynamic outdoor lighting.
PILLAR 2

Access Control & Zero-Trust Architecture

  • Cryptographically Signed IPC: Inter-service communication across Unix domain sockets verified using HMAC-SHA256 signatures to prevent unauthorized privilege escalation.
  • Verified Unlock Sequencing: Asynchronous deadlock-safe handshake with August Smart Lock API—requires verified LockStatus.UNLOCKED state confirmation before triggering access signals.
  • Fail-Safe Fallbacks: Automated auto-relock timers, token auto-refresh daemon, and tamper/dwell alarms guarding against mechanical stall.
PILLAR 3

Full-Duplex Two-Way Audio & Edge TTS

  • Hardware Talk Protocol Engine: Reverse-engineered binary talk protocols (Reolink Baichuan) delivering raw ADPCM audio packets directly to outdoor IP camera speakers over TCP.
  • Dynamic Neural Speech Synthesis: Pre-cached binary speech buffer (~0.2ms latency) that dynamically strips internal database IDs to speak personalized greetings (“Hello Michael. System unlocked.”).
  • Synchronized Audio Archival: Live porch microphone stream captured in universal 48 kHz stereo AAC alongside GPU-encoded video.
PILLAR 4

Multi-Zone Spatial Intelligence ("Package Guard")

  • Ray-Casting Polygon Geofencing: Real-time spatial tracking distinguishing between public sidewalks, approach yards, and private porch zones.
  • Dwell & Theft Heuristics: Automated package arrival/departure detection with candidate incident clip staging, auto-discarding known household members.
  • Dual-Lens Field Staging: Sub-pixel rectification stitching panoramic optical streams for continuous approach trajectory modeling.
End-to-End Edge Execution Pipeline
Dual RTSP Capture
Reolink IP Streams
→
POSIX /dev/shm
Zero-Copy Ring Buffers
→
TensorRT / CUDA
512-D ArcFace Embeddings
→
HMAC-SHA256 IPC
Temporal Streak Verification
→
August API & TCP Audio
Stateful Deadbolt & TTS