2026 · AR product

GrokEye

Hands-free Grok-augmented reality that watches through the camera, speaks the fix, and points at exactly where things go.

Problem

Physical work is hands-busy. Looking down at a phone to follow a manual breaks flow, and a generic chatbot cannot point at the part in front of you.

Approach

GrokEye looks through a webcam or catalog clip, routes voice to Grok, and draws boxes plus connection arrows on the frame while Carina speaks the next move. Web how-tos become step JSON; “check my work” compares before and after frames. A local tracker keeps overlays glued as the video plays.

Outcome

Built for Grokathon 2026. Demoed live espresso portafilter coaching and IKEA assembly overlays—voice in, overlays on the part, spoken verdict on the step.

Stack

React · Vite · Grok 4.5 · xAI TTS · Web Speech API