SteadyPT

About this project

SteadyPT is an AI physical-therapy coach that runs on nothing but a webcam. The problem: patients do 80%+ of their PT alone at home, where adherence and form fall apart, and clinicians get no objective data on what actually happened between visits, just "yeah, I did my exercises." SteadyPT closes that loop. How it works: MediaPipe pose tracking watches a patient do their prescribed exercise. But we didn't hard-code one movement like most "AI fitness" demos. Instead, a PT pastes their written exercise documentation or uploads a prescription PDF, and Gemini Flash compiles it once into a structured tracker spec: target joint, range of motion, tempo, and rep/set goals. A new exercise is supported in seconds, with no retraining and no LLM anywhere near the real-time loop. During a set, a fast, deterministic rule-based engine counts reps and coaches form, speaking cues aloud via ElevenLabs text-to-speech. A live web dashboard streams a skeleton overlay, a ticking rep counter, a depth gauge, and real-time form flags over a FastAPI WebSocket at 30 Hz. At the end of each set, Gemini reasons over the full rep-by-rep data and speaks a debrief plus a concrete next-set adjustment. The patient drives the entire app hands-free by voice. A "Hey coach" wake word and Web Speech API let them start sets, ask questions, and move through check-in, live tracking, and debrief without ever touching the laptop, which matters when you're mid-squat across the room from the camera. Every session is persisted to BigQuery for cross-session trends, and exported as a real FHIR Observation, an interoperable clinical record that drops straight into a clinician's workflow via a shareable handoff page. The architecture is the differentiator: the LLM compiles the prescription up front and reasons between sets, while a deterministic engine handles every real-time frame. That keeps coaching fast and reliable while still being infinitely flexible. New exercise in seconds, objective range-of-motion an

Gallery