e6685203fe
- Added REPO_INVENTORY.md with all repos, branches, remotes, and staging info - Added .gitignore - Synced all existing docs from local workspace - Centralized documentation hub for GrowQR team
6.8 KiB
6.8 KiB
| GrowQR Roleplay Service — One-Pager Avatar-Based AI + Real-Time Video/Voice + Screen Share | v3 | ROLEPLAY | | :---- | :---: |
| PURPOSE |
|---|
| Avatar-based AI roleplay with real-time video/audio/screen interaction. AI personas speak using voice synthesis. Users respond via voice/text and can share screen (specific window) for coding/presentation demos. System analyzes voice (including filler word count), facial expressions, body language, technical setup quality (lighting, framing), and screen content to provide comprehensive performance feedback and certificates. |
| CORE FEATURES | |
|---|---|
| 1 | Avatar-Based AI Personas: Visual characters with voice synthesis, lip-sync, emotional expressions |
| 2 | Real-Time Voice Synthesis: AI persona SPEAKS all questions and responses (not just text bubbles) |
| 3 | Video/Audio Recording: Capture user's camera and microphone throughout entire session |
| 4 | Screen Share: User shares specific window (IDE, slides, demo app) - NOT full desktop |
| 5 | AI Screen Content Analysis: Analyzes code quality (syntax, logic), presentation slides, demos |
| 6 | Comprehensive Analysis: Voice (tone, pace, filler word count), Facial (engagement, eye contact), Body (posture, gestures), Technical Quality (lighting, framing, background), Screen content |
| 7 | Performance Scoring: Voice 35%, Facial 25%, Body Language 20%, Technical Quality 10%, Content 10% |
| USER JOURNEY | |||
|---|---|---|---|
| Step | User Action | System Response | Data Captured |
| 1 | User selects scenario | Load AI persona avatar, request camera/mic/screen permissions | Scenario ID, User ID |
| 2 | Permissions granted | Initialize avatar rendering, voice synthesis, recording systems | Media access granted |
| 3 | Session starts | Deduct 1 credit, start all recording (video/audio/screen), avatar speaks first question | Session ID, Start time, Recording streams |
| 4 | User shares screen (optional) | Capture specific window, begin screen content analysis | Screen recording, Content type detected |
| 5 | User responds (voice/text) | Capture response, AI analyzes voice/facial/body/technical/screen in real-time or queued | Transcript, Voice metrics (including filler count), Facial timeline, Body timeline, Technical quality (lighting/framing), Screen analysis |
| 6 | AI persona responds | Voice synthesis speaks follow-up, avatar lip-syncs | Conversation flow |
| 7 | Session completes | Stop all recordings, finalize analysis, calculate scores across all metrics | Overall score, Voice 88%, Facial 75%, Body 82%, Technical 90%, Screen 85% |
| KEY STATES TO HANDLE | ||
|---|---|---|
| State | Condition | System Behavior |
| Permission Denied | User blocks camera/mic/screen | Show error: 'Camera, microphone required. Screen share optional but recommended.' |
| Media Loading | Avatar/voice/recording initializing | Show: 'Preparing your session...' with progress (5-10s) |
| Connection Unstable | Poor network, dropped frames/audio | Degrade quality or switch to text-only mode, continue session |
| Voice Synthesis Error | TTS service fails | Fallback: Display text bubbles instead of speaking, avatar remains visible |
| Screen Share Stopped | User stops sharing mid-session | Continue session without screen analysis, note in results |
| Active Session | Roleplay in progress | Avatar speaks, user responds, all recording/analysis systems active |
| Analysis Complete | Post-session processing done | Display all scores: Voice, Facial, Body Language, Technical Quality, Screen Content (if shared), Overall |
| ANALYSIS METRICS | |
|---|---|
| Metric | What's Analyzed |
| Voice (35%) | Tone (friendly/professional), Pace (WPM), Clarity (articulation), Confidence (vocal strength), Filler words COUNT (um, uh, like, you know) |
| Facial Expression (25%) | Smile frequency, Eye contact (camera gaze), Emotional state (neutral/happy/stressed), Engagement level |
| Body Language (20%) | Posture (upright/slouched), Gestures (open/closed), Fidgeting, Professionalism rating |
| Technical Quality (10%) | Lighting: Well-lit/Acceptable/Poor |
| Content Quality (10%) | Relevance, Completeness, Accuracy of responses |
| Screen Content (if shared) | Code: Syntax, structure, logic, best practices |
| Overall Performance | Weighted combination of all metrics with historical comparison to track improvement |
| DATA SHARING WITH OTHER SERVICES | ||
|---|---|---|
| Target Service | Data Shared | Purpose |
| Interview Service | Voice/facial/body/technical/screen scores, weak communication areas, filler word patterns | Tailor interview questions to address roleplay gaps |
| Assessment Service | Topic performance, behavioral competencies, communication patterns, technical setup quality | Recommend targeted assessments for weak areas |
| Course Service | Skill gaps (communication, presentation, technical), low metric scores, setup quality issues | Recommend soft skills, communication, technical courses, setup guidance |
| Dashboard Service | Session count, certificates earned, performance scores (all metrics), credits remaining | Update Q-Score based on performance, show progress achievements, suggest improvements |
| ACCEPTANCE CRITERIA | ||
|---|---|---|
| # | Must Pass | Pass/Fail |
| 1 | Avatar renders correctly. Voice synthesis works with lip-sync within 100ms. Fallback to text if voice fails. | [ ] PASS [ ] FAIL |
| 2 | User camera/microphone captured throughout session. All recordings stored in cloud. Download links work. | [ ] PASS [ ] FAIL |
| 3 | User can share specific window. Screen recorded alongside video/audio. User can stop sharing anytime. | [ ] PASS [ ] FAIL |
| 4 | AI analyzes voice including filler word COUNT (um, uh, like, you know). Counts displayed in feedback. | [ ] PASS [ ] FAIL |
| 5 | AI analyzes technical quality: Lighting (well-lit/acceptable/poor), Framing (centered/off-center), Background (professional/distracting), Audio quality (clear/noisy). | [ ] PASS [ ] FAIL |
| 6 | AI analyzes screen content: Code quality (syntax, structure, logic) for IDEs, Slide quality (design, clarity) for presentations, Demo completeness for apps. | [ ] PASS [ ] FAIL |
| 7 | All metrics calculated correctly (Voice, Facial, Body, Technical Quality, Screen Content). Overall weighted score accurate. Certificate generated for 70%+ on eligible scenarios. | [ ] PASS [ ] FAIL |