# GrowQR Interview Service - Server/Client Responsibility Split

**Version:** 1.0  
**Date:** March 2, 2026  
**Purpose:** Define clear boundaries between server and client implementations

---

## 🎯 Quick Summary

| Aspect | Server | Client |
|--------|--------|--------|
| **Primary Role** | Orchestrator & Intelligence | Media Processor & User Interface |
| **Heavy Lifting** | AI logic, business rules, data persistence | Recording, encoding, speech processing |
| **Data Flow** | Text-based (transcripts, questions, metadata) | Media-based (video, audio, screen capture) |
| **Technology** | Python + FastAPI | JavaScript/TypeScript + Browser APIs |
| **Scalability** | Horizontal scaling via stateless design | Scales with user devices |

---

## 📦 SERVER-SIDE: What Goes on Server

### Core Principle
**Server = Stateless orchestrator that handles AI, business logic, auth, and data persistence**

### Server Responsibilities Overview

```
┌─────────────────────────────────────────────────────────────┐
│                      SERVER SIDE                             │
│                                                              │
│  ┌──────────────────────────────────────────────────────┐  │
│  │  1. AUTHENTICATION & AUTHORIZATION                   │  │
│  │     • JWT token validation                           │  │
│  │     • User identity verification                     │  │
│  │     • Permission checks                              │  │
│  └──────────────────────────────────────────────────────┘  │
│                          ↓                                   │
│  ┌──────────────────────────────────────────────────────┐  │
│  │  2. BUSINESS LOGIC & ORCHESTRATION                   │  │
│  │     • Session state machine                          │  │
│  │     • Interview flow control                         │  │
│  │     • Credits/entitlements management                │  │
│  │     • Timer and progress tracking                    │  │
│  └──────────────────────────────────────────────────────┘  │
│                          ↓                                   │
│  ┌──────────────────────────────────────────────────────┐  │
│  │  3. AI LOGIC (LLM Integration)                       │  │
│  │     • Question generation                            │  │
│  │     • Follow-up question logic                       │  │
│  │     • Transcript analysis                            │  │
│  │     • Narrative feedback generation                  │  │
│  └──────────────────────────────────────────────────────┘  │
│                          ↓                                   │
│  ┌──────────────────────────────────────────────────────┐  │
│  │  4. STORAGE BROKER                                   │  │
│  │     • Pre-signed URL generation                      │  │
│  │     • Artifact metadata registry                     │  │
│  │     • Retention policy enforcement                   │  │
│  └──────────────────────────────────────────────────────┘  │
│                          ↓                                   │
│  ┌──────────────────────────────────────────────────────┐  │
│  │  5. POST-PROCESSING & ANALYSIS                       │  │
│  │     • Scoring algorithms (7 metrics)                 │  │
│  │     • Report generation                              │  │
│  │     • Historical trends calculation                  │  │
│  │     • Event emission (integrations)                  │  │
│  └──────────────────────────────────────────────────────┘  │
└──────────────────────────────────────────────────────────────┘
```

---

## 🖥️ SERVER MODULES (Detailed Breakdown)

### Module 1: Authentication & Authorization

**Purpose:** Verify user identity and permissions

**Responsibilities:**
- ✅ Validate JWT tokens (signature, expiry, claims)
- ✅ Extract user_id from token
- ✅ Check user permissions (org visibility, feature access)
- ✅ Rate limiting per user

**Technology Stack:**
- JWT library (PyJWT)
- Middleware pattern (FastAPI dependencies)
- Redis for rate limiting

**Inputs:**
- JWT token (from Authorization header)

**Outputs:**
- Validated User object
- HTTP 401 if invalid
- HTTP 429 if rate limited

**TODO:**
- Implement JWT validation middleware
- Define token claims structure
- Setup rate limiting rules

---

### Module 2: Credits & Entitlements

**Purpose:** Manage user credits and interview quotas

**Responsibilities:**
- ✅ Check available credits before starting interview
- ✅ Deduct credits atomically (prevent double-spend)
- ✅ Handle different interview types (different credit costs)
- ✅ Support refunds (if interview fails)

**Technology Stack:**
- PostgreSQL transactions
- Database-level locks
- Optional: External credits service API

**Database Schema:**
```
user_credits:
  - user_id
  - credits_balance
  - last_updated

credit_transactions:
  - id
  - user_id
  - session_id
  - amount (positive or negative)
  - transaction_type (deduct, refund, purchase)
  - created_at
```

**Inputs:**
- User ID
- Interview type

**Outputs:**
- Success/failure
- Remaining credits

**TODO:**
- Implement atomic deduction logic
- Define credit costs per interview type
- Create refund workflow

---

### Module 3: Session State Machine

**Purpose:** Manage interview session lifecycle

**Responsibilities:**
- ✅ Create new sessions
- ✅ Track session state transitions
- ✅ Enforce state rules (can't submit turn if not started)
- ✅ Handle timeouts and abandonment

**States:**
```
configured → in_progress → completed → processing → ready
                    ↓
                abandoned/timeout
```

**Technology Stack:**
- PostgreSQL for persistence
- Redis for active session cache
- FSM (Finite State Machine) pattern

**Database Schema:**
```
interview_sessions:
  - id (UUID)
  - user_id
  - interview_type
  - status (enum: configured, in_progress, completed, processing, ready, abandoned)
  - started_at
  - completed_at
  - created_at
  - metadata (JSONB)
```

**Inputs:**
- Session ID
- State transition request

**Outputs:**
- Updated session state
- Validation errors

**TODO:**
- Implement state machine logic
- Define transition rules
- Create timeout handling (background job)

---

### Module 4: Question Generation (AI)

**Purpose:** Generate personalized interview questions using LLM

**Responsibilities:**
- ✅ Generate 8-10 questions based on interview type
- ✅ Personalize based on user profile (role, experience, skills)
- ✅ Create avatar scripts (conversational versions)
- ✅ Mix difficulty levels (easy/medium/hard)
- ✅ Store questions in database

**Technology Stack:**
- Anthropic Claude 3.5 Sonnet OR OpenAI GPT-4
- LLM client library
- Prompt templates

**Prompt Strategy:**
```
Input:
- Interview type (technical, behavioral, coding, warmup)
- User profile (role, experience, skills, previous interviews)
- Programming language (for coding interviews)

Output (JSON):
[
  {
    "text": "Question for interviewer",
    "avatar_script": "Conversational version",
    "category": "technical|behavioral|problem_solving",
    "difficulty": "easy|medium|hard",
    "expected_duration": 120
  }
]
```

**Database Schema:**
```
interview_questions:
  - id (UUID)
  - session_id
  - question_order (int)
  - question_text
  - avatar_script
  - category
  - difficulty
  - expected_duration_seconds
  - is_followup (boolean)
  - parent_question_id (UUID, nullable)
```

**Inputs:**
- Interview type
- User profile
- Programming language (optional)

**Outputs:**
- List of questions (JSON)
- Stored in database

**TODO:**
- Create prompt templates
- Implement LLM API integration
- Define question validation rules
- Handle LLM failures (retry logic)

---

### Module 5: Follow-up Question Logic (AI)

**Purpose:** Decide whether to ask follow-up or proceed to next question

**Responsibilities:**
- ✅ Analyze user's answer transcript
- ✅ Determine if follow-up is needed (incomplete, interesting, or unclear answer)
- ✅ Generate contextual follow-up question
- ✅ Limit follow-ups (max 1-2 per main question)

**Technology Stack:**
- LLM (Claude/GPT-4)
- Context window management
- Prompt templates

**Decision Logic:**
```
Input:
- Previous question
- User's transcript
- Session context (time remaining, questions completed)

LLM Analysis:
1. Is the answer complete?
2. Did user mention something interesting worth exploring?
3. Is clarification needed?
4. Do we have time for follow-up?

Output:
- followup_needed: true/false
- followup_question: "..." (if needed)
```

**Inputs:**
- Session ID
- Previous question + answer transcript
- Time remaining

**Outputs:**
- Next question (follow-up or sequential)

**TODO:**
- Create follow-up analysis prompt
- Implement decision logic
- Set follow-up limits per session

---

### Module 6: Pre-signed URL Generation

**Purpose:** Generate secure, time-limited upload URLs for client

**Responsibilities:**
- ✅ Generate S3/R2 pre-signed URLs
- ✅ Scope URLs to session + chunk number
- ✅ Set expiry (15 minutes)
- ✅ Generate URLs in batches (5-10 at a time)
- ✅ Support URL refresh requests

**Technology Stack:**
- Boto3 (S3 SDK) or Cloudflare R2 SDK
- S3-compatible API (signature v4)

**URL Structure:**
```
https://storage.growqr.com/sessions/{session_id}/{media_type}/chunk_{n}.webm?
  X-Amz-Algorithm=AWS4-HMAC-SHA256&
  X-Amz-Credential=...&
  X-Amz-Date=...&
  X-Amz-Expires=900&
  X-Amz-Signature=...
```

**Security:**
- PUT-only permission
- Content-Type restricted (video/webm)
- Path scoped to session
- Short expiry (15 min)

**Inputs:**
- Session ID
- Chunk numbers
- Media type (video, screen, audio)

**Outputs:**
- Array of pre-signed URLs with metadata

**TODO:**
- Implement URL generation function
- Configure S3/R2 bucket permissions
- Create URL refresh endpoint

---

### Module 7: Turn Management

**Purpose:** Store and manage Q&A turns during interview

**Responsibilities:**
- ✅ Store transcript for each answer
- ✅ Record metadata (duration, chunks uploaded, confidence)
- ✅ Link turn to question
- ✅ Validate turn completeness

**Technology Stack:**
- PostgreSQL

**Database Schema:**
```
interview_turns:
  - id (UUID)
  - session_id
  - question_id
  - transcript (TEXT)
  - duration_seconds (INT)
  - chunks_uploaded (INT[])
  - metadata (JSONB) - STT confidence, language detected, etc.
  - created_at
```

**Inputs:**
- Session ID
- Question ID
- Transcript
- Duration
- Chunks uploaded

**Outputs:**
- Stored turn record
- Validation result

**TODO:**
- Implement turn storage endpoint
- Add validation rules
- Create turn query APIs

---

### Module 8: Media Artifact Registry

**Purpose:** Track uploaded media chunks and metadata

**Responsibilities:**
- ✅ Register chunk uploads (after client uploads to S3)
- ✅ Track upload completeness
- ✅ Store metadata (size, duration, storage path)
- ✅ Support retention policies

**Technology Stack:**
- PostgreSQL

**Database Schema:**
```
media_chunks:
  - id (UUID)
  - session_id
  - chunk_number
  - media_type (video, screen, audio)
  - storage_key (S3 path)
  - file_size_bytes
  - duration_seconds
  - uploaded_at
  - expires_at (based on retention policy)
```

**Inputs:**
- Session ID
- Chunk metadata (from client callback)

**Outputs:**
- Registered chunk record

**TODO:**
- Implement chunk registration endpoint
- Create retention policy job
- Setup deletion workflow

---

### Module 9: Scoring Engine

**Purpose:** Calculate 7 performance metrics from interview data

**Responsibilities:**
- ✅ Communication clarity (0-100)
- ✅ Technical accuracy (0-100)
- ✅ Problem-solving approach (0-100)
- ✅ Confidence (0-100)
- ✅ Professionalism (0-100)
- ✅ Engagement (0-100)
- ✅ Overall score (weighted average)

**Scoring Algorithms:**

**1. Communication Clarity:**
- Word count / duration = words per minute
- Filler word detection (um, uh, like, you know)
- Penalty for excessive fillers
- Score: 100 - (filler_count × 2)

**2. Technical Accuracy:**
- LLM analysis of answers vs expected concepts
- Keyword/concept matching
- Code correctness (if applicable)

**3. Problem-Solving:**
- Answer structure analysis
- Breaking down complex problems
- Logical flow

**4. Confidence:**
- Speech rate consistency
- Pause analysis
- Filler word ratio

**5. Professionalism:**
- Language formality
- No inappropriate content
- Proper terminology

**6. Engagement:**
- Answer completeness
- Enthusiasm markers
- Eye contact (if using video analysis)

**Technology Stack:**
- Python (NumPy for calculations)
- LLM for qualitative analysis
- Optional: Speech analysis libraries

**Inputs:**
- All turns (transcripts)
- Session metadata
- Optional: Media files for advanced analysis

**Outputs:**
- Scores object (JSON)
- Metric breakdowns

**TODO:**
- Implement deterministic scoring logic
- Integrate LLM for qualitative metrics
- Define scoring weights
- Create testing suite for score consistency

---

### Module 10: Report Generation (AI)

**Purpose:** Generate narrative feedback using LLM

**Responsibilities:**
- ✅ Analyze complete interview transcript
- ✅ Generate strengths (top 3 with examples)
- ✅ Generate improvements (top 3 with examples)
- ✅ Overall assessment (2-3 sentences)
- ✅ Actionable recommendations

**Technology Stack:**
- LLM (Claude/GPT-4)
- Structured output (JSON)
- Report templates

**Prompt Strategy:**
```
Input:
- Full interview transcript (all Q&A pairs)
- Quantitative scores
- User profile

Prompt:
"Analyze this interview performance. Provide:
1. Top 3 strengths (with specific examples from transcript)
2. Top 3 areas for improvement (with specific examples)
3. Overall assessment (2-3 sentences)
4. Actionable next steps

Be specific, constructive, and reference actual quotes."

Output (JSON):
{
  "strengths": ["...", "...", "..."],
  "improvements": ["...", "...", "..."],
  "overall_assessment": "...",
  "recommendations": ["...", "..."]
}
```

**Database Schema:**
```
interview_reports:
  - id (UUID)
  - session_id (unique)
  - scores (JSONB)
  - feedback (JSONB)
  - metrics (JSONB)
  - processing_duration_seconds
  - created_at
```

**Inputs:**
- Session ID
- All turns
- Scores

**Outputs:**
- Complete report (JSON)
- Stored in database

**TODO:**
- Create feedback prompt template
- Implement LLM integration
- Define report structure
- Add caching for identical transcripts

---

### Module 11: Historical Trends & Comparison

**Purpose:** Track user progress over time

**Responsibilities:**
- ✅ Retrieve previous interview scores
- ✅ Calculate improvement/decline trends
- ✅ Identify consistent strengths/weaknesses
- ✅ Generate progress charts data

**Technology Stack:**
- PostgreSQL aggregations
- Window functions (SQL)

**Calculations:**
```
- Average score change (last 5 interviews)
- Metric-specific trends (e.g., communication improving)
- Consistency analysis (score variance)
- Time between interviews (engagement)
```

**Inputs:**
- User ID
- Current session scores

**Outputs:**
- Trend data (JSON)
- Comparison with previous average

**TODO:**
- Implement trend calculation queries
- Define minimum interviews for trends (need 2+)
- Create visualization data format

---

### Module 12: Background Jobs (Celery Workers)

**Purpose:** Async processing for heavy tasks

**Responsibilities:**
- ✅ Post-interview analysis (fetch media, score, report)
- ✅ Retention policy enforcement (delete old media)
- ✅ Session timeout handling
- ✅ Abandoned session cleanup

**Technology Stack:**
- Celery (Python task queue)
- Redis (broker + result backend)
- Worker processes

**Jobs:**

**1. analyze_interview(session_id)**
- Fetch all turns
- Run scoring engine
- Generate LLM report
- Calculate trends
- Update session status → "ready"
- Emit events

**2. cleanup_expired_media()**
- Query expired chunks
- Delete from S3
- Delete from database

**3. handle_session_timeout(session_id)**
- Mark session as "abandoned"
- Partial refund (if applicable)
- Cleanup resources

**Inputs:**
- Job-specific (session_id, etc.)

**Outputs:**
- Task results
- Updated database records

**TODO:**
- Setup Celery configuration
- Implement worker tasks
- Define retry policies
- Setup monitoring (Flower)

---

### Module 13: Event Publishing & Integrations

**Purpose:** Share interview results with other services

**Responsibilities:**
- ✅ Emit events after interview completion
- ✅ Outbox pattern for reliability
- ✅ Integration with Dashboard, Courses, Assessment, Roleplay

**Technology Stack:**
- Outbox table pattern
- Message queue (RabbitMQ, Kafka, or SQS)
- Event schemas

**Event Schema:**
```json
{
  "event_type": "interview.completed",
  "event_version": "1.0",
  "timestamp": "2026-03-02T15:30:00Z",
  "user_id": "uuid",
  "session_id": "uuid",
  "payload": {
    "interview_type": "technical",
    "overall_score": 81,
    "scores": {...},
    "skill_gaps": [
      {
        "skill_id": "python-async",
        "proficiency": 65,
        "target": 80
      }
    ],
    "recommended_courses": ["course-id-1", "course-id-2"]
  }
}
```

**Database Schema:**
```
outbox_events:
  - id (UUID)
  - event_type
  - payload (JSONB)
  - published (boolean)
  - published_at
  - created_at
```

**Inputs:**
- Session results

**Outputs:**
- Published events

**TODO:**
- Define event schemas
- Implement outbox pattern
- Setup message queue
- Create consumer contracts

---

### Module 14: Compliance & Audit

**Purpose:** GDPR/CCPA compliance and security auditing

**Responsibilities:**
- ✅ Audit log for sensitive operations
- ✅ Data deletion workflow
- ✅ Retention policy enforcement
- ✅ Access logging

**Technology Stack:**
- PostgreSQL (audit log)
- S3 lifecycle policies
- Scheduled jobs

**Database Schema:**
```
audit_logs:
  - id (UUID)
  - user_id
  - action (viewed_report, deleted_session, etc.)
  - resource_id (session_id, etc.)
  - ip_address
  - user_agent
  - created_at
```

**Operations to Audit:**
- Session creation
- Report viewing
- Media access
- Session deletion
- Admin actions

**TODO:**
- Implement audit logging middleware
- Create deletion API
- Setup retention jobs
- Define retention periods per market

---

## 🌐 CLIENT-SIDE: What Goes on Client

### Core Principle
**Client = Smart media processor that handles all recording, encoding, and speech processing**

### Client Responsibilities Overview

```
┌─────────────────────────────────────────────────────────────┐
│                      CLIENT SIDE                             │
│                                                              │
│  ┌──────────────────────────────────────────────────────┐  │
│  │  1. MEDIA CAPTURE                                     │  │
│  │     • Camera/Microphone recording                    │  │
│  │     • Screen capture (for coding)                    │  │
│  │     • Device permission management                   │  │
│  └──────────────────────────────────────────────────────┘  │
│                          ↓                                   │
│  ┌──────────────────────────────────────────────────────┐  │
│  │  2. MEDIA PROCESSING                                 │  │
│  │     • Video/audio compression                        │  │
│  │     • Chunking (30-second segments)                  │  │
│  │     • Format encoding (WebM/VP9/Opus)                │  │
│  └──────────────────────────────────────────────────────┘  │
│                          ↓                                   │
│  ┌──────────────────────────────────────────────────────┐  │
│  │  3. SPEECH PROCESSING                                │  │
│  │     • Speech-to-Text (Whisper WASM)                  │  │
│  │     • Text-to-Speech (Web Speech API)                │  │
│  │     • Real-time transcription                        │  │
│  └──────────────────────────────────────────────────────┘  │
│                          ↓                                   │
│  ┌──────────────────────────────────────────────────────┐  │
│  │  4. AVATAR RENDERING                                 │  │
│  │     • 2D/3D avatar display                           │  │
│  │     • Lip sync (viseme mapping)                      │  │
│  │     • Facial expressions                             │  │
│  └──────────────────────────────────────────────────────┘  │
│                          ↓                                   │
│  ┌──────────────────────────────────────────────────────┐  │
│  │  5. UPLOAD MANAGEMENT                                │  │
│  │     • Direct S3/R2 uploads                           │  │
│  │     • Background upload (non-blocking)               │  │
│  │     • Retry logic                                    │  │
│  │     • Progress tracking                              │  │
│  └──────────────────────────────────────────────────────┘  │
│                          ↓                                   │
│  ┌──────────────────────────────────────────────────────┐  │
│  │  6. USER INTERFACE                                   │  │
│  │     • Interview flow UI                              │  │
│  │     • Real-time feedback                             │  │
│  │     • Progress indicators                            │  │
│  │     • Error handling                                 │  │
│  └──────────────────────────────────────────────────────┘  │
└──────────────────────────────────────────────────────────────┘
```

---

## 💻 CLIENT MODULES & TECHNOLOGIES

### Module 1: Media Recorder

**Purpose:** Capture video, audio, and screen

**Technology:**
- **MediaRecorder API** (native browser)
- **getUserMedia API** (camera/mic access)
- **getDisplayMedia API** (screen capture)

**Features:**
- Configure video resolution (1280x720)
- Audio quality settings (48kHz, echo cancellation)
- Chunked recording (30-second timeslices)
- Format: WebM with VP9 video codec and Opus audio codec

**Implementation:**
```javascript
navigator.mediaDevices.getUserMedia({
  video: { width: 1280, height: 720 },
  audio: { echoCancellation: true, noiseSuppression: true }
})
```

**TODO:**
- Handle device permissions
- Implement fallback for unsupported browsers
- Add device selection UI

---

### Module 2: Speech-to-Text (STT)

**Purpose:** Convert speech to text in real-time

**Technology Options:**

**Primary: Whisper.cpp WASM**
- Local processing (privacy)
- Model sizes: tiny (40MB), base (140MB)
- Offline capable
- High accuracy

**Fallback: Web Speech Recognition API**
- Native browser API
- Online only
- Variable accuracy across browsers

**Decision:** Start with Web Speech API, add Whisper WASM as enhancement

**Features:**
- Real-time or near-real-time transcription
- Confidence scores
- Language detection
- Punctuation

**TODO:**
- Integrate Whisper.cpp WASM bindings
- Implement Web Speech API fallback
- Add language selection
- Handle low-confidence results

---

### Module 3: Text-to-Speech (TTS)

**Purpose:** Synthesize avatar voice from text

**Technology Options:**

**Primary: Web Speech Synthesis API**
- Native browser voices
- Zero cost
- Immediate playback
- Variable quality across platforms

**Premium: Azure Speech SDK (optional)**
- Consistent quality
- Neural voices
- Viseme data (for lip sync)
- Cost: ~$0.016 per 1K characters

**Features:**
- Voice selection
- Rate/pitch/volume control
- Playback callbacks
- SSML support (optional)

**TODO:**
- Implement Web Speech Synthesis
- Add voice selection UI
- Optionally integrate Azure SDK for premium users
- Handle speech interruption

---

### Module 4: Avatar Renderer

**Purpose:** Render AI interviewer with lip sync

**Technology Options:**

**Option 1: Canvas 2D**
- Simple 2D animated avatar
- Lightweight
- Good browser support

**Option 2: WebGL/Three.js**
- 3D realistic avatar
- More engaging
- Higher performance requirements

**Lip Sync Technology:**
- **Viseme mapping** (phoneme → mouth shape)
- Sync with TTS timing
- Facial expressions (optional)

**Features:**
- Smooth animations
- Lip sync accuracy
- Idle animations
- Expression changes (friendly, thinking, etc.)

**TODO:**
- Choose 2D vs 3D approach
- Design/source avatar assets
- Implement viseme mapping
- Add facial expressions

---

### Module 5: Upload Manager

**Purpose:** Upload media chunks to S3/R2 via pre-signed URLs

**Technology:**
- **Fetch API** (native)
- Chunked uploads
- Background processing

**Features:**
- Direct PUT to pre-signed URLs (no server proxy)
- Progress tracking per chunk
- Retry logic (exponential backoff)
- Queue management (upload in order)
- Handle URL expiry (refresh from server)

**Flow:**
```
1. Receive pre-signed URL from server
2. Chunk ready → PUT request to URL
3. Track upload progress
4. On success: notify server, mark chunk complete
5. On failure: retry 3 times, then report error
6. If URL expired: request new URL, retry
```

**TODO:**
- Implement upload queue
- Add retry logic with exponential backoff
- Create progress indicator UI
- Handle network failures gracefully

---

### Module 6: State Management

**Purpose:** Manage client-side interview state

**Technology:**
- React Context API (if React)
- Zustand / Redux (for complex state)
- LocalStorage (persistence)

**State to Manage:**
- Current session config
- Current question
- Recording status
- Upload progress
- User responses (transcripts)
- Timer/progress
- Connection status

**TODO:**
- Define state structure
- Implement state management
- Add state persistence (survive page refresh)
- Handle state sync with server

---

### Module 7: WebSocket Handler

**Purpose:** Real-time state sync with server

**Technology:**
- Native WebSocket API
- Reconnection logic

**Events from Server:**
- Next question
- Time warnings
- Session end
- Error messages

**Events to Server:**
- Ready for next question
- Recording status updates
- Pause requests

**TODO:**
- Implement WebSocket connection
- Add reconnection logic
- Handle connection failures
- Define message protocols

---

### Module 8: User Interface Components

**Purpose:** Interview experience UI

**Technology:**
- React / Vue / Svelte (framework choice)
- Tailwind CSS / Material-UI (styling)
- Responsive design

**Components:**
- Interview setup screen
- Question display
- Avatar display
- Recording indicator
- Timer/progress bar
- Answer input (transcript display)
- Navigation controls
- Results display

**TODO:**
- Design UI mockups
- Implement component library
- Add responsive layouts
- Create loading states
- Add error states

---

### Module 9: Optional - MediaPipe Integration

**Purpose:** Extract facial and body landmarks (client-side CV)

**Technology:**
- **MediaPipe Face Mesh** (468 landmarks)
- **MediaPipe Pose** (33 landmarks)
- WASM-based (runs in browser)

**Use Case:**
- Extract features locally
- Send compact feature data (not raw frames)
- Privacy-friendly
- Reduces upload bandwidth

**Features:**
- Real-time landmark detection
- Feature extraction (eye contact, posture)
- Export compact JSON (not video frames)

**TODO:**
- Evaluate if needed for v1
- Integrate MediaPipe libraries
- Define features to extract
- Create feature export format

---

## 📊 Data Flow Summary

### During Interview

```
CLIENT                           SERVER
  │                                │
  │  1. POST /configure           │
  │ ───────────────────────────> │ Generate questions
  │ <─────────────────────────── │ Return config + URLs
  │                                │
  │  2. Record user response      │
  │     (local processing)        │
  │                                │
  │  3. PUT chunk to S3          │
  │ ──────────────────────> [S3] │ (direct, no server)
  │                                │
  │  4. POST /turn (transcript)   │
  │ ───────────────────────────> │ Store transcript
  │ <─────────────────────────── │ Return next question
  │                                │
  │  5. Repeat steps 2-4          │
  │                                │
  │  6. POST /complete            │
  │ ───────────────────────────> │ Queue analysis job
  │ <─────────────────────────── │ Return job ID
  │                                │
  │  7. Poll GET /report          │
  │ ───────────────────────────> │
  │ <─────────────────────────── │ Return report (when ready)
```

### Media Never Touches Server

```
CLIENT                    S3/R2                SERVER
  │                         │                     │
  │  Record & Encode        │                     │
  │  ─────────────>         │                     │
  │                         │                     │
  │  Upload (PUT)           │                     │
  │  ──────────────────────>│                     │
  │                         │                     │
  │  (text only) POST /turn │                     │
  │  ─────────────────────────────────────────> │
  │                         │                     │
  │                         │  Fetch for analysis │
  │                         │ <─────────────────  │
```

---

## 🔧 Technology Stack Summary

### Server Stack

| Layer | Technology | Purpose |
|-------|------------|---------|
| **Language** | Python 3.11+ | Backend logic |
| **Framework** | FastAPI | REST API |
| **Package Manager** | UV | Fast dependency management |
| **Database** | PostgreSQL 15+ | Data persistence |
| **Cache** | Redis | Session cache, queue broker |
| **Storage** | S3 / Cloudflare R2 | Media storage |
| **Queue** | Celery + Redis | Background jobs |
| **LLM** | Claude 3.5 / GPT-4 | AI logic |
| **Auth** | JWT | Stateless authentication |
| **Deployment** | Docker + ECS/EC2 | Containerized deployment |

### Client Stack

| Layer | Technology | Purpose |
|-------|------------|---------|
| **Language** | JavaScript/TypeScript | Frontend logic |
| **Framework** | React / Vue / Svelte | UI framework |
| **Recording** | MediaRecorder API | Native recording |
| **STT** | Whisper.cpp WASM / Web Speech | Speech-to-text |
| **TTS** | Web Speech Synthesis / Azure | Text-to-speech |
| **Avatar** | Canvas 2D / Three.js | Avatar rendering |
| **Upload** | Fetch API | Direct S3 uploads |
| **State** | Context / Zustand | State management |
| **Styling** | Tailwind CSS | UI styling |
| **Build** | Vite / Webpack | Bundling |

---

## ✅ Decision Checklist

### Server Decisions Needed:
- [ ] LLM provider: Claude vs GPT-4?
- [ ] Storage: S3 vs Cloudflare R2?
- [ ] Deployment: AWS vs GCP vs Azure?
- [ ] Auth: Internal JWT or external service?
- [ ] Database: Self-hosted PostgreSQL or managed (RDS)?

### Client Decisions Needed:
- [ ] Framework: React vs Vue vs Svelte?
- [ ] STT: Start with Web Speech or go straight to Whisper WASM?
- [ ] TTS: Web Speech only or add Azure premium option?
- [ ] Avatar: 2D simple or 3D realistic?
- [ ] MediaPipe: Include in v1 or later?

---

## 📋 Implementation Priority

### Phase 1 (Week 1): Foundation
**Server:**
- [ ] Basic API structure
- [ ] JWT validation (TODO: design)
- [ ] Database setup
- [ ] Session management
- [ ] Pre-signed URL generation

**Client:**
- [ ] Media recording
- [ ] Basic UI
- [ ] Upload manager
- [ ] Web Speech API integration

### Phase 2 (Week 2): Intelligence
**Server:**
- [ ] LLM question generation
- [ ] Follow-up logic
- [ ] Celery worker setup
- [ ] Scoring algorithms (basic)

**Client:**
- [ ] Avatar rendering
- [ ] TTS integration
- [ ] WebSocket connection
- [ ] State management

### Phase 3 (Week 3): Polish
**Server:**
- [ ] Complete scoring (7 metrics)
- [ ] Report generation
- [ ] Historical trends
- [ ] Event publishing
- [ ] Compliance features

**Client:**
- [ ] UI polish
- [ ] Error handling
- [ ] Progress indicators
- [ ] Report display
- [ ] Offline handling

---

## 🎯 Success Criteria

**Server:**
- ✅ API latency p95 < 200ms
- ✅ Can handle 100 concurrent sessions
- ✅ Analysis completes in < 5 minutes
- ✅ Zero data loss
- ✅ GDPR/CCPA compliant

**Client:**
- ✅ Recording works on Chrome, Firefox, Safari, Edge
- ✅ STT accuracy > 90%
- ✅ Upload success rate > 99%
- ✅ Avatar lip sync smooth (no lag)
- ✅ Works on 5 Mbps connection

---

## 📝 Notes

1. **Authentication is placeholder**: JWT validation needs to be designed based on your auth system
2. **Cost optimization**: Client-side processing saves ~70% on server costs
3. **Privacy**: Media stays client-side or in storage, never processed on server in real-time
4. **Scalability**: Stateless server design enables horizontal scaling
5. **Offline capability**: Client can record offline, upload later (future enhancement)