matchmaking-v2/research/docs/SIGNAL_AUDIT_V2.md

Signal Audit v2 — Evidence-Based Input Inventory

Companion to ENGINE_DESIGN.md. Supersedes its §2/§5 signal audit. Written 2026-06-16, grounded in live prod-DB evidence (sampled over SSH) rather than the original conservative assumptions. Key correction: the v1 doc flagged interview/roleplay/assessment as "❌ GAP — build transport." The data is live and queryable today — richer engine is mostly wiring + discipline, not new builds.

---

0. Governing discipline (unchanged — the v1-collapse guardrails)

More signals make the engine stronger only if we keep these rules. Breaking them is how v1 died.

1. Trust ⟂ coverage. A signal earns SCORE WEIGHT only if coverage is high enough that absence isn't the norm. Otherwise it is an ADDITIVE BOOSTER (bonus when present, never a penalty when absent). Most new signals are low-coverage → boosters, not weights.
2. Filter degenerate data. Verified live: the sampled interview review was a no-show (score 30), the roleplay an 8-word low-talk. These would poison the profile. Drop no_show / low_talk sessions (the data already flags exclude_from_trend: true / scoring_kind: no_show).
3. Coverage-aware renormalization (ENGINE_DESIGN.md §3.3): score over present signals only; a missing signal redistributes its weight, never flattens.

1. The signal inventory (by tier)

Tier A — Backbone (high coverage → filter + weight)

Signal	Source (verified)	Role	Flows in via
years_experience (date-derived)	resume-builder parse	weight	context-resolver
current_role / title	resume + verified LinkedIn	filter + weight	context-resolver
skills (→ ESCO normalized)	resume-builder parse	core weight (skill match)	context-resolver
location + work mode	user prefs (Configure must-haves) + user-service	hard filter + 30% weight	request prefs
target_roles / industries	Configure (user-stated want)	filter, not score	request prefs

Location logic — PORT from the old engine (matchmaking/app/engine/core.py, it did this better than v1's one-liner). Bring forward _score_location_fit + _location_filter_blocks_match: remote job → 100 (0 if seeker is onsite-only); else city+country substring match vs preferred locations (so an add-your-own typed city "just works"); travel-mode adds travel locations; unknown → neutral 60. As a hard gate: block remote for onsite-only seekers, block no-fit jobs (clamp below threshold) unless the seeker set no location prefs ("Anywhere" = open). Weighted 30%.

Tier B — Verified enrichers (medium coverage → real weight when present)

Signal	Source (verified)	Role	Notes
verified title / company / industry	social-branding live LinkedIn fetch	weight (present)	doc said "partial" — it's live
degree / field / institution	resume parse / LinkedIn education	weight (present)
certifications	resume / LinkedIn	booster
assessment scores	user-service (assessment tables)	booster	doc called it a "gap"; it exists
QScore quotient vector (10-D)	qscore-service	competence proxy (weight, coverage-penalized)	see §2 — this is the key aggregate
Market context (demand, hiring velocity, posting freshness, comp band)	Perplexity API (online) — TESTED 2026-06-16, returns sourced numbers + honest "can't-verify" on layoffs	enricher / context for ranking + the UI "why"	cache per role+location (latency/cost). Honest by design — Perplexity flags what it can't confirm (e.g. layoff signals). Replaces the fabricated market-intel that was in the UI signal list.

Tier C — Behavioral / competence boosters (LOW coverage → ADDITIVE ONLY)

Signal	Source (verified)	Role	Notes
Course watch behavior (watch-time, completion, interest profile)	growqr_course	booster — target-role inference + FIT/STRETCH direction	NEW class. Highest-trust: revealed preference > stated. "Completing React content → step toward frontend."
improvement velocity / coachability	interview/roleplay trend_data, historical_comparison	booster	growth trajectory
video on-camera presence	interview/roleplay video_analysis (Gemini 2.5 Pro)	booster	<1% relevant; rarely surfaced

Interview 8-D rubric + roleplay EQ are not listed as independent inputs — see §2 (already folded into QScore).

Labels (training targets, NOT scoring inputs)

Signal	Source	Use
SAVE / DISMISS / APPLY / RSVP feedback	old matchmaking (Record Feedback) + our apply engine outcomes	accrue labels → train pairwise/listwise ranker (ENGINE_DESIGN.md §3.1)

2. Entanglement: QScore already consumes interview/roleplay

Do not double-count. QScore is consumed by interview & roleplay, and the interview/roleplay competence is digested into the QScore quotient vector. So for the engine:

• v1 (recommended): use the QScore quotient vector as the competence proxy — it already incorporates the interview/roleplay rubric signals. This matches ENGINE_DESIGN.md §5: "launch on READY rails (Tier A + QScore quotients), add direct rubric access later as a precision upgrade."
• Later (precision upgrade): read the raw interview/roleplay rubrics directly as boosters — only once we've confirmed they add lift beyond the QScore aggregate (else it's redundant weight).

This means course watch-behavior is the one genuinely new, non-redundant signal to wire in v1.

3. Transport: mostly already solved

The résumé + LinkedIn → service transport already exists — interview & roleplay services already receive résumé/LinkedIn context (they're résumé/LinkedIn-aware), assembled by the orchestrator's context_resolver into user_context. matchmaking-v2 reuses that exact pattern: consume the assembled user_context (resume + verified LinkedIn + QScore + assessment) the same way interview/ roleplay do. So the rich user-side profile is a wiring job against a proven, existing transport, not a new build. Course watch-behavior is the one extra read (from growqr_course).

The user-side profile is then scored against the on-demand fetched listings (LinkedIn/Naukri/ Indeed via Apify) — the job side stays on-demand; the user side is what these signals enrich.

4. Cut / discard

• Pathways (entire) — CUT. Feature dropped (weak link: buggy skill-gap, impractical roles) AND its questionnaire is not used as a signal. The Configure screen already captures stated prefs.
• Carry-over discards (ENGINE_DESIGN.md §2): ATS overall_score (resume quality ≠ fit), parser confidence, vanity metrics, self-declared salary, institution_tier (pedigree bias).
• UI overclaims with no backing signal: "live status" (tap-time liveness unreliable, §6.1), "in-network" (no referral agent), "Closes in N days" (no deadline data).

5. Net: stronger engine, same discipline

v1 input set = Backbone (resume skills/experience + Configure prefs/filters) + verified LinkedIn + QScore quotient vector (competence proxy) + course watch-behavior (new, high-trust booster), fused coverage-aware, scored against on-demand listings, with save/apply feedback accruing as labels. That is materially richer than the doc's "Tier-A + QScore proxy" — and it's reachable now because the transport exists and the discipline (boosters for sparse, filter degenerate) keeps it from collapsing.

Honesty ledger

Now proven available (was assumed a gap)	Still must validate
Interview/roleplay competence (live in DB; via QScore)	Does raw rubric add lift beyond QScore?
Course watch-behavior as a signal	Coverage of active learners; interest→role mapping quality
Verified LinkedIn identity (live fetch)	India LinkedIn fetch reliability
Feedback labels exist (save/apply)	Volume sufficient to train a ranker (was 21 users)
Transport reuse (context_resolver)	matchmaking-v2 wiring into the mesh