matchmaking-v2/research/docs/SIGNAL_AUDIT_V2.md

# Signal Audit v2 — Evidence-Based Input Inventory

> Companion to `ENGINE_DESIGN.md`. Supersedes its §2/§5 signal audit. Written 2026-06-16, grounded
> in **live prod-DB evidence** (sampled over SSH) rather than the original conservative assumptions.
> Key correction: the v1 doc flagged interview/roleplay/assessment as "❌ GAP — build transport."
> The data is **live and queryable today** — richer engine is mostly *wiring + discipline*, not new builds.

---

## 0. Governing discipline (unchanged — the v1-collapse guardrails)
More signals make the engine **stronger only if** we keep these rules. Breaking them is how v1 died.
1. **Trust ⟂ coverage.** A signal earns SCORE WEIGHT only if coverage is high enough that absence
   isn't the norm. Otherwise it is an **ADDITIVE BOOSTER** (bonus when present, never a penalty when
   absent). Most new signals are low-coverage → boosters, not weights.
2. **Filter degenerate data.** Verified live: the sampled interview review was a *no-show* (score 30),
   the roleplay an *8-word low-talk*. These would poison the profile. Drop `no_show` / `low_talk`
   sessions (the data already flags `exclude_from_trend: true` / `scoring_kind: no_show`).
3. **Coverage-aware renormalization** (`ENGINE_DESIGN.md` §3.3): score over *present* signals only;
   a missing signal redistributes its weight, never flattens.

## 1. The signal inventory (by tier)

### Tier A — Backbone (high coverage → filter + weight)
| Signal | Source (verified) | Role | Flows in via |
|---|---|---|---|
| years_experience (date-derived) | resume-builder parse | weight | context-resolver |
| current_role / title | resume + verified LinkedIn | filter + weight | context-resolver |
| skills (→ ESCO normalized) | resume-builder parse | **core weight** (skill match) | context-resolver |
| **location + work mode** | user prefs (Configure must-haves) + user-service | **hard filter + 30% weight** | request prefs |
| target_roles / industries | Configure (user-stated *want*) | filter, not score | request prefs |

> **Location logic — PORT from the old engine** (`matchmaking/app/engine/core.py`, it did this better
> than v1's one-liner). Bring forward `_score_location_fit` + `_location_filter_blocks_match`:
> remote job → 100 (0 if seeker is onsite-only); else **city+country substring match** vs preferred
> locations (so an add-your-own typed city "just works"); travel-mode adds travel locations;
> unknown → neutral 60. As a **hard gate**: block remote for onsite-only seekers, block no-fit jobs
> (clamp below threshold) unless the seeker set no location prefs ("Anywhere" = open). Weighted 30%.

### Tier B — Verified enrichers (medium coverage → real weight when present)
| Signal | Source (verified) | Role | Notes |
|---|---|---|---|
| verified title / company / **industry** | **social-branding live LinkedIn fetch** | weight (present) | doc said "partial" — it's *live* |
| degree / field / institution | resume parse / LinkedIn education | weight (present) | |
| certifications | resume / LinkedIn | booster | |
| assessment scores | user-service (assessment tables) | booster | doc called it a "gap"; it exists |
| **QScore quotient vector (10-D)** | qscore-service | **competence proxy (weight, coverage-penalized)** | see §2 — this is the key aggregate |
| **Market context** (demand, hiring velocity, posting freshness, comp band) | **Perplexity API (online)** — TESTED 2026-06-16, returns sourced numbers + honest "can't-verify" on layoffs | enricher / context for ranking + the UI "why" | **cache per role+location** (latency/cost). Honest by design — Perplexity flags what it can't confirm (e.g. layoff signals). Replaces the fabricated market-intel that was in the UI signal list. |

### Tier C — Behavioral / competence boosters (LOW coverage → ADDITIVE ONLY)
| Signal | Source (verified) | Role | Notes |
|---|---|---|---|
| **Course watch behavior** (watch-time, completion, interest profile) | `growqr_course` | **booster — target-role inference + FIT/STRETCH direction** | **NEW class. Highest-trust: revealed preference > stated.** "Completing React content → step toward frontend." |
| improvement velocity / coachability | interview/roleplay `trend_data`, `historical_comparison` | booster | growth trajectory |
| video on-camera presence | interview/roleplay `video_analysis` (Gemini 2.5 Pro) | booster | <1% relevant; rarely surfaced |

> Interview 8-D rubric + roleplay EQ are **not listed as independent inputs** — see §2 (already folded into QScore).

### Labels (training targets, NOT scoring inputs)
| Signal | Source | Use |
|---|---|---|
| SAVE / DISMISS / APPLY / RSVP feedback | old matchmaking (`Record Feedback`) + **our apply engine outcomes** | accrue labels → train pairwise/listwise ranker (`ENGINE_DESIGN.md` §3.1) |

## 2. Entanglement: QScore already consumes interview/roleplay
**Do not double-count.** QScore is *consumed by* interview & roleplay, and the interview/roleplay
competence is *digested into* the QScore quotient vector. So for the engine:
- **v1 (recommended):** use the **QScore quotient vector as the competence proxy** — it already
  incorporates the interview/roleplay rubric signals. This matches `ENGINE_DESIGN.md` §5: "launch on
  READY rails (Tier A + QScore quotients), add direct rubric access later as a precision upgrade."
- **Later (precision upgrade):** read the **raw** interview/roleplay rubrics directly as boosters —
  only once we've confirmed they add lift *beyond* the QScore aggregate (else it's redundant weight).

This means course watch-behavior is the **one genuinely new, non-redundant signal** to wire in v1.

## 3. Transport: mostly already solved
**The résumé + LinkedIn → service transport already exists** — interview & roleplay services already
receive résumé/LinkedIn context (they're résumé/LinkedIn-aware), assembled by the orchestrator's
`context_resolver` into `user_context`. **matchmaking-v2 reuses that exact pattern**: consume the
assembled `user_context` (resume + verified LinkedIn + QScore + assessment) the same way interview/
roleplay do. So the rich user-side profile is a *wiring* job against a **proven, existing transport**,
not a new build. Course watch-behavior is the one extra read (from `growqr_course`).

The user-side profile is then scored against the **on-demand fetched listings** (LinkedIn/Naukri/
Indeed via Apify) — the job side stays on-demand; the *user* side is what these signals enrich.

## 4. Cut / discard
- **Pathways (entire) — CUT.** Feature dropped (weak link: buggy skill-gap, impractical roles) AND
  its questionnaire is **not used as a signal**. The Configure screen already captures stated prefs.
- Carry-over discards (`ENGINE_DESIGN.md` §2): ATS overall_score (resume quality ≠ fit), parser
  confidence, vanity metrics, self-declared salary, **institution_tier** (pedigree bias).
- UI overclaims with **no backing signal**: "live status" (tap-time liveness unreliable, §6.1),
  "in-network" (no referral agent), "Closes in N days" (no deadline data).

## 5. Net: stronger engine, same discipline
v1 input set = **Backbone (resume skills/experience + Configure prefs/filters) + verified LinkedIn +
QScore quotient vector (competence proxy) + course watch-behavior (new, high-trust booster)**, fused
coverage-aware, scored against on-demand listings, with save/apply feedback accruing as labels. That
is materially richer than the doc's "Tier-A + QScore proxy" — and it's reachable now because the
transport exists and the discipline (boosters for sparse, filter degenerate) keeps it from collapsing.

## Honesty ledger
| Now proven available (was assumed a gap) | Still must validate |
|---|---|
| Interview/roleplay competence (live in DB; via QScore) | Does raw rubric add lift beyond QScore? |
| Course watch-behavior as a signal | Coverage of active learners; interest→role mapping quality |
| Verified LinkedIn identity (live fetch) | India LinkedIn fetch reliability |
| Feedback labels exist (save/apply) | Volume sufficient to train a ranker (was 21 users) |
| Transport reuse (context_resolver) | matchmaking-v2 wiring into the mesh |