89ca9ad647
On-demand Scout service (replaces the nightly aggregator): - FastAPI agent-mesh service; card name "matchmaking-service" (HTTP /a2a/tasks + Redis-stream worker matching the sibling-service pattern) - run_search: parallel multi-board sweep (Naukri/blackfalcondata + Foundit + LinkedIn), city-filtered at the board, cheapest-per-result first - per-board adapters + normalizers -> ScoutJob with rich read-only details and offsite apply links; recall mode + MVQ guard - result cache for dev replay ($0); per-board cost knobs; retry-on-5xx - research/: engine design, board cost economics, India-actor shortlist, POC
8.0 KiB
Signal Audit v2 — Evidence-Based Input Inventory
Companion to
ENGINE_DESIGN.md. Supersedes its §2/§5 signal audit. Written 2026-06-16, grounded in live prod-DB evidence (sampled over SSH) rather than the original conservative assumptions. Key correction: the v1 doc flagged interview/roleplay/assessment as "❌ GAP — build transport." The data is live and queryable today — richer engine is mostly wiring + discipline, not new builds.
0. Governing discipline (unchanged — the v1-collapse guardrails)
More signals make the engine stronger only if we keep these rules. Breaking them is how v1 died.
- Trust ⟂ coverage. A signal earns SCORE WEIGHT only if coverage is high enough that absence isn't the norm. Otherwise it is an ADDITIVE BOOSTER (bonus when present, never a penalty when absent). Most new signals are low-coverage → boosters, not weights.
- Filter degenerate data. Verified live: the sampled interview review was a no-show (score 30),
the roleplay an 8-word low-talk. These would poison the profile. Drop
no_show/low_talksessions (the data already flagsexclude_from_trend: true/scoring_kind: no_show). - Coverage-aware renormalization (
ENGINE_DESIGN.md§3.3): score over present signals only; a missing signal redistributes its weight, never flattens.
1. The signal inventory (by tier)
Tier A — Backbone (high coverage → filter + weight)
| Signal | Source (verified) | Role | Flows in via |
|---|---|---|---|
| years_experience (date-derived) | resume-builder parse | weight | context-resolver |
| current_role / title | resume + verified LinkedIn | filter + weight | context-resolver |
| skills (→ ESCO normalized) | resume-builder parse | core weight (skill match) | context-resolver |
| location + work mode | user prefs (Configure must-haves) + user-service | hard filter + 30% weight | request prefs |
| target_roles / industries | Configure (user-stated want) | filter, not score | request prefs |
Location logic — PORT from the old engine (
matchmaking/app/engine/core.py, it did this better than v1's one-liner). Bring forward_score_location_fit+_location_filter_blocks_match: remote job → 100 (0 if seeker is onsite-only); else city+country substring match vs preferred locations (so an add-your-own typed city "just works"); travel-mode adds travel locations; unknown → neutral 60. As a hard gate: block remote for onsite-only seekers, block no-fit jobs (clamp below threshold) unless the seeker set no location prefs ("Anywhere" = open). Weighted 30%.
Tier B — Verified enrichers (medium coverage → real weight when present)
| Signal | Source (verified) | Role | Notes |
|---|---|---|---|
| verified title / company / industry | social-branding live LinkedIn fetch | weight (present) | doc said "partial" — it's live |
| degree / field / institution | resume parse / LinkedIn education | weight (present) | |
| certifications | resume / LinkedIn | booster | |
| assessment scores | user-service (assessment tables) | booster | doc called it a "gap"; it exists |
| QScore quotient vector (10-D) | qscore-service | competence proxy (weight, coverage-penalized) | see §2 — this is the key aggregate |
| Market context (demand, hiring velocity, posting freshness, comp band) | Perplexity API (online) — TESTED 2026-06-16, returns sourced numbers + honest "can't-verify" on layoffs | enricher / context for ranking + the UI "why" | cache per role+location (latency/cost). Honest by design — Perplexity flags what it can't confirm (e.g. layoff signals). Replaces the fabricated market-intel that was in the UI signal list. |
Tier C — Behavioral / competence boosters (LOW coverage → ADDITIVE ONLY)
| Signal | Source (verified) | Role | Notes |
|---|---|---|---|
| Course watch behavior (watch-time, completion, interest profile) | growqr_course |
booster — target-role inference + FIT/STRETCH direction | NEW class. Highest-trust: revealed preference > stated. "Completing React content → step toward frontend." |
| improvement velocity / coachability | interview/roleplay trend_data, historical_comparison |
booster | growth trajectory |
| video on-camera presence | interview/roleplay video_analysis (Gemini 2.5 Pro) |
booster | <1% relevant; rarely surfaced |
Interview 8-D rubric + roleplay EQ are not listed as independent inputs — see §2 (already folded into QScore).
Labels (training targets, NOT scoring inputs)
| Signal | Source | Use |
|---|---|---|
| SAVE / DISMISS / APPLY / RSVP feedback | old matchmaking (Record Feedback) + our apply engine outcomes |
accrue labels → train pairwise/listwise ranker (ENGINE_DESIGN.md §3.1) |
2. Entanglement: QScore already consumes interview/roleplay
Do not double-count. QScore is consumed by interview & roleplay, and the interview/roleplay competence is digested into the QScore quotient vector. So for the engine:
- v1 (recommended): use the QScore quotient vector as the competence proxy — it already
incorporates the interview/roleplay rubric signals. This matches
ENGINE_DESIGN.md§5: "launch on READY rails (Tier A + QScore quotients), add direct rubric access later as a precision upgrade." - Later (precision upgrade): read the raw interview/roleplay rubrics directly as boosters — only once we've confirmed they add lift beyond the QScore aggregate (else it's redundant weight).
This means course watch-behavior is the one genuinely new, non-redundant signal to wire in v1.
3. Transport: mostly already solved
The résumé + LinkedIn → service transport already exists — interview & roleplay services already
receive résumé/LinkedIn context (they're résumé/LinkedIn-aware), assembled by the orchestrator's
context_resolver into user_context. matchmaking-v2 reuses that exact pattern: consume the
assembled user_context (resume + verified LinkedIn + QScore + assessment) the same way interview/
roleplay do. So the rich user-side profile is a wiring job against a proven, existing transport,
not a new build. Course watch-behavior is the one extra read (from growqr_course).
The user-side profile is then scored against the on-demand fetched listings (LinkedIn/Naukri/ Indeed via Apify) — the job side stays on-demand; the user side is what these signals enrich.
4. Cut / discard
- Pathways (entire) — CUT. Feature dropped (weak link: buggy skill-gap, impractical roles) AND its questionnaire is not used as a signal. The Configure screen already captures stated prefs.
- Carry-over discards (
ENGINE_DESIGN.md§2): ATS overall_score (resume quality ≠ fit), parser confidence, vanity metrics, self-declared salary, institution_tier (pedigree bias). - UI overclaims with no backing signal: "live status" (tap-time liveness unreliable, §6.1), "in-network" (no referral agent), "Closes in N days" (no deadline data).
5. Net: stronger engine, same discipline
v1 input set = Backbone (resume skills/experience + Configure prefs/filters) + verified LinkedIn + QScore quotient vector (competence proxy) + course watch-behavior (new, high-trust booster), fused coverage-aware, scored against on-demand listings, with save/apply feedback accruing as labels. That is materially richer than the doc's "Tier-A + QScore proxy" — and it's reachable now because the transport exists and the discipline (boosters for sparse, filter degenerate) keeps it from collapsing.
Honesty ledger
| Now proven available (was assumed a gap) | Still must validate |
|---|---|
| Interview/roleplay competence (live in DB; via QScore) | Does raw rubric add lift beyond QScore? |
| Course watch-behavior as a signal | Coverage of active learners; interest→role mapping quality |
| Verified LinkedIn identity (live fetch) | India LinkedIn fetch reliability |
| Feedback labels exist (save/apply) | Volume sufficient to train a ranker (was 21 users) |
| Transport reuse (context_resolver) | matchmaking-v2 wiring into the mesh |