matchmaking-v2/research/docs/ENGINE_BUILD_ANALYSIS.md

Engine Build Analysis — from 90 raw cards to a ranked, explained shortlist

Written 2026-06-19. Companion to ENGINE_DESIGN.md (the locked design) + SIGNAL_AUDIT_V2.md (the signal inventory). This doc is the build-readiness analysis: what we have today, the gap, and the smallest engine that turns the live multi-board sweep into "only the recommended ones, with aligned stats." It translates the design into a phased, buildable plan against the real data surface.

---

1. The gap (current vs target)

Today (Slice 1a): fetch ~90 live jobs → search.run_sweep assigns a placeholder score (matchScore = max(55, 95-i), i.e. rank = fetch order) → all 90 shown. Critically:

• user_context (the user's resume/skills/experience) is fetched by the orchestrator but ignored by handle_run_search — so nothing is actually matched to the user.
• The card's "stats" are fabricated in the frontend: scoutJobToMatchRole() builds the QX breakdown from required_skills[:3] + fixed deltas [10,-7,-32], derives growth from that, sets reason = "Live from Naukri", and proofReady = matchScore≥84. None of it reflects the user.

Target: fetch ~90 → engine (filter + rank against the user + select) → top ~20–30 → each card carries a real match %, a real "why picked", a real skill breakdown, and a FIT/STRETCH tag.

Two asks map to two engine outputs:

• "only show the ones recommended" → hard filter + score + top-N selection.
• "stats more aligned" → move the breakdown/why/growth from fabricated-frontend to real-backend, and have the card render what the engine emits.

---

2. The input surface we actually have NOW

Job side — per fetched ScoutJob (already extracted by the normalizers)

title · required_skills (full list in details.skills) · seniority_level · location_city + location_mode · salary_lpa / payLabel · industry · role_category · employment_type · description (clean text) · note (board) · offsite_apply · freshness (all on-demand = fresh by construction; we already drop isExpired).

User side

• Configure prefs (ScoutPrefs) — always present: title, role[], location[], workMode[], experience/seniority, industry[], years, targetComp, companyStage, availability, dealBreakers[], priorities[] (rank), stretch (safe/balanced/reach).
• user_context (orchestrator-assembled, currently ignored) — present when the user has a profile: skills[] (resume + live LinkedIn, deduped), current_role, experience_history, education, linkedin_headline/summary/experience. Plus the QScore quotient vector (qscore-service).

Coverage reality: Configure prefs are guaranteed; user_context may be sparse/empty (the "No resume" state). The engine must degrade gracefully (design §3.3 coverage-renormalization) — rank on what's present, never fabricate, never collapse to a neutral 60 (the v1 bug).

---

3. The design (recap) vs what's buildable first

ENGINE_DESIGN.md §3 cascade: ① normalize → ② hard-filter → ③ white-box utility → ④ rank-fusion → ⑤ LLM rerank+explain → ⑥ output → (log → learned ranker).

The full vision needs ESCO taxonomy + depth-2 KG, contrastive skill embeddings, GPT-5.4 rerank, and a learned pairwise ranker. None of that is needed to ship the core value. Stages ② + ③ + ⑥ — deterministic, white-box, no LLM, no taxonomy service — already deliver filter + rank + select + explain against the current data. That is the first engine; the rest are precision upgrades.

---

4. Phased build

Phase 0 — Deterministic white-box engine (app/engine/rank.py), no LLM, no user_context required

The shippable core. Filters 90 → top-N, scores, emits real stats. Works on Configure prefs alone, gets sharper when user_context lands (Phase 1).

② Hard filter (deterministic gate — the design's point: embeddings can't do this):

• Location / work-mode fit — port _score_location_fit + _location_filter_blocks_match from the old matchmaking/app/engine/core.py (the audit says it did this better): remote→100 (0 if seeker onsite-only); else city/country substring vs preferred locations; unknown→neutral 60; gate blocks no-fit unless prefs are "Anywhere".
• Seniority band — drop jobs far outside the user's level (e.g. a 2-yr seeker vs a Lead role), governed by stretch (reach allows +1 level).
• Deal-breakers — exclude on stated breakers where detectable (on-call, on-site, etc.).
• Freshness — already fresh; drop any isExpired/dead.

③ White-box utility, coverage-renormalized — U = Σ_present c_f·w_f·φ_f / Σ_present c_f·w_f:

factor	prior w	φ (0–1) from
skill_match	.35	overlap(user.skills, job.required_skills), synonym-normalized; required>preferred
experience	.25	user years/seniority vs job seniority_level band
location	.15	ported location-fit score
salary	.10	job salary_lpa vs targetComp band — present-only (most are "Not disclosed" → weight redistributes)
industry/role	.05+.05	job industry/role_category vs prefs
semantic	.10	cheap title/description keyword overlap (NOT embeddings in v0)

Sparse profile (no skills) → skill weight redistributes to location/experience/recency. Never penalize absence.

⑤′ Selection (the "only recommended" lever) — sort by U, cut by stretch: safe→ U≥0.85 · balanced→ U≥0.70 (+ a few stretch) · reach→ U≥0.55. Floor of ~8–10 so the deck is never empty (relax threshold if too few pass); cap ~25–30 (never the full 90).

FIT vs STRETCH (§3.5): FIT = skill-gap≈0 (score current overlap, mild over-qual penalty); STRETCH = one level up / adjacent, scored by reachability (fraction of required skills the user has or is one hop from). Tag each card; the stretch pref sets how many STRETCH cards ride along.

⑥ Aligned stats (emitted by the engine, rendered by the card):

• match = round(U×100) — real, not fetch-order.
• breakdown[] = the real top dims (matched skills Strong / partial Solid / missing Light) — from user↔job skill overlap, replacing the fabricated [10,-7,-32].
• growth = the weakest reachable dim + the lift it buys.
• reason = a real templated "why" ("Matches 6/8 must-have skills · Senior fit · fresh from Naukri").
• proofReady = user is Strong on the job's key skill (real).
• fit | stretch tag.

Phase 1 — Wire user_context (the accuracy unlock)

handle_run_search reads params["user_context"] (already delivered by the worker) → feeds skills/current_role/experience_history/education + QScore quotient into ③. QScore/assessments are boosters (present-only, coverage-penalized), never a penalty when absent. This is the user-arm the whole product is about — Phase 0 ranks on prefs; Phase 1 ranks on who the user actually is.

Phase 2 — LLM rerank + explanation (GPT-5.4 on top ~20)

Calibrated 0–1 fit + a genuine 1-line "why" + growth nudge; replaces the templated reason on the top slice. Log soft-scores as cold-start supervision. Only the top ~20, cached per (role, location) — never LLM all 90 (cost/latency).

Phase 3 — Learned ranker

Accrue SAVE/DISMISS/APPLY labels (the record_feedback action) → pairwise/listwise ranker → shift fusion weight off the expert priors.

---

5. The stats-alignment refactor (a contract change)

Right now the card's "intelligence" is invented in scoutJobToMatchRole. For stats to be honest, the engine must own them and the ScoutJob must carry a match block:

match: {
  score: number,                 // U×100
  fit: "fit" | "stretch",
  reason: string,                // real why-picked
  breakdown: [{name, score, level}],   // real user↔job skill dims
  growth?: {text, from, to},
  proofReady: boolean,
}

The frontend scoutJobToMatchRole then reads job.match.* instead of fabricating it. Net: delete the fake-breakdown logic; the card becomes a pure renderer of engine output. (Same modal/stats UI — it just gets real numbers.)

---

6. Key decisions & risks

• Skill normalization without ESCO (v0): a seed synonym map (k8s→Kubernetes, React.js→React, ML→Machine Learning, JS→JavaScript…) + India tech terms + lowercase/punctuation fold. Beats exact match; upgrade to ESCO+KG+contrastive later. This is the single biggest accuracy lever (§3.2).
• Sparse profiles are the norm, not the exception — coverage-renormalization is load-bearing, not a nicety. Test the "No resume" path first.
• Never an empty deck — selection has a floor; if few pass, lower the bar and label them honestly ("a few reaches").
• Don't double-count QScore (already digests interview/roleplay) — §2 of the audit.
• LLM only on the top slice, cached — the §3 "retrieve cheap, rerank expensive" discipline.
• Freshness is free here — on-demand fetch means every shown listing is seconds old (§6.1).

---

7. Recommendation (smallest first step)

Build Phase 0 + Phase 1 together as app/engine/rank.py — a deterministic, coverage-aware, white-box ranker that consumes user_context, filters 90 → top ~25, and emits the real match block. No LLM, no ESCO service, no new infra; it ships both asks ("only recommended" + "aligned stats"). Then add Phase 2 (LLM rerank/explain) once the deterministic core + the frontend match-contract are in. Phase 3 (learned) follows the live save/apply loop.

Open scoping questions for the build: (a) wire user_context now or rank on Configure prefs first? (b) how aggressive should default selection be (target deck size)? (c) include the GPT-5.4 rerank in v1 or ship deterministic-only first? (d) seed-synonym map now vs minimal exact+fold to start?