matchmaking-v2/research/docs/ENGINE_BUILD_ANALYSIS.md

# Engine Build Analysis — from 90 raw cards to a ranked, explained shortlist

> Written 2026-06-19. Companion to `ENGINE_DESIGN.md` (the locked design) + `SIGNAL_AUDIT_V2.md`
> (the signal inventory). This doc is the **build-readiness analysis**: what we have *today*, the gap,
> and the smallest engine that turns the live multi-board sweep into "only the recommended ones, with
> aligned stats." It translates the design into a phased, buildable plan against the real data surface.

---

## 1. The gap (current vs target)

**Today (Slice 1a):** fetch ~90 live jobs → `search.run_sweep` assigns a **placeholder** score
(`matchScore = max(55, 95-i)`, i.e. rank = fetch order) → **all 90** shown. Critically:
- `user_context` (the user's resume/skills/experience) is **fetched by the orchestrator but ignored**
  by `handle_run_search` — so nothing is actually matched *to the user*.
- The card's "stats" are **fabricated in the frontend**: `scoutJobToMatchRole()` builds the QX
  `breakdown` from `required_skills[:3]` + fixed deltas `[10,-7,-32]`, derives `growth` from that, sets
  `reason = "Live from Naukri"`, and `proofReady = matchScore≥84`. **None of it reflects the user.**

**Target:** fetch ~90 → **engine** (filter + rank *against the user* + select) → **top ~20–30** →
each card carries a **real** match %, a real "why picked", a real skill breakdown, and a FIT/STRETCH tag.

Two asks map to two engine outputs:
- *"only show the ones recommended"* → **hard filter + score + top-N selection**.
- *"stats more aligned"* → **move the breakdown/why/growth from fabricated-frontend to real-backend**,
  and have the card render what the engine emits.

---

## 2. The input surface we actually have NOW

### Job side — per fetched `ScoutJob` (already extracted by the normalizers)
`title` · `required_skills` (full list in `details.skills`) · `seniority_level` · `location_city` +
`location_mode` · `salary_lpa` / `payLabel` · `industry` · `role_category` · `employment_type` ·
`description` (clean text) · `note` (board) · `offsite_apply` · freshness (**all on-demand = fresh by
construction**; we already drop `isExpired`).

### User side
- **Configure prefs (`ScoutPrefs`) — always present:** `title`, `role[]`, `location[]`, `workMode[]`,
  `experience`/seniority, `industry[]`, `years`, `targetComp`, `companyStage`, `availability`,
  `dealBreakers[]`, `priorities[]` (rank), `stretch` (safe/balanced/reach).
- **`user_context` (orchestrator-assembled, currently ignored) — present when the user has a profile:**
  `skills[]` (resume + live LinkedIn, deduped), `current_role`, `experience_history`, `education`,
  `linkedin_headline/summary/experience`. Plus the **QScore quotient vector** (qscore-service).

**Coverage reality:** Configure prefs are guaranteed; `user_context` may be sparse/empty (the "No
resume" state). The engine **must** degrade gracefully (design §3.3 coverage-renormalization) — rank on
what's present, never fabricate, never collapse to a neutral 60 (the v1 bug).

---

## 3. The design (recap) vs what's buildable first

`ENGINE_DESIGN.md` §3 cascade: ① normalize → ② hard-filter → ③ white-box utility → ④ rank-fusion →
⑤ LLM rerank+explain → ⑥ output → (log → learned ranker).

The **full** vision needs ESCO taxonomy + depth-2 KG, contrastive skill embeddings, GPT-5.4 rerank, and
a learned pairwise ranker. **None of that is needed to ship the core value.** Stages ② + ③ + ⑥ —
deterministic, white-box, no LLM, no taxonomy service — already deliver *filter + rank + select +
explain* against the current data. That is the first engine; the rest are precision upgrades.

---

## 4. Phased build

### Phase 0 — Deterministic white-box engine (`app/engine/rank.py`), no LLM, no `user_context` required
The shippable core. Filters 90 → top-N, scores, emits real stats. Works on Configure prefs alone, gets
sharper when `user_context` lands (Phase 1).

**② Hard filter (deterministic gate — the design's point: embeddings can't do this):**
- **Location / work-mode fit** — port `_score_location_fit` + `_location_filter_blocks_match` from the
  *old* `matchmaking/app/engine/core.py` (the audit says it did this better): remote→100 (0 if seeker
  onsite-only); else city/country substring vs preferred locations; unknown→neutral 60; **gate** blocks
  no-fit unless prefs are "Anywhere".
- **Seniority band** — drop jobs far outside the user's level (e.g. a 2-yr seeker vs a Lead role),
  governed by `stretch` (reach allows +1 level).
- **Deal-breakers** — exclude on stated breakers where detectable (on-call, on-site, etc.).
- **Freshness** — already fresh; drop any `isExpired`/dead.

**③ White-box utility, coverage-renormalized** — `U = Σ_present c_f·w_f·φ_f / Σ_present c_f·w_f`:
| factor | prior w | φ (0–1) from |
|---|---|---|
| skill_match | .35 | overlap(user.skills, job.required_skills), synonym-normalized; required>preferred |
| experience | .25 | user `years`/seniority vs job `seniority_level` band |
| location | .15 | ported location-fit score |
| salary | .10 | job `salary_lpa` vs `targetComp` band — **present-only** (most are "Not disclosed" → weight redistributes) |
| industry/role | .05+.05 | job `industry`/`role_category` vs prefs |
| semantic | .10 | cheap title/description keyword overlap (NOT embeddings in v0) |

Sparse profile (no `skills`) → skill weight redistributes to location/experience/recency. Never penalize absence.

**⑤′ Selection (the "only recommended" lever)** — sort by U, cut by `stretch`:
`safe`→ U≥0.85 · `balanced`→ U≥0.70 (+ a few stretch) · `reach`→ U≥0.55. **Floor of ~8–10 so the deck
is never empty** (relax threshold if too few pass); **cap ~25–30** (never the full 90).

**FIT vs STRETCH (§3.5):** FIT = skill-gap≈0 (score current overlap, mild over-qual penalty);
STRETCH = one level up / adjacent, scored by **reachability** (fraction of required skills the user has
or is one hop from). Tag each card; the `stretch` pref sets how many STRETCH cards ride along.

**⑥ Aligned stats (emitted by the engine, rendered by the card):**
- `match` = round(U×100) — **real**, not fetch-order.
- `breakdown[]` = the real top dims (matched skills Strong / partial Solid / missing Light) — **from
  user↔job skill overlap**, replacing the fabricated `[10,-7,-32]`.
- `growth` = the weakest *reachable* dim + the lift it buys.
- `reason` = a real templated "why" ("Matches 6/8 must-have skills · Senior fit · fresh from Naukri").
- `proofReady` = user is Strong on the job's key skill (real).
- `fit | stretch` tag.

### Phase 1 — Wire `user_context` (the accuracy unlock)
`handle_run_search` reads `params["user_context"]` (already delivered by the worker) → feeds
`skills/current_role/experience_history/education` + QScore quotient into ③. QScore/assessments are
**boosters** (present-only, coverage-penalized), never a penalty when absent. This is the user-arm the
whole product is about — Phase 0 ranks on prefs; Phase 1 ranks on *who the user actually is*.

### Phase 2 — LLM rerank + explanation (GPT-5.4 on top ~20)
Calibrated 0–1 fit + a genuine 1-line "why" + growth nudge; replaces the templated reason on the top
slice. Log soft-scores as cold-start supervision. **Only the top ~20, cached per (role, location)** —
never LLM all 90 (cost/latency).

### Phase 3 — Learned ranker
Accrue SAVE/DISMISS/APPLY labels (the `record_feedback` action) → pairwise/listwise ranker → shift
fusion weight off the expert priors.

---

## 5. The stats-alignment refactor (a contract change)

Right now the card's "intelligence" is **invented in `scoutJobToMatchRole`**. For stats to be honest,
the engine must own them and the ScoutJob must carry a `match` block:

```
match: {
  score: number,                 // U×100
  fit: "fit" | "stretch",
  reason: string,                // real why-picked
  breakdown: [{name, score, level}],   // real user↔job skill dims
  growth?: {text, from, to},
  proofReady: boolean,
}
```
The frontend `scoutJobToMatchRole` then **reads `job.match.*`** instead of fabricating it. Net: delete
the fake-breakdown logic; the card becomes a pure renderer of engine output. (Same modal/stats UI — it
just gets real numbers.)

---

## 6. Key decisions & risks
- **Skill normalization without ESCO (v0):** a **seed synonym map** (k8s→Kubernetes, React.js→React,
  ML→Machine Learning, JS→JavaScript…) + India tech terms + lowercase/punctuation fold. Beats exact
  match; upgrade to ESCO+KG+contrastive later. This is the single biggest accuracy lever (§3.2).
- **Sparse profiles are the norm, not the exception** — coverage-renormalization is load-bearing, not
  a nicety. Test the "No resume" path first.
- **Never an empty deck** — selection has a floor; if few pass, lower the bar and label them honestly
  ("a few reaches").
- **Don't double-count QScore** (already digests interview/roleplay) — §2 of the audit.
- **LLM only on the top slice, cached** — the §3 "retrieve cheap, rerank expensive" discipline.
- **Freshness is free here** — on-demand fetch means every shown listing is seconds old (§6.1).

---

## 7. Recommendation (smallest first step)

Build **Phase 0 + Phase 1 together** as `app/engine/rank.py` — a deterministic, coverage-aware,
white-box ranker that consumes `user_context`, filters 90 → top ~25, and emits the real `match` block.
No LLM, no ESCO service, no new infra; it ships *both* asks ("only recommended" + "aligned stats").
Then add Phase 2 (LLM rerank/explain) once the deterministic core + the frontend `match`-contract are
in. Phase 3 (learned) follows the live save/apply loop.

**Open scoping questions for the build:** (a) wire `user_context` now or rank on Configure prefs first?
(b) how aggressive should default selection be (target deck size)? (c) include the GPT-5.4 rerank in v1
or ship deterministic-only first? (d) seed-synonym map now vs minimal exact+fold to start?