The measurement showed the USER-FACING Opus score is already well-calibrated — the real bug was curate
FAILING and dropping to the bad white-box fallback:
- curate.py: _salvage_kept() brace-scans + parses each kept object independently, so one bad char or a
max-tokens truncation no longer drops the whole shortlist. max_tokens 4000 → 8000 (28 sifted jobs ×
full report cards overflowed). This fixed the sales case (was failing → fallback).
- curate.py: calibration nudge in the prompt — a genuinely strong current-state match is a real low-80s,
partial/stretch 60s-70s; don't under-sell strong matches into the 60s (without inflating weak ones).
- normalize.py + search.py: _seniority_from_title() infers seniority from the title (conservative) when a
board omits it, applied centrally in the sweep → the experience factor + Opus brief aren't blind.
Verified (live Opus on the regression cases): tech MAE 9.8→4.5, sales FAILED→MAE 5.3 (kept 13),
ops 4.2→3.3 — user-facing scores now MAE 3-5 with zero curate failures. 31 tests pass.
Phase 0 — measurement gate:
- tests/test_contracts.py: BOARDS↔NORMALIZERS alignment, funnel-shape guards, overlap() None
contract, and the frontend↔backend EXACT-STRING vocab check (scout.ts options must resolve in
coerce.py, else silent keyword-fold).
- tests/run_regression.py + fixtures/regression_set.json (+ _seed_regression.py): a frozen 120-pair
set (tech + non-tech) Opus-judged for recruiter-fit targets. Two metrics: score MAE and SIFT TOP-K
MEMBERSHIP RECALL (do the best jobs reach Opus?). Baseline: MAE 24.7, recall 0.68 — i.e. ~32% of
the genuinely-best jobs are cut before the curator ever sees them (worse for non-tech).
Phase 1 — coverage (India non-tech):
- Enable Indeed (misceres) + enrich indeed_to_scoutjob with details.description + location_mode
(jobType is employment type, not skills → required_skills now []).
- Add WorkIndia (shahidirfan) — India blue/grey-collar non-tech: build_workindia_input +
workindia_to_scoutjob (real skills + description; per-job apply URL from job_id since source_url
is generic and would dedup-collapse the deck).
- Drop Wellfound from the stack (US-startup-heavy + 400s); kept registered-but-off.
- BOARDS_ENABLED = naukri,foundit,linkedin,indeed,workindia. Pool ~90 → ~140 jobs/search.
Regression scores unchanged (engine scoring untouched). 31 tests pass.
- OpportunityState model + repo: persist applied / dismissed / restored status and the
generated resume-builder doc ids (never re-pay to re-tailor). annotate_and_filter drops
dismissed + annotates applied/docs on read; handlers for dismiss/restore/mark_applied/
save_apply_docs.
- Unique decks per run (spend only for NEW jobs, no cache, no re-fetch+dedup band-aid):
per-(user,query) search cursor → LinkedIn page++ and Naukri incremental+stateKey (async
run path in apify_client, since the actor's crawl exceeds the run-sync window); seen-net
excludes already-shown ids (covers Foundit, which can't paginate).
- normalize: every board offsite-first (apply_url prefers the employer/ATS redirect over the
board listing) + offsite_apply flag; real company logos (_logo_url across Naukri logoPath /
LinkedIn company.logo / etc.).
- A1: fire the hard filter (Apify is precise); embedding vibe-ranker (embed.py)
blends with the floor-free white-box (rank.py) to sift ~90 -> top 18
- A2: Opus curator (curate.py) reads the 18 -> honest <=count + Gemini-voiced
report cards; graceful fallback to the white-box + templated cards
- LLM via opencode.ai/zen gateway (llm.py): Opus chat + direct-OpenAI embeddings
- Run LLM work off the event loop (asyncio.to_thread) so the Redis response publishes
- C1: dedicated Postgres (app/db/) persists the per-user feed; get_scout_feed
replays it so matches survive navigation/refresh
- match contract + report-card fields (schema.py); skills.py; tests/
