5 Commits

Author SHA1 Message Date
raulgupta
13faed0a06 Engine: signal-weighted India stack + cursorless pool + honest-card hardening
Boards (signal-weighted, ~150/sweep):
- Add TimesJobs (shahidirfan) — richest board: skills+salary+experience+offsite. Build/normalize +
  register; legacy city alias (New Delhi→Delhi, Bengaluru→Bangalore, Gurugram→Gurgaon) — verified live
  it returns 0 for "New Delhi", 35 for "Delhi".
- Swap LinkedIn harvestapi → curious_coder (structured industry/jobFunction/applicants); harvestapi kept
  registered as linkedin_v1 fallback.
- Drop Indeed from enabled (0% skills, redundant). Budgets: Foundit 40, TimesJobs 35, Naukri 30,
  LinkedIn 30, WorkIndia 15.

Pool — cursorless freshness + safety:
- REMOVE pagination/cursors: curious_coder only honors start=0 (start>0 → empty), proven; the rest are
  date feeds. Freshness now = boards' date-sort + the per-user seen-net + pool_save id-dedup.
- Exhaustion guard: a refill adding < POOL_MIN_NEW_PER_REFILL new jobs flags the (user,query) exhausted →
  the gate relaxes the floor instead of block-fetching dupes (makes the 100/90 floor safe on niche queries).
- Storage cap: pool_save trims beyond POOL_MAX_PER_QUERY freshest (kills DB swell). 72h TTL verified.
- Thresholds 100/90 (deep pool, pay-for-volume).

Scoring / cards:
- Evidence-based fallback prose: when Haiku's cards stage gives nothing, the card uses Opus's REAL
  dimension notes (not a generic "Strong on X") + coverage logging + generic salvage parser.
- "Skills & requirements" → "Skills fit" (consistent dimension labels).

Bug fixes:
- mark_seen dedups ids (was CardinalityViolationError on a duplicate id in one batch).

Tests: deep stack contracts + committed board-sample fixture (no cache-pollution flakiness) + offline
e2e + opt-in live e2e. 64 pass.
2026-06-26 15:56:27 +05:30
raulgupta
7c2640a999 Engine: honest high-90s scoring + warm pool + 2-stage curation (cost-down)
A search-quality + cost epic. Headline: genuine high-90s matches (88→96 on a senior
fintech-sales test), honestly, with no added spend.

SCORING — honest, computed, calibrated:
- rubric.py (NEW): the published rubric — 6 weighted dimensions + anchors. The overall is COMPUTED
  (rubric.aggregate), never model-emitted. A genuinely-aligned match arithmetically reaches the 90s.
- curate.py: TWO-STAGE — Opus SCORES the rubric dimensions (integrity-critical judgment), Haiku WRITES
  the report-card prose from Opus's evidence notes (cheap output, never judges). ~25% cheaper Opus +
  tighter calibration + better latency. Robust salvage parse; growth parse tolerant.
- rubric.calibrate: transparent presentation curve on the headline score — MONOTONIC, FLOOR-ANCHORED,
  UNIFORM (50→50, 70→74, 90→94, 95→97). A match% is a calibrated judgment; weak NEVER becomes strong,
  the breakdown stays raw evidence. Gated by CALIBRATION_ENABLED/GAMMA.
- MATCH_FLOOR=50 hard filter; floor checked on the RAW score before calibration.

RETRIEVAL — righter jobs (the honest score-lifter), cost-neutral:
- build_keyword(seniority, industry, skills): the recall boards (naukri-feed, foundit) + LinkedIn title
  now target right-level/industry/skill jobs instead of bare-title breadth → they align on more rubric
  dimensions → honestly higher scores. Verified live: no over-narrowing (138 jobs fetched, unchanged).
- Richer _profile_brief (resume skills/experience/education) so the rubric SEES requirements are met.

WARM POOL — stop re-paying Apify every search:
- UserJobPool: bank surplus fetched jobs per (user,query); serve from the pool, sweep only when fresh-
  unseen dips. Gate reorders at 80 / hard-floors at 70; background refill. (Saves Apify, not Opus.)

ACTORS / LATENCY (earlier in the epic):
- Indeed misceres(52s)→valig(7s); lean LLM payloads; per-board timeout. 48 tests pass.
2026-06-26 12:48:06 +05:30
raulgupta
be921559e7 Real Scout dashboard backend + per-board cursor rework
- get_scout_stats aggregator (app/engine/stats): assembles the REAL Summary metrics —
  funnel + cohort engagement rank + active-window (activity timestamps by hour) from our DB,
  accumulating salary band (avg of each deck's peak, ₹L), match/competition stats from the
  feed, Momentum/QX + Q-Score trend (qscore-service), day streak (user-service). Honest:
  unsourced cards return None so the UI omits/locks them, never faked. posted_date extractor.
- Activity tracking: viewed/saved flags + search_count → funnel (Matches→Viewed→Shortlisted→
  Applied) + engagement percentile. matchesFound = all-time count.
- Per-board search cursors {board: page} (replaces the single cursor): only boards that truly
  paginate (LinkedIn) get one; cursor = LAST page fetched (1st search of a new query → 1).
  Resets on query change OR >24h (boards refresh ~daily). Dropped Naukri incremental/stateKey
  (opaque, exhausting, cross-account dedup state) — dedup is the PER-USER seen-net only.
- tests: stats helpers (posting-age, histogram, active-window, engagement score).
2026-06-21 12:12:28 +05:30
raulgupta
75b2e538ef Manual-apply backend: per-opportunity state, unique-decks engine, offsite-first + logos
- OpportunityState model + repo: persist applied / dismissed / restored status and the
  generated resume-builder doc ids (never re-pay to re-tailor). annotate_and_filter drops
  dismissed + annotates applied/docs on read; handlers for dismiss/restore/mark_applied/
  save_apply_docs.
- Unique decks per run (spend only for NEW jobs, no cache, no re-fetch+dedup band-aid):
  per-(user,query) search cursor → LinkedIn page++ and Naukri incremental+stateKey (async
  run path in apify_client, since the actor's crawl exceeds the run-sync window); seen-net
  excludes already-shown ids (covers Foundit, which can't paginate).
- normalize: every board offsite-first (apply_url prefers the employer/ATS redirect over the
  board listing) + offsite_apply flag; real company logos (_logo_url across Naukri logoPath /
  LinkedIn company.logo / etc.).
2026-06-20 17:52:06 +05:30
raulgupta
b1e9cdd182 Engine v2: embeddings sift + Opus curator + Postgres feed persistence
- A1: fire the hard filter (Apify is precise); embedding vibe-ranker (embed.py)
  blends with the floor-free white-box (rank.py) to sift ~90 -> top 18
- A2: Opus curator (curate.py) reads the 18 -> honest <=count + Gemini-voiced
  report cards; graceful fallback to the white-box + templated cards
- LLM via opencode.ai/zen gateway (llm.py): Opus chat + direct-OpenAI embeddings
- Run LLM work off the event loop (asyncio.to_thread) so the Redis response publishes
- C1: dedicated Postgres (app/db/) persists the per-user feed; get_scout_feed
  replays it so matches survive navigation/refresh
- match contract + report-card fields (schema.py); skills.py; tests/
2026-06-19 15:51:16 +05:30