Engine: signal-weighted India stack + cursorless pool + honest-card hardening

Boards (signal-weighted, ~150/sweep):
- Add TimesJobs (shahidirfan) — richest board: skills+salary+experience+offsite. Build/normalize +
  register; legacy city alias (New Delhi→Delhi, Bengaluru→Bangalore, Gurugram→Gurgaon) — verified live
  it returns 0 for "New Delhi", 35 for "Delhi".
- Swap LinkedIn harvestapi → curious_coder (structured industry/jobFunction/applicants); harvestapi kept
  registered as linkedin_v1 fallback.
- Drop Indeed from enabled (0% skills, redundant). Budgets: Foundit 40, TimesJobs 35, Naukri 30,
  LinkedIn 30, WorkIndia 15.

Pool — cursorless freshness + safety:
- REMOVE pagination/cursors: curious_coder only honors start=0 (start>0 → empty), proven; the rest are
  date feeds. Freshness now = boards' date-sort + the per-user seen-net + pool_save id-dedup.
- Exhaustion guard: a refill adding < POOL_MIN_NEW_PER_REFILL new jobs flags the (user,query) exhausted →
  the gate relaxes the floor instead of block-fetching dupes (makes the 100/90 floor safe on niche queries).
- Storage cap: pool_save trims beyond POOL_MAX_PER_QUERY freshest (kills DB swell). 72h TTL verified.
- Thresholds 100/90 (deep pool, pay-for-volume).

Scoring / cards:
- Evidence-based fallback prose: when Haiku's cards stage gives nothing, the card uses Opus's REAL
  dimension notes (not a generic "Strong on X") + coverage logging + generic salvage parser.
- "Skills & requirements" → "Skills fit" (consistent dimension labels).

Bug fixes:
- mark_seen dedups ids (was CardinalityViolationError on a duplicate id in one batch).

Tests: deep stack contracts + committed board-sample fixture (no cache-pollution flakiness) + offline
e2e + opt-in live e2e. 64 pass.

app/agent/session.py

@@ -120,22 +120,23 @@ class MatchmakingAgentSession:
         # Bank surplus fetched jobs per (user, query); serve from the pool and re-pay Apify only when the
         # fresh-unseen count dips below the floor. Reorder EARLY (background top-up) so it never runs thin.
         pooled = await _repo.pool_load_fresh_unseen(self.user_id, sig, seen) if s.POOL_ENABLED else []
-        decision = search.pool_decision(len(pooled), fresh, s.POOL_SAFETY_FLOOR, s.POOL_REORDER_AT)
-        cursors = await _repo.get_search_cursors(self.user_id, sig)
+        exhausted = await _repo.is_pool_exhausted(self.user_id, sig) if s.POOL_ENABLED else False
+        decision = search.pool_decision(len(pooled), fresh, s.POOL_SAFETY_FLOOR, s.POOL_REORDER_AT, exhausted)
 
         if decision == "sweep":
-            # Pool below the hard floor (or forced): blocking sweep → bank ALL of it → advance cursors.
+            # Pool below the hard floor (or forced): blocking sweep → bank ALL of it. No cursors — every
+            # fetch pulls the freshest page; the seen-net dedups + daily board refresh brings new jobs.
             await self.push("agent_thinking",
                             message="Fetching fresh roles across job boards…" if fresh
                             else "Scanning live roles across job boards…")
-            prefs["_cursors"] = cursors
             sweep = await self._with_heartbeat(
                 search.run_sweep(prefs, fresh=fresh),
                 ["Scanning live roles across the boards…", "Pulling the latest Naukri + LinkedIn + Foundit roles…",
                  "Gathering fresh postings for you…"])
-            await _repo.pool_save(self.user_id, sig, sweep["opportunities"])
-            cursors = {b: int(cursors.get(b, 0)) + 1
-                       for b in (sweep.get("sources") or {}) if b in search.PAGINATING_BOARDS}
+            new_count = await _repo.pool_save(self.user_id, sig, sweep["opportunities"])
+            # tapped out? a sweep that added almost nothing new → flag it so we stop block-fetching dupes.
+            if new_count < s.POOL_MIN_NEW_PER_REFILL:
+                await _repo.mark_pool_exhausted(self.user_id, sig)
             pooled = await _repo.pool_load_fresh_unseen(self.user_id, sig, seen)
             sources = sweep["sources"]
         else:
@@ -167,11 +168,10 @@ class MatchmakingAgentSession:
         result = {"opportunities": curated, "sources": sources,
                   "shortlisted": len(curated), "scanned": dbg["scored"], "engine": engine,
                   "has_profile": bool(user_context and user_context.get("skills"))}
-        # Persist the deck + the cursors (advanced only if we swept this run; unchanged on a pool-serve —
-        # the cursor advances exactly once per real fetch, foreground or background).
+        # Persist the deck (no cursors — pagination removed; freshness is date-sort + the seen-net).
         clean_prefs = {k: v for k, v in prefs.items()
                        if k not in ("user_context", "_fresh", "_cursors")}
-        await _repo.save_feed(self.user_id, clean_prefs, result, query_sig=sig, cursors=cursors)
+        await _repo.save_feed(self.user_id, clean_prefs, result, query_sig=sig)
         # Accumulate the salary band — each deck contributes its peak salary, averaged over decks
         # (survives decks that disclose nothing, like Naukri "Not disclosed").
         await _repo.update_salary_band(self.user_id, [o.get("salary_lpa") for o in (result["opportunities"] or [])])
@@ -189,18 +189,18 @@ class MatchmakingAgentSession:
             asyncio.create_task(self._background_refill(clean_prefs, sig))
 
     async def _background_refill(self, prefs: dict, sig: str):
-        """Proactive pool top-up: sweep the next page in the background + bank it, so the fresh-unseen
-        count climbs back above the reorder mark before the user's next search. Best-effort, non-fatal."""
+        """Proactive pool top-up: re-fetch the freshest jobs in the background + bank the new ones, so the
+        fresh-unseen count climbs back above the reorder mark before the user's next search. No cursors —
+        the seen-net dedups and daily board refresh supplies new jobs. Best-effort, non-fatal."""
         from app.engine import search
         from app.db import repo as _repo
         try:
-            cursors = await _repo.get_search_cursors(self.user_id, sig)
-            sweep = await search.run_sweep({**prefs, "_cursors": cursors}, fresh=True)
-            await _repo.pool_save(self.user_id, sig, sweep["opportunities"])
-            advanced = {b: int(cursors.get(b, 0)) + 1
-                        for b in (sweep.get("sources") or {}) if b in search.PAGINATING_BOARDS}
-            await _repo.save_cursors(self.user_id, sig, advanced)
-            logger.info("pool refill (bg): +%d jobs, sig=%s", len(sweep.get("opportunities") or []), sig[:8])
+            from app.config import get_settings
+            sweep = await search.run_sweep(prefs, fresh=True)
+            new_count = await _repo.pool_save(self.user_id, sig, sweep["opportunities"])
+            if new_count < get_settings().POOL_MIN_NEW_PER_REFILL:   # tapped out → relax the floor
+                await _repo.mark_pool_exhausted(self.user_id, sig)
+            logger.info("pool refill (bg): +%d NEW jobs, sig=%s", new_count, sig[:8])
         except Exception as e:  # noqa: BLE001
             logger.warning("background refill failed (non-fatal): %s", e)
 

app/config.py

@@ -58,28 +58,36 @@ class Settings(BaseSettings):
     # Active boards — "Balanced" India stack: all three filter to the exact city at the board
     # (hyper-relevant) and surface offsite apply links. Naukri=blackfalcondata feed,
     # Foundit=Monster India, LinkedIn=geoId/city. Others (ats/indeed/naukri_v1) stay registered, off.
-    BOARDS_ENABLED: str = "naukri,foundit,linkedin,indeed,workindia"
+    BOARDS_ENABLED: str = "naukri,foundit,linkedin,timesjobs,workindia"   # signal-weighted; indeed dropped (0% skills, redundant)
     BOARD_TIMEOUT_S: float = 55.0      # per-board cap — a slow/hung board is skipped, the search stays fast
 
     # Warm pool ("the shelf") — bank surplus fetched jobs per (user, query) so we don't re-pay Apify for
     # a full sweep on every search. Serve from the pool; refill only when it dips. Reorder EARLY (top up
     # in the background) and hard-floor a blocking fetch so the pool never runs thin.
     POOL_ENABLED: bool = True
-    POOL_REORDER_AT: int = 80          # fresh-unseen < this → background top-up (while still serving)
-    POOL_SAFETY_FLOOR: int = 70        # fresh-unseen < this → BLOCK + fetch before serving (hard floor)
+    POOL_REORDER_AT: int = 100         # fresh-unseen < this → background top-up (while still serving)
+    POOL_SAFETY_FLOOR: int = 90        # fresh-unseen < this → BLOCK + fetch before serving (hard floor)
+                                       # 100/90 keeps a DEEP pool (richer decks) at the cost of more
+                                       # frequent refills — we pay for volume, deliberately.
     POOL_TTL_HOURS: int = 72           # banked jobs older than this are stale (likely filled) → dropped
-    POOL_MAX_PER_QUERY: int = 400      # cap the bank per (user, query) so it can't grow unbounded
+    POOL_MAX_PER_QUERY: int = 400      # cap the bank per (user, query) — STORAGE cap (kills DB swell) + load cap
+    # Exhaustion guard — a refill that adds fewer than this many NEW jobs means the query is tapped out;
+    # we then RELAX the floor (serve what's pooled, stop block-fetching duplicates) until it expires.
+    POOL_MIN_NEW_PER_REFILL: int = 8
+    POOL_EXHAUSTION_TTL_HOURS: int = 12   # re-try a tapped query after this (boards post new jobs ~daily)
 
     # Per-board fetch budget (the #1 Apify cost lever). Demo-sized to conserve credits.
-    NAUKRI_MAX_JOBS: int = 15       # blackfalcondata maxResults (no floor); fetchDetails=True for offsite apply links
-    FOUNDIT_MAX_JOBS: int = 50
-    LINKEDIN_MAX_JOBS: int = 25
+    # Signal-weighted budgets → ~150 jobs/sweep, tilted to the high-signal boards.
+    NAUKRI_MAX_JOBS: int = 30       # blackfalcondata maxResults; fetchDetails=True for offsite apply links
+    FOUNDIT_MAX_JOBS: int = 40
+    LINKEDIN_MAX_JOBS: int = 30
+    TIMESJOBS_MAX_JOBS: int = 35    # shahidirfan — the richest board (skills+salary+experience)
     # ATS lane (off by default) — company-seeded global/remote "dream companies".
     ATS_MAX_PER_COMPANY: int = 4
     ATS_COMPANIES: str = "stripe,databricks,gitlab,figma,ramp,notion,razorpay,zerodha"
     INDEED_MAX_JOBS: int = 30
     WELLFOUND_MAX_JOBS: int = 15     # blackfalcondata/wellfound-scraper — registered, OFF (US-heavy)
-    WORKINDIA_MAX_JOBS: int = 20     # shahidirfan/workindia-jobs-scraper — India blue/grey-collar non-tech
+    WORKINDIA_MAX_JOBS: int = 15     # shahidirfan/workindia-jobs-scraper — India blue/grey-collar (niche, low yield)
 
     # QScore (competence proxy) — consumed, not computed. Dashboard Momentum/QX reads it via REST.
     QSCORE_BASE_URL: str = "http://localhost:8004"

app/db/models.py

@@ -27,6 +27,9 @@ class UserFeed(Base):
     # Dedup is purely per-user (the seen-net below) — never cross-user, never opaque actor-side state.
     query_sig: Mapped[str] = mapped_column(String, default="")
     cursors: Mapped[dict] = mapped_column(JSON, default=dict)
+    # Set when a refill for `query_sig` returned almost no NEW jobs (the query is tapped out). While set
+    # (within POOL_EXHAUSTION_TTL), the pool gate relaxes the floor instead of block-fetching duplicates.
+    exhausted_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
     search_count: Mapped[int] = mapped_column(Integer, default=0, server_default="0")  # total searches run (engagement)
     # Salary band = AVERAGE of each deck's HIGHEST disclosed salary (₹L). Each deck contributes one
     # peak; the band is the running mean of those peaks. Survives decks that disclose nothing.

app/db/repo.py

@@ -346,7 +346,10 @@ async def mark_seen(user_id: str, opportunity_ids: list[str]) -> None:
     factory = session_factory()
     if factory is None or not user_id or not opportunity_ids:
         return
-    rows = [{"user_id": user_id, "opportunity_id": oid, "seen": True} for oid in opportunity_ids if oid]
+    # DEDUP the ids — a duplicate (user, oppid) in ONE insert trips PG's "ON CONFLICT cannot affect row
+    # a second time" (CardinalityViolationError). A deck can carry the same id twice (cross-board dupes).
+    seen_ids = list(dict.fromkeys(o for o in opportunity_ids if o))
+    rows = [{"user_id": user_id, "opportunity_id": oid, "seen": True} for oid in seen_ids]
     if not rows:
         return
     try:
@@ -363,39 +366,96 @@ async def mark_seen(user_id: str, opportunity_ids: list[str]) -> None:
 
 
 # ── Warm pool (the shelf) ────────────────────────────────────────────────────
-async def pool_save(user_id: str, query_sig: str, jobs: list[dict]) -> None:
-    """Bank fetched jobs into the per-(user, query) pool (upsert by job id; refresh pooled_at). Then
-    sweep out anything older than the TTL so the pool stays fresh + bounded. No-op if the DB is off."""
+async def pool_save(user_id: str, query_sig: str, jobs: list[dict]) -> int:
+    """Bank fetched jobs into the per-(user, query) pool (upsert by job id; refresh pooled_at). Returns
+    the count of GENUINELY NEW job ids (not already pooled) — the exhaustion signal. Then TTL-sweeps stale
+    rows AND caps storage to POOL_MAX_PER_QUERY freshest (kills DB swell). 0 if the DB is off."""
     from datetime import datetime, timedelta, timezone
     from app.config import get_settings
     factory = session_factory()
     if factory is None or not user_id or not jobs:
-        return
+        return 0
     rows = [{"user_id": user_id, "query_sig": query_sig, "job_id": j["id"], "job": j}
             for j in jobs if j.get("id")]
     if not rows:
-        return
+        return 0
     s = get_settings()
     cutoff = datetime.now(timezone.utc) - timedelta(hours=s.POOL_TTL_HOURS)
+    incoming = {r["job_id"] for r in rows}
     try:
         async with factory() as sess:
+            # how many of these are NEW (not already pooled for this query)?
+            existing = set((await sess.execute(
+                select(UserJobPool.job_id).where(
+                    (UserJobPool.user_id == user_id) & (UserJobPool.query_sig == query_sig)
+                    & (UserJobPool.job_id.in_(incoming)))
+            )).scalars().all())
+            new_count = len(incoming - existing)
+
             stmt = pg_insert(UserJobPool).values(rows)
             stmt = stmt.on_conflict_do_update(
                 index_elements=[UserJobPool.user_id, UserJobPool.query_sig, UserJobPool.job_id],
                 set_={"job": stmt.excluded.job, "pooled_at": func.now()},
             )
             await sess.execute(stmt)
-            # TTL sweep for this (user, query) — drop stale postings (likely filled).
-            await sess.execute(
-                UserJobPool.__table__.delete().where(
-                    (UserJobPool.user_id == user_id)
-                    & (UserJobPool.query_sig == query_sig)
-                    & (UserJobPool.pooled_at < cutoff)
-                )
-            )
+            # TTL sweep (stale postings, likely filled).
+            await sess.execute(UserJobPool.__table__.delete().where(
+                (UserJobPool.user_id == user_id) & (UserJobPool.query_sig == query_sig)
+                & (UserJobPool.pooled_at < cutoff)))
+            # STORAGE cap: keep only the POOL_MAX_PER_QUERY freshest for this (user, query).
+            keep = (await sess.execute(
+                select(UserJobPool.job_id)
+                .where((UserJobPool.user_id == user_id) & (UserJobPool.query_sig == query_sig))
+                .order_by(UserJobPool.pooled_at.desc()).limit(s.POOL_MAX_PER_QUERY)
+            )).scalars().all()
+            if len(keep) >= s.POOL_MAX_PER_QUERY:
+                await sess.execute(UserJobPool.__table__.delete().where(
+                    (UserJobPool.user_id == user_id) & (UserJobPool.query_sig == query_sig)
+                    & (UserJobPool.job_id.notin_(keep))))
             await sess.commit()
+            return new_count
     except Exception as e:  # noqa: BLE001
         logger.warning("pool_save failed (non-fatal): %s", e)
+        return 0
+
+
+async def mark_pool_exhausted(user_id: str, query_sig: str) -> None:
+    """Flag this (user, query) as tapped out (a refill added almost no new jobs) → the gate relaxes the
+    floor until POOL_EXHAUSTION_TTL passes. Upsert on the one-row-per-user feed."""
+    factory = session_factory()
+    if factory is None or not user_id:
+        return
+    try:
+        async with factory() as s:
+            stmt = pg_insert(UserFeed).values(user_id=user_id, query_sig=query_sig, exhausted_at=func.now())
+            stmt = stmt.on_conflict_do_update(
+                index_elements=[UserFeed.user_id],
+                set_={"exhausted_at": func.now(), "query_sig": query_sig},
+            )
+            await s.execute(stmt)
+            await s.commit()
+    except Exception as e:  # noqa: BLE001
+        logger.warning("mark_pool_exhausted failed (non-fatal): %s", e)
+
+
+async def is_pool_exhausted(user_id: str, query_sig: str) -> bool:
+    """True if this exact query was marked exhausted within POOL_EXHAUSTION_TTL (boards refresh ~daily,
+    so we re-try after it expires). False on any other query / DB off / expired."""
+    from datetime import datetime, timedelta, timezone
+    from app.config import get_settings
+    factory = session_factory()
+    if factory is None or not user_id:
+        return False
+    try:
+        async with factory() as s:
+            row = (await s.execute(select(UserFeed).where(UserFeed.user_id == user_id))).scalar_one_or_none()
+    except Exception as e:  # noqa: BLE001
+        logger.warning("is_pool_exhausted failed (non-fatal): %s", e)
+        return False
+    if not row or row.query_sig != query_sig or not row.exhausted_at:
+        return False
+    ttl = get_settings().POOL_EXHAUSTION_TTL_HOURS
+    return (datetime.now(timezone.utc) - row.exhausted_at).total_seconds() < ttl * 3600
 
 
 async def pool_load_fresh_unseen(user_id: str, query_sig: str, seen: set[str]) -> list[dict]:

app/engine/board_adapters/adapters.py

@@ -7,6 +7,7 @@ System knobs (budget / freshness) are arguments with sane defaults, not user inp
 """
 from __future__ import annotations
 from typing import Any
+from urllib.parse import urlencode
 
 from . import coerce as C
 
@@ -86,13 +87,15 @@ def build_naukri_feed_input(prefs: Prefs, *, max_jobs: int = 50, fetch_details:
     }
 
 
-# ── Foundit / Monster India (city-precise; apply_url is the offsite redirect) ──
-# Foundit uses legacy city names — it returns 0 for "New Delhi"/"Bengaluru".
-_FOUNDIT_CITY = {
+# Legacy India job boards (Foundit/Monster, TimesJobs) use OLD city names — they return 0 for the
+# modern "New Delhi"/"Bengaluru"/"Gurugram". Map to their canonical names. (Verified live: TimesJobs
+# returns 0 for "New Delhi", 15 for "Delhi".)
+_LEGACY_IN_CITY = {
     "new delhi": "Delhi", "delhi ncr": "Delhi", "bengaluru": "Bangalore",
     "gurugram": "Gurgaon", "mumbai": "Mumbai", "pune": "Pune", "hyderabad": "Hyderabad",
     "chennai": "Chennai", "kolkata": "Kolkata", "noida": "Noida",
 }
+_FOUNDIT_CITY = _LEGACY_IN_CITY   # back-compat alias
 
 
 def build_foundit_input(prefs: Prefs, *, results_wanted: int = 50) -> dict:
@@ -215,6 +218,36 @@ def build_ats_input(prefs: Prefs, *, companies: list[dict], recent_days: int = 3
 
 
 # ── WorkIndia (India blue/grey-collar — the non-tech workhorse: sales, BPO, retail, field, ops) ──
+def build_linkedin_cc_input(prefs: Prefs, *, count: int = 30) -> dict:
+    # curious_coder~linkedin-jobs-scraper takes LinkedIn SEARCH URLs (structured industry+jobFunction+
+    # applicants). NO pagination — the actor only honors start=0 (start>0 returns empty). Every fetch
+    # pulls the freshest page-1 by date; the per-user seen-net dedups, and daily board refresh feeds new jobs.
+    kw = C.build_keyword(prefs, seniority=True, industry=True)   # targeted search string
+    loc = next((f"{lp['city']}, {lp['country']}" if lp["country"] else lp["city"]
+                for lp in C.parsed_locations(prefs) if lp["city"]), None) \
+        or next((lp["country"] for lp in C.parsed_locations(prefs) if lp["country"]), "India")
+    q = urlencode({"keywords": kw, "location": loc, "sortBy": "DD"})  # DD = newest first
+    return {"urls": [f"https://www.linkedin.com/jobs/search/?{q}"],
+            "count": max(10, count), "scrapeCompany": False}   # count min 10 (actor constraint)
+
+
+def build_timesjobs_input(prefs: Prefs, *, results_wanted: int = 35) -> dict:
+    # shahidirfan/Timesjobs-Scraper — FULL signal (skills+salary+experience+offsite external_job_url).
+    # keyword + city + an experience BAND ("5-10") for right-level targeting. Clean keyword input, no page.
+    city = next((lp["city"] for lp in C.parsed_locations(prefs) if lp["city"]), "")
+    city = _LEGACY_IN_CITY.get(city.lower(), city)   # TimesJobs needs "Delhi" not "New Delhi" (verified)
+    out: dict[str, Any] = {
+        "keyword": C.build_keyword(prefs, seniority=True, industry=True, skills=True),
+        "results_wanted": results_wanted,
+    }
+    if city:
+        out["location"] = city
+    yrs = C.years_of(prefs)
+    if yrs is not None:
+        out["experience"] = f"{max(0, yrs - 2)}-{yrs + 3}"   # a band around the candidate's years
+    return out
+
+
 def build_workindia_input(prefs: Prefs, *, results_wanted: int = 20) -> dict:
     # keyword + city; includeDetails=True pulls the real skills + full description (verified live Phase 0).
     # No page param → per-user seen-net dedups. City is a lowercased name slug ("mumbai", "pune").

app/engine/curate.py

@@ -121,12 +121,14 @@ def _parse_json(text: str) -> dict:
     return json.loads(t)
 
 
-def _salvage_kept(text: str) -> list[dict]:
-    """Recover valid kept-job objects from a malformed/truncated Opus JSON: brace-scan each top-level
-    {...} after "kept" and json.loads it on its own, skipping the one broken/cut-off object. Stops a
-    single bad character (or a max-tokens truncation) from dropping the WHOLE shortlist to the fallback."""
-    i = text.find('"kept"')
-    s = text[i:] if i != -1 else text
+def _salvage_objects(text: str, *, after: str | None = None) -> list[dict]:
+    """Recover valid objects from malformed/truncated model JSON: brace-scan each top-level {...} and
+    json.loads it on its own, skipping the one broken/cut-off object. `after` anchors the scan past a key
+    (e.g. '"kept"'); omit it to scan the whole text (the cards stage uses '"cards"', not '"kept"')."""
+    s = text
+    if after:
+        i = text.find(after)
+        s = text[i:] if i != -1 else text
     out: list[dict] = []
     depth, start = 0, None
     for idx, ch in enumerate(s):
@@ -145,6 +147,11 @@ def _salvage_kept(text: str) -> list[dict]:
     return out
 
 
+def _salvage_kept(text: str) -> list[dict]:
+    """Score-stage salvage — recover kept-job objects (anchored after the '"kept"' key)."""
+    return _salvage_objects(text, after='"kept"')
+
+
 def _scored_card(k: dict) -> dict | None:
     """Stage-1 result → a partial card carrying the COMPUTED overall + the rubric breakdown (no prose yet).
     Returns None for a malformed entry (no dimensions). The engine computes the score — never the model."""
@@ -173,13 +180,21 @@ def _to_match(sc: dict, prose: dict | None) -> MatchResult:
             growth = Growth(text=g["text"], **{"from": int(g["from"]), "to": int(g["to"])})
         except (KeyError, ValueError, TypeError):
             growth = None
-    top = max(sc["dims"], key=lambda d: d["score"]) if sc["dims"] else None
-    one_line = prose.get("one_line") or (f"Strong on {top['name'].lower()}." if top else "Curated by Scout.")
+    # Fallback prose (when Haiku gave nothing) is built from Opus's REAL evidence notes — never a generic
+    # "Strong on X". A graceful card with real content, even if the writer stage failed.
+    strong = [d for d in sc["dims"] if d.get("note") and d["score"] >= 72]
+    weak = min((d for d in sc["dims"] if d.get("note")), key=lambda d: d["score"], default=None)
+    fb_line = "; ".join(d["note"] for d in strong[:2]) if strong else (
+        (max(sc["dims"], key=lambda d: d["score"])["note"] if sc["dims"] and sc["dims"][0].get("note")
+         else "A solid match worth a look."))
+    fb_coach = (f"Shore up {weak['name'].lower()}: {weak['note']}" if weak and weak["score"] < 72 else
+                "Lead with your strongest, most relevant experience.")
+    one_line = prose.get("one_line") or fb_line
     return MatchResult(
         score=sc["overall"], fit=sc["fit"],
         archetype=prose.get("archetype") or ("The Strong Match" if sc["overall"] >= 80 else "Worth a Look"),
         one_line=one_line, reason=one_line,
-        breakdown=dims, growth=growth, coach_note=prose.get("coach_note"),
+        breakdown=dims, growth=growth, coach_note=prose.get("coach_note") or fb_coach,
         proofReady=any(d.level == "Strong" for d in dims), factors={},
     )
 
@@ -239,13 +254,18 @@ def _cards_stage(prefs: dict, ctx: dict | None, scored: list[dict], by_id: dict,
         )
         content = resp.choices[0].message.content
     except Exception as e:  # noqa: BLE001 — prose is non-critical; degrade to templated cards
-        logger.warning("cards stage (Haiku) failed, using templated prose: %s", e)
+        logger.warning("cards stage (Haiku) call failed, using evidence-based fallback prose: %s", e)
         return {}
     try:
         cards = _parse_json(content).get("cards") or []
     except Exception:  # noqa: BLE001
-        cards = _salvage_kept(content)   # brace-scanner works for any [{…}] list
-    return {c.get("id"): c for c in cards if c.get("id")}
+        cards = _salvage_objects(content)   # brace-scan any [{…}] (works regardless of the wrapper key)
+    prose = {c.get("id"): c for c in cards if c.get("id")}
+    # Coverage telemetry — a silent partial/empty cards stage is exactly what shipped generic reads.
+    if len(prose) < len(scored):
+        logger.warning("cards stage covered %d/%d jobs — %d will use evidence-based fallback prose",
+                       len(prose), len(scored), len(scored) - len(prose))
+    return prose
 
 
 def curate(prefs: dict, ctx: dict | None, sifted_jobs: list[dict]) -> list[dict] | None:

app/engine/normalize.py

@@ -470,6 +470,123 @@ def foundit_to_scoutjob(it: dict, board: str = "Foundit") -> dict | None:
     }
 
 
+# LinkedIn seniorityLevel → our band (often "Not Applicable" for India → None, then title-inferred centrally).
+_LI_SENIORITY = {"internship": "junior", "entry level": "junior", "associate": "mid",
+                 "mid-senior level": "senior", "director": "lead", "executive": "lead"}
+
+
+def linkedin_cc_to_scoutjob(it: dict, board: str = "LinkedIn") -> dict | None:
+    """curious_coder~linkedin-jobs-scraper — structured INDUSTRY + jobFunction (role) + applicantsCount.
+    (seniorityLevel is present but often 'Not Applicable' for India; we map it when real, else infer.)"""
+    title = (it.get("title") or "").strip()
+    org = (it.get("companyName") or "").strip()
+    if not title or not org:
+        return None
+    # location is "City, State, Country" — first part is the city, last is the country.
+    parts = [p.strip() for p in str(it.get("location") or "").split(",") if p.strip()]
+    city = parts[0] if parts else None
+    country = parts[-1] if len(parts) >= 2 else "India"
+    sen = _LI_SENIORITY.get(str(it.get("seniorityLevel") or "").strip().lower())   # None if "Not Applicable"
+    inds = [s.strip() for s in str(it.get("industries") or "").split(",") if s.strip()]
+    fn = (it.get("jobFunction") or "").strip() or None
+    etype = (it.get("employmentType") or "").strip()
+    mode = "remote" if "remote" in etype.lower() else None
+    ext = it.get("applyUrl")
+    offsite = bool(ext and "linkedin.com" not in ext)        # applyUrl (when present) = offsite employer
+    apply_url = ext if offsite else it.get("link")           # else the LinkedIn job page
+    details = _compact({
+        "description": _plain(it.get("descriptionText") or it.get("descriptionHtml")),
+        "industry": inds[0] if inds else None,
+        "role_category": fn,
+        "employment_type": etype or None,
+    })
+    return {
+        "id": f"linkedin-{it.get('id')}",
+        "title": title,
+        "organization": org,
+        "logo": _logo(org, it.get("companyLogo")),
+        "posted_date": it.get("postedAt"),
+        "location_city": city,
+        "location_country": country,
+        "location_mode": mode,
+        "matchScore": 0,
+        "salary_lpa": None,
+        "payLabel": "Not disclosed",
+        "seniority_level": sen,
+        "required_skills": [],          # LinkedIn exposes no skill tags; description + jobFunction carry it
+        "industry": inds[0] if inds else None,
+        "tags": [fn] if fn else [],
+        "applicants": it.get("applicantsCount"),
+        "note": f"Live from {board}",
+        "apply_url": apply_url,
+        "offsite_apply": offsite,
+        "details": details,
+        "qx": 10,
+    }
+
+
+def timesjobs_to_scoutjob(it: dict, board: str = "TimesJobs") -> dict | None:
+    """shahidirfan/Timesjobs-Scraper — the richest India board: skills + salary + experience +
+    description + an OFFSITE employer link (external_job_url). Clean structured fields."""
+    title = (it.get("title") or "").strip()
+    title = re.sub(r"^title:\s*", "", title, flags=re.I).strip()   # some rows carry a stray "Title:" prefix
+    org = (it.get("company") or "").strip()
+    if not title or not org:
+        return None
+    if org.islower():
+        org = org.title()
+    # location is "City" or "City, City2" — take the first concrete city.
+    city = next((p.strip() for p in str(it.get("location") or "").split(",") if p.strip()), None)
+    skills = it.get("skills") or []
+    if isinstance(skills, str):
+        skills = [s.strip() for s in skills.split(",") if s.strip()]
+    skills = [str(s) for s in skills if s]
+    # experience "6 - 10 Yrs" → seniority from the LOW end of the band.
+    seniority = None
+    m = re.search(r"\d+", str(it.get("experience") or ""))
+    if m:
+        seniority = _seniority_from_years(int(m.group()))
+    # salary is usually the text "As per industry standard"; surface a ₹L band only if a real number is there.
+    sal_raw = str(it.get("salary") or "")
+    lpa, pay = (None, sal_raw or "Not disclosed")
+    if re.search(r"\d", sal_raw):
+        lpa, pay = _salary_from_amount(_digits(sal_raw), per_year=True)
+    types = [t.strip() for t in str(it.get("job_type") or "").split(",") if t.strip()]
+    mode = "remote" if any("remote" in t.lower() for t in types) else None
+    ext = it.get("external_job_url")
+    offsite = bool(ext and "timesjobs.com" not in (ext or ""))   # external_job_url = real employer site
+    apply_url = ext if offsite else it.get("url")
+    details = _compact({
+        "description": _plain(it.get("description_text") or it.get("description_html")),
+        "experience": it.get("experience"),
+        "employment_type": ", ".join(types) or None,
+        "company_about": _plain(it.get("company_description"), limit=400),
+        "skills": skills,
+    })
+    return {
+        "id": f"timesjobs-{it.get('job_id')}",
+        "title": title,
+        "organization": org,
+        "logo": _logo(org),
+        "posted_date": it.get("date_posted"),
+        "location_city": city,
+        "location_country": "India",
+        "location_mode": mode,
+        "matchScore": 0,
+        "salary_lpa": lpa,
+        "payLabel": pay,
+        "seniority_level": seniority,
+        "required_skills": skills[:8],
+        "tags": skills[:3],
+        "applicants": None,
+        "note": f"Live from {board}",
+        "apply_url": apply_url,
+        "offsite_apply": offsite,
+        "details": details,
+        "qx": 10,
+    }
+
+
 def workindia_to_scoutjob(it: dict, board: str = "WorkIndia") -> dict | None:
     """shahidirfan/workindia-jobs-scraper (includeDetails=True) — India blue/grey-collar non-tech.
     Verified live (Phase 0): returns real `skills` + `profile_job_description`. Apply is on-platform."""
@@ -577,10 +694,12 @@ def wellfound_to_scoutjob(it: dict, board: str = "Wellfound") -> dict | None:
 NORMALIZERS = {
     "naukri": naukri_feed_to_scoutjob,   # blackfalcondata (primary)
     "foundit": foundit_to_scoutjob,
-    "linkedin": linkedin_to_scoutjob,
+    "linkedin": linkedin_cc_to_scoutjob,   # curious_coder (structured industry+jobFunction+applicants)
+    "linkedin_v1": linkedin_to_scoutjob,   # harvestapi (fallback)
     "naukri_v1": naukri_to_scoutjob,     # muhammetakkurtt (fallback)
     "indeed": indeed_to_scoutjob,
     "ats": ats_to_scoutjob,
     "wellfound": wellfound_to_scoutjob,    # blackfalcondata (startup roles)
     "workindia": workindia_to_scoutjob,    # shahidirfan (India blue/grey-collar non-tech)
+    "timesjobs": timesjobs_to_scoutjob,    # shahidirfan (India — full signal: skills+salary+experience)
 }

app/engine/rubric.py

@@ -11,7 +11,7 @@ from __future__ import annotations
 DIMENSIONS = [
     {"key": "role", "label": "Role fit", "weight": 0.25,
      "anchor": "100 = same function · 75 = adjacent function · 45 = different function · 15 = unrelated"},
-    {"key": "skills", "label": "Skills & requirements", "weight": 0.25,
+    {"key": "skills", "label": "Skills fit", "weight": 0.25,
      "anchor": "share of the job's stated requirements the candidate genuinely meets (100 = all · 50 = half · 0 = none)"},
     {"key": "seniority", "label": "Seniority fit", "weight": 0.15,
      "anchor": "100 = same level · 70 = one band off · 35 = two+ bands off"},

app/engine/search.py

@@ -21,13 +21,22 @@ def _ats_companies() -> list[dict]:
     return [{"company": c.strip()} for c in get_settings().ATS_COMPANIES.split(",") if c.strip()]
 
 
-def pool_decision(n_fresh: int, force_fresh: bool, floor: int, reorder: int) -> str:
+def pool_decision(n_fresh: int, force_fresh: bool, floor: int, reorder: int,
+                  exhausted: bool = False) -> str:
     """The warm-pool gate. Given how many FRESH, UNSEEN jobs are banked, decide where candidates come from:
       'sweep'       — blocking Apify fetch (pool below the hard floor, or a forced fresh search)
       'pool+refill' — serve from the pool NOW + top it up in the BACKGROUND (dipped below the reorder mark)
       'pool'        — serve from the pool, no fetch at all (pool healthy)
-    Reorder EARLY and floor HIGH so the pool never runs thin (floor ≤ reorder)."""
-    if force_fresh or n_fresh < floor:
+    Reorder EARLY and floor HIGH so the pool never runs thin (floor ≤ reorder).
+
+    EXHAUSTION guard: when the query is tapped out (a recent refill added almost no NEW jobs), a blocking
+    sweep would just re-buy duplicates — so as long as the pool isn't EMPTY, serve what's there instead of
+    block-fetching. This is what makes a high floor safe on niche queries. A forced-fresh always sweeps."""
+    if force_fresh:
+        return "sweep"
+    if exhausted and n_fresh > 0:
+        return "pool"                       # tapped out but we have something → serve, don't re-buy dupes
+    if n_fresh < floor:
         return "sweep"
     if n_fresh < reorder:
         return "pool+refill"
@@ -37,7 +46,9 @@ def pool_decision(n_fresh: int, force_fresh: bool, floor: int, reorder: int) ->
 # Boards whose actor takes a real page/offset → they get a per-board cursor that advances each run.
 # Everyone else (Naukri/Foundit feeds — no page param) relies purely on the per-user seen-net; we do
 # NOT track a meaningless cursor for them.
-PAGINATING_BOARDS = {"linkedin"}   # valig-indeed uses `limit` (no page) → seen-net, not a cursor
+# NO board paginates: curious_coder only honors start=0 (start>0 → empty), and the rest are date feeds.
+# Freshness comes from the boards' "latest by date" + the per-user seen-net — NOT page cursors.
+PAGINATING_BOARDS: set[str] = set()
 
 # board key → (actor_id, build_input, normalizer). "Balanced" India stack = naukri+foundit+linkedin,
 # all city-filtered at the board. The rest stay registered (code ready) but off unless enabled.
@@ -49,11 +60,13 @@ BOARDS = {
     "foundit": ("shahidirfan~Foundit-Jobs-Scraper",
                 lambda p: A.build_foundit_input(p, results_wanted=get_settings().FOUNDIT_MAX_JOBS),
                 N.foundit_to_scoutjob),
-    "linkedin": ("harvestapi~linkedin-job-search",
-                 # cursor stores the LAST page fetched (0 = none yet) → next page = last + 1.
-                 lambda p: A.build_linkedin_input(p, recall=True, max_items=get_settings().LINKEDIN_MAX_JOBS,
-                                                  page=int((p.get("_cursors") or {}).get("linkedin", 0)) + 1),
-                 N.linkedin_to_scoutjob),
+    "linkedin": ("curious_coder~linkedin-jobs-scraper",   # structured industry+jobFunction+applicants
+                 # NO cursor — actor only honors start=0; freshness via date sort + the seen-net.
+                 lambda p: A.build_linkedin_cc_input(p, count=get_settings().LINKEDIN_MAX_JOBS),
+                 N.linkedin_cc_to_scoutjob),
+    "linkedin_v1": ("harvestapi~linkedin-job-search",   # registered fallback (jobTitles input)
+                    lambda p: A.build_linkedin_input(p, recall=True, max_items=get_settings().LINKEDIN_MAX_JOBS),
+                    N.linkedin_to_scoutjob),
     # ── registered fallbacks / other lanes (off unless added to BOARDS_ENABLED) ──
     "ats": ("bovi~greenhouse-lever-ashby-job-scraper",
             lambda p: A.build_ats_input(p, recall=True, companies=_ats_companies(),
@@ -72,6 +85,9 @@ BOARDS = {
                   # India blue/grey-collar non-tech; no page param → per-user seen-net dedups.
                   lambda p: A.build_workindia_input(p, results_wanted=get_settings().WORKINDIA_MAX_JOBS),
                   N.workindia_to_scoutjob),
+    "timesjobs": ("shahidirfan~Timesjobs-Scraper",   # India full-signal; no page param → seen-net dedups.
+                  lambda p: A.build_timesjobs_input(p, results_wanted=get_settings().TIMESJOBS_MAX_JOBS),
+                  N.timesjobs_to_scoutjob),
 }
 
 

tests/test_e2e_integration.py

@@ -0,0 +1,112 @@
+"""E2E integration — the full curate machinery (deterministic, offline) + an opt-in LIVE stack run.
+
+The offline test mocks the gateway so the WHOLE two-stage curate path runs in CI: Opus-score JSON →
+rubric.aggregate → calibration → floor filter → best-first sort → Haiku prose merge. It asserts the
+contract that matters: an aligned job reaches the 90s, a weak job is DROPPED at the floor, and every
+surviving match decomposes to real evidence.
+
+The live test (RUN_LIVE_E2E=1) runs the real new stack + Opus/Haiku — documents the manual verification
+(sweep → ~133 jobs incl. TimesJobs full-signal → scores reach 96).
+"""
+from __future__ import annotations
+
+import json
+import os
+
+import pytest
+
+from app.engine import curate as C
+from app.config import get_settings
+
+
+# ── deterministic offline e2e (mocks the gateway) ────────────────────────────
+class _Msg:
+    def __init__(self, content): self.content = content
+class _Choice:
+    def __init__(self, content): self.message = _Msg(content); self.finish_reason = "stop"
+class _Resp:
+    def __init__(self, content):
+        self.choices = [_Choice(content)]
+        self.usage = type("U", (), {"prompt_tokens": 50, "completion_tokens": 50})()
+class _Completions:
+    def __init__(self, score_model, score_json, cards_json):
+        self._sm, self._s, self._c = score_model, score_json, cards_json
+    def create(self, *, model, **kw):
+        return _Resp(self._s if model == self._sm else self._c)
+class _Chat:
+    def __init__(self, *a): self.completions = _Completions(*a)
+class _FakeClient:
+    def __init__(self, score_model, score_json, cards_json): self.chat = _Chat(score_model, score_json, cards_json)
+
+
+def test_curate_e2e_scores_calibrate_floor_and_decompose(monkeypatch):
+    s = get_settings()
+    sifted = [
+        {"id": "j1", "title": "Senior Sales Manager", "organization": "Acme", "details": {"description": "lead B2B sales"}},
+        {"id": "j2", "title": "Sales Manager", "organization": "Beta", "details": {"description": "manage accounts"}},
+        {"id": "j3", "title": "Junior Analyst", "organization": "Gamma", "details": {"description": "data entry"}},
+    ]
+    # Opus rates dimensions; j1 = strongly aligned (→ 90s), j2 = good, j3 = weak (raw < floor → dropped).
+    score_json = json.dumps({"kept": [
+        {"id": "j1", "fit": "fit", "dimensions": {"role": {"score": 100, "note": "same function"},
+            "skills": {"score": 92, "note": "B2B + CRM"}, "seniority": {"score": 95, "note": "right level"},
+            "location": {"score": 100, "note": "target city"}, "industry": {"score": 90, "note": "fintech"},
+            "experience": {"score": 90, "note": "9 yrs"}}},
+        {"id": "j2", "fit": "fit", "dimensions": {"role": {"score": 90}, "skills": {"score": 75},
+            "seniority": {"score": 80}, "location": {"score": 100}, "experience": {"score": 75}}},
+        {"id": "j3", "fit": "stretch", "dimensions": {"role": {"score": 20}, "skills": {"score": 25},
+            "seniority": {"score": 30}, "location": {"score": 100}, "experience": {"score": 20}}},
+    ]})
+    cards_json = json.dumps({"cards": [
+        {"id": "j1", "archetype": "The Perfect Fit", "one_line": "Strong on role and seniority.",
+         "coach_note": "Lead with revenue numbers.", "growth": {"text": "x", "from": 90, "to": 96}},
+        {"id": "j2", "archetype": "The Solid Step", "one_line": "Good role match, lighter on skills."},
+    ]})
+    monkeypatch.setattr(C, "curate_enabled", lambda: True)
+    monkeypatch.setattr(C, "gateway_client", lambda: _FakeClient(s.CURATE_MODEL, score_json, cards_json))
+
+    out = C.curate({"title": "Sales Manager"}, {"skills": ["B2B Sales"]}, sifted)
+    assert out is not None
+    ids = [o["id"] for o in out]
+    assert "j3" not in ids, "weak job (raw < floor) must be DROPPED — floor honesty"
+    assert ids == ["j1", "j2"], "best-first by the COMPUTED score"
+    j1 = out[0]
+    assert j1["matchScore"] >= 90, f"aligned job should reach the 90s, got {j1['matchScore']}"
+    assert all(o["matchScore"] >= s.MATCH_FLOOR for o in out), "no sub-floor match surfaces"
+    # decomposes to real evidence
+    bd = {d["name"]: d["score"] for d in j1["match"]["breakdown"]}
+    assert len(bd) >= 5 and j1["match"]["archetype"] == "The Perfect Fit"
+    assert j1["match"]["coach_note"] == "Lead with revenue numbers."
+
+
+def test_curate_floor_only_returns_empty_not_unfiltered_fallback(monkeypatch):
+    # Opus succeeds but keeps nothing ≥ floor → honest empty deck, NEVER the unfiltered white-box
+    s = get_settings()
+    sifted = [{"id": "j1", "title": "X", "organization": "Y", "details": {}}]
+    score_json = json.dumps({"kept": [{"id": "j1", "fit": "stretch",
+        "dimensions": {"role": {"score": 10}, "skills": {"score": 10}, "seniority": {"score": 10},
+                       "location": {"score": 10}, "experience": {"score": 10}}}]})
+    monkeypatch.setattr(C, "curate_enabled", lambda: True)
+    monkeypatch.setattr(C, "gateway_client", lambda: _FakeClient(s.CURATE_MODEL, score_json, "{}"))
+    out = C.curate({"title": "X"}, None, sifted)
+    assert out == [], "all-below-floor → empty, not a fallback that could leak sub-floor matches"
+
+
+# ── opt-in LIVE e2e (real stack + Opus) ──────────────────────────────────────
+@pytest.mark.skipif(os.environ.get("RUN_LIVE_E2E") != "1", reason="live stack run — set RUN_LIVE_E2E=1")
+def test_live_stack_reaches_90s_with_full_signal():
+    import asyncio
+    from app.engine import search, sift as _sift
+    prefs = {"title": "Sales Manager", "location": ["Mumbai · India"], "role": ["Sales"],
+             "industry": ["Fintech"], "experience": ["Senior"],
+             "user_context": {"id": "live", "skills": ["B2B Sales", "Account Management", "CRM"],
+                              "current_role": "Senior Sales Manager", "years_experience": 9}}
+    ctx = prefs["user_context"]
+    sweep = asyncio.run(search.run_sweep({**prefs, "_fresh": True}, fresh=True))
+    assert len(sweep["sources"]) >= 4, "most of the stack should return"
+    assert len(sweep["opportunities"]) >= 80, "deck should be deep"
+    top, _ = _sift.sift(prefs, ctx, [dict(j) for j in sweep["opportunities"]])
+    cur = C.curate(prefs, ctx, top) or []
+    scores = [o["matchScore"] for o in cur]
+    assert scores and max(scores) >= 90, f"aligned profile should reach 90s, got max {max(scores) if scores else None}"
+    assert min(scores) >= get_settings().MATCH_FLOOR, "floor honored"

tests/test_pool.py

@@ -1,26 +1,37 @@
 """Warm-pool gate — the reorder/floor decision that keeps the shelf stocked without over-fetching."""
 from app.engine.search import pool_decision
 
-FLOOR, REORDER = 70, 80   # the configured thresholds (POOL_SAFETY_FLOOR / POOL_REORDER_AT)
+FLOOR, REORDER = 90, 100   # the configured thresholds (POOL_SAFETY_FLOOR / POOL_REORDER_AT)
 
 
 def test_pool_healthy_serves_no_fetch():
     assert pool_decision(150, False, FLOOR, REORDER) == "pool"
-    assert pool_decision(80, False, FLOOR, REORDER) == "pool"     # exactly at reorder → still no fetch
+    assert pool_decision(100, False, FLOOR, REORDER) == "pool"    # exactly at reorder → still no fetch
 
 
 def test_pool_dipping_serves_plus_background_refill():
-    assert pool_decision(79, False, FLOOR, REORDER) == "pool+refill"   # below reorder → top up in bg
-    assert pool_decision(70, False, FLOOR, REORDER) == "pool+refill"   # at the floor → still serve, refill
+    assert pool_decision(99, False, FLOOR, REORDER) == "pool+refill"   # below reorder → top up in bg
+    assert pool_decision(90, False, FLOOR, REORDER) == "pool+refill"   # at the floor → still serve, refill
 
 
 def test_pool_below_floor_blocks_and_fetches():
-    assert pool_decision(69, False, FLOOR, REORDER) == "sweep"    # under the hard floor → blocking fetch
+    assert pool_decision(89, False, FLOOR, REORDER) == "sweep"    # under the hard floor → blocking fetch
     assert pool_decision(0, False, FLOOR, REORDER) == "sweep"     # empty pool (first search) → fetch
 
 
 def test_forced_fresh_always_fetches():
     assert pool_decision(999, True, FLOOR, REORDER) == "sweep"    # dev _fresh toggle bypasses the pool
+    assert pool_decision(5, True, FLOOR, REORDER, exhausted=True) == "sweep"  # force overrides exhaustion
+
+
+def test_exhausted_query_serves_instead_of_block_fetching():
+    # tapped-out niche query below the floor: serve what's pooled, DON'T re-buy duplicates
+    assert pool_decision(30, False, FLOOR, REORDER, exhausted=True) == "pool"   # below floor but exhausted
+    assert pool_decision(1, False, FLOOR, REORDER, exhausted=True) == "pool"    # even 1 job → serve it
+    # but an EMPTY exhausted pool must still try a fetch (nothing to serve)
+    assert pool_decision(0, False, FLOOR, REORDER, exhausted=True) == "sweep"
+    # exhaustion does NOT suppress the healthy/refill paths
+    assert pool_decision(200, False, FLOOR, REORDER, exhausted=True) == "pool"
 
 
 def test_floor_never_above_reorder_invariant():

tests/test_stack_contracts.py

@@ -0,0 +1,158 @@
+"""Step-0 DEEP contract tests for the new signal-weighted stack.
+
+These lock the load-bearing contracts BEFORE the actors land, so nothing is missed:
+  1. Stack alignment — BOARDS_ENABLED == the target stack; Indeed off-but-registered; every enabled
+     board has a builder + normalizer (an enabled board with no normalizer = silent drop).
+  2. Builder contract — every enabled board's input builder emits a NON-EMPTY query (no actor 400s).
+  3. Normalizer contract (on REAL cached data) — every board's normalizer turns real raw items into
+     valid ScoutJobs with the required fields AND DISTINCT dedup keys (the WorkIndia 20→1 collapse class).
+  4. Signal contract — each board's normalizer actually yields the signals we claim it does (skills /
+     salary / seniority), so a regression in the actor or mapping fails loudly.
+  5. Pool contract — floor ≤ reorder, and the gate's three outcomes are well-defined.
+
+Data contracts read .apify_cache (real captured sweeps). A target board with no cache FAILS (so we can't
+'forget' to validate a new actor) — except where explicitly allowed pre-build.
+"""
+from __future__ import annotations
+
+import glob
+import json
+import os
+
+import pytest
+
+from app.engine import search, normalize as N
+from app.engine.search import pool_decision
+from app.config import get_settings
+
+# ── the TARGET stack (what we're building toward) ────────────────────────────
+TARGET_STACK = ["naukri", "foundit", "linkedin", "timesjobs", "workindia"]
+OFF_BUT_REGISTERED = "indeed"            # dropped from enabled, kept registered
+
+# board → cache-file actor prefix (real raw data to validate the normalizer against)
+CACHE_PREFIX = {
+    "naukri": "blackfalcondata_naukri-jobs-feed",
+    "foundit": "shahidirfan_Foundit-Jobs-Scraper",
+    "linkedin": "curious_coder_linkedin-jobs-scraper",
+    "timesjobs": "shahidirfan_Timesjobs-Scraper",
+    "workindia": "shahidirfan_workindia-jobs-scraper",
+}
+# signals each board MUST confirm (guards actor / mapping regressions). 'offsite' = a real apply link.
+SIGNAL_MAP = {
+    "naukri": {"skills", "offsite"},
+    "foundit": {"skills", "offsite"},
+    "linkedin": {"industry"},            # curious_coder structured industries (100%); seniority often N/A in IN
+    "timesjobs": {"skills", "salary"},
+    "workindia": {"skills"},
+}
+
+SAMPLE_PREFS = {
+    "title": "Sales Manager", "location": ["Mumbai · India"], "role": ["Sales"],
+    "industry": ["Fintech"], "experience": ["Senior"],
+    "user_context": {"skills": ["B2B Sales", "Account Management", "CRM"]},
+}
+REQUIRED_FIELDS = {"id", "title", "organization", "location_country", "required_skills", "details"}
+# Stable, COMMITTED per-board raw samples (curated from real sweeps) — so the contract is reproducible
+# and not at the mercy of whatever is currently in the volatile .apify_cache.
+_SAMPLES_PATH = os.path.join(os.path.dirname(__file__), "fixtures", "board_samples.json")
+_SAMPLES = json.load(open(_SAMPLES_PATH)) if os.path.exists(_SAMPLES_PATH) else {}
+
+
+def _board_sample(board: str):
+    return _SAMPLES.get(board)
+
+
+def _dedup_key(j: dict) -> str:
+    return (j.get("apply_url") or "").strip() or f"{j['organization']}|{j['title']}|{j.get('location_city')}".lower()
+
+
+def _has_signal(sj: dict, signal: str) -> bool:
+    d = sj.get("details") or {}
+    if signal == "skills":
+        return bool(d.get("skills") or sj.get("required_skills"))
+    if signal == "salary":
+        return sj.get("salary_lpa") is not None or bool(sj.get("payLabel") and "disclos" not in (sj["payLabel"] or "").lower())
+    if signal == "seniority":
+        return bool(sj.get("seniority_level"))
+    if signal == "industry":
+        return bool(sj.get("industry") or (sj.get("details") or {}).get("industry"))
+    if signal == "offsite":
+        return bool(sj.get("offsite_apply"))
+    return False
+
+
+# ── 1. Stack alignment ───────────────────────────────────────────────────────
+def test_enabled_equals_target_stack():
+    enabled = [b.strip() for b in get_settings().BOARDS_ENABLED.split(",") if b.strip()]
+    assert set(enabled) == set(TARGET_STACK), f"BOARDS_ENABLED {enabled} != target {TARGET_STACK}"
+
+
+def test_indeed_off_but_registered():
+    enabled = [b.strip() for b in get_settings().BOARDS_ENABLED.split(",")]
+    assert OFF_BUT_REGISTERED not in enabled, "indeed must be DROPPED from enabled"
+    assert OFF_BUT_REGISTERED in search.BOARDS, "indeed must stay REGISTERED (off, not deleted)"
+
+
+def test_every_target_board_has_builder_and_normalizer():
+    for b in TARGET_STACK:
+        assert b in search.BOARDS, f"target board '{b}' not registered in BOARDS"
+        assert b in N.NORMALIZERS, f"target board '{b}' has no normalizer (silent drop)"
+
+
+# ── 2. Builder contract ──────────────────────────────────────────────────────
+def test_every_enabled_builder_emits_nonempty_query():
+    enabled = [b for b in TARGET_STACK if b in search.BOARDS]
+    for b in enabled:
+        inp = search.BOARDS[b][1](dict(SAMPLE_PREFS))
+        assert isinstance(inp, dict) and inp, f"{b} builder returned empty input"
+        q = inp.get("keyword") or inp.get("title") or (inp.get("jobTitles") or [None])[0] or inp.get("includeKeyword") or inp.get("urls")
+        assert q, f"{b} builder emitted no query/keyword/title → actor would 400"
+
+
+# ── 3 + 4. Normalizer + signal contracts on REAL cached data ─────────────────
+@pytest.mark.parametrize("board", TARGET_STACK)
+def test_normalizer_contract_on_real_data(board):
+    items = _board_sample(board)
+    if items is None:
+        pytest.fail(f"no committed sample for '{board}' in {_SAMPLES_PATH} — capture one")
+    fn = N.NORMALIZERS[board]
+    jobs = [sj for it in items[:25] if (sj := fn(it))]
+    assert len(jobs) >= 2, f"{board}: normalizer produced < 2 jobs from {len(items)} raw items"
+    # required fields present
+    for j in jobs:
+        missing = REQUIRED_FIELDS - set(j)
+        assert not missing, f"{board} ScoutJob missing {missing}"
+        assert isinstance(j["required_skills"], list) and isinstance(j["details"], dict)
+    # ANTI-COLLAPSE: distinct dedup keys (the WorkIndia 20→1 bug class)
+    keys = [_dedup_key(j) for j in jobs]
+    assert len(set(keys)) >= max(2, int(0.8 * len(jobs))), \
+        f"{board}: dedup keys collapse ({len(set(keys))} unique of {len(jobs)}) — apply_url not unique"
+    # SIGNAL contract: the board confirms what we claim, on a majority of jobs
+    for sig in SIGNAL_MAP[board]:
+        hits = sum(_has_signal(j, sig) for j in jobs)
+        assert hits >= 0.5 * len(jobs), f"{board}: signal '{sig}' present on only {hits}/{len(jobs)} (claimed)"
+
+
+# ── 4b. Legacy-board city aliasing (TimesJobs/Foundit need "Delhi" not "New Delhi") ──
+@pytest.mark.parametrize("builder", ["build_timesjobs_input", "build_foundit_input"])
+def test_legacy_boards_alias_modern_city_names(builder):
+    from app.engine.board_adapters import adapters as A
+    fn = getattr(A, builder)
+    for modern, legacy in [("New Delhi · India", "Delhi"), ("Bengaluru · India", "Bangalore"),
+                           ("Gurugram · India", "Gurgaon")]:
+        loc = fn({"title": "Data Architect", "location": [modern], "role": ["Data"]}).get("location")
+        assert loc == legacy, f"{builder}: {modern!r} must alias to {legacy!r} (board returns 0 otherwise), got {loc!r}"
+
+
+# ── 5. Pool contract ─────────────────────────────────────────────────────────
+def test_pool_floor_le_reorder():
+    s = get_settings()
+    assert s.POOL_SAFETY_FLOOR <= s.POOL_REORDER_AT, "floor must be ≤ reorder (reorder early, floor backstop)"
+
+
+def test_pool_gate_outcomes_well_defined():
+    f, r = 90, 100
+    assert pool_decision(200, False, f, r) == "pool"
+    assert pool_decision(95, False, f, r) == "pool+refill"
+    assert pool_decision(50, False, f, r) == "sweep"
+    assert pool_decision(9999, True, f, r) == "sweep"      # forced fresh always sweeps