Two-Phase Native Brain

The native engine (feature/issues-research-harness = ON) replaces the legacy single-call OpenAI agent with a two-phase Gemini pipeline: a fast verified-source screen runs alongside a coverage-cell planner, then a streaming deep-research phase emits issues as each cell resolves. This page covers brain selection, orchestration, both phases, the engine kernel, models, and degradation handling. See Issue Research Pipeline for how this engine fits into the pipeline as a whole.

Brain selection

Every run of the organisation-generate-issues Inngest function resolves which engine to use exactly once, via resolveIssuesResearchHarness (application/services/issues-research-harness.ts). The resolution order is:

1. Request-scope override — event.data.useNative, threaded onto the event by the org layout's flag() read (the only way a per-session Vercel toolbar override reaches this cookieless background job).
2. ISSUES_RESEARCH_HARNESS_LOCAL — a local-dev bypass env var, since Edge Config isn't wired up locally by default.
3. Edge Config feature/issues-research-harness — default OFF.

The lookup in generate-issues.ts uses ??, not ||:

const useNative = await step.run(
  'resolve-issues-research-harness',
  async () => useNativeFromEvent ?? resolveIssuesResearchHarness(),
);

?? matters because useNativeFromEvent can legitimately be false — an explicit toolbar toggle-OFF must win over whatever Edge Config says. || would treat that false as "absent" and fall through to Edge Config instead.

The decision is wrapped in step.run('resolve-issues-research-harness', …) so it is memoised for the life of the run. Code between Inngest steps re-executes on every replay/retry, and resolveIssuesResearchHarness reads mutable Edge Config — without the step, a flag flip mid-run could re-route an already-partially-native run onto the legacy path on retry (a new step id), causing a double write. The step fixes the routing decision once; a flag flip only takes effect on the organisation's next trigger. resolveIssuesStreaming (whether onProgress publishes issues-landed) is memoised the same way, immediately after, for the same reason.

Orchestration

Once the native path is selected, generate-issues.ts runs Phase 1 and the Phase-2 planner as parallel Inngest steps:

const [phase1Result, plan] = await Promise.all([
  step.run('generate-issues-phase-1', () => new GenerateOrganisationIssuesTwoPhase(twoPhaseDeps).executePhase1({ organisationId })),
  step.run('plan-issue-cells', () => new GenerateOrganisationIssuesTwoPhase(twoPhaseDeps).planPhase2Cells({ organisationId })),
]);

const phase2Result = await step.run('generate-issues-phase-2', () =>
  new GenerateOrganisationIssuesTwoPhase(twoPhaseDeps).executePhase2({ organisationId, plan }),
);

planPhase2Cells (the planCells structured-agent call) takes around 10.7 seconds. Running it in parallel with Phase 1 — rather than at the start of Phase 2 — hides that prefix behind Phase 1's own research time, so by the time Phase 2 starts, its plan is already computed and checkpointed. Phase 2 then consumes the precomputed plan directly instead of calling planCells inline.

Phase 1 — fast screening

GenerateOrganisationIssuesTwoPhase.executePhase1 fans out eight sources in parallel: BHRRC (runBhrrcResearch) plus the seven SANCTIONS_PROFILES entries (OFAC, UN, UK, UFLPA, BIS, EU, DFAT). Each source, as it resolves, enqueues a materialise-then-persist task onto one serial persist chain (createPersistChain, p-limit(1)) — so persists never interleave within the phase, while research for every source still runs concurrently. A single reconcile pool (loadPool) is fetched once at the start of the phase and threaded through every persist so later persists in the same phase see earlier ones' creates without re-reading Neo4j.

Each persisted batch is tagged with a source (bhrrc, ofac, un-sc, uk-sanctions, uflpa, bis-entity, eu-sanctions, dfat-sanctions) so provenance is traceable back to the originating list. See Data Sources & Screening for how each source is fetched and matched.

Phase-2 planner

planIssueCells (issues-core/run-research.ts) builds the scoping brief for the organisation and calls planCells, a PlanSchema structured-output agent (issues-core/plan-cells.ts, createPlanCells) bound to the modern-slavery pack's coverage skeleton (issues-research/plan-config.ts). The mandatory skeleton has four cells:

The planner tailors one search query per cell to the organisation's sector, products, and geography. If the model call fails, createPlanCells falls back to planCellsFallback — a deterministic plan built straight from the skeleton's own guidance text, so a planner outage degrades to generic-but-usable queries rather than losing the round entirely. matchPlanCells reconciles the model's output back into the skeleton's stable slots by key, backfilling any empty or unmatched slot from the fallback.

Phase 2 — streaming deep research

executePhase2 calls streamIssuesResearch (issues-research/run-issues-research.ts, wrapping issues-core/run-research.ts's streamIssuesResearch), passing the precomputed plan so no inline planning call is needed. It first loads a fresh reconcile pool — never Phase 1's — because Phase 1 and Phase 2 are separate Inngest steps: a step retry re-runs the whole phase against current DB state (see ADR-0022), and reusing Phase 1's in-memory pool would reconcile Phase 2 against a stale or foreign snapshot instead of what's actually in Neo4j when the step starts.

For each cell, the streaming orchestrator:

1. Runs mapCell — an agentic worker (issues-core/map-worker.ts) that calls the web_search_batch tool (expanding the cell's angle into up to five Tavily sub-queries in parallel) and then save_issues, which validates and captures the extracted issues in-tool.
2. Reduces just that cell's raw issues via reduceIssues (per-cell, not a global reduce) as soon as the map resolves.
3. Emits the result through onCell if it produced at least one issue.

onCell, defined in executePhase2, enqueues a materialise-then-persist task onto Phase 2's own serial persist chain — the same enqueue-and-return-promptly pattern as Phase 1, so cells never block each other while persists still run one at a time.

Per-cell failures are isolated: one cell's map or search failing never rejects the wave. StreamResearchOutcome reports this back to the caller as mapFailures (cells whose mapCell threw), searchErrors (Tavily errors across all cells), and searchWipeoutCells (cells whose search leg fully failed — errored with zero chunks returned, distinct from a healthy cell that legitimately found nothing).

The engine kernel (issues-core)

The kernel is domain-agnostic — it knows nothing about modern slavery specifically. The modern-slavery pack (issues-research/) and the fast-issues sources supply all domain knowledge; the kernel supplies the mechanics.

Models & search config

Both are routed through the Vercel AI Gateway (ai.gateway(...), authenticated via AI_GATEWAY_API_KEY or VERCEL_OIDC_TOKEN) and wrapped with a shared process-wide concurrency limiter plus MODEL_MAX_RETRIES (4 retries) for 429 backoff and transient "corrupted thought signature" recovery.

The default search budget, resolved by resolveSearchBudget (issues-core/run-research.ts), is:

{ rounds: 1, cells: 4, depthBudget: 0 }

rounds: 1 means the coverage analyst never runs — Phase 2 is a single breadth pass over the four skeleton cells. Setting SEARCH_ROUNDS=2 opts into an adaptive-depth round (the analyst selects up to depthBudget follow-up leads for a second map wave).

PHASE2_STREAM_FLASH_LITE_REDUCE (default false) selects the per-cell streaming reducer: OFF (parity) uses the tagged full-flash reducer, matching the batch reduce model; true switches to the tagged flash-lite reducer for faster, comparably-accurate reduction. See Configuration & Operations for the full environment-variable table.

Degradation handling

Two independent emitters can report an "engine degraded but didn't crash" outcome: Phase 2's deep-research engine (phase2-degraded/phase2-degraded-total, in generate-organisation-issues-two-phase.ts) and BHRRC's map-reduce (bhrrc-degraded/bhrrc-degraded-total, in persist-bhrrc-issues.ts). isPartialDegradationError (helpers/degradation-errors.ts) classifies a phase2-degraded:/bhrrc-degraded: message as partial — a bounded, transient failure (one Tavily 429, one BHRRC article fetch failure) that a clean run can plausibly absorb.

generate-issues.ts excludes partial-degradation errors from the FAILED decision:

const fatalErrors = combined.errors.filter((e) => !isPartialDegradationError(e));
const hasErrors = fatalErrors.length > 0 && combined.issues.length === 0;

Only errors that are both fatal (not partial-degradation) and accompanied by zero issues across the whole run mark the organisation FAILED. The -total wipeout variants — every cell or source lost to infrastructure — remain fatal, as does every other error string. Visibility is unaffected either way: combined.errors still carries every error, partial or fatal, in the job's return value and logs; only the FAILED marking is narrowed.