Configuration & Operations
The native pipeline's behaviour is governed by three independent Edge Config flags, a set of environment variables, and one hard deploy prerequisite. This page is the canonical reference for all three, plus the recommended rollout order and where to look next. See Issue Research Pipeline for how these flags fit into the pipeline as a whole.
Feature flags
Each flag is stored inside the flags object of a single Edge Config item — not one item per flag, since Edge Config top-level keys can't contain the slash in feature/... (see getFeatureFlag, infrastructure/edge-config/feature-flags.ts). Every flag is an independent, reversible kill switch, defaults to OFF in every environment, and reverts its slice of behaviour byte-for-byte the moment it is flipped back off. Local dev has no EDGE_CONFIG connection string by default, so each flag's own *_LOCAL environment variable is checked ahead of Edge Config — this is how a developer forces a flag on/off without a dashboard.
| Edge Config key | FEATURE_FLAG_KEYS constant | Resolver | Local override env | Effect |
|---|---|---|---|---|
feature/issues-research-harness | ISSUES_RESEARCH_HARNESS | resolveIssuesResearchHarness (application/services/issues-research-harness.ts) | ISSUES_RESEARCH_HARNESS_LOCAL | Native two-phase Gemini brain vs legacy OpenAI agent (full engine switch) |
feature/issues-streaming | ISSUES_STREAMING | resolveIssuesStreaming (application/services/issues-streaming.ts) | ISSUES_STREAMING_LOCAL | Live UI streaming: issues-landed publishes, the on-load trigger, and the live count poll |
feature/bhrrc-corpus | BHRRC_CORPUS | resolveBhrrcCorpus (application/services/bhrrc-corpus.ts) | BHRRC_CORPUS_LOCAL | BHRRC article bodies from the Blob corpus vs live RSS |
All three default OFF everywhere, and OFF is byte-for-byte identical to pre-flag behaviour. See Two-Phase Native Brain for how feature/issues-research-harness is resolved and memoised inside a run, and Data Sources & Screening for what feature/bhrrc-corpus changes underneath.
Environment variables
Blob & token
| Env var | Where read | Default | Note |
|---|---|---|---|
SOURCE_DOCS_BLOB_READ_WRITE_TOKEN | getSourceDocumentsToken() (infrastructure/source-indexes/source-documents-store.ts) | none — required | Authenticates every read against the dedicated source-documents Blob store. Deliberately separate from BLOB_READ_WRITE_TOKEN, which now serves only user-generated content (reports, uploads). Missing or blank throws SourceDocumentsTokenError — see Deploy prerequisite. |
AI gateway & models
| Env var | Where read | Default | Note |
|---|---|---|---|
AI_GATEWAY_API_KEY / VERCEL_OIDC_TOKEN | Vercel AI Gateway provider (ai.gateway(...), issues-core/model.ts) | none — required | Either satisfies gateway auth; both models route through the same gateway. |
ISSUES_RESEARCH_MODEL | issues-core/model.ts | google/gemini-3.5-flash | Overrides the planner/map/reduce/analyst model — e.g. for local testing against google/gemini-3.1-flash-lite. |
TAVILY_API_KEY | TavilyRepository (infrastructure/repositories/tavily-repository.ts) | none — required | Backs Phase 2's web_search_batch tool and the BHRRC live-RSS path. |
PHASE2_STREAM_FLASH_LITE_REDUCE | resolveStreamReducer (issues-research/run-issues-research.ts) | false | false = parity (tagged full-flash streaming reducer, matching the batch reduce model); true switches Phase 2's per-cell reducer to flash-lite for speed. |
Search tuning
| Env var | Where read | Default | Note |
|---|---|---|---|
SEARCH_ROUNDS | resolveSearchBudget (issues-core/run-research.ts) | 1 | Only the literal value 2 opts into the Round-2 adaptive-depth pass (the coverage analyst proposes follow-up leads). |
MAX_SEARCHES | resolveSearchBudget | 4 | Round-1 breadth (coverage cells). Clamped to [4, 20]. |
SEARCH_DEPTH_BUDGET | resolveSearchBudget | 0 at rounds:1, 5 at rounds:2 | Round-2 follow-up query budget; only consulted when SEARCH_ROUNDS=2. Clamped to ≤ 20. |
SEARCH_SCORE_FLOOR | resolveSearchConfig (issues-core/search.ts) | 0.4 | Tavily results scoring below this are dropped (falls back to all results if the floor would drop everything). Clamped to [0, 1]. |
SEARCH_MAX_RESULTS | resolveSearchConfig | 5 | Max Tavily results kept per sub-query. Clamped to [1, 10]. |
SEARCH_DEPTH | resolveSearchConfig | advanced | Tavily search depth; any value other than basic resolves to advanced. |
SEARCH_CONTENT_CHARS | resolveSearchConfig | 1200 | Characters of article content kept per result. Clamped to [100, 100000]. |
Concurrency
| Env var | Where read | Default | Note |
|---|---|---|---|
MODEL_GLOBAL_CONCURRENCY | resolveModelMaxConcurrency (issues-core/model.ts) | 8 | Process-wide cap on concurrent Gemini calls (shared by both model tiers). Account-wide concurrency scales with the number of concurrent processes. |
SEARCH_MAX_CONCURRENCY | resolveSearchMaxConcurrency (issues-core/search.ts) | 4 | Process-wide cap on concurrent live Tavily calls; cache hits pass through without consuming the budget. |
MODEL_MAX_RETRIES (4 retries — 429 backoff plus transient "corrupted thought signature" recovery) is a hardcoded constant in issues-core/model.ts, not an environment variable.
Cache TTLs
| Env var | Where read | Default | Note |
|---|---|---|---|
SOURCE_INDEX_TTL_MS | load-source-index.ts | 6 hours | Per-pathname TTL for the 8 cached sanctions/BHRRC indexes. Unbounded by size — only 8 small files exist, so there is no LRU eviction. |
BHRRC_ARTICLES_TTL_MS | load-bhrrc-articles.ts | 6 hours | Per-contentId TTL for cached BHRRC article records. |
BHRRC_ARTICLES_CACHE_BYTES | load-bhrrc-articles.ts | 150 MB | Byte budget for the article-record LRU. Least-recently-used entries evict first; at least one entry is always retained. |
Models & providers
| Model | Constant | Role |
|---|---|---|
google/gemini-3.5-flash | ISSUES_RESEARCH_MODEL (default) | Planner, map worker, batch/streaming reduce, coverage analyst |
google/gemini-3.1-flash-lite | FLASH_LITE_MODEL | BHRRC Phase-1 map + reduce; optional Phase-2 streaming reducer (PHASE2_STREAM_FLASH_LITE_REDUCE=true) |
Both models route through the Vercel AI Gateway (ai.gateway(...)), authenticated by AI_GATEWAY_API_KEY or VERCEL_OIDC_TOKEN. Phase 2's web_search_batch tool and the BHRRC live-RSS fallback both search via Tavily (TAVILY_API_KEY).
For contrast: the legacy engine — still the production default while feature/issues-research-harness is OFF, see methodology — runs an entirely different provider stack: an OpenAI agent plus SerpAPI, Tavily, and Diffbot extraction. The two engines share no model configuration; Tavily is the only provider both use.
Deploy prerequisite
Before any deploy, SOURCE_DOCS_BLOB_READ_WRITE_TOKEN must be provisioned in every environment, and the 8 source-indexes/*.json blobs (7 sanctions lists plus bhrrc-index.json) must be uploaded to the source-documents Blob store. Index reads (load-source-index.ts) are always-on and ungated — every sanctions/BHRRC screen, in every environment, reads these indexes unconditionally, with no feature flag anywhere in that path. A missing or blank token, or an absent/empty index blob, throws and breaks all sanctions/BHRRC screening in that environment. feature/bhrrc-corpus does not relax this — it gates only the BHRRC article-body source (corpus vs live RSS), never the index reads. The upload and verification scripts are untracked-by-convention; run them from a checkout that has them, not from a fresh clone. See Data Sources & Screening for the store's key layout.
Rollout sequencing
Recommended order — each step is independently reversible:
- Provision the token and upload the indexes. Set
SOURCE_DOCS_BLOB_READ_WRITE_TOKENin the target environment and upload all 8source-indexes/*.jsonblobs. This is a hard prerequisite, not gated by any flag — see Deploy prerequisite. - Flip
feature/issues-research-harnessto routeorganisation-generate-issuesonto the native two-phase Gemini brain instead of the legacy OpenAI agent. - Flip
feature/issues-streamingto turn on live UI streaming — the issue count ticking up and the Analyst/Spotlight lists streaming in — instead of results jumping in once at the end of the run. See Live Streaming for the full mechanics. - Optionally flip
feature/bhrrc-corpus, once the BHRRC article corpus is populated and on a recurring re-scrape-and-upload cadence. The corpus is a point-in-time scrape and needs refreshing to hold recall at live-RSS parity — see Corpus vs live RSS.
Flipping any one flag back off returns that slice of behaviour to byte-for-byte parity with the pre-flag state, without touching the other two.
Reference
- ADR-0022 — the concurrency and additive-merge rationale behind the two-phase harness's retry semantics.
- The modern-slavery skill directory
application/ai/issues-research/skills/modern-slavery-assessment/— the classification framework the pipeline reasons with.