E2E stack identity and two-shard execution

Slice 2 of the E2E throughput plan. Slice 1 (PR #3473) cut the CI backend build from about 200 s to 40 s; the full suite still exceeds the 45-minute contract because 59 tests need roughly 33 minutes of Playwright time and one runner executes them serially. This slice makes two stacks able to coexist and splits the suite into two shards. It does not add runner capacity: both shards run one after the other on the existing pomegranate-01 lane until slice 3 activates a second listener.

No feature flag: this changes CI execution only. Browser coverage, retries, timeouts, performance budgets and trusted-source gates are unchanged.

Why a stack identity

CI today runs the stack at port offset 0 under the literal Compose project syrf-e2e. Every step that inspects, budgets, monitors or cleans the stack names syrf-e2e, syrf-e2e_default, syrf-e2e-<service> and ports 8080, 8081, 8083 and 15673 directly. The owner label is repo:run:attempt:job with no shard component, and ci-resources.py aborts when any live object has a different owner. Two runs on one host therefore cannot start, let alone run.

The local allocator already parameterises most surfaces from E2E_PORT_OFFSET through E2E_STACK. The gap is that CI fixes the offset, E2E_STACK cannot be set independently, the owner has no shard, and several steps rewrite files inside the checkout.

Contract

E2E_STACK_ID = <run_id>-<run_attempt>-<shard> (shard is 1 or 2). It is the only input; everything below derives from it.

Surface	Today	After
Port offset	E2E_PORT_OFFSET=0 fixed in CI	Allocated per shard from the reservation root; CI passes the resolved offset to every step through ports.sh exports, never literal numbers
Compose project / network / containers	syrf-e2e, syrf-e2e_default, syrf-e2e-*	syrf-e2e-<stack-id> and derived names; E2E_STACK accepts an explicit override
Owner label	repo:run:attempt:e2e-tests	repo:run:attempt:e2e-tests:<shard>; cleanup treats a live sibling shard of the same run and attempt as legitimate and still aborts on unknown owners
Job root	$RUNNER_TEMP/syrf-e2e-job (guarded)	$RUNNER_TEMP/syrf-e2e-job/<stack-id>; guard updated; shared caches stay one level up
Checkout mutations	compose, deploy-lambda.sh, setup.sh, appConfig.env.json, e2e/.auth written in place	Compose, mock-oidc/, ports.sh, setup.sh, teardown.sh, deploy-lambda.sh and service-env.sh are copied into <job root>/exec and patched and run from there; browser auth state lives in <job root>/exec/e2e/.auth (E2E_AUTH_DIR). No workflow step writes into the checkout after the build. Exception: ng serve still runs in the checkout, so runtime-config.ts writes the resolved appConfig.env.json overlay (and its backup) there and the Angular build cache stays shared; redirecting either would need src/services/web changes. Each CI job has its own checkout, so shards never share them
.NET service endpoints	overlay defaults (27018, 5673, 4566, 9000, 1434, 4200)	__ environment overrides from the resolved ports, as run-local.sh already does
Cookie names	literal syrf-session-syrf-e2e	derived from E2E_STACK (already exported by ports.sh)
Feature-flag lock	keyed by GITHUB_RUN_ID	keyed by E2E_STACK_ID; flags are per API instance, so the lock must be too
Results, report, artifacts	e2e/test-results, e2e/playwright-report, fixed artifact names	<job root>/results, artifact names suffixed with the shard
PR comment	one result, one duration	per-shard row plus merged verdict

Sharding

The full suite runs as three e2e-tests matrix entries; smoke stays one.

Entry	Playwright selection	Tests
functional-1	--project=@full --shard=1/2	half of 61
functional-2	--project=@full --shard=2/2	the other half
perf	--project=@perf	4
smoke	--project=smoke	label-triggered, unsharded

• Two shards alone did not work. tests/perf/annotation-form-perf.spec.ts holds four budget gates and the AF2 200-unit pagination gate alone takes about nine minutes per attempt; file-order sharding puts all four in shard 2, whose retries then exhaust the Playwright budget. On every complete two-shard main run since 2026-09-15 (runs 35542181815, 35039263359, 35026392154) shard 1 finished Playwright in 15-19 minutes while shard 2 was budget-killed. The @full project therefore ignores perf/ and a new @perf project owns those specs, with the same setup dependency, browser, auth state, retries and timeouts.
• The config sets fullyParallel: true, so Playwright shards the functional entries by test group rather than by file: a serial describe group stays inside one shard, other tests of one file may land in different shards.
• Implemented as the e2e-tests matrix include list (three entries for full, one for smoke, max-parallel: 1, fail-fast: false). Each entry keeps the full 45-minute job contract, gets its own stack id (<run>-<attempt>-<entry name>), owner label, port offset and job root, and uploads e2e-shard-report-<entry name> (shard.json with the entry name and project, the entry's JSON report and a whole-suite --list inventory covering @full and @perf).
• Lane time is unchanged: three entries still run one after another on the single pomegranate-e2e listener, so the full suite now occupies three sequential 45-minute job contracts on one lane until slice 3 activates a second listener. The win is that no entry is starved by another's work.
• Not yet done: balancing the two functional entries by measured healthy first-attempt durations from handover evidence (run 34905067814). Playwright 1.58 splits groups by count only; with the perf file out of the way the Dockview and annotation specs dominate, so the first main run's per-entry durations decide whether a weighted split is needed.
• merge-e2e-reports (ordinary Juniper route, read-only, no secrets, base-branch merger e2e/scripts/merge-shard-reports.cjs, --expected-entries) is red when the job result is not success, any expected entry is missing, failed or cancelled, entries disagree on the inventory, or any inventoried test ran in zero or several entries (retries are results of one test entry; the setup project runs in every entry and is excluded). Test ids carry the Playwright project name, so @full and @perf selections never collide. The PR comment shows a row per entry -- name, status, duration and test count -- plus the merged verdict, and is green only when both are.

Acceptance

1. Contract tests reject any literal syrf-e2e name or fixed port in a CI step and prove every surface above derives from E2E_STACK_ID.
2. Local proof on one checkout: two stacks with different ids run at once; mutating a runtime flag in one does not affect the other; killing one owner mid-run leaves the other's containers, network and reservation intact; cleanup removes only the terminated stack.
3. Report merger unit tests: missing, failed or cancelled shard is red; duplicate or absent test ids are red.
4. First main run after merge (the workflow executes from main only): both shards complete their inventory, per-phase timings are reported, and the sum of shard Playwright time is compared with the serial baseline.

Out of scope

Second listener activation and host provisioning (slice 3, coordinated with the server-config owner), performance-lane isolation, prebuilt web bundle and recording policy (slice 4).