Flaky E2E test in rpc_session_state.e2e.test.ts (model switchto)

[Morabbin]

The test should call session rpc model switchto is flaky, particularly on Windows CI.

It has a race condition where it awaits session.model_change after switchTo completes, frequently missing the event if it fires too quickly. It also incorrectly hardcodes gpt-4.1 expectations which fails on environments without auth.

[github-actions]

Investigation findings

Verdict: confirmed bug — the test should call session rpc model switchto in nodejs/test/e2e/rpc_session_state.e2e.test.ts has two distinct problems that together explain the flaky behaviour on Windows CI.

1. Wrong assertion (expect(after.modelId).toBe(before.modelId))

After calling session.rpc.model.switchTo({ modelId: "gpt-4.1", ... }), the test calls session.rpc.model.getCurrent() and expects the result to still equal the original model:

expect(after.modelId).toBe(before.modelId);   // expects OLD model — wrong

This directly contradicts:

• The skipped sibling test in rpc.e2e.test.ts (line 104), which was written to test the same feature and correctly expects the new model:

  expect(after.modelId).toBe("gpt-4.1");   // expects NEW model — correct

• The behaviour confirmed by session.e2e.test.ts tests (should set model on existing session, should set model with reasoningEffort), which verify via the session.model_change event that the model is updated to the switched value.

The wrong assertion is what makes the test flaky: because the CLI may update its internal state and emit the session.model_change event asynchronously after returning the JSON-RPC response to switchTo, getCurrent() sometimes races with that state update. When getCurrent() wins the race (state not yet updated) the wrong assertion accidentally passes; when the state update wins (more likely under different OS scheduling on Windows), getCurrent() returns "gpt-4.1" and the wrong assertion fails.

2. Missing event-subscription pattern (race condition)

session.e2e.test.ts subscribes to session.model_change before triggering the switch:

// nodejs/test/e2e/session.e2e.test.ts:960
const modelChangePromise = getNextEventOfType(session, "session.model_change");
await session.setModel("gpt-4.1");
const event = await modelChangePromise;

The rpc_session_state test never subscribes to this event at all. The correct pattern for verifying the post-switch state would be to wait for the model_change event (to confirm the CLI has committed the switch) before calling getCurrent().

Files investigated

File	Relevance
nodejs/test/e2e/rpc_session_state.e2e.test.ts (lines 52–71)	Bug location — active test with wrong assertion
nodejs/test/e2e/rpc.e2e.test.ts (lines 84–105)	Skipped test with the correct expect(after.modelId).toBe("gpt-4.1") assertion
nodejs/test/e2e/session.e2e.test.ts (lines 956–981)	Reference for correct subscribe-before-switch pattern
test/snapshots/rpc_session_state/should_call_session_rpc_model_switchto.yaml	Snapshot only has claude-sonnet-4.5; gpt-4.1 works because the CLI does not validate model IDs against the /models endpoint

Suggested fix

1. Subscribe to session.model_change before calling switchTo.
2. After awaiting the event, call getCurrent() and assert after.modelId === "gpt-4.1".
3. Remove or move the it.skip in rpc.e2e.test.ts once the active test is fixed (it already has the correct assertions).

Generated by Bug Handler for issue #1629 · sonnet46 2.4M · ◷