singularity-forge/web/lib/pty-manager.ts
Andrew 815be0a698 feat: managed RTK integration with opt-in preference and web UI toggle (#2620)
* feat: integrate managed RTK across shell workflows

* fix(rtk): unify managed fallback and live savings wiring

* fix(rtk): improve TUI status visibility

* fix(tests): make portability tests independent of pi-coding-agent dist build

The CI portability test runs don't guarantee that
packages/pi-coding-agent has been compiled. Any test that
imported files pulling in @gsd/pi-coding-agent (resource-loader,
preferences-skills, async-bash-tool, etc.) crashed with
ERR_MODULE_NOT_FOUND pointing at dist/index.js.

Two changes to dist-redirect.mjs (the Node ESM loader hook used by
all unit tests):
- Redirect the bare @gsd/pi-coding-agent specifier to the workspace
  source entrypoint (src/index.ts) so no dist/ artifact is needed.
- Extend the load() hook to transpile *.ts files under
  packages/pi-coding-agent/src/ through TypeScript's transpileModule.
  Node's --experimental-strip-types can't handle parameter properties
  and similar syntax present in that package's source; full transpilation
  avoids the ERR_UNSUPPORTED_TYPESCRIPT_SYNTAX crash.

Also fix the dashboard.tsx responsive grid:
- xl:grid-cols-5 → xl:grid-cols-4 2xl:grid-cols-5
  (5 metric cards no longer fit at xl without overflow; test contract
  expected xl:grid-cols-4)
- Keep loading-skeletons.tsx in sync with the same breakpoints.

Add src/tests/resolve-ts-loader.test.ts to guard the loader behaviour:
- bare @gsd/pi-coding-agent redirect points to workspace source
- direct source-entry rewrite (.js → .ts)
- transpilation removes TS parameter property syntax that strip-only
  mode cannot parse

* fix(tests): redirect all workspace package imports to source in portability tests

The previous fix only redirected @gsd/pi-coding-agent to its
source entrypoint. In CI, pi-coding-agent/src itself imports
@gsd/pi-ai (and other workspace packages) which were still pointing
at dist/. Since no workspace dist is built during the portability
test run, any transitive resolution hit the same ERR_MODULE_NOT_FOUND.

Changes to dist-redirect.mjs:
- Redirect @gsd/pi-ai, @gsd/pi-ai/oauth, @gsd/pi-agent-core, and
  @gsd/pi-tui bare imports to their workspace src/ entrypoints.
- Broaden the load() transpilation condition from
  '/packages/pi-coding-agent/src/' to '/packages/*/src/' so that
  all workspace source files are run through TypeScript's
  transpileModule, handling parameter properties and other syntax
  that Node's strip-only mode rejects.

Verified by hiding all four workspace dist/ directories locally and
running the failing test set — 96/96 pass.

* fix(tests): redirect @gsd/native sub-paths; fix Windows .cmd spawnSync

Two more portability failures after the previous fix:

1. @gsd/native sub-path imports (@gsd/native/fd, @gsd/native/text, etc.)
   were not redirected — the loader only handled the bare specifier.
   Added a prefix-match redirect for @gsd/native/* → packages/native/src/<sub>/index.ts.

2. Windows RTK tests failed because createFakeRtk produces a .cmd wrapper
   on Windows, and spawnSync(binaryPath, [...]) without shell:true silently
   returns non-zero when the binary is a .cmd file.
   Added shell: /\.(cmd|bat)$/i.test(binaryPath) to the spawnSync calls in:
   - src/resources/extensions/shared/rtk.ts (rewriteCommandWithRtk)
   - src/resources/extensions/shared/rtk-session-stats.ts (readCurrentRtkGainSummary)
   - packages/pi-coding-agent/src/utils/rtk.ts (rewriteCommandForGsd)
   Production use of rtk.exe is unaffected; the shell flag is only true for
   .cmd/.bat paths.

Verified: all 93 portability tests pass with all workspace dist/ directories
removed (simulating CI portability environment).

* fix(tests): Windows portability fixes — HOME env, managed RTK path, perf threshold

Four Windows-specific failures fixed:

1. app-smoke.test.ts: process.env.HOME is undefined on Windows (uses
   USERPROFILE instead). Changed to homedir() from node:os which works
   cross-platform.

2. Managed RTK path tests on Windows: tests placed a fake RTK as rtk.exe
   (by copying a .cmd script into a .exe filename), which Windows cannot
   execute. Two-part fix:
   - resolveRtkBinaryPath() in both rtk.ts files now falls back to rtk.cmd
     in the managed dir on Windows when rtk.exe is absent.
   - withManagedFakeRtk and equivalent patterns in rtk.test.ts,
     rtk-session-stats.test.ts, rtk-execution-seams.test.ts changed to
     place the fake at rtk.cmd instead of rtk.exe on Windows.

3. bg_shell RTK test on Windows: requires bash (for shell sessions), which
   is not available on the blacksmith-4vcpu-windows-2025 runner without
   Git Bash installed. Test now skips on win32.

4. derive-state-db perf assertion: 10ms threshold was too tight for Windows
   CI runners (measured 12ms under load). Raised to 25ms — still catches
   real regressions (baseline is 3ms locally and ~12ms on stressed runners).

* fix(tests): fix managed RTK path fallback on Windows in src/rtk.ts + fix copyable fake

Two remaining Windows failures:

1. src/rtk.ts was never patched with the rtk.cmd managed-dir fallback
   (only the shared/rtk.ts and pi-coding-agent/src/utils/rtk.ts were updated).
   Added the same rtk.cmd fallback and shell:.cmd detection to src/rtk.ts,
   which is what rtk.test.ts imports from.

2. createFakeRtk on Windows wrote '%~dp0\fake-rtk.js' in the .cmd content —
   this resolves relative to the .cmd file's own directory. When the test
   copies rtk.cmd to a different managed dir, %~dp0 resolves to the copy
   destination where fake-rtk.js does not exist. Fixed by embedding the
   absolute path to fake-rtk.js directly in the .cmd content so the fake
   works correctly regardless of where the .cmd is copied.

* feat(experimental): add RTK opt-in preference with web UI toggle

- Add `experimental` category to GSDPreferences with `rtk: boolean` (default: false)
- RTK is now opt-in: disabled by default for all projects unless explicitly enabled
- Validate experimental.* keys; unknown experimental keys produce warnings

Web UI:
- Add ExperimentalPanel component with animated toggle switch per flag
- Add /api/experimental route (GET/PATCH) to read/write flags in preferences.md
- Add 'Experimental' tab to settings dialog sidebar nav (FlaskConical icon)
- Include ExperimentalPanel at bottom of gsd-prefs mega-scroll
- Fix toggle disabled state: trigger loadSettingsData for 'experimental' section
  and self-fetch on mount when data is absent

Dashboard:
- Gate RTK Saved metric card on rtkEnabled from live auto state (web)
- Gate TUI dashboard RTK savings row on rtkEnabled
- Gate TUI footer RTK status updates on experimental.rtk preference
- Propagate rtkEnabled through AutoDashboardData → bridge-service → store

Build:
- Add scripts/build-if-stale.cjs: incremental build driver that skips each
  step (packages, root tsc, copy-resources, web) when output is newer than
  source; replaces full rebuild chain in gsd:web
- Add scripts/web-stop.cjs: robust stop with registry + legacy PID + orphan
  sweep via pgrep; handles crash/restart orphaned next-server processes
- gsd:web now uses build-if-stale.cjs (fast cold starts, instant when unchanged)
- gsd:web:stop / gsd:web:stop:all use web-stop.cjs directly

Fix: correct import path in rtk-status.ts (./preferences.js not ../preferences.js)

* fix: restore em-dash encoding in package.json to match upstream

* refactor(rtk): move command rewrite out of pi-coding-agent into GSD extension

Per review feedback from igouss: pi-coding-agent should not be modified to add
GSD-specific logic. Instead, add a proper extension point and wire RTK through it.

Changes to packages/pi-coding-agent (extension API only — no RTK logic):
- Add BashTransformEvent + BashTransformEventResult types to extension API
- Add on('bash_transform') overload to ExtensionAPI interface
- Add emitBashTransform() to ExtensionRunner (chains all handlers in order)
- Call emitBashTransform() in wrapToolWithExtensions before bash tool execution
- Export new types from extensions/index.ts and package index.ts
- Revert all RTK-specific changes from bash-executor.ts, tools/bash.ts
- Remove packages/pi-coding-agent/src/utils/rtk.ts entirely

Changes to GSD extension:
- Register bash_transform handler in register-hooks.ts that calls
  rewriteCommandWithRtk() from the existing shared/rtk.ts module
- Handler is a no-op when RTK is disabled or not installed

* fix: correct import path for shared/rtk.js in register-hooks

* fix(tests): remove deleted pi-coding-agent/utils/rtk imports from execution seams test

The RTK rewrite logic was moved out of pi-coding-agent into the GSD
extension (bash_transform hook). Tests that directly imported the
deleted utils/rtk.ts are removed; remaining tests verify the shared
RTK module and GSD-layer surfaces that still call rewriteCommandWithRtk.
2026-03-26 09:33:07 -06:00

440 lines
12 KiB
TypeScript

/**
* Server-side PTY manager — spawns and manages pseudo-terminal instances.
*
* Each terminal session gets a unique ID. PTY output is buffered and streamed
* to clients via SSE; input arrives via POST.
*/
import { chmodSync, existsSync, statSync } from "node:fs";
import { basename, join, dirname } from "node:path";
import type { IPty } from "node-pty";
import { resolveGsdCliEntry } from "../../src/web/cli-entry.ts";
// Webpack escape hatch — this global exists at runtime in webpack bundles and
// forwards to Node's native require(), bypassing webpack's module resolution.
declare const __non_webpack_require__: NodeRequire;
export interface PtySession {
id: string;
pty: IPty;
listeners: Set<(data: string) => void>;
alive: boolean;
buffer: string[];
bufferedBytes: number;
}
interface LoadedNodePty {
nodePtyModule: typeof import("node-pty");
packageRoot: string;
}
// Use globalThis to persist across Turbopack/HMR module re-evaluations in dev
const GLOBAL_KEY = "__gsd_pty_sessions__" as const;
const CLEANUP_GUARD_KEY = "__gsd_pty_cleanup_installed__" as const;
const MAX_SESSION_BUFFER_BYTES = 1024 * 1024;
function getSessions(): Map<string, PtySession> {
const g = globalThis as Record<string, unknown>;
if (!g[GLOBAL_KEY]) {
g[GLOBAL_KEY] = new Map<string, PtySession>();
}
return g[GLOBAL_KEY] as Map<string, PtySession>;
}
function getChunkByteLength(data: string): number {
return Buffer.byteLength(data, "utf8");
}
function appendToSessionBuffer(session: PtySession, data: string): void {
if (!data) return;
session.buffer.push(data);
session.bufferedBytes += getChunkByteLength(data);
while (session.bufferedBytes > MAX_SESSION_BUFFER_BYTES && session.buffer.length > 1) {
const removed = session.buffer.shift();
if (!removed) break;
session.bufferedBytes -= getChunkByteLength(removed);
}
}
function destroyAllSessions(): void {
const map = getSessions();
for (const [sessionId, session] of map.entries()) {
session.alive = false;
try {
session.pty.kill();
} catch {
// Already dead.
}
session.listeners.clear();
map.delete(sessionId);
}
}
function ensureProcessCleanupHandlers(): void {
const g = globalThis as Record<string, unknown>;
if (g[CLEANUP_GUARD_KEY]) return;
g[CLEANUP_GUARD_KEY] = true;
const cleanup = () => {
destroyAllSessions();
};
process.once("exit", cleanup);
process.once("SIGINT", () => {
cleanup();
process.exit(130);
});
process.once("SIGTERM", () => {
cleanup();
process.exit(143);
});
process.once("SIGHUP", () => {
cleanup();
process.exit(129);
});
}
function getDefaultShell(): string {
if (process.platform === "win32") return "powershell.exe";
return process.env.SHELL || "/bin/zsh";
}
function getProjectCwd(): string {
return process.env.GSD_WEB_PROJECT_CWD || process.cwd();
}
function getShellArgs(): string[] {
// Launch an interactive login shell with the user's normal config.
// Previously we passed -f / --norc to skip rc files, but that removed the
// user's prompt, PATH, aliases, etc. — making the terminal feel broken.
// History pollution is already prevented via HISTFILE=/dev/null in the env.
return [];
}
interface TerminalSpawnSpec {
executable: string;
args: string[];
label: string;
}
const ALLOWED_TERMINAL_COMMANDS = new Set([
"gsd",
process.env.SHELL || "/bin/zsh",
"/bin/bash",
"/bin/zsh",
"/bin/sh",
]);
export function isAllowedTerminalCommand(command?: string): boolean {
if (!command) return true;
return ALLOWED_TERMINAL_COMMANDS.has(command);
}
function resolveTerminalSpawnSpec(cwd: string, command?: string, commandArgs: string[] = []): TerminalSpawnSpec {
if (!command) {
const shell = getDefaultShell();
return {
executable: shell,
args: getShellArgs(),
label: basename(shell),
};
}
if (command === "gsd") {
try {
const cliEntry = resolveGsdCliEntry({
packageRoot: process.env.GSD_WEB_PACKAGE_ROOT || process.cwd(),
cwd,
execPath: process.execPath,
hostKind: process.env.GSD_WEB_HOST_KIND,
mode: "interactive",
messages: commandArgs,
});
return {
executable: cliEntry.command,
args: cliEntry.args,
label: "gsd",
};
} catch (error) {
console.warn(
"[pty] Falling back to PATH-resolved gsd:",
error instanceof Error ? error.message : String(error),
);
}
}
return {
executable: command,
args: commandArgs,
label: basename(command),
};
}
function getNodePtyCandidateRoots(): string[] {
const roots = new Set<string>();
roots.add(process.cwd());
const packageRoot = process.env.GSD_WEB_PACKAGE_ROOT;
if (packageRoot) {
roots.add(packageRoot);
roots.add(join(packageRoot, "dist", "web", "standalone"));
roots.add(join(packageRoot, "web"));
}
return Array.from(roots);
}
function hasNativeAssets(packageRoot: string): boolean {
const prebuildDir = join(packageRoot, "prebuilds", `${process.platform}-${process.arch}`);
return (
existsSync(join(prebuildDir, "pty.node")) ||
existsSync(join(packageRoot, "build", "Release", "pty.node")) ||
existsSync(join(packageRoot, "build", "Debug", "pty.node"))
);
}
function loadNodePty(): LoadedNodePty {
const failures: string[] = [];
for (const root of getNodePtyCandidateRoots()) {
// Probe for node-pty's package.json directly in node_modules under this root.
// We avoid createRequire from node:module because webpack mangles it in
// Next.js standalone builds — the import gets swallowed/replaced with
// undefined since webpack treats `module` as its own internal concept.
const candidate = join(root, "node_modules", "node-pty", "package.json");
if (!existsSync(candidate)) {
failures.push(`${root}: node-pty not found`);
continue;
}
try {
const packageRoot = dirname(candidate);
if (!hasNativeAssets(packageRoot)) {
failures.push(`${packageRoot}: missing native assets`);
continue;
}
// node-pty is listed in serverExternalPackages, but webpack still
// processes require() calls with computed paths — it replaces them with
// a "module not found" stub. We use __non_webpack_require__ (webpack's
// escape hatch) so the require passes through to Node's native loader
// at runtime.
//
// The bare `require` fallback is wrapped in Function() to prevent
// webpack from statically analyzing it and emitting a "critical
// dependency" warning. At runtime in non-webpack environments (e.g.
// tests) this produces an identical NodeRequire function.
const nativeRequire: NodeRequire = typeof __non_webpack_require__ !== "undefined"
? __non_webpack_require__
: new Function("return require")() as NodeRequire;
const nodePtyModule = nativeRequire(join(packageRoot, "lib", "index.js")) as typeof import("node-pty");
return { nodePtyModule, packageRoot };
} catch (error) {
failures.push(
`${root}: ${error instanceof Error ? error.message : String(error)}`,
);
}
}
throw new Error(
`Failed to load node-pty with native assets. Tried: ${failures.join(" | ") || "no candidate roots"}`,
);
}
export function getOrCreateSession(sessionId: string, projectCwd?: string, command?: string, commandArgs: string[] = []): PtySession {
ensureProcessCleanupHandlers();
if (!isAllowedTerminalCommand(command)) {
throw new Error(`Command not allowed: ${command}`);
}
const map = getSessions();
const existing = map.get(sessionId);
if (existing?.alive) return existing;
// Clean up dead session if it exists
if (existing) {
map.delete(sessionId);
}
const { nodePtyModule: pty, packageRoot: nodePtyRoot } = loadNodePty();
// Ensure the spawn-helper binary is executable (npm doesn't always preserve permissions)
try {
const helperPath = join(
nodePtyRoot,
"prebuilds",
`${process.platform}-${process.arch}`,
"spawn-helper",
);
if (existsSync(helperPath)) {
const st = statSync(helperPath);
if ((st.mode & 0o111) === 0) {
chmodSync(helperPath, st.mode | 0o755);
console.log("[pty] Fixed spawn-helper permissions:", helperPath);
}
}
} catch (e) {
console.warn("[pty] Could not check spawn-helper:", e);
}
const cwd = projectCwd || getProjectCwd();
const spawnSpec = resolveTerminalSpawnSpec(cwd, command, commandArgs);
console.log("[pty] Spawning command:", spawnSpec.label, "cwd:", cwd, "node-pty:", nodePtyRoot);
// Build a clean env — remove GSD-specific vars that would confuse a shell
const cleanEnv: Record<string, string> = {};
for (const [key, value] of Object.entries(process.env)) {
if (value !== undefined && !key.startsWith("GSD_WEB_")) {
cleanEnv[key] = value;
}
}
cleanEnv.TERM = "xterm-256color";
cleanEnv.COLORTERM = "truecolor";
cleanEnv.HISTFILE = "/dev/null";
cleanEnv.HISTSIZE = "0";
cleanEnv.SAVEHIST = "0";
cleanEnv.LESSHISTFILE = "/dev/null";
cleanEnv.NODE_REPL_HISTORY = "/dev/null";
if (command) {
cleanEnv.GSD_WEB_PTY = "1";
}
let ptyProcess: IPty;
try {
ptyProcess = pty.spawn(spawnSpec.executable, spawnSpec.args, {
name: "xterm-256color",
cols: 120,
rows: 30,
cwd,
env: cleanEnv,
});
console.log("[pty] Spawned pid:", ptyProcess.pid);
} catch (spawnError) {
console.error("[pty] Spawn failed:", spawnError);
console.error("[pty] Command:", spawnSpec.executable, "Args:", spawnSpec.args, "CWD:", cwd);
console.error("[pty] CWD exists:", existsSync(cwd));
throw spawnError;
}
const session: PtySession = {
id: sessionId,
pty: ptyProcess,
listeners: new Set(),
alive: true,
buffer: [],
bufferedBytes: 0,
};
ptyProcess.onData((data: string) => {
appendToSessionBuffer(session, data);
for (const listener of session.listeners) {
try {
listener(data);
} catch {
// Listener may have been removed during iteration
}
}
});
ptyProcess.onExit(({ exitCode, signal }) => {
session.alive = false;
// Notify listeners about exit
const exitMessage = `\r\n\x1b[90m[Process exited with code ${exitCode}${signal ? `, signal ${signal}` : ""}]\x1b[0m\r\n`;
appendToSessionBuffer(session, exitMessage);
for (const listener of session.listeners) {
try {
listener(exitMessage);
} catch {
// ignore
}
}
});
map.set(sessionId, session);
return session;
}
export function writeToSession(sessionId: string, data: string): boolean {
const session = getSessions().get(sessionId);
if (!session?.alive) return false;
session.pty.write(data);
return true;
}
export function resizeSession(
sessionId: string,
cols: number,
rows: number,
): boolean {
const session = getSessions().get(sessionId);
if (!session?.alive) return false;
try {
session.pty.resize(cols, rows);
return true;
} catch {
return false;
}
}
export function destroySession(sessionId: string): boolean {
const map = getSessions();
const session = map.get(sessionId);
if (!session) return false;
session.alive = false;
try {
session.pty.kill();
} catch {
// Already dead
}
session.listeners.clear();
map.delete(sessionId);
return true;
}
export function addListener(
sessionId: string,
listener: (data: string) => void,
): (() => void) | null {
const session = getSessions().get(sessionId);
if (!session) return null;
const snapshot = session.buffer.slice();
session.listeners.add(listener);
for (const chunk of snapshot) {
try {
listener(chunk);
} catch {
session.listeners.delete(listener);
return null;
}
}
return () => {
session.listeners.delete(listener);
};
}
export function isSessionAlive(sessionId: string): boolean {
const session = getSessions().get(sessionId);
return session?.alive ?? false;
}
export interface PtySessionInfo {
id: string;
alive: boolean;
pid: number | undefined;
}
export function listSessions(): PtySessionInfo[] {
const map = getSessions();
return Array.from(map.values()).map((s) => ({
id: s.id,
alive: s.alive,
pid: s.pty.pid,
}));
}