Docs: hosting CliHost from a GUI-subsystem (WinExe) app leaks literal ANSI; AttachConsole alone is not enough

[tig]

What happened

I hosted CliHost inside a WinForms app (tig/mcec#252): OutputType=WinExe, with the classic hybrid pattern of AttachConsole(ATTACH_PARENT_PROCESS) when args are present so --help/--version/--opencli print to the parent terminal.

--version and --opencli were fine, but myapp help printed the rendered help with ANSI escapes as literal text ([39m[49m[1m## Commands...), and the mangled lines wrapped, wrecking the table layout.

Root cause

MarkdownRenderer.RenderToAnsi writes VT escape sequences produced by Driver.ToAnsi(). Every shipped host (greet, clet, the survey example in #9) is a console-subsystem app, which inherits a console session whose output mode the shell/terminal already set up (VT processing on). A GUI-subsystem process that attaches to a console does not get ENABLE_VIRTUAL_TERMINAL_PROCESSING on its output, so conhost renders the escape bytes as text.

The fix on the host side is one call after AttachConsole:

IntPtr stdout = GetStdHandle(STD_OUTPUT_HANDLE);
if (GetConsoleMode(stdout, out uint mode)) {
    SetConsoleMode(stdout, mode | ENABLE_VIRTUAL_TERMINAL_PROCESSING);
}

Suggestions

1. Docs: add a "Hosting from a GUI app (WinExe)" section to the README/library spec: AttachConsole + enable VT processing, plus the caveat that the shell prompt returns immediately for a WinExe. None of the examples cover this shape, and it is a natural one for TG.Cli (an existing GUI app that wants a scriptable CLI surface without becoming a console app).
2. Library (optional but nicer): MarkdownRenderer.RenderToAnsi already fixes up the console when writing to the real console (Console.OutputEncoding → UTF-8). Enabling VT processing on Windows in that same spot would make every host correct by construction, including hybrids, with no behavior change for console apps (the flag is already set there).

Happy to PR either.

[tig]

Correction to the suggested snippet after testing it for real: GetStdHandle(STD_OUTPUT_HANDLE) can return NULL even after a successful AttachConsole; a GUI-subsystem process's std-handle table is launcher-dependent and AttachConsole does not reliably reinitialize it. The robust form opens the attached console's active screen buffer directly; the output mode is a property of the buffer, not of any handle, so this covers every writer:

IntPtr conout = CreateFile("CONOUT$", GENERIC_READ | GENERIC_WRITE,
    FILE_SHARE_READ | FILE_SHARE_WRITE, IntPtr.Zero, OPEN_EXISTING, 0, IntPtr.Zero);
if (conout != IntPtr.Zero && conout != INVALID_HANDLE_VALUE) {
    if (GetConsoleMode(conout, out uint mode)) {
        SetConsoleMode(conout, mode | ENABLE_VIRTUAL_TERMINAL_PROCESSING);
    }
    CloseHandle(conout);
}

If the library takes the "enable VT in MarkdownRenderer" route, it should use CONOUT$ too for the same reason. Working implementation: https://github.com/tig/mcec/blob/claude/headless-estop/src/Helpers/ConsoleNativeMethods.cs

[tig]

Follow-up with the full root cause, after building a ConPTY harness that types commands into an interactive pwsh and captures the terminal-bound byte stream:

1. The VT-processing point above is real but only affects SGR color rendering on legacy conhost. It was NOT the main symptom.
2. The main symptom: bare myapp help dispatches HelpCommand (a Viewer) WITHOUT --cat, which launches the full interactive Terminal.Gui help browser. From a console-subsystem host that is a feature. From a GUI-subsystem (WinExe) host it can never work: the shell resumed reading input the moment the process launched (shells do not wait for GUI-subsystem exes), so the TUI fights the shell for the same console input queue while painting absolute-positioned frames (ESC[H, ESC[<n>X erases, an ESC]0;Help title change) over the shell's freshly-drawn prompt. The user experiences it as "ANSI leaking through and the layout is wrong."

Our fix: the host appends --cat to viewer invocations, so help/agent-guide render via RenderCatAsync as flowing text. Verified in the harness: zero cursor-home/title/frame sequences after the change. Filing a separate issue proposing a first-class CliHostOptions switch for this plus docs.