Supporting React-based Shiny components

> [!NOTE]
> This issue was mostly written over a few rounds of conversation with Claude after exploring the shiny-react code base. I've read through it and largely agree with the recommendations. But I haven't pre-confirmed that they are certainly going to work (I'm happy to find out the hard way 😄)

## Summary

The current shiny-react library assumes the entire page is a single React application. It does not support a use case where multiple independent React components (each with its own `createRoot()`) are embedded within a page that is not itself a React app. This is a common pattern for Shiny custom widgets and module integration.

## Current Architecture

The library uses several global singletons and patterns that prevent multiple React roots from coexisting properly:

### 1. Global Singleton Registries

**react-registry.ts:10** - Single global registry:
```typescript
let reactRegistry: ShinyReactRegistry | undefined = undefined;
```

**message-registry.ts:112** - Single global message registry:
```typescript
const messageRegistry = new ShinyMessageRegistry();
```

**use-shiny.ts:245** - Single initialization flag:
```typescript
let shinyReactInitialized = false;
```

### 2. Single Output Container

**output-registry.ts:52-57** - Creates one hidden container in `document.body`:
```typescript
constructor() {
  const div = document.createElement("div");
  div.className = "shiny-react-output-container";
  div.style.visibility = "hidden";
  this.container = div;
  document.body.appendChild(this.container);
}
```

### 3. Flat ID Maps Without Namespacing

**input-registry.ts:61** - All inputs stored in a single flat map:
```typescript
private inputs: Map<string, InputRegistryEntry<any>> = new Map();
```

**output-registry.ts:48** - All outputs stored in a single flat map:
```typescript
private outputs: Map<string, OutputRegistryEntry<any>> = new Map();
```

### 4. Single Output Binding Registration

**output-registry.ts:171** - One global output binding:
```typescript
shiny.outputBindings.register(new ReactOutputBinding(), "shiny.reactOutput");
```

## Problems

### 1. ID Collisions

If two React sub-apps both use `useShinyInput("counter", 0)`, they will conflict because the InputRegistry uses a flat map keyed by ID. The second component will get the value from the first component's registry entry.

### 2. No Shiny Module Support

Shiny modules use namespacing to allow reusable UI components:
- Server: `ns <- NS(id); output[[ns("plot")]]`
- Client: IDs become `moduleId-plot`

The current shiny-react hooks have no way to specify a namespace prefix, making them incompatible with Shiny modules.

### 3. Shared State Across Roots

Multiple React roots share the same registries. If one root updates `input$foo`, all roots see that change - even if they're supposed to be independent components.

### 4. Lifecycle Conflicts

When a React root unmounts, cleanup in `useEffect` runs:

**use-shiny.ts:103-111**:
```typescript
return () => {
  inputRegistryEntry.removeUseStateSetValueFn(setValue);
  // Registry entry still exists...
};
```

If one root unmounts while another still uses the same ID, the remaining root may have stale or missing data.

### 5. Single Hidden Container

All outputs share one hidden container div. When `bindAll()` is called on this container, it processes all outputs regardless of which React root they belong to. This can cause unnecessary re-binding and potential race conditions.

## Use Cases This Proposal Enables

### A. Multiple Custom Widgets on a Traditional Shiny Page

```r
# ui.R
fluidPage(
  myReactWidget("widget1"),  # Each is its own React root
  myReactWidget("widget2"),  # with createRoot()
  myReactWidget("widget3")
)
```

### B. React Widgets Inside Shiny Modules

```r
# R module
myModuleUI <- function(id) {
  ns <- NS(id)
  myReactWidget(ns("chart"))  # ID should be "moduleId-chart"
}

myModuleServer <- function(id) {
  moduleServer(id, function(input, output, session) {
    output$chart <- renderReactOutput({...})
  })
}
```

### C. Dynamic Widget Creation/Destruction

```r
# Widgets created/destroyed based on user interaction
observeEvent(input$addWidget, {
  insertUI(selector = "#container", ui = myReactWidget(paste0("widget_", input$addWidget)))
})
```

## Proposed Solution

> **Implementation Note:** This section is divided into **REQUIRED** changes that must be implemented, and **FUTURE/OPTIONAL** changes that may be implemented later if needed.

---

## REQUIRED: Core Namespace Support

These changes MUST be implemented to support multiple React roots and Shiny module integration.

### 1. Add Namespace Support to Hooks

Add an optional `namespace` parameter to all hooks:

```typescript
function useShinyInput<T>(
  id: string,
  defaultValue: T,
  options?: {
    debounceMs?: number;
    priority?: EventPriority;
    namespace?: string;  // NEW: prefix for Shiny module support
  }
): [T, (value: T) => void]

function useShinyOutput<T>(
  outputId: string,
  defaultValue?: T,
  options?: {
    namespace?: string;  // NEW
  }
): [T | undefined, boolean]

function useShinyMessageHandler<T>(
  messageType: string,
  handler: (data: T) => void,
  options?: {
    namespace?: string;  // NEW
  }
): void
```

The namespace is prepended to the ID when communicating with Shiny, using `-` as the separator (consistent with Shiny's `NS()` function):
```typescript
const fullId = namespace ? `${namespace}-${id}` : id;
```

**Namespace handling is flat, not nested.** When Shiny modules are nested, the full namespace path (e.g., `"outer-inner"`) is constructed by Shiny's `ns()` function on the server side and passed to the React widget. The React side simply receives and uses the complete namespace string.

### 2. Add ShinyModuleProvider Context

Create a React context that provides the namespace to all child hooks, eliminating the need to pass it to every hook:

```typescript
const ShinyModuleContext = createContext<string | undefined>(undefined);

export function ShinyModuleProvider({
  namespace,
  children
}: {
  namespace: string;
  children: React.ReactNode
}) {
  return (
    <ShinyModuleContext.Provider value={namespace}>
      {children}
    </ShinyModuleContext.Provider>
  );
}

// In hooks:
function useShinyInput<T>(id: string, defaultValue: T, options?: {...}) {
  const contextNamespace = useContext(ShinyModuleContext);
  const namespace = options?.namespace ?? contextNamespace;
  const fullId = namespace ? `${namespace}-${id}` : id;
  // ...
}
```

**Note:** `ShinyModuleProvider` does NOT support nesting. Each provider sets the complete namespace string. For nested Shiny modules, Shiny's `ns()` function already constructs the full path (e.g., `"outer-inner-widget"`), so the React widget receives this complete namespace.

### 3. Update ImageOutput Component

The `ImageOutput` component must read namespace from context and apply it to all internal IDs.

**Note:** `useShinyInitialized` is NOT modified. It returns global Shiny readiness state (WebSocket connected, bindings registered), which is not module-specific.

```typescript
export function ImageOutput({
  id,
  namespace: explicitNamespace,  // NEW: optional override
  // ... other props
}: {
  id: string;
  namespace?: string;  // NEW
  // ...
}) {
  const contextNamespace = useContext(ShinyModuleContext);
  const namespace = explicitNamespace ?? contextNamespace;
  const fullId = namespace ? `${namespace}-${id}` : id;

  const [imgWidth, setImgWidth] = useShinyInput<number | null>(
    ".clientdata_output_" + fullId + "_width", null
  );
  const [imgHeight, setImgHeight] = useShinyInput<number | null>(
    ".clientdata_output_" + fullId + "_height", null
  );
  const [imgHidden] = useShinyInput<boolean>(
    ".clientdata_output_" + fullId + "_hidden", false
  );
  const [imgData, imgRecalculating] = useShinyOutput<ImageData>(fullId, undefined);
  // ...
}
```

### 4. Update Server-Side post_message() Functions

The `post_message()` helper functions in R and Python must automatically namespace message types using `session$ns()` (R) or `session.ns()` (Python).

**R (shinyreact.R):**
```r
post_message <- function(session, type, data) {
  # Apply namespace to message type
  namespaced_type <- session$ns(type)

  session$sendCustomMessage("shinyReactMessage", list(
    type = namespaced_type,
    data = data
  ))
}
```

**Python (shinyreact.py):**
```python
async def post_message(session: Session, type: str, data: Jsonifiable):
    # Apply namespace to message type
    namespaced_type = session.ns(type)

    await session.send_custom_message("shinyReactMessage", {
        "type": namespaced_type,
        "data": data
    })
```

This ensures that when a module server calls `post_message(session, "logEvent", ...)`, the message type becomes `"moduleId-logEvent"` automatically, matching what the client-side `useShinyMessageHandler("logEvent", ...)` expects within a `ShinyModuleProvider`.

---

## Known Limitations of the Shared Registry Approach

The required changes above use namespace prefixing with a shared global registry. This approach is simpler to implement and sufficient for typical use cases, but has known limitations:

### 1. Registry Entries Persist After Unmount

When a React component unmounts, its cleanup removes the `useStateSetValueFn` from the registry entry, but the entry itself persists. This is intentional - it prevents losing state during React re-renders (where cleanup runs between renders).

**Consequences:**
- **Memory accumulation:** Apps that create and destroy many widgets over their lifetime will accumulate stale registry entries that are never cleaned up.
- **Unexpected state restoration:** If a new widget mounts with the same namespace as a previously unmounted widget, it inherits the old widget's state instead of starting fresh. This could be desirable (state persistence) or surprising (stale data), depending on intent.

**Why this is acceptable for typical use:**
- Most Shiny apps don't create/destroy hundreds of widgets dynamically
- Memory per entry is small (a few hundred bytes)
- Shiny module namespaces are typically unique per module instance
- If fresh state is needed, use a unique namespace

### 2. Shared Output Container Rebinds All Outputs

All outputs are placed in a single hidden container. When any output is added or removed, `bindAll()` rebinds *all* outputs in the container:

```typescript
requestAnimationFrame(() => {
  shiny.unbindAll?.(this.container);
  shiny.bindAll?.(this.container);  // Processes ALL outputs, not just the changed one
});
```

**Consequences:**
- Adding/removing one output triggers rebinding of all outputs
- Cost scales with total number of outputs across all widgets

**Why this is acceptable for typical use:**
- Binding operations are cheap (DOM queries, no data re-fetching)
- Most apps have tens of outputs, not hundreds
- Functionally correct - Shiny routes data by ID, no cross-talk

### 3. When These Limitations Matter

These limitations become significant in:
- **Long-running apps** with many dynamically created/destroyed widgets (memory accumulation)
- **Apps with many outputs** where frequent add/remove causes noticeable rebind overhead
- **Scenarios requiring state isolation** where namespace reuse should NOT restore old state

The FUTURE/OPTIONAL section below describes solutions for these scenarios.

---

## FUTURE/OPTIONAL: Scoped Registries

> **Status:** Not required for initial implementation. These options address the limitations described above if real-world usage reveals them to be problematic.

### Future Option: Scoped Registries Per Root

**What it does:** Creates separate `InputRegistry` and `OutputRegistry` instances for each React root, plus separate hidden output containers.

**When to consider:**
- **Memory accumulation:** Scoped registries are garbage-collected when the root unmounts, eliminating stale entries.
- **State isolation:** New widgets with reused namespaces start fresh instead of inheriting old state.
- **Output binding efficiency:** Each root's container only contains its own outputs, so rebinding is scoped.
- **Debugging:** Easier to inspect a specific widget's registry state in isolation.

```typescript
interface ShinyReactOptions {
  namespace?: string;
  container?: HTMLElement;  // Custom container for outputs
}

export function createShinyReactRoot(options?: ShinyReactOptions): ShinyReactRoot {
  return {
    registry: new ShinyReactRegistry(options?.namespace),
    outputContainer: options?.container || createHiddenContainer(),
    namespace: options?.namespace,
  };
}
```

### Future Option: Initialization Per Root

**What it does:** Tracks initialization state per root instead of globally, allowing each root to have its own output binding registration and message registry setup.

**When to consider:**
- **Clean teardown:** With global initialization, registry entries persist forever even after widgets unmount. Per-root initialization allows complete cleanup when a root unmounts - its registry and all entries can be garbage collected.
- **Output binding isolation:** Each root's output binding would only process outputs for that root's container, rather than all outputs in the shared global container.

```typescript
const initializedRoots = new WeakSet<ShinyReactRoot>();

function ensureRootInitialized(root: ShinyReactRoot) {
  if (initializedRoots.has(root)) return;

  initializeRegistryForRoot(root);
  createOutputBindingForRoot(root);
  initializeMessageRegistryForRoot(root);

  initializedRoots.add(root);
}
```

### Future Option: Root Context Provider

**What it does:** Provides a React context that gives components access to their specific root's registry and configuration.

**When to consider:**
- **Required if scoped registries are implemented** - hooks need to know which registry to use.
- **Advanced customization:** If different roots need different configurations (e.g., different debounce defaults, different output containers).

```typescript
const ShinyReactRootContext = createContext<ShinyReactRoot | undefined>(undefined);

export function ShinyReactRootProvider({
  root,
  children
}: {
  root: ShinyReactRoot;
  children: React.ReactNode;
}) {
  return (
    <ShinyReactRootContext.Provider value={root}>
      {children}
    </ShinyReactRootContext.Provider>
  );
}

// Usage in widget entry point:
const root = createShinyReactRoot({ namespace: "myWidget_1" });
const reactRoot = createRoot(container);
reactRoot.render(
  <ShinyReactRootProvider root={root}>
    <MyWidget />
  </ShinyReactRootProvider>
);
```

---

## API Examples After Implementation

### Basic Widget with Namespace

```typescript
// Widget entry point
export function initializeWidget(container: HTMLElement, namespace: string) {
  const reactRoot = createRoot(container);
  reactRoot.render(
    <ShinyModuleProvider namespace={namespace}>
      <MyWidget />
    </ShinyModuleProvider>
  );
}

// Inside the widget - hooks automatically use namespace from context
function MyWidget() {
  const [value, setValue] = useShinyInput("slider", 50);
  const [data] = useShinyOutput("chartData");

  // These communicate as:
  // - input${namespace}-slider
  // - output${namespace}-chartData
}
```

### Multiple Independent Widgets

```typescript
// Each widget gets its own namespace and operates independently
initializeWidget(document.getElementById("widget1"), "mod1");
initializeWidget(document.getElementById("widget2"), "mod2");

// widget1 uses input$mod1-slider, output$mod1-chartData
// widget2 uses input$mod2-slider, output$mod2-chartData
```

### Backward Compatibility

Existing code without namespaces continues to work:

```typescript
// No namespace - works exactly as before
function App() {
  const [value, setValue] = useShinyInput("myInput", "");
  // Uses input$myInput
}
```

## Implementation Summary

### REQUIRED (Must Implement)

**Client-side (shiny-react npm package):**

| Component | Change |
|-----------|--------|
| `useShinyInput` | Add optional `namespace` parameter to options |
| `useShinyOutput` | Add optional `namespace` parameter to options |
| `useShinyMessageHandler` | Add optional `namespace` parameter to options |
| `ShinyModuleProvider` | New context provider component |
| `ShinyModuleContext` | New React context (exported for advanced use) |
| `ImageOutput` | Add `namespace` prop, read from context |
| `useShinyInitialized` | No changes (global Shiny state, not module-specific) |

**Server-side (shinyreact.R / shinyreact.py):**

| Component | Change |
|-----------|--------|
| `post_message()` (R) | Use `session$ns()` to namespace message types |
| `post_message()` (Python) | Use `session.ns()` to namespace message types |

### FUTURE/OPTIONAL (Do Not Implement Now)

| Component | Change | When to Consider |
|-----------|--------|------------------|
| `createShinyReactRoot()` | Scoped registry factory | If shared registry causes issues |
| `ShinyReactRootProvider` | Root isolation context | If full isolation is needed |
| Per-root initialization | Separate init per root | If shared init causes conflicts |

## Documentation Requirements

After implementation, documentation should cover:

1. **Full-page React apps** - Using shiny-react without namespaces (existing pattern, unchanged)

2. **Widget-style usage** - Creating React widgets that can be embedded in traditional Shiny pages:
   - How to wrap widget entry point with `ShinyModuleProvider`
   - How namespace flows from R/Python UI to React

3. **Shiny module integration** - Using React widgets inside Shiny modules:
   - How `ns()` on the server creates the namespace string
   - How to pass that namespace to `ShinyModuleProvider`
   - Example showing nested modules

4. **Message handling in modules** - How `post_message()` and `useShinyMessageHandler` work with namespaces


Component	Change
`useShinyInput`	Add optional `namespace` parameter to options
`useShinyOutput`	Add optional `namespace` parameter to options
`useShinyMessageHandler`	Add optional `namespace` parameter to options
`ShinyModuleProvider`	New context provider component
`ShinyModuleContext`	New React context (exported for advanced use)
`ImageOutput`	Add `namespace` prop, read from context
`useShinyInitialized`	No changes (global Shiny state, not module-specific)

Component	Change
`post_message()` (R)	Use `session$ns()` to namespace message types
`post_message()` (Python)	Use `session.ns()` to namespace message types

Component	Change	When to Consider
`createShinyReactRoot()`	Scoped registry factory	If shared registry causes issues
`ShinyReactRootProvider`	Root isolation context	If full isolation is needed
Per-root initialization	Separate init per root	If shared init causes conflicts

Supporting React-based Shiny components #2

Description

Summary

Current Architecture

1. Global Singleton Registries

2. Single Output Container

3. Flat ID Maps Without Namespacing

4. Single Output Binding Registration

Problems

1. ID Collisions

2. No Shiny Module Support

3. Shared State Across Roots

4. Lifecycle Conflicts

5. Single Hidden Container

Use Cases This Proposal Enables

A. Multiple Custom Widgets on a Traditional Shiny Page

B. React Widgets Inside Shiny Modules

C. Dynamic Widget Creation/Destruction

Proposed Solution

REQUIRED: Core Namespace Support

1. Add Namespace Support to Hooks

2. Add ShinyModuleProvider Context

3. Update ImageOutput Component

4. Update Server-Side post_message() Functions

Known Limitations of the Shared Registry Approach

1. Registry Entries Persist After Unmount

2. Shared Output Container Rebinds All Outputs

3. When These Limitations Matter

FUTURE/OPTIONAL: Scoped Registries

Future Option: Scoped Registries Per Root

Future Option: Initialization Per Root

Future Option: Root Context Provider

API Examples After Implementation

Basic Widget with Namespace

Multiple Independent Widgets

Backward Compatibility

Implementation Summary

REQUIRED (Must Implement)

FUTURE/OPTIONAL (Do Not Implement Now)

Documentation Requirements

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions