Skip to content

Commit 2a2745a

Browse files
committed
Update design docs to reflect completed work
- implementation-plan.md: Mark Phase 5, 7, 9C steps as DONE, update API refs from engine/server to backend, fix selection descriptions, check off paginateSubRows/Parquet/Web Worker deferred items, add checkbox maintenance note - duckdb-wasm-engine.md: Add API rename note, update tables and summary to backend = backendDuckDB(), update CDN decision, mark paginateSubRows DONE - server-side-data.md: Update backend options and usage examples, fix test matrix (Group/Selection bugs fixed, add DuckDB rows), mark Phase 4 selection DONE, resolve Key Challenges, update expanded/select-all descriptions
1 parent c4046f7 commit 2a2745a

File tree

3 files changed

+114
-104
lines changed

3 files changed

+114
-104
lines changed

design/duckdb-wasm-engine/duckdb-wasm-engine.md

Lines changed: 29 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,9 @@
11
# DuckDB-WASM Data Engine for Reactable
22

3+
> **API note:** This doc was written before the Phase 7 `backend =` refactor. Code examples using
4+
> `engine = "duckdb"` and `server = "duckdb"` should be read as `backend = backendDuckDB()` and
5+
> `backend = backendDuckDB("server")` respectively. The `engine` param was removed and never released.
6+
37
## Problem Statement
48

59
reactable has three modes for handling data, each with significant limitations:
@@ -815,13 +819,17 @@ DuckDB-WASM runs in a Web Worker:
815819
816820
### CDN vs. self-hosted
817821
818-
Can use jsDelivr CDN for the WASM files (simplest, no self-hosting required) or bundle them with the R package.
822+
~~Can use jsDelivr CDN for the WASM files (simplest, no self-hosting required) or bundle them with the R package.~~
823+
824+
~~CDN pros: cached across sites, no package size increase, always latest version.~~
825+
~~CDN cons: requires internet, reliability depends on CDN, not available in air-gapped environments.~~
819826
820-
CDN pros: cached across sites, no package size increase, always latest version.
821-
CDN cons: requires internet, reliability depends on CDN, not available in air-gapped environments.
827+
~~**Recommendation:** Default to jsDelivr CDN. Add `options(reactable.duckdb.wasm.url = "local/path")` for
828+
self-hosted/offline use.~~
822829
823-
**Recommendation:** Default to jsDelivr CDN. Add `options(reactable.duckdb.wasm.url = "local/path")` for
824-
self-hosted/offline use.
830+
**Decision:** Self-hosted/bundled. WASM files are bundled in `inst/htmlwidgets/lib/duckdb-wasm/`. CDN was
831+
rejected because it fails for air-gapped corporate environments. CRAN size limit (~5 MB) is an open concern
832+
since `duckdb-eh.wasm` is ~32.7 MB.
825833
826834
### Query execution is sequential
827835
@@ -933,12 +941,12 @@ directly from the R data frame's memory without copying.
933941
934942
### When to use which
935943
936-
| Scenario | Recommended mode |
937-
| ------------------------------------------ | --------------------------------------------- |
938-
| Static HTML, R Markdown, Quarto (<2M rows) | `engine = "duckdb"` (WASM, client-side) |
939-
| Shiny, data must stay on server | `server = "duckdb"` (R backend) |
940-
| Shiny, don't mind client seeing data | `engine = "duckdb"` (WASM, offloads R server) |
941-
| Very large data (>2M rows, static doc) | `engine = "duckdb"` + Parquet sidecar file |
944+
| Scenario | Recommended mode |
945+
| ------------------------------------------ | ---------------------------------------------------------- |
946+
| Static HTML, R Markdown, Quarto (<2M rows) | `backend = backendDuckDB()` (WASM, client-side) |
947+
| Shiny, data must stay on server | `backend = backendDuckDB("server")` (R backend) |
948+
| Shiny, don't mind client seeing data | `backend = backendDuckDB()` (WASM, offloads R server) |
949+
| Very large data (>2M rows, static doc) | `backend = backendDuckDB()` + Parquet sidecar file |
942950
| Small tables (<10K rows) | Default JSON path (no engine needed) |
943951
944952
---
@@ -1028,13 +1036,10 @@ special handling:
10281036

10291037
This is more complex than flat pagination but entirely feasible with the SQL model.
10301038

1031-
### `paginateSubRows` not supported
1032-
1033-
`paginateSubRows = TRUE` is not implemented for the DuckDB engine (WASM or R server). When groups are expanded,
1034-
sub-rows are added on top of the page size (the default `paginateSubRows = FALSE` behavior). This matches the df
1035-
and dt server backends — only the V8 backend supports `paginateSubRows`.
1039+
### ~~`paginateSubRows` not supported~~ DONE
10361040

1037-
See [paginate-sub-rows.md](paginate-sub-rows.md) for the design plan to implement this.
1041+
`paginateSubRows = TRUE` is now implemented for all backends: DuckDB WASM (client), DuckDB R server, and df server.
1042+
See [paginate-sub-rows.md](paginate-sub-rows.md) for the design.
10381043

10391044
---
10401045

@@ -1081,12 +1086,12 @@ and memory than it saves on queries.
10811086

10821087
### Recommended target range
10831088

1084-
| Use case | Recommended approach |
1085-
| -------------------------------------------------- | -------------------------------------------------------------------- |
1086-
| Static HTML (R Markdown / Quarto) up to ~200K rows | `engine = "duckdb"` (WASM) all operations <300ms |
1087-
| Static HTML, 200K-1M rows | `engine = "duckdb"` (WASM) pagination/sort instant, search may lag |
1088-
| Shiny with data that must stay on server | `server = "duckdb"` (R backend) no WASM limit |
1089-
| Shiny with >1M rows or sensitive data | `server = "duckdb"` (R backend) |
1089+
| Use case | Recommended approach |
1090+
| -------------------------------------------------- | ----------------------------------------------------------------------------- |
1091+
| Static HTML (R Markdown / Quarto) up to ~200K rows | `backend = backendDuckDB()` (WASM) -- all operations <300ms |
1092+
| Static HTML, 200K-1M rows | `backend = backendDuckDB()` (WASM) -- pagination/sort instant, search may lag |
1093+
| Shiny with data that must stay on server | `backend = backendDuckDB("server")` (R backend) -- no WASM limit |
1094+
| Shiny with >1M rows or sensitive data | `backend = backendDuckDB("server")` (R backend) |
10901095

10911096
### Future search optimizations
10921097

@@ -1215,6 +1220,6 @@ The same DuckDB engine powers both sides:
12151220
- **Server-side (DuckDB R):** For Shiny apps where data must stay private. Zero-copy data registration, ~100x faster
12161221
than the current data.frame backend, same SQL as the WASM engine.
12171222

1218-
The implementation is additive (opt-in via `engine = "duckdb"` or `server = "duckdb"`), backward compatible (existing
1223+
The implementation is additive (opt-in via `backend = backendDuckDB()`), backward compatible (existing
12191224
tables unchanged), and phased (Arrow-only is valuable on its own, DuckDB-WASM pagination is the first milestone, and
12201225
the R backend shares the same query logic).

0 commit comments

Comments
 (0)