|
1 | 1 | # DuckDB-WASM Data Engine for Reactable |
2 | 2 |
|
| 3 | +> **API note:** This doc was written before the Phase 7 `backend =` refactor. Code examples using |
| 4 | +> `engine = "duckdb"` and `server = "duckdb"` should be read as `backend = backendDuckDB()` and |
| 5 | +> `backend = backendDuckDB("server")` respectively. The `engine` param was removed and never released. |
| 6 | +
|
3 | 7 | ## Problem Statement |
4 | 8 |
|
5 | 9 | reactable has three modes for handling data, each with significant limitations: |
@@ -815,13 +819,17 @@ DuckDB-WASM runs in a Web Worker: |
815 | 819 |
|
816 | 820 | ### CDN vs. self-hosted |
817 | 821 |
|
818 | | -Can use jsDelivr CDN for the WASM files (simplest, no self-hosting required) or bundle them with the R package. |
| 822 | +~~Can use jsDelivr CDN for the WASM files (simplest, no self-hosting required) or bundle them with the R package.~~ |
| 823 | +
|
| 824 | +~~CDN pros: cached across sites, no package size increase, always latest version.~~ |
| 825 | +~~CDN cons: requires internet, reliability depends on CDN, not available in air-gapped environments.~~ |
819 | 826 |
|
820 | | -CDN pros: cached across sites, no package size increase, always latest version. |
821 | | -CDN cons: requires internet, reliability depends on CDN, not available in air-gapped environments. |
| 827 | +~~**Recommendation:** Default to jsDelivr CDN. Add `options(reactable.duckdb.wasm.url = "local/path")` for |
| 828 | +self-hosted/offline use.~~ |
822 | 829 |
|
823 | | -**Recommendation:** Default to jsDelivr CDN. Add `options(reactable.duckdb.wasm.url = "local/path")` for |
824 | | -self-hosted/offline use. |
| 830 | +**Decision:** Self-hosted/bundled. WASM files are bundled in `inst/htmlwidgets/lib/duckdb-wasm/`. CDN was |
| 831 | +rejected because it fails for air-gapped corporate environments. CRAN size limit (~5 MB) is an open concern |
| 832 | +since `duckdb-eh.wasm` is ~32.7 MB. |
825 | 833 |
|
826 | 834 | ### Query execution is sequential |
827 | 835 |
|
@@ -933,12 +941,12 @@ directly from the R data frame's memory without copying. |
933 | 941 |
|
934 | 942 | ### When to use which |
935 | 943 |
|
936 | | -| Scenario | Recommended mode | |
937 | | -| ------------------------------------------ | --------------------------------------------- | |
938 | | -| Static HTML, R Markdown, Quarto (<2M rows) | `engine = "duckdb"` (WASM, client-side) | |
939 | | -| Shiny, data must stay on server | `server = "duckdb"` (R backend) | |
940 | | -| Shiny, don't mind client seeing data | `engine = "duckdb"` (WASM, offloads R server) | |
941 | | -| Very large data (>2M rows, static doc) | `engine = "duckdb"` + Parquet sidecar file | |
| 944 | +| Scenario | Recommended mode | |
| 945 | +| ------------------------------------------ | ---------------------------------------------------------- | |
| 946 | +| Static HTML, R Markdown, Quarto (<2M rows) | `backend = backendDuckDB()` (WASM, client-side) | |
| 947 | +| Shiny, data must stay on server | `backend = backendDuckDB("server")` (R backend) | |
| 948 | +| Shiny, don't mind client seeing data | `backend = backendDuckDB()` (WASM, offloads R server) | |
| 949 | +| Very large data (>2M rows, static doc) | `backend = backendDuckDB()` + Parquet sidecar file | |
942 | 950 | | Small tables (<10K rows) | Default JSON path (no engine needed) | |
943 | 951 |
|
944 | 952 | --- |
@@ -1028,13 +1036,10 @@ special handling: |
1028 | 1036 |
|
1029 | 1037 | This is more complex than flat pagination but entirely feasible with the SQL model. |
1030 | 1038 |
|
1031 | | -### `paginateSubRows` not supported |
1032 | | - |
1033 | | -`paginateSubRows = TRUE` is not implemented for the DuckDB engine (WASM or R server). When groups are expanded, |
1034 | | -sub-rows are added on top of the page size (the default `paginateSubRows = FALSE` behavior). This matches the df |
1035 | | -and dt server backends — only the V8 backend supports `paginateSubRows`. |
| 1039 | +### ~~`paginateSubRows` not supported~~ DONE |
1036 | 1040 |
|
1037 | | -See [paginate-sub-rows.md](paginate-sub-rows.md) for the design plan to implement this. |
| 1041 | +`paginateSubRows = TRUE` is now implemented for all backends: DuckDB WASM (client), DuckDB R server, and df server. |
| 1042 | +See [paginate-sub-rows.md](paginate-sub-rows.md) for the design. |
1038 | 1043 |
|
1039 | 1044 | --- |
1040 | 1045 |
|
@@ -1081,12 +1086,12 @@ and memory than it saves on queries. |
1081 | 1086 |
|
1082 | 1087 | ### Recommended target range |
1083 | 1088 |
|
1084 | | -| Use case | Recommended approach | |
1085 | | -| -------------------------------------------------- | -------------------------------------------------------------------- | |
1086 | | -| Static HTML (R Markdown / Quarto) up to ~200K rows | `engine = "duckdb"` (WASM) — all operations <300ms | |
1087 | | -| Static HTML, 200K-1M rows | `engine = "duckdb"` (WASM) — pagination/sort instant, search may lag | |
1088 | | -| Shiny with data that must stay on server | `server = "duckdb"` (R backend) — no WASM limit | |
1089 | | -| Shiny with >1M rows or sensitive data | `server = "duckdb"` (R backend) | |
| 1089 | +| Use case | Recommended approach | |
| 1090 | +| -------------------------------------------------- | ----------------------------------------------------------------------------- | |
| 1091 | +| Static HTML (R Markdown / Quarto) up to ~200K rows | `backend = backendDuckDB()` (WASM) -- all operations <300ms | |
| 1092 | +| Static HTML, 200K-1M rows | `backend = backendDuckDB()` (WASM) -- pagination/sort instant, search may lag | |
| 1093 | +| Shiny with data that must stay on server | `backend = backendDuckDB("server")` (R backend) -- no WASM limit | |
| 1094 | +| Shiny with >1M rows or sensitive data | `backend = backendDuckDB("server")` (R backend) | |
1090 | 1095 |
|
1091 | 1096 | ### Future search optimizations |
1092 | 1097 |
|
@@ -1215,6 +1220,6 @@ The same DuckDB engine powers both sides: |
1215 | 1220 | - **Server-side (DuckDB R):** For Shiny apps where data must stay private. Zero-copy data registration, ~100x faster |
1216 | 1221 | than the current data.frame backend, same SQL as the WASM engine. |
1217 | 1222 |
|
1218 | | -The implementation is additive (opt-in via `engine = "duckdb"` or `server = "duckdb"`), backward compatible (existing |
| 1223 | +The implementation is additive (opt-in via `backend = backendDuckDB()`), backward compatible (existing |
1219 | 1224 | tables unchanged), and phased (Arrow-only is valuable on its own, DuckDB-WASM pagination is the first milestone, and |
1220 | 1225 | the R backend shares the same query logic). |
0 commit comments