Skip to content

Document CLI dataset commands #141

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
simantak-dabhade merged 1 commit into from
Jun 11, 2026
Merged

Document CLI dataset commands #141

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 64 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,70 @@ Useful local options:
| `bigset --app-port 4500 --backend-port 4501` | Use alternate app/backend ports |
| `bigset --home ~/.bigset-dev` | Use a separate local cache directory |

### CLI dataset commands

Once `bigset` is running and setup is complete, you can create and export
datasets directly from your terminal.

Create a dataset:

```bash
bigset create "fintech startups in the bay area"
```

Create a dataset, wait for population to finish, and export CSV:

```bash
bigset create "fintech startups in the bay area" --rows 10 --wait --csv fintech.csv
```

List datasets:

```bash
bigset list
```

Inspect an existing dataset:

```bash
bigset status <datasetId>
bigset rows <datasetId> --json
```
Comment on lines +137 to +142

@coderabbitai coderabbitai Bot Jun 11, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Document the --json flag or clarify flag table scope.

Line 141 uses --json with the rows command, but this flag is not documented in the "Useful create flags" table below (lines 158-166). Consider either:

  1. Expanding the flags table to cover all commands and include --json for rows and export, or
  2. Renaming the table to clarify it only documents create command flags
📝 Option 1: Expand the table to document flags for all commands

Replace the table at lines 157-166 with:

-Useful create flags:
+Useful CLI flags:

 | Flag | What it does |
 |------|-------------|
 | `--rows <n>` | Maximum rows to collect |
 | `--wait` | Keep the command open until population finishes |
 | `--csv <path>` | Write populated rows to a CSV file |
+| `--json` | Output results as JSON (works with `rows` command) |
 | `--skip-populate` | Create the dataset without starting population |
 | `--cadence manual\|30m\|6h\|12h\|daily\|weekly` | Set refresh cadence |
 | `--backend-port <port>` | Target a non-default local backend port |
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@README.md` around lines 137 - 142, The README currently shows using the
--json flag with the rows command but the "Useful create flags" table only
documents create-related flags; update the documentation to avoid confusion by
either renaming that table header to clarify it only covers create (e.g.,
"Useful create flags") or, preferably, expand the flags table to include
command-scoped flags such as --json (used by rows and export) and list any other
global or command-specific flags; locate references to rows and export and add
an entry for --json (describe its effect) or change the table title accordingly
so the scope is clear.


Export an existing dataset:

```bash
bigset export <datasetId> --csv out.csv
```

Manage a run:

```bash
bigset populate <datasetId>
bigset stop <datasetId>
```

Useful create flags:

| Flag | What it does |
|------|-------------|
| `--rows <n>` | Maximum rows to collect |
| `--wait` | Keep the command open until population finishes |
| `--csv <path>` | Write populated rows to a CSV file |
| `--skip-populate` | Create the dataset without starting population |
| `--cadence manual\|30m\|6h\|12h\|daily\|weekly` | Set refresh cadence |
| `--backend-port <port>` | Target a non-default local backend port |

Agents such as Codex or Claude Code can use the same commands. A typical agent
workflow is:

```bash
bigset create "AI infrastructure startups hiring backend engineers" --rows 30 --wait --csv ai-infra-hiring.csv
```

The agent can then inspect `ai-infra-hiring.csv` and summarize or transform the
results.

---

## Developing From Source
Expand Down