From d76ed0b35ba83092af5e4979adcd4ab6a5259d98 Mon Sep 17 00:00:00 2001 From: Brandon Honeycutt <697896+brandonhon@users.noreply.github.com> Date: Fri, 5 Jun 2026 12:50:37 -0500 Subject: [PATCH] docs: document TT-RSS import, multi-feed picker & schemeless add-feed - getting-started: 'Migrating from another reader' section (OPML + TT-RSS file/live import); add-feed note for schemeless URLs + the multi-feed picker - architecture: internal/ttrss package + DiscoverAll/normalize in internal/feed - security: TT-RSS import + /feeds/discover in the rate-limit and SSRF lists, TT-RSS credential handling note, 50 MiB import cap - index: mention library migration (OPML + Tiny Tiny RSS) --- docs/architecture.md | 3 ++- docs/getting-started.md | 11 ++++++++++- docs/index.md | 2 +- docs/security.md | 5 ++++- 4 files changed, 17 insertions(+), 4 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index ad850bc..5393adf 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -27,10 +27,11 @@ internal/auth/ argon2id passwords, securecookie sessions, WebAuth internal/config/ env-var loading (typed Config) internal/db/ SQLite open, pragmas, embedded migrations (goose) internal/digest/ SMTP daily-digest builder + sender (multipart/alt + STARTTLS) -internal/feed/ gofeed wrapper + readability fallback fetcher + Discover (homepage → feed URL) +internal/feed/ gofeed wrapper + readability fallback fetcher + Discover / DiscoverAll (homepage → one or many feed URLs) + URL normalize (schemeless → https) internal/filters/ matcher (field/op/value), apply outcome combiner internal/models/ data types shared across packages internal/opml/ OPML import + export + discovery → subscribe +internal/ttrss/ Tiny Tiny RSS import — XML export parser + live JSON API client → one non-polling "Imported" feed internal/poller/ adaptive scheduler, fetch dispatch, summary queue internal/store/ SQLite CRUD, FTS5 search, app_settings KV, dbops, passkeys, digests internal/summarize/ Summarizer interface + Ollama implementation + noop for tests diff --git a/docs/getting-started.md b/docs/getting-started.md index aa55124..a468ece 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -112,11 +112,20 @@ Migrations are embedded in the binary and apply automatically on startup; no man 2. Open **Settings → Language model** and confirm the recommendation matches your hardware. The `ember probe` subcommand (or that section) reports detected RAM/CPU/GPU and the suggested model. 3. Open **Settings → Database**, schedule a daily backup, and pick a cleanup cadence. 4. Open **Settings → Preferences** and pick your theme + article density. -5. Click **Browse starter packs** or paste a feed URL — or a homepage URL — into the sidebar "+ Add feed". Ember auto-discovers the feed link. +5. Click **Browse starter packs** or paste a feed URL — or just a domain like `example.com` (no `https://` needed) — into the sidebar "+ Add feed". Ember auto-discovers the feed (it reads `` and probes common paths like `/feed`, `/rss`, `/atom.xml`); if the site publishes more than one, a picker lets you choose which to add. 6. (Optional) **Settings → Passkeys** to register a passkey for password-less sign-in. Requires `EMBER_PUBLIC_URL` to be set. 7. (Optional) Configure SMTP env vars (see [Configuration](/configuration#optional-env-vars)) and enable a daily digest email from your profile. 8. (Optional) Install Ember as a PWA — Chrome / Edge / Safari "Install app" menu. Once installed, new articles trigger an OS-level numeric badge on the app icon (taskbar / dock / launcher) in addition to the in-tab favicon dot. +## Migrating from another reader + +**Settings → Import & migrate** brings your existing library across — none of it touches feeds you've already added: + +- **OPML** — import your subscription list (or export Ember's at any time). +- **Tiny Tiny RSS** — import your **starred & archived articles**, either by uploading the export file or pulling them live from a running instance. For the live pull, enable *Settings → Preferences → Enable API access* in TT-RSS first; if it's served under a subpath (e.g. `https://example.com/tt-rss`), include that — Ember appends `/api/`. Credentials are used only for the import and are never stored. + +Imported articles arrive starred (and marked read) in a dedicated, non-polling feed, so they show up in **Starred** immediately without re-fetching your old sources. + ## Run from a pre-built binary Bare-metal, VM, NAS, systemd — anywhere you'd rather not run Docker. Each release ships four tarballs (`linux-{amd64,arm64}`, `darwin-{amd64,arm64}`) plus a `SHA256SUMS` checksum file at the same URL. diff --git a/docs/index.md b/docs/index.md index 700e41d..33e56c4 100644 --- a/docs/index.md +++ b/docs/index.md @@ -50,7 +50,7 @@ features: details: Five actions (mark_read, star, hide, tag, add_to_board), eight match fields including feed, tags, published_at, has_image. Per-rule priority and a Preview button that counts last-7-day matches before you save. --- -And plenty more under the hood: **Fever-compatible API** (Reeder, FeedMe & co. via a random per-user token), **passkey sign-in** (Touch ID / Face ID / hardware keys), an opt-in **daily digest email**, **subscribe-by-URL** discovery (including YouTube channels and Mastodon profiles), **15-second auto-refresh** with a favicon unread dot, and **live admin controls** for hot-swapping the LLM model, tuning generation params, and scheduling backups / cleanup / OPML exports. +And plenty more under the hood: **migrate your library** in (OPML subscriptions + starred/archived articles from Tiny Tiny RSS), a **Fever-compatible API** (Reeder, FeedMe & co. via a random per-user token), **passkey sign-in** (Touch ID / Face ID / hardware keys), an opt-in **daily digest email**, **subscribe-by-URL** discovery (paste a homepage; pick from multiple feeds), **15-second auto-refresh** with a favicon unread dot, and **live admin controls** for hot-swapping the LLM model, tuning generation params, and scheduling backups / cleanup / OPML exports. ## Why? diff --git a/docs/security.md b/docs/security.md index ddc13eb..d09907b 100644 --- a/docs/security.md +++ b/docs/security.md @@ -9,7 +9,7 @@ For vulnerability reporting, see [SECURITY.md](https://github.com/brandonhon/emb - **Passwords**: argon2id (`time=3`, `memory=64 MiB`, `parallelism=2`, salt=16 bytes). Meets OWASP 2024 recommendations. - **Sessions**: 64-hex-char random IDs signed via `gorilla/securecookie`. Server-side row in `sessions` table backs every cookie. Default lifetime 24 hours; override via `EMBER_SESSION_TTL` env var or **Settings → Sessions** (bounded 5 min – 90 days). Destroyed on logout, login (fixation defense), and on password change (both self-service and admin). - **Cookies**: `HttpOnly`, `Secure`, `SameSite=Strict`, scoped to `/`. -- **Rate limiting**: per-IP token bucket on `POST /api/auth/login` and `POST /api/auth/passkey/*` (burst 10/min). A second, higher-burst bucket (30/min) guards the expensive authenticated endpoints — add-feed, refresh-feed, resummarize(-all), OPML import, starter-pack import, article extract, and search — which spawn outbound fetches / goroutines / FTS work. Buckets key on the real client IP (see Transport / proxy expectations for how that's resolved). +- **Rate limiting**: per-IP token bucket on `POST /api/auth/login` and `POST /api/auth/passkey/*` (burst 10/min). A second, higher-burst bucket (30/min) guards the expensive authenticated endpoints — add-feed, feed discovery, refresh-feed, resummarize(-all), OPML import, TT-RSS import (file + live API), starter-pack import, article extract, and search — which spawn outbound fetches / goroutines / FTS work. Buckets key on the real client IP (see Transport / proxy expectations for how that's resolved). - **Passkeys / WebAuthn**: optional second sign-in method (FIDO2). Credentials are bound to a relying-party ID derived from `EMBER_PUBLIC_URL`; ceremonies expire after 5 minutes; a stale `webauthn_sessions` row is reaped on a 15-minute cadence. Credentials never leave the device — only the public key is stored. ## Authorization @@ -49,7 +49,9 @@ Every outbound URL fetch passes through `internal/urlcheck.Check`: Surfaces covered: - `POST /api/feeds` (add feed) +- `POST /api/feeds/discover` (multi-feed picker — the target page and every advertised feed link is validated before fetching) - `POST /api/feeds/import` (OPML import — each `xmlUrl` is filtered) +- `POST /api/feeds/import-ttrss-api` (TT-RSS live pull — the API endpoint is validated and the client carries `feed.RedirectGuard`; the user-supplied TT-RSS credentials are held only for the request and never persisted). The file upload `POST /api/feeds/import-ttrss` makes no outbound request, and `javascript:`/`data:` article links in the export are dropped. - Poller readability enrichment (Lobsters / HN aggregator → external link) - Feed fetcher redirects - `POST /api/articles/{id}/extract` (on-demand Re-extract) — runs the same `urlcheck.Check` before fetching the article URL through readability. @@ -67,6 +69,7 @@ Outbound mail (digests + the test message) never sends credentials or message bo - `decodeJSON` wraps the body in `http.MaxBytesReader` capped at **1 MiB**. - OPML import body capped at **8 MiB** (and the parser reads at most 10 MiB). +- TT-RSS file import capped at **50 MiB** (the streaming XML parser reads at most that); the live API pull paginates and stops at 100k articles. - Fever (`/fever`) form body capped at **64 KiB**. - Request headers capped at **64 KiB** (`http.Server.MaxHeaderBytes`). - `/api/articles/read` (and other bulk endpoints) accept at most **1000** ids per request; `/api/articles?limit=` is clamped to **200**.