Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
434 changes: 31 additions & 403 deletions README.md

Large diffs are not rendered by default.

211 changes: 211 additions & 0 deletions docs/api.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,211 @@
# API Reference

All write endpoints (`POST` / `PUT` / `PATCH` / `DELETE`) require the `API_KEY` env var when set; pass it as `Authorization: Bearer <key>`.

## Presigned Uploads

```
POST /v1/uploads/presign
```

Generates a presigned URL for direct upload — to S3/R2 by default, or directly to Cloudflare Stream when the profile has `delivery: stream`.

**Request body:**
```json
{
"key_base": "unique-file-id",
"ext": "jpg",
"mime": "image/jpeg",
"size_bytes": 1024000,
"kind": "image",
"profile": "avatar",
"multipart": "auto"
}
```

| Field | Notes |
|---|---|
| `key_base` | Unique identifier. Ignored for `delivery: stream` profiles — Stream assigns the UID. |
| `ext` | File extension (used in `storage_path` template). |
| `mime` | Validated against profile `allowed_mimes`. |
| `size_bytes` | Validated against profile `size_max_bytes`. For Stream, also chooses POST (≤200MB) vs TUS. |
| `kind` | `image` or `video`. Must match the profile's `kind`. |
| `profile` | Name from `storage-config.yaml`. |
| `multipart` | `auto` / `force` / `off`. Ignored for Stream delivery. |

**Response — single PUT (R2):**
```json
{
"object_key": "originals/avatars/ab/unique-file-id.jpg",
"upload": {
"single": {
"method": "PUT",
"url": "https://presigned-s3-url",
"headers": { "Content-Type": "image/jpeg", "If-None-Match": "*" },
"expires_at": "2024-01-01T12:00:00Z"
}
}
}
```

**Response — multipart (R2):**
```json
{
"object_key": "originals/avatars/ab/unique-file-id.jpg",
"upload": {
"multipart": {
"upload_id": "abc123xyz",
"part_size": 8388608,
"parts": [
{
"part_number": 1,
"method": "PUT",
"url": "https://presigned-s3-part-url-1",
"headers": { "Content-Type": "image/jpeg" },
"expires_at": "2024-01-01T12:00:00Z"
}
],
"complete": {
"method": "POST",
"url": "https://your-api/v1/uploads/.../complete/abc123xyz",
"headers": { "Content-Type": "application/json" },
"expires_at": "2024-01-01T12:00:00Z"
},
"abort": {
"method": "DELETE",
"url": "https://your-api/v1/uploads/.../abort/abc123xyz",
"headers": {},
"expires_at": "2024-01-01T12:00:00Z"
}
}
}
}
```

**Response — Cloudflare Stream:**
```json
{
"object_key": "8f2a3c4d5e6f7890abcdef1234567890",
"upload": {
"stream": {
"method": "POST",
"url": "https://upload.videodelivery.net/...",
"uid": "8f2a3c4d5e6f7890abcdef1234567890",
"expires_at": "2024-01-01T12:00:00Z"
}
}
}
```

For `delivery: stream`, `object_key` is the Stream video UID — use it as the asset's stable handle for probe + delete. `method` is `"POST"` for ≤200MB, `"TUS"` for larger.

## Multipart Upload Completion

```
POST /v1/uploads/{object_key}/complete/{upload_id}
```

```json
{
"parts": [
{ "part_number": 1, "etag": "\"d41d8cd98f00b204e9800998ecf8427e\"" },
{ "part_number": 2, "etag": "\"098f6bcd4621d373cade4e832627b4f6\"" }
]
}
```

Response: `{ "status": "completed", "object_key": "..." }`

## Multipart Upload Abort

```
DELETE /v1/uploads/{object_key}/abort/{upload_id}
```

Response: `{ "status": "aborted", "upload_id": "..." }`

## Probe Asset (Video)

```
POST /v1/assets/{profile}/{key_base}/probe
```

Validates an uploaded video against the profile's constraint fields. Required: profile must be `kind: video`. Returns 200 for both pass and fail — `ok` is the gate, not the status code.

- **R2 profiles** — mediaflow runs `ffprobe` over a presigned GET URL.
- **Stream profiles** — `key_base` is the Stream UID; mediaflow reads metadata from `GET /accounts/{id}/stream/{uid}`. Returns `202` with `{ok: false, ready: false, state: "queued"|"inprogress"}` if Stream is still encoding.

**Pass:**
```json
{
"ok": true,
"ready": true,
"video": { "duration_seconds": 42.18, "width": 1920, "height": 1080, "codec": "h264" },
"reasons": []
}
```

**Fail:**
```json
{
"ok": false,
"video": { "duration_seconds": 67.4, "width": 854, "height": 480 },
"reasons": [
{ "code": "duration_exceeded", "limit": 45, "actual": 67.4 },
{ "code": "width_too_low", "limit": 1280, "actual": 854 }
]
}
```

**Reason codes:** `duration_exceeded`, `width_too_low`, `height_too_low`, `width_too_high`, `height_too_high`, `codec_not_allowed`, `no_video_stream`.

**Other status codes:** `404` (asset/UID not found), `422` (profile is not `kind: video`), `502` (ffprobe crash or Stream API error).

## Delete Asset

```
DELETE /v1/assets/{profile}/{key_base}
```

R2 profiles: deletes the original + every object under `thumb_folder/{key_base}*`.
Stream profiles: `key_base` is the UID; mediaflow calls Stream's `DELETE /stream/{uid}`.

**R2 response:**
```json
{ "status": "deleted", "profile": "avatar", "key_base": "abc123", "objects_deleted": 4 }
```

**Stream response:**
```json
{ "status": "deleted", "profile": "trailer", "uid": "8f2a3c4d5e6f7890..." }
```

## Thumbnails

```
GET /thumb/{type}/{image_id}?width=512
POST /thumb/{type}/{image_id}
```

Generates and serves thumbnails. POST requires `API_KEY` auth.

**GET parameters:**
- `type`: profile name (avatar, photo, banner, …)
- `image_id`: unique identifier
- `width`: pixels — defaults to the profile's `default_size`

## Original Images

```
GET /originals/{type}/{image_id}
```

Serves the original from storage.

## Health Check

```
GET /health
```

Returns `OK`.
134 changes: 134 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
# Configuration

## storage-config.yaml

Profile-based config combining upload, processing, probe, and delivery rules. Loaded from the path in `STORAGE_CONFIG_PATH` (defaults to `examples/storage-config.yaml`). Can also be loaded from S3 with an `s3://bucket/key` path.

```yaml
profiles:
avatar:
kind: "image"
allowed_mimes: ["image/jpeg", "image/png", "image/webp"]
size_max_bytes: 5242880 # 5MB
multipart_threshold_mb: 15
part_size_mb: 8
token_ttl_seconds: 900
storage_path: "originals/avatars/{shard?}/{key_base}"
enable_sharding: true
thumb_folder: "thumbnails/avatars"
sizes: ["128", "256"]
default_size: "256"
quality: 90
convert_to: "webp"

video:
# R2-delivered video — mediaflow runs ffprobe at attach
kind: "video"
allowed_mimes: ["video/mp4", "video/quicktime", "video/webm"]
size_max_bytes: 104857600 # 100MB
multipart_threshold_mb: 15
part_size_mb: 8
token_ttl_seconds: 1800
storage_path: "originals/videos/{shard?}/{key_base}"
enable_sharding: true
max_duration_seconds: 600
min_width: 640
min_height: 360
allowed_codecs: ["h264", "hevc"]

trailer:
# Stream-delivered video — bytes go directly to Cloudflare Stream
kind: "video"
delivery: "stream"
allowed_mimes: ["video/mp4", "video/quicktime"]
size_max_bytes: 78643200 # 75MB; Stream caps plain POST at 200MB, TUS at 30GB
token_ttl_seconds: 1800
max_duration_seconds: 45
min_width: 1280
min_height: 720
```

## Field reference

### Upload

| Field | Notes |
|---|---|
| `kind` | `image` or `video`. |
| `delivery` | `""` / `"r2"` (default — presigned R2 PUT) or `"stream"` (Cloudflare Stream Direct Creator Upload). Stream requires `STREAM_ACCOUNT_ID` + `STREAM_API_TOKEN`. |
| `allowed_mimes` | Whitelist of MIME types accepted at presign. |
| `size_max_bytes` | Hard cap on upload size, enforced at presign. |
| `multipart_threshold_mb` | Files above this trigger multipart (R2 only). |
| `part_size_mb` | Multipart chunk size (R2 only). |
| `token_ttl_seconds` | Presigned URL expiration. |
| `storage_path` | Template for R2 key. Required for R2 profiles, omitted for `delivery: stream`. Supports `{key_base}`, `{ext}`, `{shard}`, `{shard?}`. |
| `enable_sharding` | When true, mediaflow generates a 2-char shard from `key_base` hash. |

### Image processing

| Field | Notes |
|---|---|
| `thumb_folder` | Where thumbnails are written. |
| `sizes` | Available widths (e.g. `["128", "256"]`). |
| `default_size` | Width used when client doesn't specify. |
| `quality` | 1–100. |
| `convert_to` | Output format (`webp`, `jpeg`, etc.). |

### Video probe constraints

All optional. Unset fields are skipped during probe; only set fields are enforced.

| Field | Notes |
|---|---|
| `max_duration_seconds` | Reject longer videos. For `delivery: stream`, also passed to Stream's `direct_upload` API as `maxDurationSeconds`. |
| `min_width` / `min_height` | Resolution floors. |
| `max_width` / `max_height` | Resolution ceilings. |
| `allowed_codecs` | Whitelist (e.g. `["h264", "hevc"]`). Stream profiles ignore this — Stream re-encodes to H.264 regardless. |

## Storage path templates

`storage_path` is a string template:

| Placeholder | Meaning |
|---|---|
| `{key_base}` | Unique file identifier from the request. |
| `{ext}` | File extension. |
| `{shard}` | Always replaced. Use when `enable_sharding: true`. |
| `{shard?}` | Removed (along with surrounding `/`) when `enable_sharding: false`. |

**Auto-sharding** (`enable_sharding: true`):
- `"originals/{shard?}/{key_base}"` → `originals/ab/my-file`
- Shards auto-generated from a SHA1 prefix of `key_base`.
- Clients can optionally pass `shard` in the request to override.

**Fixed organization** (`enable_sharding: false`):
- `"originals/user123/{key_base}"` → `originals/user123/my-file`
- `{shard?}` placeholders are stripped.
- `shard` field in requests is ignored.

## Environment variables

```bash
# Required
S3_BUCKET=your-bucket-name

# AWS / R2 credentials — pick one
AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-key
# or use IAM role (ECS/EC2)

# Optional
S3_REGION=us-east-1
S3_ENDPOINT= # Override for non-AWS S3 (R2, MinIO, etc.)
PUBLIC_S3_ENDPOINT= # Host used in presigned URLs (defaults to S3_ENDPOINT)
PORT=8080
CACHE_MAX_AGE=86400
STORAGE_CONFIG_PATH=storage-config.yaml # Or s3://bucket/path/storage-config.yaml

# API authentication (write endpoints reject if unset on a write)
API_KEY=

# Cloudflare Stream — required if any profile uses delivery: stream
STREAM_ACCOUNT_ID=
STREAM_API_TOKEN= # Token with Stream:Edit on the account
```
Loading
Loading