diff --git a/app/authzed/guides/_meta.ts b/app/authzed/guides/_meta.ts index 845c8f75..d60c6855 100644 --- a/app/authzed/guides/_meta.ts +++ b/app/authzed/guides/_meta.ts @@ -1,4 +1,5 @@ export default { "picking-a-product": "Picking a Product", cloud: "Getting Started with Authzed Cloud", + "postgres-fdw": "Using Postgres FDW with AuthZed", }; diff --git a/app/authzed/guides/postgres-fdw/page.mdx b/app/authzed/guides/postgres-fdw/page.mdx new file mode 100644 index 00000000..01f4e993 --- /dev/null +++ b/app/authzed/guides/postgres-fdw/page.mdx @@ -0,0 +1,480 @@ +import { Callout, Steps } from "nextra/components"; + +# Using Postgres FDW with AuthZed Cloud/Dedicated + +This guide shows you how to query your AuthZed permissions system using standard SQL through the SpiceDB Postgres Foreign Data Wrapper (FDW). + +The Postgres FDW acts as a translation layer that implements the PostgreSQL wire protocol and converts SQL queries into SpiceDB API calls. +This allows you to query permissions, relationships, and schema using familiar SQL syntax. + + + The Postgres FDW is an **experimental feature** that has been tested but may have issues in + certain scenarios. It is subject to change and should be used with caution in production + environments. Please report any issues you encounter on the [SpiceDB GitHub + repository](https://github.com/authzed/spicedb/issues). + + +## Prerequisites + +- An AuthZed Dedicated or Cloud account with a Permissions System created +- A SpiceDB endpoint (provided by AuthZed) +- PostgreSQL installed (for connecting to the FDW server) +- Docker or the SpiceDB binary + +## Overview + +The setup process involves: + +1. Creating a permissions system on AuthZed +2. Generating an API token with appropriate permissions +3. Starting the FDW proxy server +4. Configuring PostgreSQL to connect to the FDW server + + + +### Create a Permissions System on AuthZed + +If you don't already have a Permissions System, create one by following the [Getting Started with AuthZed Cloud](/authzed/guides/cloud) guide. + +Make note of your **SpiceDB endpoint**, which will look like: + +``` +grpc.authzed.com:443 +``` + +Or for Dedicated clusters: + +``` +your-cluster-name.authzed.com:443 +``` + +### Generate an API Token using Restricted API Access + +You'll need to create a token for the FDW to access your SpiceDB API. +AuthZed provides [Restricted API Access](/authzed/concepts/restricted-api-access) to apply least-privilege access control. + +#### Create a Service Account + +Navigate to your Permissions System in the AuthZed dashboard and go to the **Access** tab. + +Create a **Service Account**: + +- **Name**: `postgres-fdw` +- **Description**: Service account for Postgres FDW access + +#### Create a Role + +Create a **Role** that defines what permissions the FDW will have. + +**For full access** (all operations): + +Create a role named `fdw-full-access` with these permissions: + +``` +authzed.api/ReadSchema +authzed.api/WriteSchema +authzed.api/ReadRelationships +authzed.api/WriteRelationships +authzed.api/DeleteRelationships +authzed.api/CheckPermission +authzed.api/LookupResources +authzed.api/LookupSubjects +authzed.api/ExpandPermissionTree +authzed.api/Watch +authzed.api/ExportBulkRelationships +authzed.api/BulkExportRelationships +authzed.api/ImportBulkRelationships +authzed.api/BulkImportRelationships +``` + +**For read-only access** (recommended for analytics/reporting): + +Create a role named `fdw-read-only` with these permissions: + +``` +authzed.api/ReadSchema +authzed.api/ReadRelationships +authzed.api/CheckPermission +authzed.api/LookupResources +authzed.api/LookupSubjects +authzed.api/ExpandPermissionTree +``` + + + The FDW will return an error if it attempts an operation not granted by the token. Start with + read-only access and expand permissions as needed. + + +#### Create a Policy + +Create a **Policy** to bind the role to your service account: + +- **Name**: `fdw-policy` +- **Principal**: Select the `postgres-fdw` service account +- **Roles**: Select the role you created (`fdw-full-access` or `fdw-read-only`) + +#### Generate a Token + +Navigate to the `postgres-fdw` service account and create a **Token**: + +- **Name**: `fdw-token-1` +- **Description**: Token for FDW access + + + Save the token immediately after creation. It will look like: `sdbst_h256_yoursecrettoken`. You + won't be able to view it again. + + +### Start the FDW Proxy Server + +The FDW proxy server acts as a bridge between PostgreSQL and your AuthZed SpiceDB API. + +#### Using Docker (Recommended) + +```bash +docker run --rm -p 5432:5432 \ + authzed/spicedb \ + postgres-fdw \ + --spicedb-api-endpoint grpc.authzed.com:443 \ + --spicedb-access-token-secret "sdbst_h256_yoursecrettoken" \ + --postgres-endpoint ":5432" \ + --postgres-username "postgres" \ + --postgres-access-token-secret "your-fdw-password" +``` + +#### Using the SpiceDB Binary + +```bash +spicedb postgres-fdw \ + --spicedb-api-endpoint grpc.authzed.com:443 \ + --spicedb-access-token-secret "sdbst_h256_yoursecrettoken" \ + --postgres-endpoint ":5432" \ + --postgres-username "postgres" \ + --postgres-access-token-secret "your-fdw-password" +``` + +#### Using Environment Variables + +```bash +export SPICEDB_SPICEDB_API_ENDPOINT="grpc.authzed.com:443" +export SPICEDB_SPICEDB_ACCESS_TOKEN_SECRET="sdbst_h256_yoursecrettoken" +export SPICEDB_POSTGRES_ENDPOINT=":5432" +export SPICEDB_POSTGRES_USERNAME="postgres" +export SPICEDB_POSTGRES_ACCESS_TOKEN_SECRET="your-fdw-password" + +spicedb postgres-fdw +``` + + + Replace `grpc.authzed.com:443` with your specific SpiceDB endpoint and + `sdbst_h256_yoursecrettoken` with your actual token from the previous step. + + +#### Configuration Options + +| Flag | Description | Default | +| -------------------------------- | ----------------------------------------------- | ----------------- | +| `--spicedb-api-endpoint` | Your AuthZed SpiceDB endpoint | `localhost:50051` | +| `--spicedb-access-token-secret` | Your AuthZed API token (required) | - | +| `--postgres-endpoint` | FDW server listen address | `:5432` | +| `--postgres-username` | Username for Postgres authentication | `postgres` | +| `--postgres-access-token-secret` | Password for Postgres authentication (required) | - | + +### Configure PostgreSQL Foreign Data Wrapper + +Connect to your PostgreSQL database and run the following SQL commands: + +```sql +-- Install the postgres_fdw extension +CREATE EXTENSION IF NOT EXISTS postgres_fdw; + +-- Create a foreign server pointing to the FDW proxy +CREATE SERVER spicedb_server + FOREIGN DATA WRAPPER postgres_fdw + OPTIONS ( + host 'localhost', + port '5432', + dbname 'ignored' + ); + +-- Create user mapping with authentication credentials +CREATE USER MAPPING FOR CURRENT_USER + SERVER spicedb_server + OPTIONS ( + user 'postgres', + password 'your-fdw-password' + ); + +-- Import foreign tables +IMPORT FOREIGN SCHEMA public + LIMIT TO (permissions, relationships, schema) + FROM SERVER spicedb_server + INTO public; +``` + + + Replace `your-fdw-password` with the password you set when starting the FDW proxy server. If your + FDW proxy is running on a different host, update the `host` parameter accordingly. + + +### Query Your Permissions + +You can now query your AuthZed permissions system using SQL! + +#### Check Permissions + +```sql +-- Check if user:alice has permission to view document:readme +SELECT has_permission +FROM permissions +WHERE resource_type = 'document' + AND resource_id = 'readme' + AND permission = 'view' + AND subject_type = 'user' + AND subject_id = 'alice'; +``` + +#### Lookup Resources + +```sql +-- Find all documents that user:alice can view +SELECT resource_id +FROM permissions +WHERE resource_type = 'document' + AND permission = 'view' + AND subject_type = 'user' + AND subject_id = 'alice'; +``` + +#### Lookup Subjects + +```sql +-- Find all users who can view document:readme +SELECT subject_id +FROM permissions +WHERE resource_type = 'document' + AND resource_id = 'readme' + AND permission = 'view' + AND subject_type = 'user'; +``` + +#### Query Relationships + +```sql +-- Read relationships for a specific resource +SELECT resource_type, resource_id, relation, subject_type, subject_id +FROM relationships +WHERE resource_type = 'document' + AND resource_id = 'readme'; +``` + +#### Read Schema + +```sql +-- Get all schema definitions +SELECT definition FROM schema; +``` + + + +## Available Tables + +The FDW provides three virtual tables: + +### `permissions` Table + +Used for checking permissions and looking up resources or subjects. + +| Column | Type | Description | +| --------------------------- | ------- | -------------------------------- | +| `resource_type` | text | Resource type (e.g., 'document') | +| `resource_id` | text | Resource ID | +| `permission` | text | Permission name | +| `subject_type` | text | Subject type (e.g., 'user') | +| `subject_id` | text | Subject ID | +| `optional_subject_relation` | text | Optional subject relation | +| `has_permission` | boolean | Whether permission is granted | +| `consistency` | text | Consistency token (ZedToken) | + +**Supported Operations:** SELECT only + +### `relationships` Table + +Used for reading, writing, and deleting relationships. + +| Column | Type | Description | +| --------------------------- | ----- | ---------------------------- | +| `resource_type` | text | Resource type | +| `resource_id` | text | Resource ID | +| `relation` | text | Relation name | +| `subject_type` | text | Subject type | +| `subject_id` | text | Subject ID | +| `optional_subject_relation` | text | Optional subject relation | +| `optional_caveat_name` | text | Optional caveat name | +| `optional_caveat_context` | jsonb | Optional caveat context | +| `consistency` | text | Consistency token (ZedToken) | + +**Supported Operations:** SELECT, INSERT, DELETE + +### `schema` Table + +Used for reading your schema definition. + +| Column | Type | Description | +| ------------ | ---- | ------------------------------- | +| `definition` | text | Schema definition in Zed format | + +**Supported Operations:** SELECT only + +## Advanced Features + +### Consistency Control + +Control read consistency using the `consistency` column: + +```sql +-- Get a consistent view +SELECT resource_id, consistency +FROM permissions +WHERE resource_type = 'document' + AND permission = 'view' + AND subject_type = 'user' + AND subject_id = 'alice' + AND consistency = 'fully_consistent'; +``` + +Available consistency modes: + +- `minimize_latency`: Default, uses the newest available snapshot +- `fully_consistent`: Waits for a fully consistent view +- ``: Uses a specific consistency token +- `@`: Uses exact snapshot matching + +### Writing Relationships + +If you created a token with write access, you can insert and delete relationships: + +#### Insert Relationships + +```sql +-- Add a new relationship +INSERT INTO relationships (resource_type, resource_id, relation, subject_type, subject_id) +VALUES ('document', 'readme', 'viewer', 'user', 'alice'); +``` + +#### Delete Relationships + +```sql +-- Remove a relationship +DELETE FROM relationships +WHERE resource_type = 'document' + AND resource_id = 'readme' + AND relation = 'viewer' + AND subject_type = 'user' + AND subject_id = 'alice'; +``` + +### Joining with Local Tables + +One powerful feature of the FDW is the ability to join FDW tables with local PostgreSQL tables. +This allows you to enrich permission data with local application data. + +```sql +-- First, create a local table with document metadata +CREATE TABLE document ( + id text PRIMARY KEY, + title text NOT NULL, + contents text NOT NULL +); + +-- Insert some documents +INSERT INTO document (id, title, contents) VALUES + ('firstdoc', 'Document 1', 'Contents of document 1'), + ('seconddoc', 'Document 2', 'Contents of document 2'), + ('thirddoc', 'Document 3', 'Contents of document 3'); + +-- Join local documents with permissions to find which documents a user can access +SELECT document.id, document.title +FROM document +JOIN permissions ON permissions.resource_id = document.id +WHERE permissions.resource_type = 'document' + AND permissions.permission = 'view' + AND permissions.subject_type = 'user' + AND permissions.subject_id = 'alice' +ORDER BY document.title DESC; +``` + +This pattern is useful for: + +- Building filtered lists based on permissions +- Enriching permission checks with application metadata +- Creating permission-aware reports and dashboards + +### Using Cursors for Large Result Sets + +For queries that return many results, use cursors to paginate: + +```sql +BEGIN; + +DECLARE my_cursor CURSOR FOR + SELECT resource_id FROM permissions + WHERE resource_type = 'document' + AND permission = 'view' + AND subject_type = 'user' + AND subject_id = 'alice'; + +FETCH 100 FROM my_cursor; +FETCH 100 FROM my_cursor; + +CLOSE my_cursor; +COMMIT; +``` + +## Limitations + +The FDW has some limitations to be aware of: + +- **Joins between FDW tables**: Joins between FDW tables (e.g., `permissions` JOIN `relationships`) are not supported. However, joins between FDW tables and local PostgreSQL tables work as expected. +- **Aggregations**: SUM, COUNT, etc. are performed client-side by PostgreSQL +- **Ordering**: ORDER BY clauses are performed client-side by PostgreSQL +- **Subqueries**: Not supported +- **Complex WHERE clauses**: Only simple equality predicates and AND conditions are pushed down to SpiceDB + + + For super-fast joins or checks on large datasets, consider [AuthZed + Materialize](/authzed/concepts/authzed-materialize). Once set up, Materialize works seamlessly + with the FDW with no SQL changes required to your queries. + + +## Troubleshooting + +### Connection Refused + +If you get a connection error, verify: + +1. The FDW proxy server is running +2. The port is accessible (not blocked by firewall) +3. The host and port in your PostgreSQL configuration match the FDW server + +### Permission Denied Errors + +If you get permission denied errors: + +1. Verify your API token is correct +2. Check that your role includes the necessary permissions +3. Ensure your policy binds the role to your service account + +### Empty Results + +If queries return no results: + +1. Verify your schema and relationships exist in AuthZed +2. Check that you're using the correct resource types and permission names +3. Try querying the `schema` table to see your current schema + +## Next Steps + +- Learn more about [Restricted API Access](/authzed/concepts/restricted-api-access) +- Explore [SpiceDB concepts](/spicedb/concepts/schema) to better understand your data +- Review [best practices](/best-practices) for production deployments diff --git a/app/spicedb/getting-started/installing-zed/page.mdx b/app/spicedb/getting-started/installing-zed/page.mdx index df4abc96..83b0a0ee 100644 --- a/app/spicedb/getting-started/installing-zed/page.mdx +++ b/app/spicedb/getting-started/installing-zed/page.mdx @@ -1,4 +1,4 @@ -import { Callout } from 'nextra/components' +import { Callout } from "nextra/components"; # Installing Zed @@ -123,7 +123,6 @@ You can find more commands for tasks such as testing, linting in the repository' [CONTRIBUTING.md]: https://github.com/authzed/zed/blob/main/CONTRIBUTING.md - ## Reference: `zed` A command-line client for managing SpiceDB clusters. @@ -161,17 +160,16 @@ zed permission check --explain document:firstdoc writer user:emilia ### Children commands -- [zed backup](#reference-zed-backup) - Create, restore, and inspect permissions system backups -- [zed context](#reference-zed-context) - Manage configurations for connecting to SpiceDB deployments -- [zed import](#reference-zed-import) - Imports schema and relationships from a file or url -- [zed mcp](#reference-zed-mcp) - MCP (Model Context Protocol) server commands -- [zed permission](#reference-zed-permission) - Query the permissions in a permissions system -- [zed relationship](#reference-zed-relationship) - Query and mutate the relationships in a permissions system -- [zed schema](#reference-zed-schema) - Manage schema for a permissions system -- [zed use](#reference-zed-use) - Alias for `zed context use` -- [zed validate](#reference-zed-validate) - Validates the given validation file (.yaml, .zaml) or schema file (.zed) -- [zed version](#reference-zed-version) - Display zed and SpiceDB version information - +- [zed backup](#reference-zed-backup) - Create, restore, and inspect permissions system backups +- [zed context](#reference-zed-context) - Manage configurations for connecting to SpiceDB deployments +- [zed import](#reference-zed-import) - Imports schema and relationships from a file or url +- [zed mcp](#reference-zed-mcp) - MCP (Model Context Protocol) server commands +- [zed permission](#reference-zed-permission) - Query the permissions in a permissions system +- [zed relationship](#reference-zed-relationship) - Query and mutate the relationships in a permissions system +- [zed schema](#reference-zed-schema) - Manage schema for a permissions system +- [zed use](#reference-zed-use) - Alias for `zed context use` +- [zed validate](#reference-zed-validate) - Validates the given validation file (.yaml, .zaml) or schema file (.zed) +- [zed version](#reference-zed-version) - Display zed and SpiceDB version information ## Reference: `zed backup` @@ -210,13 +208,12 @@ zed backup [flags] ### Children commands -- [zed backup create](#reference-zed-backup-create) - Backup a permission system to a file -- [zed backup parse-relationships](#reference-zed-backup-parse-relationships) - Extract the relationships from a backup file -- [zed backup parse-revision](#reference-zed-backup-parse-revision) - Extract the revision from a backup file -- [zed backup parse-schema](#reference-zed-backup-parse-schema) - Extract the schema from a backup file -- [zed backup redact](#reference-zed-backup-redact) - Redact a backup file to remove sensitive information -- [zed backup restore](#reference-zed-backup-restore) - Restore a permission system from a file - +- [zed backup create](#reference-zed-backup-create) - Backup a permission system to a file +- [zed backup parse-relationships](#reference-zed-backup-parse-relationships) - Extract the relationships from a backup file +- [zed backup parse-revision](#reference-zed-backup-parse-revision) - Extract the revision from a backup file +- [zed backup parse-schema](#reference-zed-backup-parse-schema) - Extract the schema from a backup file +- [zed backup redact](#reference-zed-backup-redact) - Redact a backup file to remove sensitive information +- [zed backup restore](#reference-zed-backup-restore) - Restore a permission system from a file ## Reference: `zed backup create` @@ -253,8 +250,6 @@ zed backup create [flags] --token string token used to authenticate to SpiceDB ``` - - ## Reference: `zed backup parse-relationships` Extract the relationships from a backup file @@ -289,8 +284,6 @@ zed backup parse-relationships [flags] --token string token used to authenticate to SpiceDB ``` - - ## Reference: `zed backup parse-revision` Extract the revision from a backup file @@ -318,8 +311,6 @@ zed backup parse-revision --token string token used to authenticate to SpiceDB ``` - - ## Reference: `zed backup parse-schema` Extract the schema from a backup file @@ -354,8 +345,6 @@ zed backup parse-schema [flags] --token string token used to authenticate to SpiceDB ``` - - ## Reference: `zed backup redact` Redact a backup file to remove sensitive information @@ -392,8 +381,6 @@ zed backup redact [flags] --token string token used to authenticate to SpiceDB ``` - - ## Reference: `zed backup restore` Restore a permission system from a file @@ -433,8 +420,6 @@ zed backup restore [flags] --token string token used to authenticate to SpiceDB ``` - - ## Reference: `zed context` Manage configurations for connecting to SpiceDB deployments @@ -460,11 +445,10 @@ Manage configurations for connecting to SpiceDB deployments ### Children commands -- [zed context list](#reference-zed-context-list) - Lists all available contexts -- [zed context remove](#reference-zed-context-remove) - Removes a context by name -- [zed context set](#reference-zed-context-set) - Creates or overwrite a context -- [zed context use](#reference-zed-context-use) - Sets a context as the current context - +- [zed context list](#reference-zed-context-list) - Lists all available contexts +- [zed context remove](#reference-zed-context-remove) - Removes a context by name +- [zed context set](#reference-zed-context-set) - Creates or overwrite a context +- [zed context use](#reference-zed-context-use) - Sets a context as the current context ## Reference: `zed context list` @@ -499,8 +483,6 @@ zed context list [flags] --token string token used to authenticate to SpiceDB ``` - - ## Reference: `zed context remove` Removes a context by name @@ -528,8 +510,6 @@ zed context remove --token string token used to authenticate to SpiceDB ``` - - ## Reference: `zed context set` Creates or overwrite a context @@ -557,8 +537,6 @@ zed context set --token string token used to authenticate to SpiceDB ``` - - ## Reference: `zed context use` Sets a context as the current context @@ -586,8 +564,6 @@ zed context use --token string token used to authenticate to SpiceDB ``` - - ## Reference: `zed import` Imports schema and relationships from a file or url @@ -658,8 +634,6 @@ zed import [flags] --token string token used to authenticate to SpiceDB ``` - - ## Reference: `zed mcp` MCP (Model Context Protocol) server commands. @@ -689,8 +663,7 @@ To use with Claude Code, run `zed mcp experimental-run` to start the SpiceDB Dev ### Children commands -- [zed mcp experimental-run](#reference-zed-mcp-experimental-run) - Run the Experimental MCP server - +- [zed mcp experimental-run](#reference-zed-mcp-experimental-run) - Run the Experimental MCP server ## Reference: `zed mcp experimental-run` @@ -725,8 +698,6 @@ zed mcp experimental-run [flags] --token string token used to authenticate to SpiceDB ``` - - ## Reference: `zed permission` Query the permissions in a permissions system @@ -752,12 +723,11 @@ Query the permissions in a permissions system ### Children commands -- [zed permission bulk](#reference-zed-permission-bulk) - Check permissions in bulk exist for resource-permission-subject triplets -- [zed permission check](#reference-zed-permission-check) - Check if a subject has permission on a resource -- [zed permission expand](#reference-zed-permission-expand) - Expand the structure of a permission -- [zed permission lookup-resources](#reference-zed-permission-lookup-resources) - Enumerates the resources of a given type for which a subject has permission -- [zed permission lookup-subjects](#reference-zed-permission-lookup-subjects) - Enumerates the subjects of a given type for which the subject has permission on the resource - +- [zed permission bulk](#reference-zed-permission-bulk) - Check permissions in bulk exist for resource-permission-subject triplets +- [zed permission check](#reference-zed-permission-check) - Check if a subject has permission on a resource +- [zed permission expand](#reference-zed-permission-expand) - Expand the structure of a permission +- [zed permission lookup-resources](#reference-zed-permission-lookup-resources) - Enumerates the resources of a given type for which a subject has permission +- [zed permission lookup-subjects](#reference-zed-permission-lookup-subjects) - Enumerates the subjects of a given type for which the subject has permission on the resource ## Reference: `zed permission bulk` @@ -799,8 +769,6 @@ zed permission bulk [flags] --token string token used to authenticate to SpiceDB ``` - - ## Reference: `zed permission expand` Expand the structure of a permission @@ -882,8 +848,6 @@ zed permission expand [flags] --token string token used to authenticate to SpiceDB ``` - - ## Reference: `zed permission lookup-resources` Enumerates the resources of a given type for which a subject has permission @@ -926,8 +890,6 @@ zed permission lookup-resources [flags] --token string token used to authenticate to SpiceDB ``` - - ## Reference: `zed permission lookup-subjects` Enumerates the subjects of a given type for which the subject has permission on the resource @@ -967,8 +929,6 @@ zed permission lookup-subjects [flags] zed preview schema compile schema.zed 1> compiled.zed Write to a file: zed preview schema compile root.zed --out compiled.zed - + ``` ### Options @@ -1015,8 +975,6 @@ zed preview schema compile [flags] --token string token used to authenticate to SpiceDB ``` - - ## Reference: `zed relationship` Query and mutate the relationships in a permissions system @@ -1042,13 +1000,12 @@ Query and mutate the relationships in a permissions system ### Children commands -- [zed relationship bulk-delete](#reference-zed-relationship-bulk-delete) - Deletes relationships matching the provided pattern en masse -- [zed relationship create](#reference-zed-relationship-create) - Create a relationship for a subject -- [zed relationship delete](#reference-zed-relationship-delete) - Deletes a relationship -- [zed relationship read](#reference-zed-relationship-read) - Enumerates relationships matching the provided pattern -- [zed relationship touch](#reference-zed-relationship-touch) - Idempotently updates a relationship for a subject -- [zed relationship watch](#reference-zed-relationship-watch) - Watches the stream of relationship updates and schema updates from the server - +- [zed relationship bulk-delete](#reference-zed-relationship-bulk-delete) - Deletes relationships matching the provided pattern en masse +- [zed relationship create](#reference-zed-relationship-create) - Create a relationship for a subject +- [zed relationship delete](#reference-zed-relationship-delete) - Deletes a relationship +- [zed relationship read](#reference-zed-relationship-read) - Enumerates relationships matching the provided pattern +- [zed relationship touch](#reference-zed-relationship-touch) - Idempotently updates a relationship for a subject +- [zed relationship watch](#reference-zed-relationship-watch) - Watches the stream of relationship updates and schema updates from the server ## Reference: `zed relationship bulk-delete` @@ -1085,8 +1042,6 @@ zed relationship bulk-delete < --token string token used to authenticate to SpiceDB ``` - - ## Reference: `zed relationship touch` Idempotently updates a relationship for a subject @@ -1265,8 +1214,6 @@ zed relationship touch [flags] --token string token used to authenticate to SpiceDB ``` - - ## Reference: `zed schema diff` Diff two schema files @@ -1410,8 +1352,6 @@ zed schema diff --token string token used to authenticate to SpiceDB ``` - - ## Reference: `zed schema read` Read the schema of a permissions system @@ -1445,8 +1385,6 @@ zed schema read [flags] --token string token used to authenticate to SpiceDB ``` - - ## Reference: `zed schema write` Write a schema file (.zed or stdin) to the current permissions system @@ -1492,8 +1430,6 @@ zed schema write [flags] --token string token used to authenticate to SpiceDB ``` - - ## Reference: `zed use` Alias for `zed context use` @@ -1521,8 +1457,6 @@ zed use --token string token used to authenticate to SpiceDB ``` - - ## Reference: `zed validate` Validates the given validation file (.yaml, .zaml) or schema file (.zed) @@ -1581,8 +1515,6 @@ zed validate [flags] --token string token used to authenticate to SpiceDB ``` - - ## Reference: `zed version` Display zed and SpiceDB version information @@ -1616,6 +1548,3 @@ zed version [flags] --skip-version-check if true, no version check is performed against the server --token string token used to authenticate to SpiceDB ``` - - - diff --git a/app/spicedb/ops/_meta.ts b/app/spicedb/ops/_meta.ts index 029121c5..b5e0cccc 100644 --- a/app/spicedb/ops/_meta.ts +++ b/app/spicedb/ops/_meta.ts @@ -3,6 +3,7 @@ export default { "deploying-spicedb-operator": "Deploying the SpiceDB Kubernetes Operator", eks: "Deploying to AWS EKS", data: "Writing data to SpiceDB", + "postgres-fdw": "Using Postgres FDW with SpiceDB", performance: "Improving Performance", resilience: "Improving Resilience", observability: "Observability Tooling", diff --git a/app/spicedb/ops/postgres-fdw/page.mdx b/app/spicedb/ops/postgres-fdw/page.mdx new file mode 100644 index 00000000..ac7b529e --- /dev/null +++ b/app/spicedb/ops/postgres-fdw/page.mdx @@ -0,0 +1,609 @@ +import { Callout, Steps } from "nextra/components"; + +# Using Postgres FDW with SpiceDB + +This guide shows you how to query your SpiceDB instance using standard SQL through the SpiceDB Postgres Foreign Data Wrapper (FDW). + +The Postgres FDW acts as a translation layer that implements the PostgreSQL wire protocol and converts SQL queries into SpiceDB API calls. +This allows you to query permissions, relationships, and schema using familiar SQL syntax. + + + The Postgres FDW is an **experimental feature** that has been tested but may have issues in + certain scenarios. It is subject to change and should be used with caution in production + environments. Please report any issues you encounter on the [SpiceDB GitHub + repository](https://github.com/authzed/spicedb/issues). + + +## Prerequisites + +- A running SpiceDB instance (local or remote) +- Access to the SpiceDB gRPC endpoint +- A SpiceDB preshared key or token +- PostgreSQL installed (for connecting to the FDW server) +- Docker or the SpiceDB binary with FDW support + +## Overview + +The setup process involves: + +1. Starting a SpiceDB instance (if not already running) +2. Starting the FDW proxy server +3. Configuring PostgreSQL to connect to the FDW server +4. Querying your permissions data with SQL + + + +### Start Your SpiceDB Instance + +If you don't already have SpiceDB running, start it with your preferred datastore. + +#### Using Docker with in-memory storage (development) + +```bash +docker run -d \ + --name spicedb \ + -p 50051:50051 \ + authzed/spicedb serve \ + --grpc-preshared-key "somerandomkeyhere" \ + --datastore-engine memory +``` + +#### Using Docker with PostgreSQL (production-ready) + +```bash +docker run -d \ + --name spicedb \ + -p 50051:50051 \ + authzed/spicedb serve \ + --grpc-preshared-key "somerandomkeyhere" \ + --datastore-engine postgres \ + --datastore-conn-uri "postgres://user:password@localhost:5432/spicedb?sslmode=disable" +``` + + + For production deployments, see the [Deploying SpiceDB + Operator](/spicedb/ops/deploying-spicedb-operator) guide. Make note of your preshared key - you'll + need it to configure the FDW. + + +### Start the FDW Proxy Server + +The FDW proxy server acts as a bridge between PostgreSQL and your SpiceDB instance. + +#### Using Docker (Recommended) + +```bash +docker run --rm -p 5432:5432 \ + authzed/spicedb \ + postgres-fdw \ + --spicedb-api-endpoint localhost:50051 \ + --spicedb-access-token-secret "somerandomkeyhere" \ + --spicedb-insecure \ + --postgres-endpoint ":5432" \ + --postgres-username "postgres" \ + --postgres-access-token-secret "fdw-password" +``` + +#### Using the SpiceDB Binary + +```bash +spicedb postgres-fdw \ + --spicedb-api-endpoint localhost:50051 \ + --spicedb-access-token-secret "somerandomkeyhere" \ + --spicedb-insecure \ + --postgres-endpoint ":5432" \ + --postgres-username "postgres" \ + --postgres-access-token-secret "fdw-password" +``` + +#### Using Environment Variables + +```bash +export SPICEDB_SPICEDB_API_ENDPOINT="localhost:50051" +export SPICEDB_SPICEDB_ACCESS_TOKEN_SECRET="somerandomkeyhere" +export SPICEDB_SPICEDB_INSECURE="true" +export SPICEDB_POSTGRES_ENDPOINT=":5432" +export SPICEDB_POSTGRES_USERNAME="postgres" +export SPICEDB_POSTGRES_ACCESS_TOKEN_SECRET="fdw-password" + +spicedb postgres-fdw +``` + + + The `--spicedb-insecure` flag disables TLS verification. Only use this for local development. For + production deployments with TLS, omit this flag and ensure your SpiceDB endpoint uses proper TLS + certificates. + + +#### Configuration Options + +| Flag | Description | Default | +| -------------------------------- | ----------------------------------------------- | ----------------- | +| `--spicedb-api-endpoint` | SpiceDB gRPC endpoint | `localhost:50051` | +| `--spicedb-access-token-secret` | SpiceDB preshared key or token (required) | - | +| `--spicedb-insecure` | Disable TLS verification (development only) | `false` | +| `--postgres-endpoint` | FDW server listen address | `:5432` | +| `--postgres-username` | Username for Postgres authentication | `postgres` | +| `--postgres-access-token-secret` | Password for Postgres authentication (required) | - | +| `--shutdown-grace-period` | Graceful shutdown timeout | `0s` | + +### Configure PostgreSQL Foreign Data Wrapper + +Connect to your PostgreSQL database and run the following SQL commands: + +```sql +-- Install the postgres_fdw extension +CREATE EXTENSION IF NOT EXISTS postgres_fdw; + +-- Create a foreign server pointing to the FDW proxy +CREATE SERVER spicedb_server + FOREIGN DATA WRAPPER postgres_fdw + OPTIONS ( + host 'localhost', + port '5432', + dbname 'ignored' + ); + +-- Create user mapping with authentication credentials +CREATE USER MAPPING FOR CURRENT_USER + SERVER spicedb_server + OPTIONS ( + user 'postgres', + password 'fdw-password' + ); + +-- Import foreign tables +IMPORT FOREIGN SCHEMA public + LIMIT TO (permissions, relationships, schema) + FROM SERVER spicedb_server + INTO public; +``` + + + Replace `fdw-password` with the password you set when starting the FDW proxy server. If your FDW + proxy is running on a different host, update the `host` parameter accordingly. + + +### Load a Schema and Data + +Before querying, you'll need a schema and some relationships in SpiceDB. + +#### Example Schema + +Create a file `schema.zed`: + +```zed +definition user {} + +definition document { + relation viewer: user + relation editor: user + + permission view = viewer + editor + permission edit = editor +} +``` + +#### Load Schema Using zed + +```bash +zed schema write schema.zed \ + --endpoint localhost:50051 \ + --insecure \ + --token "somerandomkeyhere" +``` + +#### Add Relationships + +```bash +# Alice is a viewer of document:readme +zed relationship create document:readme viewer user:alice \ + --endpoint localhost:50051 \ + --insecure \ + --token "somerandomkeyhere" + +# Bob is an editor of document:readme +zed relationship create document:readme editor user:bob \ + --endpoint localhost:50051 \ + --insecure \ + --token "somerandomkeyhere" +``` + +### Query Your Permissions + +You can now query your SpiceDB instance using SQL! + +#### Check Permissions + +```sql +-- Check if user:alice has permission to view document:readme +SELECT has_permission +FROM permissions +WHERE resource_type = 'document' + AND resource_id = 'readme' + AND permission = 'view' + AND subject_type = 'user' + AND subject_id = 'alice'; +``` + +#### Lookup Resources + +```sql +-- Find all documents that user:alice can view +SELECT resource_id +FROM permissions +WHERE resource_type = 'document' + AND permission = 'view' + AND subject_type = 'user' + AND subject_id = 'alice'; +``` + +#### Lookup Subjects + +```sql +-- Find all users who can view document:readme +SELECT subject_id +FROM permissions +WHERE resource_type = 'document' + AND resource_id = 'readme' + AND permission = 'view' + AND subject_type = 'user'; +``` + +#### Query Relationships + +```sql +-- Read relationships for a specific resource +SELECT resource_type, resource_id, relation, subject_type, subject_id +FROM relationships +WHERE resource_type = 'document' + AND resource_id = 'readme'; +``` + +#### Read Schema + +```sql +-- Get all schema definitions +SELECT definition FROM schema; +``` + + + +## Available Tables + +The FDW provides three virtual tables: + +### `permissions` Table + +Used for checking permissions and looking up resources or subjects. + +| Column | Type | Description | +| --------------------------- | ------- | -------------------------------- | +| `resource_type` | text | Resource type (e.g., 'document') | +| `resource_id` | text | Resource ID | +| `permission` | text | Permission name | +| `subject_type` | text | Subject type (e.g., 'user') | +| `subject_id` | text | Subject ID | +| `optional_subject_relation` | text | Optional subject relation | +| `has_permission` | boolean | Whether permission is granted | +| `consistency` | text | Consistency token (ZedToken) | + +**Supported Operations:** SELECT only + +The FDW automatically routes queries to the appropriate SpiceDB API: + +- **CheckPermission**: When all fields are specified +- **LookupResources**: When `resource_id` is not specified +- **LookupSubjects**: When `subject_id` is not specified + +### `relationships` Table + +Used for reading, writing, and deleting relationships. + +| Column | Type | Description | +| --------------------------- | ----- | ---------------------------- | +| `resource_type` | text | Resource type | +| `resource_id` | text | Resource ID | +| `relation` | text | Relation name | +| `subject_type` | text | Subject type | +| `subject_id` | text | Subject ID | +| `optional_subject_relation` | text | Optional subject relation | +| `optional_caveat_name` | text | Optional caveat name | +| `optional_caveat_context` | jsonb | Optional caveat context | +| `consistency` | text | Consistency token (ZedToken) | + +**Supported Operations:** SELECT, INSERT, DELETE + +### `schema` Table + +Used for reading your schema definition. + +| Column | Type | Description | +| ------------ | ---- | ------------------------------- | +| `definition` | text | Schema definition in Zed format | + +**Supported Operations:** SELECT only + +## Advanced Features + +### Consistency Control + +Control read consistency using the `consistency` column: + +```sql +-- Get a consistent view +SELECT resource_id, consistency +FROM permissions +WHERE resource_type = 'document' + AND permission = 'view' + AND subject_type = 'user' + AND subject_id = 'alice' + AND consistency = 'fully_consistent'; +``` + +Available consistency modes: + +- `minimize_latency`: Default, uses the newest available snapshot +- `fully_consistent`: Waits for a fully consistent view +- ``: Uses a specific consistency token +- `@`: Uses exact snapshot matching + +Learn more about [SpiceDB consistency](/spicedb/concepts/consistency). + +### Writing Relationships + +You can insert and delete relationships directly via SQL: + +#### Insert Relationships + +```sql +-- Add a new relationship +INSERT INTO relationships (resource_type, resource_id, relation, subject_type, subject_id) +VALUES ('document', 'readme', 'viewer', 'user', 'alice'); +``` + +#### Delete Relationships + +```sql +-- Remove a relationship +DELETE FROM relationships +WHERE resource_type = 'document' + AND resource_id = 'readme' + AND relation = 'viewer' + AND subject_type = 'user' + AND subject_id = 'alice'; +``` + +### Joining with Local Tables + +One powerful feature of the FDW is the ability to join FDW tables with local PostgreSQL tables. +This allows you to enrich permission data with local application data. + +```sql +-- First, create a local table with document metadata +CREATE TABLE document ( + id text PRIMARY KEY, + title text NOT NULL, + contents text NOT NULL +); + +-- Insert some documents +INSERT INTO document (id, title, contents) VALUES + ('firstdoc', 'Document 1', 'Contents of document 1'), + ('seconddoc', 'Document 2', 'Contents of document 2'), + ('thirddoc', 'Document 3', 'Contents of document 3'); + +-- Join local documents with permissions to find which documents a user can access +SELECT document.id, document.title +FROM document +JOIN permissions ON permissions.resource_id = document.id +WHERE permissions.resource_type = 'document' + AND permissions.permission = 'view' + AND permissions.subject_type = 'user' + AND permissions.subject_id = 'alice' +ORDER BY document.title DESC; +``` + +This pattern is useful for: + +- Building filtered lists based on permissions +- Enriching permission checks with application metadata +- Creating permission-aware reports and dashboards + +### Using Cursors for Large Result Sets + +For queries that return many results, use cursors to paginate: + +```sql +BEGIN; + +DECLARE my_cursor CURSOR FOR + SELECT resource_id FROM permissions + WHERE resource_type = 'document' + AND permission = 'view' + AND subject_type = 'user' + AND subject_id = 'alice'; + +FETCH 100 FROM my_cursor; +FETCH 100 FROM my_cursor; + +CLOSE my_cursor; +COMMIT; +``` + +## Docker Compose Example + +Here's a complete example using Docker Compose: + +```yaml +version: "3" + +services: + postgres: + image: postgres:16 + environment: + POSTGRES_PASSWORD: password + POSTGRES_DB: spicedb + ports: + - "5433:5432" + volumes: + - postgres-data:/var/lib/postgresql/data + + spicedb: + image: authzed/spicedb + command: serve + environment: + SPICEDB_GRPC_PRESHARED_KEY: "somerandomkeyhere" + SPICEDB_DATASTORE_ENGINE: "postgres" + SPICEDB_DATASTORE_CONN_URI: "postgres://postgres:password@postgres:5432/spicedb?sslmode=disable" + ports: + - "50051:50051" + depends_on: + - postgres + + spicedb-fdw: + image: authzed/spicedb + command: postgres-fdw + environment: + SPICEDB_SPICEDB_API_ENDPOINT: "spicedb:50051" + SPICEDB_SPICEDB_ACCESS_TOKEN_SECRET: "somerandomkeyhere" + SPICEDB_SPICEDB_INSECURE: "true" + SPICEDB_POSTGRES_ENDPOINT: ":5432" + SPICEDB_POSTGRES_USERNAME: "postgres" + SPICEDB_POSTGRES_ACCESS_TOKEN_SECRET: "fdw-password" + ports: + - "5432:5432" + depends_on: + - spicedb + +volumes: + postgres-data: +``` + +Start the stack: + +```bash +docker-compose up -d +``` + +Connect to the FDW: + +```bash +psql -h localhost -p 5432 -U postgres -d ignored +# Password: fdw-password +``` + +## Limitations + +The FDW has some limitations to be aware of: + +- **Joins between FDW tables**: Joins between FDW tables (e.g., `permissions` JOIN `relationships`) are not supported. However, joins between FDW tables and local PostgreSQL tables work as expected. +- **Aggregations**: SUM, COUNT, etc. are performed client-side by PostgreSQL +- **Ordering**: ORDER BY clauses are performed client-side by PostgreSQL +- **Subqueries**: Not supported +- **Complex WHERE clauses**: Only simple equality predicates and AND conditions are pushed down to SpiceDB + + + For complex analytics queries, consider exporting data using [bulk + operations](/spicedb/ops/data/bulk-operations) or using the [Watch API](/spicedb/concepts/watch) + to stream changes to a data warehouse. + + +## Performance Considerations + +### Query Planning + +The FDW provides basic statistics to PostgreSQL's query planner, but these are estimates. +Use `EXPLAIN` to understand how your queries are executed: + +```sql +EXPLAIN SELECT resource_id +FROM permissions +WHERE resource_type = 'document' + AND permission = 'view' + AND subject_type = 'user' + AND subject_id = 'alice'; +``` + +### Large Datasets + +For super-fast joins or checks on large datasets, consider [AuthZed Materialize](/authzed/concepts/authzed-materialize). +Once set up, Materialize works seamlessly with the FDW with no SQL changes required to your queries. + +## Troubleshooting + +### Connection Refused + +If you get a connection error, verify: + +1. The FDW proxy server is running and accessible +2. The port is not blocked by a firewall +3. The host and port in your PostgreSQL configuration match the FDW server + +```bash +# Test FDW proxy connectivity +psql -h localhost -p 5432 -U postgres -d ignored +``` + +### SpiceDB Connection Errors + +If the FDW proxy cannot connect to SpiceDB: + +1. Verify SpiceDB is running and accessible +2. Check that the endpoint and port are correct +3. Verify the preshared key matches +4. For remote connections, ensure TLS is configured correctly (omit `--spicedb-insecure`) + +```bash +# Test SpiceDB connectivity using zed +zed context set local localhost:50051 "somerandomkeyhere" --insecure +zed schema read +``` + +### Empty Results + +If queries return no results: + +1. Verify your schema is loaded: `SELECT definition FROM schema;` +2. Check relationships exist: `SELECT * FROM relationships;` +3. Ensure resource types and permission names match your schema + +### Performance Issues + +If queries are slow: + +1. Check SpiceDB performance using the [observability tools](/spicedb/ops/observability) +2. Review your datastore performance (especially important for large datasets) +3. Consider if your queries can be optimized (e.g., using specific resource IDs instead of lookups) +4. Use cursors for large result sets instead of fetching all rows at once +5. For super-fast performance on large datasets, consider [AuthZed Materialize](/authzed/concepts/authzed-materialize), which works seamlessly with the FDW + +## Security Considerations + +### Network Security + +- **Local Development**: Use `--spicedb-insecure` for convenience +- **Production**: Always use TLS for both SpiceDB and FDW connections +- **Firewall Rules**: Restrict access to the FDW proxy port to trusted clients only + +### Authentication + +- Store preshared keys securely (use environment variables or secrets management) +- Rotate preshared keys periodically +- Use different keys for different environments (dev, staging, prod) + +### Access Control + +For granular access control to SpiceDB APIs, consider: + +- Using [Restricted API Access](/authzed/concepts/restricted-api-access) with AuthZed products +- Implementing application-level access controls +- Using PostgreSQL roles and permissions to control FDW access + +## Next Steps + +- Learn about [SpiceDB datastores](/spicedb/concepts/datastores) for production deployments +- Explore [bulk operations](/spicedb/ops/data/bulk-operations) for managing large datasets +- Review [performance tuning](/spicedb/ops/performance) recommendations +- Set up [observability](/spicedb/ops/observability) for monitoring +- Deploy SpiceDB with the [Kubernetes Operator](/spicedb/ops/deploying-spicedb-operator)