diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index bcbe793730..71a503c937 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -174,13 +174,28 @@ jobs: - TYPE=find PLUGINS=pouchdb-find npm test - TYPE=mapreduce npm test runs-on: ubuntu-latest + services: + couchdb: + image: couchdb:3.1 + ports: + - 5984:5984 + env: + COUCHDB_USER: admin + COUCHDB_PASSWORD: password env: CLIENT: node + SERVER: couchdb-master + COUCH_HOST: http://admin:password@127.0.0.1:5984 + SKIP_MIGRATION: 1 ADAPTERS: ${{ matrix.adapter }} steps: - uses: actions/checkout@v4 with: persist-credentials: false + - name: Wait for CouchDB + run: ./bin/wait-for-couch.sh 20 + - name: Setup CouchDB CORS + run: curl 'http://admin:password@127.0.0.1:5984/_node/_local/_config/cors/origins' -X PUT -d '"http://127.0.0.1:8000"' - uses: ./.github/actions/install-node-package with: node-version: ${{ matrix.node }} @@ -213,7 +228,18 @@ jobs: - TYPE=find PLUGINS=pouchdb-find npm test - TYPE=mapreduce npm test runs-on: ubuntu-latest + services: + couchdb: + image: couchdb:3.1 + ports: + - 5984:5984 + env: + COUCHDB_USER: admin + COUCHDB_PASSWORD: password env: + SERVER: couchdb-master + COUCH_HOST: http://admin:password@127.0.0.1:5984 + SKIP_MIGRATION: 1 USE_MINIFIED: 1 CLIENT: ${{ matrix.client }} ADAPTERS: ${{ matrix.adapter }} @@ -221,6 +247,10 @@ jobs: - uses: actions/checkout@v4 with: persist-credentials: false + - name: Wait for CouchDB + run: ./bin/wait-for-couch.sh 20 + - name: Setup CouchDB CORS + run: curl 'http://admin:password@127.0.0.1:5984/_node/_local/_config/cors/origins' -X PUT -d '"http://127.0.0.1:8000"' - uses: ./.github/actions/install-node-package with: node-version: ${{ env.NODE_VERSION }} @@ -249,12 +279,55 @@ jobs: cmd: - CLIENT=firefox npm run test-webpack - AUTO_COMPACTION=true npm test - - TYPE=performance npm test - npm run test-unit - npm run test-component - npm run test-fuzzy - npm run verify-build runs-on: ubuntu-latest + env: + SERVER: couchdb-master + COUCH_HOST: http://admin:password@127.0.0.1:5984 + services: + couchdb: + image: couchdb:3.1 + ports: + - 5984:5984 + env: + COUCHDB_USER: admin + COUCHDB_PASSWORD: password + steps: + - uses: actions/checkout@v4 + with: + persist-credentials: false + - name: Wait for CouchDB + run: ./bin/wait-for-couch.sh 20 + - name: Setup CouchDB CORS + run: curl 'http://admin:password@127.0.0.1:5984/_node/_local/_config/cors/origins' -X PUT -d '"http://127.0.0.1:8000"' + - uses: ./.github/actions/install-node-package + with: + node-version: ${{ matrix.node }} + - uses: ./.github/actions/build-pouchdb + - id: test + run: ${{ matrix.cmd }} + continue-on-error: true + - name: First retry + id: retry + if: steps.test.outcome == 'failure' + run: git reset --hard && ${{ matrix.cmd }} + continue-on-error: true + - name: Second retry + if: steps.retry.outcome == 'failure' + run: git reset --hard && ${{ matrix.cmd }} + + nodejs-perf: + needs: lint + strategy: + fail-fast: false + matrix: + node: [20, 22] + cmd: + - TYPE=performance npm test + runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 with: diff --git a/TESTING.md b/TESTING.md index 5ab22f8981..961d8fa151 100644 --- a/TESTING.md +++ b/TESTING.md @@ -1,12 +1,35 @@ -# Running PouchDB Tests +# Running PouchDB Tests This document covers all the types of tests the PouchDB project has and how to run them. PouchDB has been primarily developed on Linux and macOS, if you are using Windows then these instructions will have problems, we would love your help fixing them though. - -## Installation +> [!WARNING] VERY IMPORTANT +> **Always set the `COUCH_HOST` env var to a proper CouchDB for the integration tests!** +> +> If you don’t, the integration tests currently fall back to `pouchdb-server`, which is +> no longer reliable and causes random test failures. (20000ms timeouts). +> +> Until this is resolved, please set the `COUCH_HOST` env var: +> `$ CLIENT=firefox COUCH_HOST=http://admin:admin@127.0.0.1:5984 npm test` +> +> This is necessary for both `npm test` and `npm run dev`. + +## Table of Contents + +- [Running the integration tests](#running-the-integration-tests) + - [Test options](#test-options) +- [Other sets of tests](#other-sets-of-tests) + - [`find` and `mapreduce`](#find-and-mapreduce) + - ["Fuzzy" tests](#fuzzy-tests) + - [Performance tests](#performance-tests) + - [Running tests in the browser](#running-tests-in-the-browser) +- [Other test tasks](#other-test-tasks) +- [Troubleshooting and useful info when working on tests](#troubleshooting-and-useful-info-when-working-on-tests) + - [Where do I get a CouchDB from?](#where-do-i-get-a-couchdb-from) + - [Utils won’t update](#utils-wont-update) + - [Flaky tests and random timeouts](#flaky-tests-and-random-timeouts) The PouchDB test suite expects an instance of CouchDB (version 1.6.1 and above) running in [Admin Party](http://guide.couchdb.org/draft/security.html#party) on @@ -38,6 +61,9 @@ The main test suite can be run using the following command: $ npm test +> [!NOTE] +> If the tests don‘t pass on `main`, try building once before running the tests: `$npm run build`. + PouchDB runs in the browser and on Node.js, and has multiple different storage backends known as _adapters_. In the browser these are `idb`, `indexeddb` and `memory` and on Node.js they're `leveldb` and `memory`. @@ -59,7 +85,7 @@ replication to a remote server, and by default we start an instance of The integration tests support the following options, configured via environment variables. -#### `ADAPTERS` (default: depends on `CLIENT`) +#### `ADAPTERS` (default: depends on `CLIENT`) Comma-separated list of preferred adapter backends that PouchDB will use for local databases. These are selected automatically based on the execution environment, @@ -74,21 +100,21 @@ a remote server, identified by `COUCH_HOST`. This is not necessary for integration tests since they use a mixture of local and remote databases to check compatibility, but it's useful for the `find` and `mapreduce` suites. -#### `AUTO_COMPACTION` (default: `0`) +#### `AUTO_COMPACTION` (default: `0`) Set this to `1` to enable automatic compaction of PouchDB databases by default. -#### `BAIL` (default: `1`) +#### `BAIL` (default: `1`) Normally the test runner will halt as soon as it discovers a failing test. Set this to `0` to prevent this behaviour. -#### `CLIENT` (default: `node`) +#### `CLIENT` (default: `node`) Sets the target platform the tests will execute on. Set this to `firefox`, `chromium` or `webkit` to execute the tests in the browser. -#### `COUCH_HOST` +#### `COUCH_HOST` Some tests perform replication between local and remote databases. When we create a remote database, we get the URL of the remote server from `COUCH_HOST`. @@ -98,25 +124,25 @@ CORS enabled. If not set explicitly, this variable is set automatically based on the other configuration values. -#### `FETCH` (default: `0`) +#### `FETCH` (default: `0`) Set this to `1` to stop PouchDB falling back to `XMLHttpRequest` if `fetch()` is not available. -#### `GREP` +#### `GREP` Use this to request that a specific test is run; if you set `GREP='name of test'` then only those tests whose names include the string `name of test` will run. Regular expressions are also supported. -#### `PLUGINS` (default: empty) +#### `PLUGINS` (default: empty) Comma-separated list of additional plugins that should be loaded into the test environment. For example: $ PLUGINS=pouchdb-find npm test -#### `POUCHDB_SRC` +#### `POUCHDB_SRC` This overrides the path used to load PouchDB in the browser. We use this in CI to select different builds of the PouchDB library, for example to test the @@ -124,19 +150,19 @@ Webpack version, etc. This is an alternative to `SRC_ROOT` and `USE_MINIFIED`. -#### `SRC_ROOT` +#### `SRC_ROOT` This overrides the path used to load all PouchDB files in the browser. We use this in performance tests to allow easily comparing two different versions of PouchDB, including plugin and adapter implementations. -#### `USE_MINIFIED` +#### `USE_MINIFIED` This changes the file extension used for loading PouchDB files in the browser. This can be used in CI and performance testing to select the minified version of PouchDB and its adapters, plugins, etc. -#### `SERVER` (default: `pouchdb-express-router`) +#### `SERVER` (default: `pouchdb-express-router`) To support remote replication tests, we start a server in the background that speaks the CouchDB replication protocol. This variable controls how that is @@ -151,11 +177,11 @@ following values: - `couchdb-master`: use this value if you already have CouchDB running; it causes `COUCH_HOST` to be set to the correct value. -#### `SKIP_MIGRATION` (default: `0`) +#### `SKIP_MIGRATION` (default: `0`) Set this to `1` to skip the migration tests. -#### `VIEW_ADAPTERS` (default: `memory`) +#### `VIEW_ADAPTERS` (default: `memory`) Comma-separated list of preferred view adapter backends that PouchDB will use. This variable overrides the default choice and causes additional adapters to @@ -226,7 +252,7 @@ This supports most of the same options as the integration suite, particularly the `CLIENT`, `ADAPTERS` and `GREP` options. It has some additional options of its own: -#### `ITERATIONS` +#### `ITERATIONS` Sets the number of iterations each test uses by default. @@ -252,6 +278,10 @@ server and opening a browser window yourself. To run the server: $ npm run dev +You will almost always want to include the `find` plugin though, without it, all the `find` tests will fail: + + $ PLUGINS=pouchdb-find npm run dev + Then you can open the page for any of the test suites via the following URLs: - `http://127.0.0.1:8000/tests/integration/` @@ -259,8 +289,9 @@ Then you can open the page for any of the test suites via the following URLs: - `http://127.0.0.1:8000/tests/mapreduce/` - `http://127.0.0.1:8000/tests/performance/` -The test options are controlled by editing the query string; some of the common -command-line options and their query string equivalents are: +You can re-run tests by reloading, and only run specific suites by clicking on the suite names, this sets the `grep` query string mentioned below. This works well in conjunction with setting `.only` on individual tests in that suite. + +The test options are controlled by editing the query string; some of the common command-line options and their query string equivalents are: | Environment variable | Query-string param | | -------------------- | ------------------ | @@ -281,29 +312,61 @@ command-line options and their query string equivalents are: There are a few other tasks we run during CI and which you will find useful to run during development. -### `npm run eslint` +### `npm run eslint` Checks that all code in the project follows our formatting and style guide. This runs before any other tasks are run during our CI build. -### `npm run test-unit` +### `npm run test-unit` Runs the unit tests; running these can give more precise feedback about key building blocks that are not working. -### `npm run test-component` +### `npm run test-component` Tests some additional components besides the core database functionality, for example authentication and read-only replication. -### `npm run test-coverage` +### `npm run test-coverage` Runs the test suite with coverage analysis turned on. -### `npm run test-webpack` +### `npm run test-webpack` Checks that the Webpack build of PouchDB works correctly. -### `npm run verify-build` +### `npm run verify-build` Checks that the build is correct. + +## Troubleshooting and useful info when working on tests + +### Where do I get a CouchDB from? + +See the [official CouchDB documentation](https://docs.couchdb.org/en/stable/install/index.html) for a guide on how to install CouchDB. + +Your CouchDB will most likely then run at `http://127.0.0.1:5984`. All you need to do now is enable CORS so CouchDB will accept requests from PouchDB in the tests. Add `http://127.0.0.1:8000`, the test server’s domain. There are several ways to do this, easiest first: + +1. In the [CouchDB admin UI](http://127.0.0.1:5984/_utils), click on `Configuration` -> `CORS`. +2. Via cURL (`_local` is the node name in single-node databases, which your local dev CouchDB probably is): + ```sh + curl 'http://admin:password@127.0.0.1:5984/_node/_local/_config/cors/origins' -X PUT -d '"http://127.0.0.1:8000"' + ``` +3. There’s also an older [npm package](https://github.com/pouchdb/add-cors-to-couchdb) to help you do this porgrammatically. + +### Utils won’t update + +If you’re modifying the test utils (`/tests/integration/utils.js` etc.) or adding logs to them, you need to rebuild these before running your tests again, eg.: + +```sh +$ npm run build && CLIENT=firefox npm test +``` + +### Flaky tests and random timeouts + +If you regularly run into random timeouts, please re-read the top of this file and set the `COUCH_HOST` env var when running tests or the dev server, eg.: + +```sh +$ CLIENT=firefox COUCH_HOST=http://admin:admin@127.0.0.1:5984 npm test +``` +