diff --git a/.github/ISSUE_TEMPLATE/README.md b/.github/ISSUE_TEMPLATE/README.md new file mode 100644 index 000000000..0ae02773e --- /dev/null +++ b/.github/ISSUE_TEMPLATE/README.md @@ -0,0 +1,10 @@ +## 'ISSUE_TEMPLATE' Directory Contents + +### Files + +bug_report.yml + +feature_request.yaml + +voxl.yml + diff --git a/.github/workflows/README.md b/.github/workflows/README.md new file mode 100644 index 000000000..7a4b38642 --- /dev/null +++ b/.github/workflows/README.md @@ -0,0 +1,6 @@ +## 'workflows' Directory Contents + +### Files + +main.yml + diff --git a/bin/README.md b/bin/README.md new file mode 100644 index 000000000..13c2e1543 --- /dev/null +++ b/bin/README.md @@ -0,0 +1,24 @@ +## 'PIXL/bin' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[linters](./linters/README.md) + +
+ +
+ +

Files

+ + + +| **User docs** | +| :--- | +README.md + +
+ diff --git a/bin/linters/README.md b/bin/linters/README.md new file mode 100644 index 000000000..84698fd12 --- /dev/null +++ b/bin/linters/README.md @@ -0,0 +1,16 @@ +A directory that contains the files used for linting. + +## 'PIXL/bin/linters' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| check_headers_exist.sh | README.md | + +
+ diff --git a/cli/README.md b/cli/README.md index f2002f133..a6aa7997f 100644 --- a/cli/README.md +++ b/cli/README.md @@ -150,3 +150,32 @@ The CLI tests require a running instance of the `rabbitmq` service, for which we pytest -vs tests #for all tests pytest -vs tests/test_docker_commands.py #e.g., for particular tests ``` + +## 'PIXL/cli' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[src](./src/README.md) + +[tests](./tests/README.md) + +
+ +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| pyproject.toml | README.md | + +
+ + + diff --git a/cli/src/README.md b/cli/src/README.md new file mode 100644 index 000000000..c1c1c9029 --- /dev/null +++ b/cli/src/README.md @@ -0,0 +1,26 @@ +This directory contains the source files for the PIXL commands line interface. + +## 'PIXL/cli/src' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[pixl_cli](./pixl_cli/README.md) + +
+ +
+ +

Files

+ + + +| **User docs** | +| :--- | +README.md + +
+ diff --git a/cli/src/pixl_cli/README.md b/cli/src/pixl_cli/README.md new file mode 100644 index 000000000..685a139c5 --- /dev/null +++ b/cli/src/pixl_cli/README.md @@ -0,0 +1,22 @@ +This directory contains the files necessary to create a command line instance of PIXL. + +## 'PIXL/cli/src/pixl_cli' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| main.py | README.md | +| _config.py | | +| _database.py | | +| _docker_commands.py | | +| _io.py | | +| _message_processing.py | | +| __init__.py | | + +
+ diff --git a/cli/tests/README.md b/cli/tests/README.md index 280d3bf6b..2d25bff75 100644 --- a/cli/tests/README.md +++ b/cli/tests/README.md @@ -1,7 +1,33 @@ +# PIXL cli tests -Remove the db container and associated data +This directory contains the code for the tests of the PIXL command line interface. + + + +In order to remove the db container and associated data after the tests have been run use the following command: ```bash docker container rm pixl-test-db -v -f ``` +## 'PIXL/cli/tests' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| conftest.py | README.md | +| test_check_env.py | | +| test_database.py | | +| test_docker_commands.py | | +| test_io.py | | +| test_messages_from_files.py | | +| test_message_processing.py | | +| test_populate.py | | + +
+ diff --git a/docker/README.md b/docker/README.md new file mode 100644 index 000000000..083e669bc --- /dev/null +++ b/docker/README.md @@ -0,0 +1,34 @@ +## 'PIXL/docker' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[export-api](./export-api/README.md) + +[hasher-api](./hasher-api/README.md) + +[imaging-api](./imaging-api/README.md) + +[orthanc-anon](./orthanc-anon/README.md) + +[orthanc-raw](./orthanc-raw/README.md) + +[postgres](./postgres/README.md) + +
+ +
+ +

Files

+ + + +| **Configuration** | **User docs** | **Housekeeping** | +| :--- | :--- | :--- | +| common.env | README.md | .dockerignore | + +
+ diff --git a/docker/hasher-api/README.md b/docker/hasher-api/README.md new file mode 100644 index 000000000..7500b7a27 --- /dev/null +++ b/docker/hasher-api/README.md @@ -0,0 +1,14 @@ +## 'PIXL/docker/hasher-api' Directory Contents + +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| Dockerfile | README.md | + +
+ diff --git a/docker/imaging-api/README.md b/docker/imaging-api/README.md new file mode 100644 index 000000000..72ff585b6 --- /dev/null +++ b/docker/imaging-api/README.md @@ -0,0 +1,14 @@ +## 'PIXL/docker/imaging-api' Directory Contents + +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| Dockerfile | README.md | + +
+ diff --git a/docker/orthanc-anon/README.md b/docker/orthanc-anon/README.md new file mode 100644 index 000000000..d2b913295 --- /dev/null +++ b/docker/orthanc-anon/README.md @@ -0,0 +1,14 @@ +## 'PIXL/docker/orthanc-anon' Directory Contents + +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| Dockerfile | README.md | + +
+ diff --git a/docker/orthanc/README.md b/docker/orthanc/README.md new file mode 100644 index 000000000..e949c46ad --- /dev/null +++ b/docker/orthanc/README.md @@ -0,0 +1,14 @@ +## 'PIXL/docker/orthanc-raw' Directory Contents + +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| Dockerfile | README.md | + +
+ diff --git a/docker/pixl-python/README.md b/docker/pixl-python/README.md new file mode 100644 index 000000000..5890f1ee2 --- /dev/null +++ b/docker/pixl-python/README.md @@ -0,0 +1,14 @@ +## 'PIXL/docker/export-api' Directory Contents + +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| Dockerfile | README.md | + +
+ diff --git a/docker/postgres/README.md b/docker/postgres/README.md new file mode 100644 index 000000000..7135418ac --- /dev/null +++ b/docker/postgres/README.md @@ -0,0 +1,14 @@ +## 'PIXL/docker/postgres' Directory Contents + +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| Dockerfile | README.md | + +
+ diff --git a/docs/README.md b/docs/README.md index 57025896a..0bdb0b573 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,5 +1,6 @@ ## 'PIXL/docs' Directory Contents +

Subdirectories with links to the relevant README

@@ -16,3 +17,4 @@
+ diff --git a/docs/design/README.md b/docs/design/README.md index 63a5f0a24..b7ac2dd9c 100644 --- a/docs/design/README.md +++ b/docs/design/README.md @@ -14,3 +14,4 @@ + diff --git a/docs/design/diagrams/README.md b/docs/design/diagrams/README.md new file mode 100644 index 000000000..7da298bcc --- /dev/null +++ b/docs/design/diagrams/README.md @@ -0,0 +1,8 @@ +## 'PIXL/docs/design/diagrams' Directory Contents + +### Files + +pixl-multi-project-config.drawio + +pixl-multi-project-config.png + diff --git a/docs/file_types/README.md b/docs/file_types/README.md new file mode 100644 index 000000000..a24a35216 --- /dev/null +++ b/docs/file_types/README.md @@ -0,0 +1,6 @@ +## 'PIXL/docs/file_types' Directory Contents + +### Files + +parquet_files.md + diff --git a/docs/joss-publication/README.md b/docs/joss-publication/README.md new file mode 100644 index 000000000..83800df8e --- /dev/null +++ b/docs/joss-publication/README.md @@ -0,0 +1,12 @@ +## 'PIXL/docs/joss-publication' Directory Contents + +### Subdirectories + +[figures](./figures/README.md) + +### Files + +paper.bib + +paper.md + diff --git a/docs/joss-publication/figures/README.md b/docs/joss-publication/figures/README.md new file mode 100644 index 000000000..c6147e300 --- /dev/null +++ b/docs/joss-publication/figures/README.md @@ -0,0 +1,2 @@ +## 'PIXL/docs/joss-publication/figures' Directory Contents + diff --git a/docs/services/README.md b/docs/services/README.md new file mode 100644 index 000000000..f0337901a --- /dev/null +++ b/docs/services/README.md @@ -0,0 +1,8 @@ +## 'PIXL/docs/services' Directory Contents + +### Files + +ftp-server.md + +pixl_database.md + diff --git a/docs/setup/README.md b/docs/setup/README.md new file mode 100644 index 000000000..fd9845552 --- /dev/null +++ b/docs/setup/README.md @@ -0,0 +1,10 @@ +## 'PIXL/docs/setup' Directory Contents + +### Files + +azure-keyvault.md + +developer.md + +uclh-infrastructure-setup.md + diff --git a/hasher/README.md b/hasher/README.md index 668374378..d571e7c3a 100644 --- a/hasher/README.md +++ b/hasher/README.md @@ -58,9 +58,40 @@ See the [Azure Key vault setup](../docs/setup/azure-keyvault.md) documentation f Save the credentials in `.secrets.env` and a LastPass `Hasher API secrets` note. +SK QUESTION: is the reference to Last Pass something that is specific to us or is it a dependency somebody else would need. actually I assume the whole Azure thing is rather how we have chosen to do that rather than a necessity for somebody i.e. they might use a different system for storing their hashes + ``` HASHER_API_AZ_CLIENT_ID= HASHER_API_AZ_CLIENT_PASSWORD= HASHER_API_AZ_TENANT_ID= HASHER_API_AZ_KEY_VAULT_NAME= ``` + +## 'PIXL/hasher' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[src](./src/README.md) + +[tests](./tests/README.md) + +
+ +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| pyproject.toml | README.md | + +
+ + + diff --git a/hasher/src/README.md b/hasher/src/README.md new file mode 100644 index 000000000..0de380702 --- /dev/null +++ b/hasher/src/README.md @@ -0,0 +1,24 @@ +## 'PIXL/hasher/src' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[hasher](./hasher/README.md) + +
+ +
+ +

Files

+ + + +| **User docs** | +| :--- | +| README.md | + +
+ diff --git a/hasher/src/hasher/README.md b/hasher/src/hasher/README.md new file mode 100644 index 000000000..736088286 --- /dev/null +++ b/hasher/src/hasher/README.md @@ -0,0 +1,17 @@ +## 'PIXL/hasher/src/hasher' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| endpoints.py | README.md | +| hashing.py | | +| main.py | | +| __init__.py | | + +
+ diff --git a/hasher/tests/README.md b/hasher/tests/README.md new file mode 100644 index 000000000..9f492bde3 --- /dev/null +++ b/hasher/tests/README.md @@ -0,0 +1,17 @@ +## 'PIXL/hasher/tests' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| conftest.py | README.md | +| test_endpoints.py | | +| test_hashing.py | | +| __init__.py | | + +
+ diff --git a/orthanc/README.md b/orthanc/README.md index 1683a2e5d..d6ea61701 100644 --- a/orthanc/README.md +++ b/orthanc/README.md @@ -9,3 +9,33 @@ PIXL defines 2 types of ORTHANC instances: For both instances we define a plugin in `orthanc-*/plugin/pixl.py` that implements the custom functionality . + +## 'PIXL/orthanc' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[assets](./assets/README.md) + +[orthanc-anon](./orthanc-anon/README.md) + +[orthanc-raw](./orthanc-raw/README.md) + +
+ +
+ +

Files

+ + + +| **User docs** | +| :--- | +| README.md | + +
+ + diff --git a/orthanc/assets/README.md b/orthanc/assets/README.md new file mode 100644 index 000000000..4ce46f006 --- /dev/null +++ b/orthanc/assets/README.md @@ -0,0 +1,16 @@ +## 'PIXL/orthanc/assets' Directory Contents + +
+ +

Files

+ + + +| **Images** | **User docs** | +| :--- | :--- | +| orthanc-anon-az-dicom.png | README.md | +| orthanc-anon-web.png | | +| orthanc-raw-web.png | | + +
+ diff --git a/orthanc/orthanc-anon/README.md b/orthanc/orthanc-anon/README.md index 784084e9c..258e5789a 100644 --- a/orthanc/orthanc-anon/README.md +++ b/orthanc/orthanc-anon/README.md @@ -1,7 +1,7 @@ # Orthanc Anon -_The Orthanc instance responsible for anonymising DICOM data from PACS/VNA and forwarding the images -to their final destination (currently the UCL Data Safe Haven)._ +Orthanc Anon is responsible for anonymising DICOM data from PACS/VNA and forwarding the images +to their final destination (at UCL/UCLH this is currently the UCL Data Safe Haven). ## Setup @@ -24,8 +24,8 @@ The following assumptions are made: ### Configuration -- The Docker image is based on `orthancteam/orthanc`. -- Configuration is driven through customised JSON config. files stored in the [config](./config/) +- Orthanc Anon is based on the Docker image: [orthancteam/orthanc](https://hub.docker.com/r/orthancteam/orthanc). +- Orthanc Anon is configured via a customised JSON config files stored in the [orthanc-anon/config](./config/) directory. - The files are populated with values from environment variables and injected into the container as secrets. Orthanc interprets all `.json` files in the `/run/secrets` mount as config. files. @@ -73,7 +73,7 @@ Start the instance via Docker compose. ### Step 3 -If you have chosen to expose the portas, you should now be able to navigate the web interface at `http://localhost:`, supply the chosen credentials and will be presented with the Orthanc web interface: +If you have chosen to expose the ports, you should now be able to navigate the web interface at `http://localhost:`, supply the chosen credentials and will be presented with the Orthanc web interface: ![Orthanc Raw Web interface](../assets/orthanc-anon-web.png) ### Step 4 @@ -103,4 +103,36 @@ start. ## References -- [Cheat sheet of the REST API](https://book.orthanc-server.com/users/rest-cheatsheet.html) +- [Cheat sheet for the REST API](https://book.orthanc-server.com/users/rest-cheatsheet.html) + + +## 'PIXL/orthanc/orthanc-anon' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[config](./config/README.md) + +[docs](./docs/README.md) + +[plugin](./plugin/README.md) + +
+ +
+ +

Files

+ + + +| **User docs** | +| :--- | +| README.md | + +
+ + + diff --git a/orthanc/orthanc-anon/config/README.md b/orthanc/orthanc-anon/config/README.md new file mode 100644 index 000000000..02fdbac03 --- /dev/null +++ b/orthanc/orthanc-anon/config/README.md @@ -0,0 +1,15 @@ +## 'PIXL/orthanc/orthanc-anon/config' Directory Contents + +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| dicom.json | README.md | +| orthanc.json | | + +
+ diff --git a/orthanc/orthanc-anon/docs/README.md b/orthanc/orthanc-anon/docs/README.md new file mode 100644 index 000000000..594e155de --- /dev/null +++ b/orthanc/orthanc-anon/docs/README.md @@ -0,0 +1,25 @@ +## 'PIXL/orthanc/orthanc-anon/docs' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[assets](./assets/README.md) + +
+ +
+ +

Files

+ + + +| **User docs** | +| :--- | +| DicomServiceViaAAD.md | +| README.md | + +
+ diff --git a/orthanc/orthanc-anon/docs/assets/README.md b/orthanc/orthanc-anon/docs/assets/README.md new file mode 100644 index 000000000..9dc439e65 --- /dev/null +++ b/orthanc/orthanc-anon/docs/assets/README.md @@ -0,0 +1,14 @@ +## 'PIXL/orthanc/orthanc-anon/docs/assets' Directory Contents + +
+ +

Files

+ + + +| **Images** | **User docs** | +| :--- | :--- | +| DicomServiceAuthority.png | README.md | + +
+ diff --git a/orthanc/orthanc-anon/plugin/README.md b/orthanc/orthanc-anon/plugin/README.md new file mode 100644 index 000000000..51a17ee9e --- /dev/null +++ b/orthanc/orthanc-anon/plugin/README.md @@ -0,0 +1,15 @@ +## 'PIXL/orthanc/orthanc-anon/plugin' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| download_dicom_spec.py | README.md | +| pixl.py | | + +
+ diff --git a/orthanc/orthanc-raw/README.md b/orthanc/orthanc-raw/README.md index 31171157f..7a76f9d6d 100644 --- a/orthanc/orthanc-raw/README.md +++ b/orthanc/orthanc-raw/README.md @@ -22,7 +22,7 @@ available shortly when the service is started). ### Configuration -- The Docker image is a deployment of `orthancteam/orthanc` with some extra configuration +- The Orthanc Raw instance is a deployment of the [orthancteam/orthanc](https://hub.docker.com/r/orthancteam/orthanc) container with the following additional configuration: - `ORTHANC_RAW_MAXIMUM_STORAGE_SIZE` to limit the storage size - `ORTHANC_RAW_JOB_HISTORY_SIZE` has been increased so that while there is concurrent processing, the job should always exist for being able to query its status @@ -109,4 +109,31 @@ team. ## References - - [Cheat sheet of the REST API](https://book.orthanc-server.com/users/rest-cheatsheet.html) + - [Cheat sheet for the REST API](https://book.orthanc-server.com/users/rest-cheatsheet.html) + +## 'PIXL/orthanc/orthanc-raw' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[config](./config/README.md) + +[plugin](./plugin/README.md) + +
+ +
+ +

Files

+ + + +| **User docs** | +| :--- | +| README.md | + +
+ diff --git a/orthanc/orthanc-raw/config/README.md b/orthanc/orthanc-raw/config/README.md new file mode 100644 index 000000000..2933d2b62 --- /dev/null +++ b/orthanc/orthanc-raw/config/README.md @@ -0,0 +1,17 @@ +## 'PIXL/orthanc/orthanc-raw/config' Directory Contents + +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| dicom.json | README.md | +| orthanc.json | | +| postgres.json | | + + +
+ diff --git a/orthanc/orthanc-raw/plugin/README.md b/orthanc/orthanc-raw/plugin/README.md new file mode 100644 index 000000000..13b57c61e --- /dev/null +++ b/orthanc/orthanc-raw/plugin/README.md @@ -0,0 +1,14 @@ +## 'PIXL/orthanc/orthanc-raw/plugin' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| pixl.py | README.md | + +
+ diff --git a/pixl_core/README.md b/pixl_core/README.md index cb1fc34a2..bf5645c5a 100644 --- a/pixl_core/README.md +++ b/pixl_core/README.md @@ -15,7 +15,7 @@ Specifically, it defines: - Pydantic models for [project configuration](./src/core/project_config/pixl_config_model.py) - [Secrets management](./src/core/project_config/secrets.py) via an Azure Key Vault. - Handling of [uploads over FTPS](./src/core/uploader/_ftps.py), used to transfer images and parquet files - to the DSH (Data Safe Haven) + to the relevant FTPS server (at UCL this is the DSH (Data Safe Haven)) - [Uploading DICOM files to a DICOMWeb server](./src/core/uploader/_dicomweb.py) - [Uploading DICOM files to XNAT](./src/core/uploader/_xnat.py) @@ -68,7 +68,7 @@ and export of stable DICOM to orthanc-anon while still pulling from the VNA. ### OMOP ES files -Public parquet exports from OMOP ES that should be transferred outside the hospital are copied to +Public parquet exports from OMOP ES that may be transferred outside the hospital are copied to the `exports` directory at the repository base. Within this directory each project has a directory, with all extracts stored in `all_extracts` and @@ -98,7 +98,7 @@ for convenience `latest` is a symlink to the most recent extract. ## Project configuration The `project_config` module provides the functionality to handle -[project configurations](../README.md#configure-a-new-project). +[project configurations](../README.md#configure-a-new-project). <== SK comment I'm not sure this goes to exactly the right place OR the name in the # is misleading ### Design @@ -114,7 +114,7 @@ such as the `FTPSUploader` implement the actual upload functionality. The creden uploading are queried from an **Azure Keyvault** instance (implemented in `core.project_config.secrets`), for which the setup instructions are in the [top-level README](../README.md#project-secrets) -When an extract is ready to be published to the DSH, the PIXL pipeline will upload the **Public** +When an extract is ready to be published to the relevant FTPS server (UCL DSH), the PIXL pipeline will upload the **Public** and **Radiology** [_parquet_ files](../docs/file_types/parquet_files.md) to the `` directory where the DICOM datasets are stored (see the directory structure below). The uploading is controlled by `upload_parquet_files` in [`upload.py`](./src/core/upload.py) which takes a `ParquetExport` @@ -123,7 +123,7 @@ by the `export-patient-data` API endpoint defined in the [Export API](../pixl_export/src/pixl_export/main.py), which in turn is called by the `export_patient_data` command in the [PIXL CLI](../cli/README.md). -Once the parquet files have been uploaded to the DSH, the directory structure will look like this: +Once the parquet files have been uploaded, the directory structure will look like this: ```sh @@ -219,7 +219,7 @@ with existing session and series data: ### XNAT testing setup -For unit testing, we use [`xnat4tests`](https://github.com/Australian-Imaging-Service/xnat4tests) to spin up an XNAT +For unit testing, we use the publicly available [`xnat4tests`](https://github.com/Australian-Imaging-Service/xnat4tests) to spin up an XNAT instance in a Docker container. Secrets are not used for these unit testing. Instead, the following environment variables are used to configure XNAT for testing: @@ -233,3 +233,31 @@ Note, it can take several minutes for the server to start up. Once the server has started, you can log in by visiting `http://localhost:8080` with the username and password set in the `XNAT_USER_NAME` and `XNAT_PASSWORD` environment variables. + +## 'PIXL/pixl_core' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[src](./src/README.md) + +[tests](./tests/README.md) + +
+ +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| pyproject.toml | README.md | + +
+ + diff --git a/pixl_core/src/README.md b/pixl_core/src/README.md new file mode 100644 index 000000000..849af740e --- /dev/null +++ b/pixl_core/src/README.md @@ -0,0 +1,24 @@ +## 'PIXL/pixl_core/src' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[core](./core/README.md) + +
+ +
+ +

Files

+ + + +| **User docs** | +| :--- | +| README.md | + +
+ diff --git a/pixl_core/src/core/README.md b/pixl_core/src/core/README.md new file mode 100644 index 000000000..a16c0b96d --- /dev/null +++ b/pixl_core/src/core/README.md @@ -0,0 +1,37 @@ +## 'PIXL/pixl_core/src/core' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[db](./db/README.md) + +[patient_queue](./patient_queue/README.md) + +[project_config](./project_config/README.md) + +[rest_api](./rest_api/README.md) + +[token_buffer](./token_buffer/README.md) + +[uploader](./uploader/README.md) + +
+ +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| dicom_tags.py | README.md | +| exceptions.py | | +| exports.py | | +| __init__.py | | + +
+ diff --git a/pixl_core/src/core/db/README.md b/pixl_core/src/core/db/README.md new file mode 100644 index 000000000..a59fe00ef --- /dev/null +++ b/pixl_core/src/core/db/README.md @@ -0,0 +1,16 @@ +## 'PIXL/pixl_core/src/core/db' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| models.py | README.md | +| queries.py | | +| __init__.py | | + +
+ diff --git a/pixl_core/src/core/patient_queue/README.md b/pixl_core/src/core/patient_queue/README.md new file mode 100644 index 000000000..3ac2b3449 --- /dev/null +++ b/pixl_core/src/core/patient_queue/README.md @@ -0,0 +1,18 @@ +## 'PIXL/pixl_core/src/core/patient_queue' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| message.py | README.md | +| producer.py | | +| subscriber.py | | +| _base.py | | +| __init__.py | | + +
+ diff --git a/pixl_core/src/core/project_config/README.md b/pixl_core/src/core/project_config/README.md new file mode 100644 index 000000000..529758a4f --- /dev/null +++ b/pixl_core/src/core/project_config/README.md @@ -0,0 +1,17 @@ +## 'PIXL/pixl_core/src/core/project_config' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| pixl_config_model.py | README.md | +| secrets.py | | +| tag_operations.py | | +| __init__.py | | + +
+ diff --git a/pixl_core/src/core/rest_api/README.md b/pixl_core/src/core/rest_api/README.md new file mode 100644 index 000000000..8dd17e3ad --- /dev/null +++ b/pixl_core/src/core/rest_api/README.md @@ -0,0 +1,15 @@ +## 'PIXL/pixl_core/src/core/rest_api' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| router.py | README.md | +| __init__.py | | + +
+ diff --git a/pixl_core/src/core/token_buffer/README.md b/pixl_core/src/core/token_buffer/README.md new file mode 100644 index 000000000..8b5d341b2 --- /dev/null +++ b/pixl_core/src/core/token_buffer/README.md @@ -0,0 +1,16 @@ +## 'PIXL/pixl_core/src/core/token_buffer' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| models.py | README.md | +| tokens.py | | +| __init__.py | | + +
+ diff --git a/pixl_core/tests/README.md b/pixl_core/tests/README.md new file mode 100644 index 000000000..94ae3ea10 --- /dev/null +++ b/pixl_core/tests/README.md @@ -0,0 +1,34 @@ +## 'PIXL/pixl_core/tests' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[data](./data/README.md) + +[patient_queue](./patient_queue/README.md) + +[project_config](./project_config/README.md) + +[token_buffer](./token_buffer/README.md) + +[uploader](./uploader/README.md) + +
+ +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| conftest.py | README.md | +| docker-compose.yml | | +| test_exports.py | | + +
+ diff --git a/pixl_core/tests/data/README.md b/pixl_core/tests/data/README.md new file mode 100644 index 000000000..8c3111207 --- /dev/null +++ b/pixl_core/tests/data/README.md @@ -0,0 +1,16 @@ +## 'PIXL/pixl_core/tests/data' Directory Contents + +
+ +

Files

+ + + +| **Data** | **User docs** | +| :--- | :--- | +| dicom_series.zip | README.md | +| public.zip | | +| xnat_project.xml | | + +
+ diff --git a/pixl_core/tests/patient_queue/README.md b/pixl_core/tests/patient_queue/README.md new file mode 100644 index 000000000..02a555e07 --- /dev/null +++ b/pixl_core/tests/patient_queue/README.md @@ -0,0 +1,16 @@ +## 'PIXL/pixl_core/tests/patient_queue' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| test_message.py | README.md | +| test_producer.py | | +| test_subscriber.py | | + +
+ diff --git a/pixl_core/tests/project_config/README.md b/pixl_core/tests/project_config/README.md new file mode 100644 index 000000000..c5751ce88 --- /dev/null +++ b/pixl_core/tests/project_config/README.md @@ -0,0 +1,15 @@ +## 'PIXL/pixl_core/tests/project_config' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| test_project_config.py | README.md | +| test_secrets.py | | + +
+ diff --git a/pixl_core/tests/token_buffer/README.md b/pixl_core/tests/token_buffer/README.md new file mode 100644 index 000000000..58e597e77 --- /dev/null +++ b/pixl_core/tests/token_buffer/README.md @@ -0,0 +1,14 @@ +## 'PIXL/pixl_core/tests/token_buffer' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| test_tokens.py | README.md | + +
+ diff --git a/pixl_core/tests/uploader/README.md b/pixl_core/tests/uploader/README.md new file mode 100644 index 000000000..a20250f05 --- /dev/null +++ b/pixl_core/tests/uploader/README.md @@ -0,0 +1,17 @@ +## 'PIXL/pixl_core/tests/uploader' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| test_base.py | README.md | +| test_dicomweb.py | | +| test_ftps.py | | +| test_xnat.py | | + +
+ diff --git a/pixl_dcmd/README.md b/pixl_dcmd/README.md index a99b18195..3c80a7cf6 100644 --- a/pixl_dcmd/README.md +++ b/pixl_dcmd/README.md @@ -8,7 +8,7 @@ For external users, the `pixl_dcmd` package provides the following functionality - `anonymise_dicom()`: Applies the [anonymisation operations](#tag-scheme-anonymisation) for the appropriate tag scheme using [Kitware Dicom Anonymizer](https://github.com/KitwareMedical/dicom-anonymizer) and deletes any tags not mentioned in the tag scheme. The dataset is updated in place. - - Will throw a `PixlSkipInstanceError` for any series based on the project config file. Specifically, an error + - Will throw a `PixlSkipInstanceError` for any series based on the project config file. {SK: this sentence doesn't quite make sense to me} Specifically, an error will be thrown if: - the series description matches any series in `series_filters` (usually to remove localiser series) - the modality of the DICOM is not in `modalities` @@ -62,3 +62,31 @@ and optionally a `manufacturer_overrides`. If a `manufacturer_overrides` is defined, it will be used to override the `base` tags, if the manufacturer of the DICOM file matches the manufacturer in the `manufacturer_overrides`. Any tags in the `manufacturer_overrides` that are not in the `base` will be added to the scheme as well. +[SK: Between this and the previous read me I'm not sure this is totally clear, is it possible to have a full example of the yml file linked so that base and a manufacturer_overrides make a bit more sense ] + +## 'PIXL/pixl_dcmd' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[src](./src/README.md) + +[tests](./tests/README.md) + +
+ +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| pyproject.toml | README.md | + +
+ diff --git a/pixl_dcmd/src/README.md b/pixl_dcmd/src/README.md new file mode 100644 index 000000000..bd4b95bbb --- /dev/null +++ b/pixl_dcmd/src/README.md @@ -0,0 +1,24 @@ +## 'PIXL/pixl_dcmd/src' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[pixl_dcmd](./pixl_dcmd/README.md) + +
+ +
+ +

Files

+ + + +| **User docs** | +| :--- | +| README.md | + +
+ diff --git a/pixl_dcmd/src/pixl_dcmd/README.md b/pixl_dcmd/src/pixl_dcmd/README.md new file mode 100644 index 000000000..312dfdca5 --- /dev/null +++ b/pixl_dcmd/src/pixl_dcmd/README.md @@ -0,0 +1,30 @@ +## 'PIXL/pixl_dcmd/src/pixl_dcmd' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[resources](./resources/README.md) + +
+ +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| main.py | README.md | +| tagrecording.py | | +| _database.py | | +| _dicom_helpers.py | | +| _tag_schemes.py | | +| _version.py | | +| __init__.py | | + +
+ diff --git a/pixl_dcmd/src/pixl_dcmd/resources/README.md b/pixl_dcmd/src/pixl_dcmd/resources/README.md new file mode 100644 index 000000000..2a9a5e9df --- /dev/null +++ b/pixl_dcmd/src/pixl_dcmd/resources/README.md @@ -0,0 +1,14 @@ +## 'PIXL/pixl_dcmd/src/pixl_dcmd/resources' Directory Contents + +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| recorded-headers.yml | README.md | + +
+ diff --git a/pixl_dcmd/tests/README.md b/pixl_dcmd/tests/README.md new file mode 100644 index 000000000..5447c3203 --- /dev/null +++ b/pixl_dcmd/tests/README.md @@ -0,0 +1,19 @@ +## 'PIXL/pixl_dcmd/tests' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| conftest.py | README.md | +| test_database.py | | +| test_dicom_validator.py | | +| test_main.py | | +| test_tagrecording.py | | +| test_tag_schemes.py | | + +
+ diff --git a/pixl_export/README.md b/pixl_export/README.md index a42fcdeb9..25979d4d2 100644 --- a/pixl_export/README.md +++ b/pixl_export/README.md @@ -3,7 +3,7 @@ The Export API provides HTTP endpoints to control the copying of EHR data from the OMOP extract to its destination (eg. FTPS). It also uploads DICOM data to its destination after it has been processed by the Imaging API and orthanc(s). -It no longer accepts messages from rabbitmq. +It no longer accepts messages from rabbitmq. ## Installation @@ -37,8 +37,36 @@ pytest ## Usage -Usage should be from the CLI driver, which calls the HTTP endpoints. +Usage should be from the [CLI driver](../cli/README.md), which calls the HTTP endpoints. ## Notes -- The height/weight/GCS value is extracted only within a 24 h time window +- The height/weight/GCS value is extracted only within a 24 h time window + +## 'PIXL/pixl_export' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[src](./src/README.md) + +[tests](./tests/README.md) + +
+ +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| pyproject.toml | README.md | + +
+ + diff --git a/pixl_export/src/README.md b/pixl_export/src/README.md new file mode 100644 index 000000000..3e4e55566 --- /dev/null +++ b/pixl_export/src/README.md @@ -0,0 +1,24 @@ +## 'PIXL/pixl_export/src' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[pixl_export](./pixl_export/README.md) + +
+ +
+ +

Files

+ + + +| **User docs** | +| :--- | +| README.md | + +
+ diff --git a/pixl_export/src/pixl_export/README.md b/pixl_export/src/pixl_export/README.md new file mode 100644 index 000000000..a40140515 --- /dev/null +++ b/pixl_export/src/pixl_export/README.md @@ -0,0 +1,17 @@ +## 'PIXL/pixl_export/src/pixl_export' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| main.py | README.md | +| _databases.py | | +| _queries.py | | +| __init__.py | | + +
+ diff --git a/pixl_export/tests/README.md b/pixl_export/tests/README.md new file mode 100644 index 000000000..e308b1232 --- /dev/null +++ b/pixl_export/tests/README.md @@ -0,0 +1,15 @@ +## 'PIXL/pixl_export/tests' Directory Contents + +
+ +

Files

+ + + +| **Configuration** | **Code** | **User docs** | +| :--- | :--- | :--- | +| docker-compose.yml | conftest.py | README.md | +| | test_app.py | | + +
+ diff --git a/pixl_imaging/README.md b/pixl_imaging/README.md index 9e8a09172..02e3590ad 100644 --- a/pixl_imaging/README.md +++ b/pixl_imaging/README.md @@ -20,14 +20,14 @@ If the study has be identified in VNA or PACS, a query to `orthanc-raw` is made exists locally. If it does exist locally, a check is made to ensure all instances exist locally and any missing instances are retrieved. If the study does not exist locally, the entire study is retrieved from the archive. -Once the study and all its instances are in `orthanc-raw`, the study is sent to `orthanc-anon` via a C-STORE +Once the study and all its instances are in `orthanc-raw`, the study is sent to `orthanc-anon` via a C-STORE operation. >[!NOTE] > When querying the archives, if we do not know the `StudyInstanceUID` we will query by MRN and Accession Number. > This may result in multiple studies being found in the archives. In this instance, all studies returned by the > query will be retrieved and sent to Orthanc Anon for anonymisation. In Orthanc Anon, the studies will be combined -> into a single study as they share the same MRN and Accession Number. +> into a single study as they share the same MRN and Accession Number. ## Installation @@ -56,3 +56,36 @@ The `SKIP_ALEMBIC` environmental variable is used to control whether migrations - `SKIP_ALEMBIC` is set to true for tests that do not use the database (e.g. `SKIP_ALEMBIC=true`). Otherwise you probably want to run this. - If you wanted to test out new migrations from a test/dev deployment on the GAE with data in, then you can redeploy just the `imaging-api` container while keeping the `postgres` container up. + +## 'PIXL/pixl_imaging' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[alembic](./alembic/README.md) + +[scripts](./scripts/README.md) + +[src](./src/README.md) + +[tests](./tests/README.md) + +
+ +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| pyproject.toml | README.md | + + +
+ + diff --git a/pixl_imaging/alembic/README.md b/pixl_imaging/alembic/README.md index fa1a34d83..4fefece03 100644 --- a/pixl_imaging/alembic/README.md +++ b/pixl_imaging/alembic/README.md @@ -10,9 +10,9 @@ as the entry point is [migrate_and_run.sh](../scripts/migrate_and_run.sh). ## Creating a new migration after editing the database model -For convenience, the [autogenerate-migration.sh](autogenerate-migration.sh) has been made. +For convenience, the [autogenerate-migration.sh](autogenerate-migration.sh) script has been created. -Which you can run giving a name for the migration like this: +It can be run giving a name for the migration like this: ```shell cd alembic @@ -22,10 +22,41 @@ cd alembic - This creates a postgres container - Runs the existing migrations - Checks for differences between the SQLAlchemy [models](../../pixl_core/src/core/db/models.py) - and creates a new migration in [versions](versions) + and creates a new migration in the [versions](versions) directory - Takes down the postgres container There's a couple of manual steps: - Check the changes autogenerated and update to match your intention - Commit the changes and review in a pull request as normal + +## 'PIXL/pixl_imaging/alembic' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[versions](./versions/README.md) + +
+ +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| alembic.ini | README.md | +| autogenerate-migration.sh | | +| env.py | | +| migrations.env | | +| script.py.mako | | + +
+ + + diff --git a/pixl_imaging/alembic/versions/README.md b/pixl_imaging/alembic/versions/README.md new file mode 100644 index 000000000..67da1fc1d --- /dev/null +++ b/pixl_imaging/alembic/versions/README.md @@ -0,0 +1,17 @@ +## 'PIXL/pixl_imaging/alembic/versions' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| 83dcb3812628_add_study_uid_column_to_image_table.py | README.md | +| bcaef54e2bfe_create_extract_and_image_tables.py | | +| cb5ee12a6e20_replace_hashed_id_with_pseudo_study_uid.py | | +| d947cc715eb1_add_pseudo_patient_id_column_to_image_table.py | | + +
+ diff --git a/pixl_imaging/scripts/README.md b/pixl_imaging/scripts/README.md new file mode 100644 index 000000000..ae1e0718a --- /dev/null +++ b/pixl_imaging/scripts/README.md @@ -0,0 +1,14 @@ +## 'PIXL/pixl_imaging/scripts' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| migrate_and_run.sh | README.md | + +
+ diff --git a/pixl_imaging/src/README.md b/pixl_imaging/src/README.md new file mode 100644 index 000000000..d56a9acbc --- /dev/null +++ b/pixl_imaging/src/README.md @@ -0,0 +1,24 @@ +## 'PIXL/pixl_imaging/src' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[pixl_imaging](./pixl_imaging/README.md) + +
+ +
+ +

Files

+ + + +| **User docs** | +| :--- | +| README.md | + +
+ diff --git a/pixl_imaging/src/pixl_imaging/README.md b/pixl_imaging/src/pixl_imaging/README.md new file mode 100644 index 000000000..dbe9a22e4 --- /dev/null +++ b/pixl_imaging/src/pixl_imaging/README.md @@ -0,0 +1,17 @@ +## 'PIXL/pixl_imaging/src/pixl_imaging' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| main.py | README.md | +| _orthanc.py | | +| _processing.py | | +| __init__.py | | + +
+ diff --git a/pixl_imaging/tests/README.md b/pixl_imaging/tests/README.md new file mode 100644 index 000000000..98447e8ec --- /dev/null +++ b/pixl_imaging/tests/README.md @@ -0,0 +1,25 @@ +## 'PIXL/pixl_imaging/tests' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[orthanc_raw_config](./orthanc_raw_config/README.md) + +
+ +
+ +

Files

+ + + +| **Configuration** | **Code** | **User docs** | +| :--- | :--- | :--- | +| docker-compose.yml | conftest.py | README.md | +| | test_imaging_processing.py | | + +
+ diff --git a/pixl_imaging/tests/orthanc_raw_config/README.md b/pixl_imaging/tests/orthanc_raw_config/README.md new file mode 100644 index 000000000..3deb7691a --- /dev/null +++ b/pixl_imaging/tests/orthanc_raw_config/README.md @@ -0,0 +1,15 @@ +## 'PIXL/pixl_imaging/tests/orthanc_raw_config' Directory Contents + +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| dicom.json | README.md | +| orthanc.json | | + +
+ diff --git a/postgres/README.md b/postgres/README.md index b161ebf78..f17a85177 100644 --- a/postgres/README.md +++ b/postgres/README.md @@ -5,3 +5,20 @@ container and the pixl_imaging api uses [alembic](https://alembic.sqlalchemy.org This allows a reproducible way to update (or rollback) alterations to the PIXL pipeline's database schema. See [/pixl_imaging/alembic](../pixl_imaging/alembic) for how these are defined and how to create new migrations. + +## 'PIXL/postgres' Directory Contents + +
+ +

Files

+ + + +| **Configuration** | **Code** | **User docs** | +| :--- | :--- | :--- | +| postgres.conf | pixl-db_init.sh | README.md | + +
+ + + diff --git a/projects/README.md b/projects/README.md new file mode 100644 index 000000000..f393a0d79 --- /dev/null +++ b/projects/README.md @@ -0,0 +1,26 @@ +## 'PIXL/projects' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[configs](./configs/README.md) + +[exports](./exports/README.md) + +
+ +
+ +

Files

+ + + +| **User docs** | +| :--- | +| README.md | + +
+ diff --git a/projects/configs/README.md b/projects/configs/README.md new file mode 100644 index 000000000..15420eb30 --- /dev/null +++ b/projects/configs/README.md @@ -0,0 +1,30 @@ +## 'PIXL/projects/configs' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[tag-operations](./tag-operations/README.md) + +
+ +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| ms-pinpoint.yaml | README.md | +| test-external-user.yaml | | +| test-extract-uclh-omop-cdm-dicomweb.yaml | | +| test-extract-uclh-omop-cdm-xnat.yaml | | +| test-extract-uclh-omop-cdm.yaml | | +| uclh-nasogastric-tube-project-ngt-only-full-dataset.yaml | | +| uclh-prostate-mri-external-dataset.yaml | | + +
+ diff --git a/projects/configs/tag-operations/README.md b/projects/configs/tag-operations/README.md new file mode 100644 index 000000000..d97782d7a --- /dev/null +++ b/projects/configs/tag-operations/README.md @@ -0,0 +1,31 @@ +## 'PIXL/projects/configs/tag-operations' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[manufacturer-overrides](./manufacturer-overrides/README.md) + +
+ +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| base.yaml | README.md | +| diffusion-weighted-mri.yaml | | +| mri.yaml | | +| ms-pinpoint.yaml | | +| test-external-user.yaml | | +| test-extract-uclh-omop-cdm.yaml | | +| uclh-nasogastric-tube-project-ngt-only-full-dataset.yaml | | +| xray.yaml | | + +
+ diff --git a/projects/configs/tag-operations/manufacturer-overrides/README.md b/projects/configs/tag-operations/manufacturer-overrides/README.md new file mode 100644 index 000000000..d0c3127b0 --- /dev/null +++ b/projects/configs/tag-operations/manufacturer-overrides/README.md @@ -0,0 +1,17 @@ +## 'PIXL/projects/configs/tag-operations/manufacturer-overrides' Directory Contents + +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| mri-diffusion.yaml | README.md | +| mri.yaml | | + +
+ + + diff --git a/pytest-pixl/README.md b/pytest-pixl/README.md index efaac6860..5da350c26 100644 --- a/pytest-pixl/README.md +++ b/pytest-pixl/README.md @@ -27,6 +27,8 @@ the following environment variables: ## Available testing utilities +Functions from 'PIXL/pytest-pixl/src/pytest_pixl': + - `dicom.write_volume`: write a volume of MRI DICOMs for testing - `dicom.generate_dicom_dataset`: generate a DICOM dataset for testing @@ -46,7 +48,7 @@ Currently we only handle the following dictionary keys - `window_width` - `pixel_data` -This is useful for example when generating DICOM datasets for a volume of slices. +This is useful when generating DICOM datasets for a volume of slices. See for example `dicom.write_volume()`. In addition to the tags dictionary, `generate_dicom_dataset()` has a `**kwargs` parameter that @@ -64,7 +66,7 @@ and `private_tags` is a list of `tuple`s with the following format: [(tag_id, VR, value), ...] ``` -where `tag` can be a `str`, `int` or `Tuple[int, int]`, `VR` is a `str` and `value` is a `str`. +where `tag_id` can be a `str`, `int` or `Tuple[int, int]`, `VR` is a `str` and `value` is a `str`. Note that this requires the [VR](https://dicom.nema.org/dicom/2013/output/chtml/part05/sect_6.2.html) of the tag to be known. @@ -102,3 +104,33 @@ default DICOM tag values used to generate fake DICOM data in `generate_dicom_dat The JSON file was created by running [`scripts/create_default_dicom_tags_json.py`](./scripts/create_default_dicom_tags_json.py), with the details implemented in [`pytest_pixl.dicom._create_default_json`](./src/pytest_pixl/dicom.py). +## 'PIXL/pytest-pixl' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[scripts](./scripts/README.md) + +[src](./src/README.md) + +[tests](./tests/README.md) + +
+ +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| pyproject.toml | README.md | + +
+ + + diff --git a/pytest-pixl/scripts/README.md b/pytest-pixl/scripts/README.md new file mode 100644 index 000000000..c9259c4e5 --- /dev/null +++ b/pytest-pixl/scripts/README.md @@ -0,0 +1,14 @@ +## 'PIXL/pytest-pixl/scripts' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| create_default_dicom_tags_json.py | README.md | + +
+ diff --git a/pytest-pixl/src/README.md b/pytest-pixl/src/README.md new file mode 100644 index 000000000..9a38c50e5 --- /dev/null +++ b/pytest-pixl/src/README.md @@ -0,0 +1,26 @@ +## 'PIXL/pytest-pixl/src' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[pytest_pixl](./pytest_pixl/README.md) + +[resources](./resources/README.md) + +
+ +
+ +

Files

+ + + +| **User docs** | +| :--- | +| README.md | + +
+ diff --git a/pytest-pixl/src/pytest_pixl/README.md b/pytest-pixl/src/pytest_pixl/README.md new file mode 100644 index 000000000..0e5186840 --- /dev/null +++ b/pytest-pixl/src/pytest_pixl/README.md @@ -0,0 +1,28 @@ +## 'PIXL/pytest-pixl/src/pytest_pixl' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[data](./data/README.md) + +
+ +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| dicom.py | README.md | +| ftpserver.py | | +| helpers.py | | +| plugin.py | | +| __init__.py | | + +
+ diff --git a/pytest-pixl/src/pytest_pixl/data/README.md b/pytest-pixl/src/pytest_pixl/data/README.md new file mode 100644 index 000000000..09b8b2a13 --- /dev/null +++ b/pytest-pixl/src/pytest_pixl/data/README.md @@ -0,0 +1,25 @@ +## 'PIXL/pytest-pixl/src/pytest_pixl/data' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[omop-resources](./omop-resources/README.md) + +
+ +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| default_dicom_tags.json | README.md | +| volume_dicom_variables.json | | + +
+ diff --git a/pytest-pixl/src/pytest_pixl/data/omop-resources/README.md b/pytest-pixl/src/pytest_pixl/data/omop-resources/README.md new file mode 100644 index 000000000..780ec047f --- /dev/null +++ b/pytest-pixl/src/pytest_pixl/data/omop-resources/README.md @@ -0,0 +1,28 @@ +## 'PIXL/pytest-pixl/src/pytest_pixl/data/omop-resources' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[omop](./omop/README.md) + +
+ +
+ +

Files

+ + + +| **Data** | **User docs** | +| :--- | :--- | +| batch_input.csv | README.md | +| duplicate_input.csv | | +| multiple_projects.csv | | +| participant_id.csv | | +| test.csv | | + +
+ diff --git a/pytest-pixl/src/pytest_pixl/data/omop-resources/omop/README.md b/pytest-pixl/src/pytest_pixl/data/omop-resources/omop/README.md new file mode 100644 index 000000000..328a0fc7c --- /dev/null +++ b/pytest-pixl/src/pytest_pixl/data/omop-resources/omop/README.md @@ -0,0 +1,26 @@ +## 'PIXL/pytest-pixl/src/pytest_pixl/data/omop-resources/omop' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[private](./private/README.md) + +[public](./public/README.md) + +
+ +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| extract_summary.json | README.md | + +
+ diff --git a/pytest-pixl/src/pytest_pixl/data/omop-resources/omop/private/README.md b/pytest-pixl/src/pytest_pixl/data/omop-resources/omop/private/README.md new file mode 100644 index 000000000..81c4746bc --- /dev/null +++ b/pytest-pixl/src/pytest_pixl/data/omop-resources/omop/private/README.md @@ -0,0 +1,14 @@ +## 'PIXL/pytest-pixl/src/pytest_pixl/data/omop-resources/omop/private' Directory Contents + +
+ +

Files

+ + + +| **Data** | **User docs** | +| :--- | :--- | +| PERSON_LINKS.parquet | README.md | +| PROCEDURE_OCCURRENCE_LINKS.parquet | | + +
diff --git a/pytest-pixl/src/pytest_pixl/data/omop-resources/omop/public/README.md b/pytest-pixl/src/pytest_pixl/data/omop-resources/omop/public/README.md new file mode 100644 index 000000000..d446f6902 --- /dev/null +++ b/pytest-pixl/src/pytest_pixl/data/omop-resources/omop/public/README.md @@ -0,0 +1,14 @@ +## 'PIXL/pytest-pixl/src/pytest_pixl/data/omop-resources/omop/public' Directory Contents + +
+ +

Files

+ + + +| **Data** | **User docs** | +| :--- | :--- | +| PROCEDURE_OCCURRENCE.parquet | README.md | + +
+ diff --git a/pytest-pixl/src/resources/README.md b/pytest-pixl/src/resources/README.md new file mode 100644 index 000000000..591013d8e --- /dev/null +++ b/pytest-pixl/src/resources/README.md @@ -0,0 +1,24 @@ +## 'PIXL/pytest-pixl/src/resources' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[ssl](./ssl/README.md) + +
+ +
+ +

Files

+ + + +| **User docs** | +| :--- | +| README.md | + +
+ diff --git a/pytest-pixl/src/resources/ssl/README.md b/pytest-pixl/src/resources/ssl/README.md new file mode 100644 index 000000000..d6d2ec754 --- /dev/null +++ b/pytest-pixl/src/resources/ssl/README.md @@ -0,0 +1,15 @@ +## 'PIXL/pytest-pixl/src/resources/ssl' Directory Contents + +
+ +

Files

+ + + +| **Keys** | **User docs** | +| :--- | :--- | +| localhost.crt | README.md | +| localhost.key | | + +
+ diff --git a/pytest-pixl/tests/README.md b/pytest-pixl/tests/README.md new file mode 100644 index 000000000..f4b67339b --- /dev/null +++ b/pytest-pixl/tests/README.md @@ -0,0 +1,25 @@ +## 'PIXL/pytest-pixl/tests' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[samples_for_fixture_tests](./samples_for_fixture_tests/README.md) + +
+ +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| conftest.py | README.md | +| test_ftpserver_fixture.py | | + +
+ diff --git a/pytest-pixl/tests/samples_for_fixture_tests/README.md b/pytest-pixl/tests/samples_for_fixture_tests/README.md new file mode 100644 index 000000000..7ff4031d4 --- /dev/null +++ b/pytest-pixl/tests/samples_for_fixture_tests/README.md @@ -0,0 +1,24 @@ +## 'PIXL/pytest-pixl/tests/samples_for_fixture_tests' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[test_ftpserver_fixture](./test_ftpserver_fixture/README.md) + +
+ +
+ +

Files

+ + + +| **User docs** | +| :--- | +| README.md | + +
+ diff --git a/pytest-pixl/tests/samples_for_fixture_tests/test_ftpserver_fixture/README.md b/pytest-pixl/tests/samples_for_fixture_tests/test_ftpserver_fixture/README.md new file mode 100644 index 000000000..ab97db7b0 --- /dev/null +++ b/pytest-pixl/tests/samples_for_fixture_tests/test_ftpserver_fixture/README.md @@ -0,0 +1,14 @@ +## 'PIXL/pytest-pixl/tests/samples_for_fixture_tests/test_ftpserver_fixture' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| test_ftpserver_login.py | README.md | + +
+ diff --git a/schemas/README.md b/schemas/README.md new file mode 100644 index 000000000..0e34f82ee --- /dev/null +++ b/schemas/README.md @@ -0,0 +1,14 @@ +## 'PIXL/schemas' Directory Contents + +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| github-issue-forms.json | README.md | + +
+ diff --git a/scripts/README.md b/scripts/README.md new file mode 100644 index 000000000..992fc993e --- /dev/null +++ b/scripts/README.md @@ -0,0 +1,17 @@ +## 'PIXL/scripts' Directory Contents + +
+ +

Files

+ + + +| **Code** | **User docs** | +| :--- | :--- | +| cmove_all_studies.py | README.md | +| delete_oldest_n_studies.py | | +| filter_cohort_for_those_present_in_raw.py | | +| list_newest_n_studies.py | | + +
+ diff --git a/test/README.md b/test/README.md index 53b2d97f2..e4acf6831 100644 --- a/test/README.md +++ b/test/README.md @@ -46,9 +46,9 @@ For CI, there is also another subcommand to run pytest, reporting coverage ## The `pytest-pixl` plugin -We provide a [`pytest` plugin](../pytest-pixl/README.md) with shared functionality for PIXL system +We provide a [`pytest` plugin](../pytest-pixl/README.md) with shared functionality for the PIXL system and unit tests. This includes an `ftp_server` fixture to spin up a lightweight FTP server, -to mock the FTP server used by the Data Safe Haven. +to mock an FTP server being used to deposit the data. ## File organisation @@ -58,7 +58,7 @@ to mock the FTP server used by the Data Safe Haven. ### Scripts -`./scripts` contains bash and Python scripts to check the individual components of the system test. +`../scripts` contains bash and Python scripts to check the individual components of the system test. ### Resources @@ -76,3 +76,36 @@ but configured to upload to a [DICOMweb server](#dicomweb-config) ### DICOMWeb config `./dicomweb_config/` contains the Orthanc configuration files for the mock [DICOMweb server](../docs/services/dicomweb-server.md). + +## 'PIXL/test' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[dicomweb_config](./dicomweb_config/README.md) + +[resources](./resources/README.md) + +[vna_config](./vna_config/README.md) + +
+ +
+ +

Files

+ + + +|**Configuration** | **Code** | **User docs** | +| :--- | :--- | :--- | +| .env | conftest.py | README.md | +| .secrets.env.sample | run-system-test.sh | | +| docker-compose.yml | system_test.py | | +| | test_parquet_exports.py | | +| | utils.py | | + +
+ diff --git a/test/dicomweb_config/README.md b/test/dicomweb_config/README.md new file mode 100644 index 000000000..23d069559 --- /dev/null +++ b/test/dicomweb_config/README.md @@ -0,0 +1,15 @@ +## 'PIXL/test/dicomweb_config' Directory Contents + +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| dicom.json | README.md | +| orthanc.json | | + +
+ diff --git a/test/resources/README.md b/test/resources/README.md new file mode 100644 index 000000000..f633fe31e --- /dev/null +++ b/test/resources/README.md @@ -0,0 +1,27 @@ +## 'PIXL/test/resources' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[omop](./omop/README.md) + +[omop-dicomweb](./omop-dicomweb/README.md) + +
+ +
+ +

Files

+ + + +| **Dicom** | **User docs** | +| :--- | :--- | +| Dicom1.dcm | README.md | +| Dicom2.dcm | | + +
+ diff --git a/test/resources/omop-dicomweb/README.md b/test/resources/omop-dicomweb/README.md new file mode 100644 index 000000000..1d1788ac7 --- /dev/null +++ b/test/resources/omop-dicomweb/README.md @@ -0,0 +1,26 @@ +## 'PIXL/test/resources/omop-dicomweb' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[private](./private/README.md) + +[public](./public/README.md) + +
+ +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| extract_summary.json | README.md | + +
+ diff --git a/test/resources/omop-dicomweb/private/README.md b/test/resources/omop-dicomweb/private/README.md new file mode 100644 index 000000000..92299c635 --- /dev/null +++ b/test/resources/omop-dicomweb/private/README.md @@ -0,0 +1,16 @@ +## 'PIXL/test/resources/omop-dicomweb/private' Directory Contents + +
+ +

Files

+ + + +| **Data** | **User docs** | +| :--- | :--- | +| PERSON_LINKS.parquet | README.md | +| PROCEDURE_OCCURRENCE_LINKS.parquet | | + +
+ + diff --git a/test/resources/omop-dicomweb/public/README.md b/test/resources/omop-dicomweb/public/README.md new file mode 100644 index 000000000..98de24d26 --- /dev/null +++ b/test/resources/omop-dicomweb/public/README.md @@ -0,0 +1,14 @@ +## 'PIXL/test/resources/omop-dicomweb/public' Directory Contents + +
+ +

Files

+ + + +| **Data** | **User docs** | +| :--- | :--- | +| PROCEDURE_OCCURRENCE.parquet | README.md | + +
+ diff --git a/test/resources/omop/README.md b/test/resources/omop/README.md new file mode 100644 index 000000000..228f6840a --- /dev/null +++ b/test/resources/omop/README.md @@ -0,0 +1,26 @@ +## 'PIXL/test/resources/omop' Directory Contents + +
+ +

Subdirectories with links to the relevant README

+ + + +[private](./private/README.md) + +[public](./public/README.md) + +
+ +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| extract_summary.json | README.md | + +
+ diff --git a/test/resources/omop/private/README.md b/test/resources/omop/private/README.md new file mode 100644 index 000000000..969599330 --- /dev/null +++ b/test/resources/omop/private/README.md @@ -0,0 +1,15 @@ +## 'PIXL/test/resources/omop/private' Directory Contents + +
+ +

Files

+ + + +| **Data** | **User docs** | +| :--- | :--- | +| PERSON_LINKS.parquet | README.md | +| PROCEDURE_OCCURRENCE_LINKS.parquet | | + +
+ diff --git a/test/resources/omop/public/README.md b/test/resources/omop/public/README.md new file mode 100644 index 000000000..e10168fbd --- /dev/null +++ b/test/resources/omop/public/README.md @@ -0,0 +1,14 @@ +## 'PIXL/test/resources/omop/public' Directory Contents + +
+ +

Files

+ + + +| **Data** | **User docs** | +| :--- | :--- | +| PROCEDURE_OCCURRENCE.parquet | README.md | + +
+ diff --git a/test/vna_config/README.md b/test/vna_config/README.md new file mode 100644 index 000000000..d5f0491e9 --- /dev/null +++ b/test/vna_config/README.md @@ -0,0 +1,15 @@ +## 'PIXL/test/vna_config' Directory Contents + +
+ +

Files

+ + + +| **Configuration** | **User docs** | +| :--- | :--- | +| dicom.json | README.md | +| orthanc.json | | + +
+