EnMAP-Box 3

The EnMAP-Box is free and open source QGIS Plugin to visualize and process remote sensing raster data. It is particularly developed to handle imaging spectroscopy data, as from the upcoming EnMAP sensor.

Highlights

• an easy-to-use graphical user interface for the visualization of vector and raster data sources in parallel and in spatially linked maps.

• collection and visualisation of spectral profiles spectral libraries. Spectral profiles can come from different sources, e.g. raster images, field spectrometer or table-sheets.

• enhances the QGIS Processing Framework with many algorithms commonly used in remote sensing and imaging spectroscopy, e.g. support vector machines or random forest based raster classification, regression, cluster approaches from the scikit-learn library.

• applications specific to imaging spectroscopy and the EnMAP program, e.g. a simulation of spectral profiles (IIVM), atmospheric correction of EnMAP data, mapping of geological classes from EnMAP data and more...

Documentation: https://enmap-box.readthedocs.io

Git Repository: https://github.com/EnMAP-Box/enmap-box

About EnMAP: https://www.enmap.org/

Run the EnMAP-Box

The EnMAP-Box is a QGIS Plugin that can be installed from the QGIS Plugin Manager.

However, the following steps show you how to run the EnMAP-Box from python without starting the QGIS Desktop application.

1. Install QGIS + Python dependencies

conda (all OS)

1. Install conda / conda mini-forge (preferred), as described here

2. Install one of the QGIS + EnMAP-Box environments listed in https://github.com/EnMAP-Box/enmap-box/tree/main/.env/conda

• enmapbox-base.yml basic QGIS installation only and minimal python requirements to start EnMAP-Box GUI and Processing Framework.
• enmapbox-full.yml QGIS + all other python requirements that allow to run all EnMAP-Box features.

Examples:

conda env create -f https://raw.githubusercontent.com/EnMAP-Box/enmap-box/main/.env/conda/enmapbox_base.yml
conda env create -f https://raw.githubusercontent.com/EnMAP-Box/enmap-box/main/.env/conda/enmapbox_full.yml

Use:

• -n to set your own environment name, e.g. -n myenmapbox.
• -y to accept all prompts.

Run conda update to update the environment to the latest version:

conda env update --prune -f https://raw.githubusercontent.com/EnMAP-Box/enmap-box/main/.env/conda/enmapbox-full.yml

• -n myenvironmentname allows to overwrite environments with names different to that specified in the *.yml file.
• --prune causes conda to remove any dependencies that are no longer required from the environment.

Windows / Linux / MacOS

Either use conda (see above), or follow the OS-specific instructions here: https://qgis.org/en/site/forusers/download.html

2. Test the QGIS environment

You need to be able to run the following commands from your shell:

1. Git (to check out the EnMAP-Box repository)

   $git --version

   Test if git can connect to the remote EnMAP-Box repository:

   • ssh (recommended): git ls-remote git@github.com:EnMAP-Box/enmap-box.git
   • https: git ls-remote https://github.com/EnMAP-Box/enmap-box.git

2. QGIS, Qt Designer (to design GUIs) and the Qt Assistant (superfast browsing of Qt / QGIS API documents)

   $qgis --version
   $designer
   $assistant

3. Check if all environmental variables are set correctly. Start Python and try to use the QGIS API:

   $python
   Python 3.9.18 | packaged by conda-forge | (main, Aug 30 2023, 03:40:31) [MSC v.1929 64 bit (AMD64)] on win32
   Type "help", "copyright", "credits" or "license" for more information.

   Print the QGIS version

   >>> from qgis.core import Qgis
   >>> print(Qgis.version())
   3.28.10-Firenze

   Import the QGIS Processing Framework

   >>> import processing
   Application path not initialized
   

   (Don't worry about the Application path not initialized message)

If one of these tests fail, check the values for the follwing variables in your local environment:

• PATH - needs to include the directories with your git and qgis executable

• PYTHONPATH - needs to include the QGIS python code directories, including the QGIS-internal plugin folder: PYTHONPATH=F:\mamba_envs\enmapbox_light_longterm\Library\python\plugins;F:\mamba_envs\enmapbox_light_longterm\Library\python;

• QT_PLUGN_PATH - the Qt plugin directory, e.g.:

  QT_PLUGIN_PATH=F:\mamba_envs\enmapbox_light_longterm\Library\qtplugins;F:\mamba_envs\enmapbox_light_longterm\Library\plugins;

3. Clone this repository

Use the following commands to clone the EnMAP-Box and update its submodules:

TLDR:

Open a shell (e.g. OSGeo4W or mamba, see above) that allows to run git and python with PyQGIS, then run:

# Clone the repository using ssh 
# See https://docs.github.com/en/authentication/connecting-to-github-with-ssh for details on SSH
git clone --recurse-submodules git@github.com:EnMAP-Box/enmap-box.git

# alternatively (but not recommended) you can use https as well:
# git clone --recurse-submodules https://github.com/EnMAP-Box/enmap-box.git

cd enmap-box
git config --local include.path ../.gitconfig

# compile the EnMAP-Box resource files and download QGIS resource files to display icons  
python scripts/setup_repository.py

# start the EnMAP-Box
python enmapbox

# if you have writing access to a submodule, set the remote-URL accordingly, e.g.
cd enmapbox/qgispluginsupport
git git remote set-url origin git@github.com:EnMAP-Box/qgispluginsupport.git

Detailed description

In the following we refer to the EnMAP-Box repository https://github.com/EnMAP-Box/enmap-box.git Replace it with your own EnMAP-Box fork from which you can create pull requests.

1. Ensure that your environment has git available and starts QGIS by calling qgis (see1. and 2.). You copy a bootstrap script like .env/OSGeo4W/qgis_env.bat (windows) or scripts/qgis_env.sh (linux) and adjust to your local settings for.

2. Clone the EnMAP-Box repository.

   git clone git@github.com:EnMAP-Box/enmap-box.git

   You might also use the url https://github.com/EnMAP-Box/enmap-box.git instead. However, ssh access is preferred.

3. Initialize submodules and pull their code, which is hosted in different repositories

   cd enmapbox
   git submodule update --init --remote --recursive

4. Once initialized, you can update submodules at any later point by:

   git submodule update --remote

   Of course cloning and submodule updating can be done in one step:

   git clone --recurse-submodules git@github.com:EnMAP-Box/enmap-box.git

   At any later point, you can pull in submodule updates by

   git submodule update --remote

   Doing so automatically when pulling the EnMAP-Box project can be enabled by:

   git config set submodule.recurse true

   This setting (and maybe more in future) is already defined in the .gitconfig. You can enable it for your local repository by:

   git config --local include.path ../.gitconfig

   Submodules use https addresses to pull and push updates (url and pushurl in .gitmodules), e.g. https://bitbucket.org/ecstagriculture/enmap-box-lmu-vegetation-apps.git. To enable ssh authentication you can replace them with SSH uris as followed:

   cd enmapbox/apps/lmuapps
   # check existing url
   git remote -v
   # change remote urls
   git remote set-url origin git@bitbucket.org:ecstagriculture/enmap-box-lmu-vegetation-apps.git
   # check changed url
   git remote -v

5. you can push changes upstream by:

   cd <submodule>
   git add .
   git commit -m "my changes"
   git push origin HEAD:master

   Finally, announce changes in a submodule to the EnMAP-Box (super) project by:

   cd <EnMAP-Box root>
   git add <submodule path>
   git commit -m "added submodule updates"
   git push

6. Ensure that PyQGIS is available to your python enviroment. (This means you can start a python shell and import qgis)

7. Compile resource files and download the test data.

   python scripts/setup_repository.py

8. Now you can start the EnMAP-Box from shell by:

   python enmapbox

4. Setup your IDE, e.g. PyCharm

tbd.

How to contribute

Our online documentation at https://enmap-box.readthedocs.io describes how you can support the development of the EnMAP-Box.

Please keep the code in a good shape.

Coding style

Use flake8 to check if the EnMAP-Box code applies to the rules defined in tox.ini:

flake8 

To check staged files only, run:

flake8 $(git status -s | grep -E '\.py$' | cut -c 4-)

Use bandit to check for security issues:

# check for issues in the entire repository
bandit -r .

# show issues marked with the nosec tag
bandit -r . --ignore-nosec 

# check for issues in a specific directory only
bandit  -r enmapbos/apps/SpecDeepMap

Testing

Run scripts/runtests.sh (Linux/macOS) or scripts\runtests.bat (Win) to start the tests defined in /tests.

Provide additional arguments to the test runner, e.g., to run tests in parallel or from a specific directory, module, -class or -method only:

# run the map canvas tests only, verbose outputs
scripts/runtests.sh -vvv tests/enmap-box/enmapbox/gui/test_mapcanvas.py

# run all tests in parallel
scripts/runtests.sh -n auto

scripts/runtests.sh -n auto tests/enmap-box/enmapbox/gui/test_mapcanvas.py::MapCanvasTests::test_mapCrosshairDistance

Some tests use the SensorProducts class in tests/enmapboxtestdata.py to load data from theEnmapBoxExternalSensorProducts folder. This folder is access-restricted, because it contains exemplary data from different sensors (e.g., Sentinel-2, Sentinel-3, EnMAP, PRISMA), which we are not allowed to share publicly. If you have access to it, ENMAPBOX_SENSOR_PRODUCT_ROOT environment variable to point to the folder containing the sensor products. This will allow you to run additional tests.

export ENMAPBOX_SENSOR_PRODUCT_ROOT=~/MyMountedDrives/EnmapBoxExternalSensorProducts

To test with docker, run:

QGIS_TEST_VERSION=3.44 docker compose \
  -f .env/docker/docker-compose.gh.yml \
  run --rm --build --user $(id -u):$(id -g) qgis \
  /usr/src/.env/docker/run_docker_tests.sh -n auto

• --rm removes the container after the tests are finished.
• --build forces a rebuild of the container
• --user sets the user id and group id of the container to the current user
• -n auto runs the tests in parallel (using pytest xdist)`

License

The EnMAP-Box is released under the GNU Public License (GPL) Version 3 or above. A copy of this licence can be found in the LICENSE.txt file that is part of the EnMAP-Box plugin folder and the EnMAP-Box repository, and also at https://www.gnu.org/licenses/

Developing the EnMAP-Box under this license means that you can (if you want to) inspect and modify the source code and guarantees that you will always have access to an EnMAP-Box software that is free of cost and can be freely modified.

FAQ

tbd.

Support

You can get support in the following ways:

• Read the EnMAP-Box documentation https://enmap-box.readthedocs.io

• Open an issue with your question, bug report, feature request or other enhancement https://github.com/EnMAP-Box/enmap-box/issues/new

• Write us an email: enmapbox@enmap.org