How to administrate upload areas and transfer data using morphic util

Why this document?

morphic-util is a command line based tool that assists with data upload to the MorPhic DRACC AWS S3 storage. This guide outlines how to do the following steps using the morphic-util tool.

• Create a new upload area where a contributor can upload their data

There are two types of users

• Administrators
  1. Can create AWS Cognito user accounts for contributors.
  2. Can create upload areas for contributors.
  3. Can delete upload areas.
  4. Can list (view) contents of any upload area.
  5. Can upload to any upload area.
  6. Can download from any upload area.
• Users
  1. Can authenticate themselves using user account details shared by administrators and can upload data and metadata files to upload areas provided by administrators.
  2. Can traverse to other upload areas within the same DPC if access has been granted by administrators.

This document is a guide for administrators. If you are a user or an administrator who wants to perform user actions please see How to upload data to an upload area using morphic util.

Prerequisites

To be able to use morphic-util you will need:

1. Python3 installed on your computer
2. Basic knowledge of how to navigate the command line and run commands in a terminal, e.g. cd, ls, pwd
3. AWS Cognito administrator username and password

Installation

Check that you have Python 3 installed by opening a terminal and running the following command:

which python3

It should return the location of your Python 3 installation e.g.

/path/to/bin/python3

If nothing is returned it means you do not have Python 3 installed.

There are many tutorials online for installing python so please follow one that makes sense to you and suits your operating system or contact your system administrator for help. One example can be found here: https://realpython.com/installing-python/

Once you have installed Python 3, in your terminal run the following command to install morphic-util

pip3 install morphic-util

Once installed, you can see the available commands by typing the following command in your terminal.

morphic-util -h

If this succeeds continue to the configuration section.

Depending on your Python installation you may need to use sudo to install the tool in your system directory. If the above command gives a Permission denied error, try using the following command:

sudo pip3 install morphic-util

You will then be prompted for your system password. If this succeeds continue to the configuration section below.

Configuration

The first time you use the tool it will need to be configured. The configuration creates an AWS profile called morphic-util that will give you the appropriate permissions to upload to your upload area.

To configure use the following command with the AWS Cognito admin username and AWS cognito password that you have obtained from another administrator.

morphic-util config <AWS COGNITO ADMIN USERNAME> <AWS COGNITO PASSWORD>

This would look something like:

$ morphic-util config myAdminUserName myAdmin#Pass#12345
Credentials saved.
Valid credentials

Actions

The actions below are only available to administrators. However, as an administrator you will also often want to perform user actions, such as listing the contents of an upload area or download its files. For these actions, please read the How to upload data to an upload area using morphic util.

Creating an upload area

One of your chief tasks as an administrator will be to create upload areas that you can pass on to users wish to upload data.

To create an upload area run the command This will be default create an upload area with upload (u) and delete (x) permissions for the user (admin will always have full permissions).

$ morphic-util create UPLOADAREANAME DPCNAME 
Created upload area with name UPLOADAREANAME and DPCNAME DPC

Replace UPLOADAREANAME with a name requested by the contributor. DPCNAME is the centre name of the contributor

To create an upload area with different permissions run the command

$ morphic-util create UPLOADAREANAME DPCNAME -p {u,ud,ux,udx}

-p {u,ud,ux,udx}   allowed actions (permissions) on new area. u for
                     upload, x for delete and d for download. Default is ux

Created upload area with name UPLOADAREANAME and DPCNAME DPC

List all upload areas

You may want to list all the MorPhic DPC folders. To do this you can use a switch on the list command, as below:

$ morphic-util list -b
morhic-msk/ ux  
morphic-jax/ ux 
morphic-nwu/ ux     
morphic-ucsf/ ux 
4 items

You may also want to list all the MorPhic upload areas within a DPC folder. To do this you can use a switch on the select command and followed by the list command, as below:

$ morphic-util select DPCFOLDER/UPLOADAREANAME
$ morphic-util list

Delete an upload area

In order to avoid paying for vast amounts of upload areas, please make every effort to delete upload areas that are no longer required.

To delete an area, select it and then run the delete command. For example,

$ morphic-util select DPCFOLDER/UPLOADAREANAME
Selected DPCFOLDER/UPLOADAREANAME

$ morphic-util delete -d
Confirm delete DPCFOLDER/UPLOADAREANAME/? Y/y to proceed: Y
Deleting...
DPCFOLDER/UPLOADAREANAME/

For more information about the select command please see How to upload data to an upload area using morphic util

Please take great care when executing this command. No backups are being kept for upload areas so when it's gone, it's gone!