New ExportProvider and ImportProvider

[lauraBahamon]

No description provided.

[lauraBahamon]

In the context of our MAUS project we forked this repository and develop further plugins for GitHub.

Export Provider

This provider plugin enables users to export project data to new or existing repositories on GitHub.

Project data is generated with RDMO's file export plugins (XML, CSV and JSON) for all projects. SMP projects have some additional export choices that are provided by our rdmo-plugins-maus. Currently, these choices are: a README.md, a CITATION.cff, a LICENSE, and an SMP report (a document containing all the project content in the form of a report).

These SMP-specific export choices are created with views that use attributes of the SMP catalog. Our hope is that, in the process of standardizing the RDMO attributes, the SMP-specific views will create usable files for every other catalog, not only SMP.

Import Provider

This provider plugin enables users to import project data from public or private GitHub repositories.

Similar to the existing GitHubImport class, the new plugin enables the import of RDMO XML project files. This new plugin also allows users to import repository metadata (license, languages, dependency graph, and CITATION.cff).

The metadata import is optimized for SMP projects because other catalogs may not have matching questions (and attributes) for the metadata or use different attributes. In those cases, the plugin only imports the usable subset of repository metadata.

SMP Catalog

All the changes use a new version of the SMP catalog. Here is our forked repository rdmo-catalog with the modified SMP catalog as well as the new views for the export choices.

Other changes

1. Radio Select displaying the most recently updated, accessible GitHub repositories for each user

The existing plugins have been modified so that their form displays an overview of repositories to choose from (instead of an input field to write the repository url).

This overview is filtered by the rights needed for the selected action (import or export). For example, when using the import plugin, the user only sees the repositories for which they have read access. The repository list for the export plugin may vary, because it lists those repositories for which the user has write access.

2. Modification to class IntegrationCreateView

In order to include the list of accessible repositories for each user in the GitHubIssueProvider class of this repository, we had to modify the RDMO class IntegrationCreateView. This involved adding a setup method that checks if the integration provider has a method called integration_setup and runs it. Here is our modified code.

3. Import and export choices form field

Both the import and export provider plugins use a custom multiple-choice field provided by our rdmo-plugins-maus. Checkout the repository's README.md for more info.

[jochenklar]

This should move to the head block.

[jochenklar]

This should move to the head block.

[jochenklar]

This should just be named validadors.py and directly to rdmo_github. Same with forms.py.

[jochenklar]

Genereally, except should not be used without a specific exception to catch. Here it would be better to use

if not response.content:
    ...

anyway.

[jochenklar]

for export in exports:

[jochenklar]

To generic, for requests you can use:

try:
    response.raise_for_status()
except requests.exceptions.HTTPError as e:
    ...

[jochenklar]

Static lists like this could go to the top of the class, maybe this could be separated into a maus and non-maus part.

[jochenklar]

We can check the settings for the plugin like this:

# in apps.py
from django.apps import AppConfig


class RDMOGitHubConfig(AppConfig):
    name = 'rdmo_github'

    def ready(self):
        import rdmo_github.checks  # noqa: F401

# in checks.py
from django.conf import settings
from django.core.checks import Error, register


@register()
def check_github_provider_settings(app_configs, **kwargs):
    errors = []

    try:
        provider = settings.GITHUB_PROVIDER
    except AttributeError:
        errors.append(
            Error('settings.GITHUB_PROVIDER does not exist.', hint='Add GITHUB_PROVIDER to config/settings/local.py')
        )
    else:
        for key in ['client_id', 'client_secret', 'app_type', 'github_app_name']:
            if not provider.get(key):
                errors.append(Error(f'Key "{key}" is missing from settings.GITHUB_PROVIDER')
        )

    return errors

[jochenklar]

For the setup and automatic formatting, you can use the files in dev of https://github.com/rdmorganiser/rdmo-plugins-orcid as template:

• https://github.com/rdmorganiser/rdmo-plugins-orcid/blob/dev/pyproject.toml for the ruff configuration
• https://github.com/rdmorganiser/rdmo-plugins-orcid/blob/dev/.pre-commit-config.yaml
• https://github.com/rdmorganiser/rdmo-plugins-orcid/blob/dev/rdmo_orcid/__init__.py to automatically determine the version of the package

pre-commit install
pre-commit run --all

should then run the hooks.

If the typos catches the translation files, it can be configured like here: https://github.com/rdmorganiser/rdmo/blob/main/.pre-commit-config.yaml#L41