Compatibility with conda-forge

[jan-janssen]

Hi @dskarls

Is is possible to release the openkim developer-platform as conda-forge package? I would be interested to compare the standardised tests in the openkim developer-platform to the ones we have in the pyiron project. Starting with simple things like comparing elastic constants and then extending it to more complex tests.

At the same time having a similar standard for different simulation environments would be beneficial for the whole community. I continue to release new packages on conda-forge, so you find my latest contribution on https://github.com/jan-janssen/conda-forge-contribution and we use those to build our docker environments at https://github.com/pyiron/docker-stacks . More recently also the AiiDA team started to integrate conda-forge in their virtual environments marvel-nccr/quantum-mobile#161 and they extended the supported features for the DFT simulation codes. So having openkim on-board would be a great extension.

As far as I see most of the packages in https://github.com/openkim/developer-platform/blob/main/docker/install/Dockerfile should already be available, so from my perspective it should be an easy transition.

Best regards,

Jan

[dskarls]

Hi @dskarls

Is is possible to release the openkim developer-platform as conda-forge package? I would be interested to compare the standardised tests in the openkim developer-platform to the ones we have in the pyiron project. Starting with simple things like comparing elastic constants and then extending it to more complex tests.

Hi Jan,

Thanks for your inquiry. As to whether the KIM developer platform is translatable to a conda package, it has three main capabilities to consider: (I) it allows you to actually run KIM tests and verification checks against our models, (II) it provides an environment to iteratively develop new models, and (III) it provides a set of convenience utilities and downloading and managing all KIM items installed within it. I think (I) should be attainable with conda as long as we can create packages for everything in our stack. (III) can probably also be done as long as we have a nice way for the user to specify the location of a controlled directory structure to use for items. However, I’m a little concerned about (II). I’ve been told that trying to use the gcc/make/cmake conda packages to compile/install models from source ends up not working very well, perhaps because of some interference with system-wide compiler installations; can you speak to whether this would be problematic?

At the same time having a similar standard for different simulation environments would be beneficial for the whole community. I continue to release new packages on conda-forge, so you find my latest contribution on https://github.com/jan-janssen/conda-forge-contribution and we use those to build our docker environments at https://github.com/pyiron/docker-stacks . More recently also the AiiDA team started to integrate conda-forge in their virtual environments marvel-nccr/quantum-mobile#161 and they extended the supported features for the DFT simulation codes. So having openkim on-board would be a great extension.

As far as I see most of the packages in https://github.com/openkim/developer-platform/blob/main/docker/install/Dockerfile should already be available, so from my perspective it should be an easy transition.

There may be some cases where licensing may be an issue. For example, MD++ has no explicit overall license defined and appears to be composed of a collection of different source code with an assortment of licenses. MD++ is also a case where we have to do some manual manipulation of the makefile to make it build headless, which might be a bit of a headache to maintain as their makefile is updated over time.

[jan-janssen]

Hi Daniel,

In terms of working with the conda compiler, I have a few examples1 mainly coupling C++ and Python. The easiest way to do it is exporting the CC environment variable and then use it in the script rather than binding to a specific compiler directly. Apart from this the conda build utility2 should allow the users to directly package their new modules locally as conda packages and maintain compatibility.

In terms of the license I am not an expert but from a technical perspective distributing a Docker container to me is the same as distributing a binary, the only difference is you add a whole operation system to the distribution. So if you have the rights to distribute a code like MD++ in your Docker container, that should also be sufficient to distribute it as binary packages. My experience is that most developers of opensource codes are very open when I suggest to create a conda package for their code.

For the headless build, I typically include those modifications as separate packages then updating them to be compatible with the latest version is less complicated. In addition you could also create two packages if there is a need for both the headless and the unmodified version.

I hope this addresses your concerns, in particular for a small community like ours I see a huge benefit if the majority of software packages are available via one package manager. I propose to use conda as it is already heavily used in the machine learning and bioinformatics community but I am also open to discuss others, I guess switching to a different package manager later is still possible once we have all the build scripts in place but at the current stage conda seems to be the most promising candidate for me.

Best,

Jan