Problems doing make rpm in reg_utils

[vetens]

Brief summary of issue

NSFD error when trying to make rpm reg_utils

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

make rpm should run uninterrupted

Current Behavior

NSFD error when looking for ``reg_utils/python/reg_interface/rpm/build/bdist.linux-x86_64/rpm/BUILDROOT/reg_interface-1.1.0-1.2879f89git.centos7.python2.7.x86_64/usr/lib/python*/site-packages/reg_utils/scripts/*.py``

"error: File not found by glob:"

Steps to Reproduce (for bugs)

1. environment variables: export ELOG_PATH=<repo directory>/elogs; export BUILD_HOME=<repo directory>; export PETA_STAGE=/data/bigdisk/sw/peta-stage/;
2. enter python virtual environment: source <repo directory>/venv/cc7/py2.7/bin/activate
3. cd <repo directory>/reg_utils/
4. make rpm
   my repo directory is: /afs/cern.ch/work/w/wvetens/private/cms-gem-daq/
   error message: error: File not found by glob: /afs/cern.ch/work/w/wvetens/private/cms-gem-daq/reg_utils/python/reg_interface/rpm/build/bdist.linux-x86_64/rpm/BUILDROOT/reg_interface-1.1.0-1.2879f89git.centos7.python2.7.x86_64/usr/lib/python*/site-packages/reg_utils/scripts/*.py

Possible Solution (for bugs)

It looks like everything up to usr exists. I tried creating a link to my /usr folder, which seems to have all the needed directories, but it looks like the makefile overwrites this before looking for usr. Looks like I'm either missing an environment variable not talked about in the readme or you can insert something like mkdir $BUILD_HOME/reg_utils/python/reg_interface/rpm/build/bdist.linux-x86_64/rpm/BUILDROOT/reg_interface-1.1.0-1.2879f89git.centos7.python2.7.x86_64/usr; ln -s /usr $BUILD_HOME/reg_utils/python/reg_interface/rpm/build/bdist.linux-x86_64/rpm/BUILDROOT/reg_interface-1.1.0-1.2879f89git.centos7.python2.7.x86_64/usr before the step where it searches for this. I couldn't find where in the makefile to do this however, as it is rather terse.

Context (for feature requests)

Your Environment

• Version used:
• Shell used: bash

[mexanick]

The error is not reproducible without the venv. Can you try to compile using the system-wide python? If this works, I would suggest you proceed as follows:

• produce rpms with system-wide python
• activate your venv
• install the rpm in your venv
• run make doc

[bdorney]

What is meant by repo directory in the statement below:

export BUILD_HOME=;

Is this case 1:

export BUILD_HOME=/some/path/reg_utils

Or case 2:

export BUILD_HOME=/some/path/
ls $BUILD_HOME
reg_utils/

In case 2 (the correct case) $BUILD_HOME is the directory that reg_utils sits in. Could you confirm this is your case?

[bdorney]

2. enter python virtual environment: source /venv/cc7/py2.7/bin/activate

I see it is not immediately apparent from the material under Building GEM Software (which is of course the problem) but you should never try to build a package or an rpm from inside a venv.

This causes (as you've seen) some issues with the make process failing to find expected filepaths.

Can you try the build process again but not from inside a $VIRTUAL_ENV?

[bdorney]

To be clear the issue here is the documentation is at fault here; not the developer (@vetens). Thanks for bringing this issue up; we will improve clarity to ensure this confusion is resolved.

[vetens]

In my case, repo directory is: /afs/cern.ch/work/w/wvetens/private/cms-gem-daq/, It is the directory in which my reg_utils and sw_utils dwell, so case 2.

Tried a system-wide compile which didn't work either. Got a few more errors, cp -rf README.md LICENSE CHANGELOG.md MANIFEST.in requirements.txt pkg cp: cannot stat ‘LICENSE’: No such file or directory cp: cannot stat ‘CHANGELOG.md’: No such file or directory
with traceback:
Traceback (most recent call last): File "setup.py", line 64, in <module> license = 'MIT', File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup _setup_distribution = dist = klass(attrs) File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 265, in __init__ self.fetch_build_eggs(attrs.pop('setup_requires')) File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 289, in fetch_build_eggs parse_requirements(requires), installer=self.fetch_build_egg File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 630, in resolve raise VersionConflict(dist,req) # XXX put more info here
Working with misha also through mattermost to try to figure out what's going on here, I'm including my machine, env output, and command history here for posterity's sake:
I am on gem904daq01, here is my env output: https://pastebin.com/EDJ3PSbF
and here is my command history: https://pastebin.com/BpXh0YyA
the gemdaq is an alias to source the python venv alias gemdaq='source sw_utils/scripts/setup_gemdaq.sh;'
the only other stuff in my bashrc are defining the environment variables ELOG_PATH, BUILD_HOME, and PETA_STAGE