Installation Error

[jhenzerling]

While installing LArCV, there seems to be an error in the assert line in "SegLabelFiveTypes2D.cxx" line 41. It calls an undefined use of assert. Removing the .cxx and .h files from ImageMod allowed the make command to complete. But is there an issue for anyone else when running make for these?

[coreyjadams]

I see the line you are talking about, but it compiles fine for me. What compiler are you using and what OS?

Seems like we could replace that with a conditional that throws an exception if false, if needed.

[jhenzerling]

OS: Linux CentOS 7.6.1810
GCC: GCC 6.2.0

I get a couple of other non compilation-breaking errors regarding unused functions, but otherwise it compiles fine. Or well, it doesn't throw an error, I am still getting some issues on LArCV imports ("unable to find larcv.Vertex"), so something else is apparently wrong.

(Example: Unusured function _import_array in PyCluster3D maker
doesn't halt compilation)

[coreyjadams]

Ok - how did you install root? If you got the tarball and didn't build from source, you probably have a GCC mismatch. You must be getting these when building larcv right?

The unused _import_array is OK, that's a numpy thing. But 'unable for find larcv.Vertex' means it failed to load the library, and I've seen this before with a gcc mismatch.

Can you build root from source with the same GCC?

[coreyjadams]

By the way ... larcv3 is coming. You should soon be able to pip install larcv and no longer need ROOT at all, and a lot of this headache will be gone. (At least, that's the plan!)

[jhenzerling]

Yeah, now that you're mentioning the GCC version, I'm seeing this at the beginning of the make:

---

make[1]: Entering directory `/user/jhenzerling/larcv2/larcv/core/Base'
<< generating dict >>
Warning in cling::IncrementalParser::CheckABICompatibility():
C++ ABI mismatch, compiled with GLIBCXX v20150623 running with v20160822
Error in TInterpreter::InspectMembers: TClass and cling disagree on the size of the class TBaseClass, respectively 128 152

Error in TInterpreter::InspectMembers: TClass and cling disagree on the size of the class TClassRef, respectively 16 40

Error in TInterpreter::InspectMembers: TClass and cling disagree on the size of the class TClassRef, respectively 16 40

Error in TInterpreter::InspectMembers: TClass and cling disagree on the size of the class TEnum, respectively 192 216

---

For ROOT I set up version 6.10.08 with no issues (I'm on a shared university system without admin privs so I have access to GCC 7.2.0 and 8.2.0 as well, along with ROOT 6.14.04).

When I tried to make with 7.2.0 and 8.2.0 they failed to even start the make, so you think it'd be a mismatch of GCC and ROOT during compilation?

[jhenzerling]

My GCC and ROOT are the same versions I used when installing larcv2 circa 3-4 months ago and it worked then without any mismatches. Is there a preferred version for all this in the current larcv2 setup?

[coreyjadams]

Yeah, gcc mismatch is definitely your problem here. I think you have two options:

1. Build larcv with the OS gcc (4.8, should be) which is almost certainly the version your root is built with, if you got it from the ROOT website.
2. Build root with gcc you need. If you go this route (I recommend it) you'll spend an extra 20 minutes building root but you'll have a newer gcc. 6.2 or 7.2 is good, 8.2 might be too new for some of the DL frameworks to have full compatibility. If you are doing distributed learning, know that some of the necessary packages like horovod won't build against pytorch if you use gcc 4.8.

If you decide to build root, you can shave down your compile time:
cmake ../root_src/root/ -Dminimal=ON -DCMAKE_INSTALL_PREFIX=../root_install -DCMAKE_CXX_COMPILER=$(which g++) -DCMAKE_C_COMPILER=$(which gcc) -Dpython=ON

[jhenzerling]

So I've been fiddling around with it, and getting some help from my group.
I've used the same GCC with root, and also tried pulling root and the corresponding gcc from fermilab via:

---

source /cvmfs/fermilab.opensciencegrid.org/products/larsoft/setup && retval="$?"
setup root v6_12_06a -q e17:prof

---

which also did not work. Using this setup for root and gcc along with the sourcing produced these errors:

---

make[1]: Entering directory /user/jhenzerling/larcv2/larcv/core/Base' << generating dict >> << compiling BaseDict.cxx >> error: invalid value 'c++1z' in '-std=c++1z' make[1]: *** [/user/jhenzerling/larcv2/build/Base/BaseDict.o] Error 1 make[1]: Leaving directory /user/jhenzerling/larcv2/larcv/core/Base'

---

So essentially, what version of ROOT and GCC are you guys running in order to compile right?

Is there a fermilab/larcv recommended versions to use?

[jhenzerling]

make[1]: Entering directory `/user/jhenzerling/larcv2/larcv/core/Base'

<< generating dict >>

<< compiling BaseDict.cxx >>

error: invalid value 'c++1z' in '-std=c++1z'

make[1]: *** [/user/jhenzerling/larcv2/build/Base/BaseDict.o] Error 1

make[1]: Leaving directory `/user/jhenzerling/larcv2/larcv/core/Base'

make: *** [obj] Error 2

(Sorry hopefully this formats the error better)

[coreyjadams]

Are you doing all of this within the ups framework? If you are just pulling root from larsoft but none of it's dependencies, you're gonna run into trouble. If you want to go that route, use the compiler that comes with that version of root via ups. Otherwise, you may have differences in the libraries at run time compared to build time which can give you trouble.

I'm not a larsoft/fermilab software user for a long time now, so I don't know what recommended settings would be there.

If you're not using larsoft with larcv, would a singularity container suit your needs?

[jhenzerling]

I was not using it in the UPS framework. Honestly those commands I was given by a colleague so I wasn't exactly certain what was going on with them.

And for settings I guess I can ask around.

And I'm not sure what a singularity container is, but I am not using LArSoft, just using LArCV to access the open data and to interface wtih deep learning.

[coreyjadams]

Ok, so I think your best option (meaning, shortest path to success) is to download the root binaries from the cernroot website. Those are built against gcc 4.8, which is the gcc installed on centos7 by default. So, use that same gcc to build larcv. Install tensorflow or pytorch with pip, and I think you will be good to go.

This is the other option:
https://github.com/DeepLearnPhysics/larcv2-singularity

I also have a fork of this repo for specifically running centos images:
https://github.com/coreyjadams/larcv2-singularity/tree/centos

You can get pre-built images for either of these from singularity hub.

Kazu wrote a blog post about singularity on the dip blog awhile ago:
http://deeplearnphysics.org/Blog/2018-09-24-Singularity.html#2018-09-24-Singularity

[jhenzerling]

Hey that actually 100% worked! I ended up using GCC 4.8.5 and ROOT 6.06.08 and that seemed to fix it. I ended up also removing some path variables that I use in my setup script that I'd originally used to link LArCV up to another bit of code, and that all seemed to work!

Thank you so much Corey I'm not sure I would have been able to figure this out on my own without you! (SegLabelFiveTypes2D still doesn't work but hey no worries from me)

Luckily now I'll know where to check if I have issues in future updates!