Pickle protocol error

[harryherold]

Hi,
I am trying to execute the VGG16 example in a Docker container:

FROM tensorflow/tensorflow:1.10.1

RUN apt update && apt install -y --no-install-recommends \
    git \
    vim \
    python \
    python3 \
    python-sklearn \
    python3-sklearn \
    python-numpy \
    python3-numpy \
    ipython \
    ipython3 \
    python3-setuptools

RUN useradd --create-home --shell /bin/bash -u 1000 appuser
USER appuser

VOLUME [/workspace]

I execute following code:

import mlpredict

VGG16 = mlpredict.import_tools.import_dnn('VGG16')
total_time, _, _ = VGG16.predict(gpu='V100', optimizer='SGD', batchsize=32)

and get following error

/home/appuser/.local/lib/python2.7/site-packages/mlpredict-0.0.1-py2.7.egg/mlpredict/api.pyc in predict(self, gpu, optimizer, batchsize, model_file, scaler_file)
    145         gpu_stats = import_gpu(gpu)
    146 
--> 147         scaler = joblib.load(scaler_file)
    148 
    149         layer, time = predict_walltime(

/usr/local/lib/python2.7/dist-packages/sklearn/externals/joblib/numpy_pickle.pyc in load(filename, mmap_mode)
    576                     return load_compatibility(fobj)
    577 
--> 578                 obj = _unpickle(fobj, filename, mmap_mode)
    579 
    580     return obj

/usr/local/lib/python2.7/dist-packages/sklearn/externals/joblib/numpy_pickle.pyc in _unpickle(fobj, filename, mmap_mode)
    506     obj = None
    507     try:
--> 508         obj = unpickler.load()
    509         if unpickler.compat_mode:
    510             warnings.warn("The file '%s' has been generated with a "

/usr/lib/python2.7/pickle.pyc in load(self)
    862             while 1:
    863                 key = read(1)
--> 864                 dispatch[key](self)
    865         except _Stop, stopinst:
    866             return stopinst.value

/usr/lib/python2.7/pickle.pyc in load_proto(self)
    890         proto = ord(self.read(1))
    891         if not 0 <= proto <= 2:
--> 892             raise ValueError, "unsupported pickle protocol: %d" % proto
    893     dispatch[PROTO] = load_proto
    894 

ValueError: unsupported pickle protocol: 3

Please, could you help me to fix it?
I am planning to use it in project.

Many thanks.

[harryherold]

What do you suggest for mlpredict, python2 or 3?

[harryherold]

Okay, I used python3 but now I get a key error while loading the file.
It seems the file is broken.

>>> joblib.load('/home/appuser/.cache/Python-Eggs/mlpredict-0.0.1-py2.7.egg-tmp/mlpredict/model/scaler_Conv_all.save')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3/dist-packages/joblib/numpy_pickle.py", line 578, in load
    obj = _unpickle(fobj, filename, mmap_mode)
  File "/usr/lib/python3/dist-packages/joblib/numpy_pickle.py", line 508, in _unpickle
    obj = unpickler.load()
  File "/usr/lib/python3.6/pickle.py", line 1050, in load
    dispatch[key[0]](self)
KeyError: 174

[james-sungjae-lee]

This problem is related to "joblib" package which scikit-learn contains, so you just downgrade scikit-learn package to "0.19.0" then reinstall mlpredict. Because this way downgrade scikit-learn package, you don't need to change "from sklearn.externals import joblib" into "import joblib".