Python 3.5 support

[wbecker]

When I try to build it with Python 3.5 I get the following error

Collecting bllipparser
  Using cached bllipparser-2015.12.3.tar.gz
Requirement already satisfied (use --upgrade to upgrade): six in /usr/lib/python3.5/site-packages (from bllipparser)
Installing collected packages: bllipparser
  Running setup.py install for bllipparser ... error
    Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-84szdo98/bllipparser/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-hkxweaja-record/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.5
    creating build/lib.linux-x86_64-3.5/bllipparser
    copying python/bllipparser/CharniakParser.py -> build/lib.linux-x86_64-3.5/bllipparser
    copying python/bllipparser/ModelFetcher.py -> build/lib.linux-x86_64-3.5/bllipparser
    copying python/bllipparser/RerankingParser.py -> build/lib.linux-x86_64-3.5/bllipparser
    copying python/bllipparser/ParsingShell.py -> build/lib.linux-x86_64-3.5/bllipparser
    copying python/bllipparser/__init__.py -> build/lib.linux-x86_64-3.5/bllipparser
    copying python/bllipparser/JohnsonReranker.py -> build/lib.linux-x86_64-3.5/bllipparser
    copying python/bllipparser/Utility.py -> build/lib.linux-x86_64-3.5/bllipparser
    copying python/bllipparser/RerankerFeatureCorpus.py -> build/lib.linux-x86_64-3.5/bllipparser
    copying python/bllipparser/__main__.py -> build/lib.linux-x86_64-3.5/bllipparser
    running build_ext
    building 'bllipparser._CharniakParser' extension
    creating build/temp.linux-x86_64-3.5
    creating build/temp.linux-x86_64-3.5/first-stage
    creating build/temp.linux-x86_64-3.5/first-stage/PARSE
    creating build/temp.linux-x86_64-3.5/first-stage/PARSE/swig
    gcc -pthread -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fPIC -Ifirst-stage/PARSE/ -I/usr/include/python3.5m -c first-stage/PARSE/swig/wrapper.C -o build/temp.linux-x86_64-3.5/first-stage/PARSE/swig/wrapper.o
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    first-stage/PARSE/swig/wrapper.C:30900:1: error: too many initializers for ‘PyAsyncMethods’
     };
     ^
    ... more errors ensue....

I've never touched swig in my life, but looking at this it's seems that wrapper.C is not being regenerated properly.

[dmcc]

Thanks for the report. swig had problems generating the wrapper code for Python 3.5 (d183ab8) but version 3.0.8 claims to fix these (haven't had time to test yet). Are you running the latest version? (swig -version)

(side note: I will update docs to make these issues clearer)

[wbecker]

I tried it with the latest version and it failed in just the same way.

I backed down and installed python 3.4 and swig 3.0.7 and it's all working now!

[jarussell]

I am having the same (or a similar) problem.

python --version
Python 3.5.1

swig -version|grep -v ^$
SWIG Version 3.0.8
Compiled with g++ [x86_64-unknown-linux-gnu]
Configured options: +pcre
Please see http://www.swig.org for reporting bugs and further information

no problems with the C++ code itself, seems to build fine with:

git clone https://github.com/BLLIP/bllip-parser
cd bllip-parser
make

This code builds but I am new to python and not sure how to get it to install from pip.

sudo pip install bllipparser --no-cache-dir #couldn't figure out how to force it to redownload since I am new to python

gives the same or similar errors to wbecker.

tox appears to build a 2.7 (successfully) and 3.4 version but is unable to build the 3.4 since I don't have 3.4 on my system.

[dmcc]

I think the issue is that the wrapper file bundled with the 2015.12.3 release uses SWIG 3.0.7 or earlier, which causes these build problems. I will make a new release soon using SWIG 3.0.8 to generate the wrapper file which will hopefully address this problem.

In the meantime, here's a possible workaround::

shell> make real-clean
shell> python setup.py build
shell> sudo python setup.py install

The first command will wipe out the SWIG-generated wrapper files (among other things) which will be regenerated during building.

[halfak]

I'm still running into this issue running python 3.5.1 on Ubuntu 16.04. pip pulls down bllipparser-2015.12.3.tar.gz.

[dmcc]

Thanks for the note @halfak. Sorry, this fell off my plate. I'll try to get a new-enough version of SWIG working this weekend and cut a new release.

[dmcc]

Alright, there's a new release (2016.9.11) which should work with Python 3.5/swig 3.0.8. Please let me know either way if it works or doesn't. Thanks!

[anupamme]

Hello @dmcc

I tried installing it and it fails:

ubuntu - 16.04 (xenial)
python - 3.5.2
swig - 3.0.8

Can @halfak confirm whether you can install because you are able to install

[halfak]

@anupamme, I am able to install bllipparser-2016.9.11 via pypi with pip.

ubuntu - 16.04
python - 3.5.1
swig - 3.0.8

[anupamme]

Okay I am not able to install. These are the things I have tried:

1. Using pypi, installing bllipparser-2016.9.11
2. git cloned this repo and ran python setup.py build, but got same errors.

I have tried it on 2 machines:
1.
ubuntu - 16.04 (xenial)
python - 3.5.2
swig - 3.0.8

Mac OS 10.11.5 (el capitan)
python - 3.5.0
swig - 3.0.10

So I am not sure what else I can try. So if someone can suggest that would be great.

[dmcc]

@anupamme Which errors are you seeing?

[anupamme]

On ubuntu 16.04 (installing using pypi):

x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Ifirst-stage/PARSE/ -I/usr/include/python3.5m -c first-stage/PARSE/swig/wrapper.C -o build/temp.linux-x86_64-3.5/first-stage/PARSE/swig/wrapper.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
first-stage/PARSE/swig/wrapper.C:172:21: fatal error: Python.h: No such file or directory
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

----------------------------------------

Command "/usr/bin/python3.5 -u -c "import setuptools, tokenize;file='/tmp/pip-build-vn848n2m/bllipparser/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-j_za0ero-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-vn848n2m/bllipparser/

On Mac 10.11.5 (installing using pypi):

/usr/bin/clang -fno-strict-aliasing -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.5/include/python3.5m -c second-stage/programs/features/simple-api.cc -o build/temp.macosx-10.6-intel-3.5/second-stage/programs/features/simple-api.o -iquote second-stage/programs/features/ -DSWIGFIX -std=c++11
In file included from second-stage/programs/features/simple-api.cc:24:
In file included from second-stage/programs/features/sp-data.h:33:
In file included from second-stage/programs/features/sptree.h:22:
In file included from second-stage/programs/features/heads.h:36:
In file included from second-stage/programs/features/tree.h:144:
second-stage/programs/features/utility.h:56:10: fatal error: 'forward_list' file not found
#include <forward_list>
^
1 error generated.
error: command '/usr/bin/clang' failed with exit status 1

----------------------------------------

Command "/Library/Frameworks/Python.framework/Versions/3.5/bin/python3.5 -u -c "import setuptools, tokenize;file='/private/tmp/pip-build-m2ez9evu/bllipparser/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-ur28p0hx-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/tmp/pip-build-m2ez9evu/bllipparser/

[dmcc]

For Ubuntu, looks like you're missing Python.h. Try using apt-get to install libpython3.5-dev.

http://stackoverflow.com/questions/16880571/why-does-clang-lack-forward-list might be part of what's going on with Mac OS 10.11.5, but it's not clear to me why this works for others.

[anupamme]

Works on Ubuntu after installing libpython3.5-dev.

On mac I am unsure why this is not working but may be I have beta version of xcode 8 and that is causing it.

But I can live with it working on ubuntu so I am good to go. Thanks for the prompt help.