Installation hangs on Mac OS X with Docker 1.38

[scottbw]

Running the launch script hangs at this point:

"Waiting for redis to be up at 172.18.0.2:6379 ... ."

Running docker ps I see the following services have launched:

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
133b354da049 eucm/kzk:0.11.0.1 "supervisord -n" About a minute ago Up About a minute 127.0.0.1:2181->2181/tcp, 127.0.0.1:9092->9092/tcp kzk
c6c564ac130b mongo:3.4 "docker-entrypoint.s…" About a minute ago Up About a minute 27017/tcp mongo
1bc2774f170d redis:4.0.2-alpine "docker-entrypoint.s…" About a minute ago Up About a minute 6379/tcp redis
d8be1b2cbc6d docker.elastic.co/elasticsearch/elasticsearch:5.6.2 "/bin/bash bin/es-do…" About a minute ago Up About a minute 127.0.0.1:9200->9200/tcp, 127.0.0.1:9300->9300/tcp elastic5

(Note that the container host/port that Redis is running on does not respond for redis-cli or netcat either when I try that manually.)

I'm using Docker 18.06.01-ce, compose 1.17.0 on Mac OS X with homebrew.

[RotaruDan]

Hello,

Could you please run ./rage-analytics.sh report (before stopping the services)

and

append the report in this issue so we can have a deeper view of the problem.

Thanks.

[scottbw]

Here's the report:
report-.txt

[RotaruDan]

Hello,

We are unable of viewing the logs of supervisor, nimbus and a2 from the report file.

Could you execute:

1. "./rage-analytics.sh logs nimbus" and copy the console output on a comment here.
2. "./rage-analytics.sh logs supervisor" and copy the console output on a comment here.
3. "./rage-analytics.sh logs a2" and copy the console output on a comment here.

Thanks.

[scottbw]

I think the main issue I'm facing here is the script relies on the "netcat hack" to check for service availability, which won't work on a Mac or any *nix system that uses a non-bridged network. But I'll happily get the logs for you!

[scottbw]

Attaching to nimbus
nimbus | --- dumping /app/apache-storm-1.1.1/conf/storm.yaml ---
nimbus | # created at 2018-10-16 10:23:20+00:00
nimbus | ui.port: 8081
nimbus | storm.zookeeper.servers:
nimbus | - kzk
nimbus | storm.local.dir: /app
nimbus | nimbus.seeds: [172.18.0.3 ]
nimbus | --- /app/apache-storm-1.1.1/conf/storm.yaml end ---
nimbus | Running: java -server -Ddaemon.name=nimbus -Dstorm.options= -Dstorm.home=/app/apache-storm-1.1.1 -Dstorm.log.dir=/app/apache-storm-1.1.1/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /app/apache-storm-1.1.1/lib/servlet-api-2.5.jar:/app/apache-storm-1.1.1/lib/objenesis-2.1.jar:/app/apache-storm-1.1.1/lib/minlog-1.3.0.jar:/app/apache-storm-1.1.1/lib/reflectasm-1.10.1.jar:/app/apache-storm-1.1.1/lib/log4j-core-2.8.2.jar:/app/apache-storm-1.1.1/lib/log4j-slf4j-impl-2.8.2.jar:/app/apache-storm-1.1.1/lib/storm-core-1.1.1.jar:/app/apache-storm-1.1.1/lib/kryo-3.0.3.jar:/app/apache-storm-1.1.1/lib/storm-rename-hack-1.1.1.jar:/app/apache-storm-1.1.1/lib/asm-5.0.3.jar:/app/apache-storm-1.1.1/lib/slf4j-api-1.7.21.jar:/app/apache-storm-1.1.1/lib/clojure-1.7.0.jar:/app/apache-storm-1.1.1/lib/log4j-over-slf4j-1.6.6.jar:/app/apache-storm-1.1.1/lib/ring-cors-0.1.5.jar:/app/apache-storm-1.1.1/lib/log4j-api-2.8.2.jar:/app/apache-storm-1.1.1/lib/disruptor-3.3.2.jar:/app/apache-storm-1.1.1/conf -Xmx1024m -Dlogfile.name=nimbus.log -DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector -Dlog4j.configurationFile=/app/apache-storm-1.1.1/log4j2/cluster.xml org.apache.storm.daemon.nimbus
nimbus exited with code 20

Attaching to supervisor
supervisor | --- dumping /app/apache-storm-1.1.1/conf/storm.yaml ---
supervisor | # created at 2018-10-16 10:25:25+00:00
supervisor | ui.port: 8081
supervisor | storm.zookeeper.servers:
supervisor | - kzk
supervisor | storm.local.dir: /app
supervisor | nimbus.seeds: [nimbus]
supervisor | --- /app/apache-storm-1.1.1/conf/storm.yaml end ---
supervisor | Running: java -server -Ddaemon.name=supervisor -Dstorm.options= -Dstorm.home=/app/apache-storm-1.1.1 -Dstorm.log.dir=/app/apache-storm-1.1.1/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /app/apache-storm-1.1.1/lib/servlet-api-2.5.jar:/app/apache-storm-1.1.1/lib/objenesis-2.1.jar:/app/apache-storm-1.1.1/lib/minlog-1.3.0.jar:/app/apache-storm-1.1.1/lib/reflectasm-1.10.1.jar:/app/apache-storm-1.1.1/lib/log4j-core-2.8.2.jar:/app/apache-storm-1.1.1/lib/log4j-slf4j-impl-2.8.2.jar:/app/apache-storm-1.1.1/lib/storm-core-1.1.1.jar:/app/apache-storm-1.1.1/lib/kryo-3.0.3.jar:/app/apache-storm-1.1.1/lib/storm-rename-hack-1.1.1.jar:/app/apache-storm-1.1.1/lib/asm-5.0.3.jar:/app/apache-storm-1.1.1/lib/slf4j-api-1.7.21.jar:/app/apache-storm-1.1.1/lib/clojure-1.7.0.jar:/app/apache-storm-1.1.1/lib/log4j-over-slf4j-1.6.6.jar:/app/apache-storm-1.1.1/lib/ring-cors-0.1.5.jar:/app/apache-storm-1.1.1/lib/log4j-api-2.8.2.jar:/app/apache-storm-1.1.1/lib/disruptor-3.3.2.jar:/app/apache-storm-1.1.1/conf -Xmx256m -Dlogfile.name=supervisor.log -DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector -Dlog4j.configurationFile=/app/apache-storm-1.1.1/log4j2/cluster.xml org.apache.storm.daemon.supervisor.Supervisor
supervisor exited with code 20

Attaching to a2
a2 | npm info it worked if it ends with ok
a2 | npm info using npm@5.3.0
a2 | npm info using node@v8.5.0
a2 | npm info lifecycle Authentication-Authorization-Module@0.0.2predocker-start: Authentication-Authorization-Module@0.0.2
a2 | npm info lifecycle Authentication-Authorization-Module@0.0.2docker-start: Authentication-Authorization-Module@0.0.2
a2 |
a2 | > Authentication-Authorization-Module@0.0.2 docker-start /app
a2 | > npm run fast-setup && npm run register-root && npm start
a2 |
a2 | npm info it worked if it ends with ok
a2 | npm info using npm@5.3.0
a2 | npm info using node@v8.5.0
a2 | npm info lifecycle Authentication-Authorization-Module@0.0.2prefast-setup: Authentication-Authorization-Module@0.0.2
a2 | npm info lifecycle Authentication-Authorization-Module@0.0.2fast-setup: Authentication-Authorization-Module@0.0.2
a2 |
a2 | > Authentication-Authorization-Module@0.0.2 fast-setup /app
a2 | > SETUP_MODE=fast node ./setup.js
a2 |
a2 | Setup complete.
a2 | npm info lifecycle Authentication-Authorization-Module@0.0.2postfast-setup: Authentication-Authorization-Module@0.0.2
a2 | npm info ok
a2 | npm info it worked if it ends with ok
a2 | npm info using npm@5.3.0
a2 | npm info using node@v8.5.0
a2 | npm info lifecycle Authentication-Authorization-Module@0.0.2preregister-root: Authentication-Authorization-Module@0.0.2
a2 | npm info lifecycle Authentication-Authorization-Module@0.0.2register-root: Authentication-Authorization-Module@0.0.2
a2 |
a2 | > Authentication-Authorization-Module@0.0.2 register-root /app
a2 | > SETUP_MODE=register-root node ./setup.js
a2 |
a2 | (node:66) DeprecationWarning: Mongoose: mpromise (mongoose's default promise library) is deprecated, plug in your own promise library instead: http://mongoosejs.com/docs/promises.html
a2 | Root user registered successfully.
a2 | npm info lifecycle Authentication-Authorization-Module@0.0.2postregister-root: Authentication-Authorization-Module@0.0.2
a2 | npm info ok
a2 | npm info it worked if it ends with ok
a2 | npm info using npm@5.3.0
a2 | npm info using node@v8.5.0
a2 | npm info lifecycle Authentication-Authorization-Module@0.0.2prestart: Authentication-Authorization-Module@0.0.2
a2 | npm info lifecycle Authentication-Authorization-Module@0.0.2start: Authentication-Authorization-Module@0.0.2
a2 |
a2 | > Authentication-Authorization-Module@0.0.2 start /app
a2 | > node ./bin/www
a2 |
a2 | Skipping login plugin _lti.js
a2 | Skipping login plugin _saml.js
a2 | Mongo db Connected!
a2 | Mongoose: users.ensureIndex({ email: 1 }, { unique: true, background: true })
a2 | Mongoose: applications.ensureIndex({ prefix: 1 }, { unique: true, background: true })
a2 | Mongoose: users.ensureIndex({ username: 1 }, { unique: true, background: true })
a2 | Mongoose: applications.ensureIndex({ host: 1 }, { unique: true, background: true })
a2 | Mongoose: users.ensureIndex({ username: 1 }, { unique: true, background: true })
a2 | Mongoose: users.ensureIndex({ email: 1 }, { unique: true, background: true })
a2 | Mongoose: applications.ensureIndex({ prefix: 1 }, { unique: true, background: true })
a2 | GET / 200 365.357 ms - 3458
a2 | GET /libs/ngStorage.min.js 200 9.267 ms - 2431
a2 | GET /controllers/controllers.js 200 8.978 ms - 35107
a2 | GET /libs/jquery.twbsPagination.min.js 200 10.203 ms - 6218
a2 | GET /controllers/app.js 200 9.907 ms - 991
a2 | GET /rageicon.png 200 9.682 ms - 2113
a2 | GET /bower/ng-file-upload/ng-file-upload.js 200 10.820 ms - 81683
a2 | GET /rageicon.png 304 0.765 ms - -
a2 | events.js:182
a2 | throw er; // Unhandled 'error' event
a2 | ^
a2 |
a2 | Error: Redis connection to redis:6379 failed - getaddrinfo ENOTFOUND redis redis:6379
a2 | at errnoException (dns.js:53:10)
a2 | at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:95:26)
a2 | npm info lifecycle Authentication-Authorization-Module@0.0.2start: Failed to exec start script
a2 | npm ERR! code ELIFECYCLE
a2 | npm ERR! errno 1
a2 | npm ERR! Authentication-Authorization-Module@0.0.2 start: node ./bin/www
a2 | npm ERR! Exit status 1
a2 | npm ERR!
a2 | npm ERR! Failed at the Authentication-Authorization-Module@0.0.2 start script.
a2 | npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
a2 |
a2 | npm ERR! A complete log of this run can be found in:
a2 | npm ERR! /app/.npm/_logs/2018-10-16T10_34_05_204Z-debug.log
a2 | npm info lifecycle Authentication-Authorization-Module@0.0.2docker-start: Failed to exec docker-start script
a2 | npm ERR! code ELIFECYCLE
a2 | npm ERR! errno 1
a2 | npm ERR! Authentication-Authorization-Module@0.0.2 docker-start: npm run fast-setup && npm run register-root && npm start
a2 | npm ERR! Exit status 1
a2 | npm ERR!
a2 | npm ERR! Failed at the Authentication-Authorization-Module@0.0.2 docker-start script.
a2 | npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
a2 |
a2 | npm ERR! A complete log of this run can be found in:
a2 | npm ERR! /app/.npm/_logs/2018-10-16T10_34_05_215Z-debug.log
a2 exited with code 1

[RotaruDan]

We think that this issue might be related with supervisor and nimbus being launched under MAC systems. Also related to the "netcat hack" to check for service availability,

Unfortunately, after analysing the logs, there is no clear error message.

We are going to try to find a solution, and strongly recommend you to use a VirtualMachine with Ubuntu 18.04 for example since we don't have MAC machines to replicate the issue nor we know if we will be able to find a solution in the near future.

[scottbw]

Well, I set up an Ubuntu VM on Azure as a backup. That fails with a different message at least :D

I'm not going to try this again until Friday now - if there's any progress do update me!