Node-RED App fails on Cloudant IAM only credentials

[johnwalicki]

The latest commit to bluemix-settings.js has unfortunate implications. It switches from cfenv to ibm-cloud-env. It sources /server/config/mappings.json file which wants to load credentials from /server/localdev-config.json

With credentials in a file, we can no longer manage an instance of node-red-app on a public github repo. The credentials would be exposed.

I wanted to build a delivery pipeline that points to a public github repo. Prior versions of node-red-app let me do this.

[johnwalicki]

Or, I've really mangled my VCAP runtime environment....

[johnwalicki]

I give up.

   2020-12-05T00:51:51.62-0500 [APP/PROC/WEB/0] OUT > node-red-app@1.1.1 start /home/vcap/app
   2020-12-05T00:51:51.62-0500 [APP/PROC/WEB/0] OUT > node --max-old-space-size=160 index.js --settings ./bluemix-settings.js -v
   2020-12-05T00:51:51.82-0500 [APP/PROC/WEB/0] OUT 5 Dec 05:51:51 - Starting Node-RED on IBM Cloud bootstrap
   2020-12-05T00:51:51.82-0500 [APP/PROC/WEB/0] OUT 5 Dec 05:51:51 - Loading bluemix-settings.js
   2020-12-05T00:51:51.88-0500 [APP/PROC/WEB/0] OUT 5 Dec 05:51:51 - Using Cloudant service: Cloudant-openeew-93573 db:openeewdevicemgmt prefix:openeewdevicemgmt
   2020-12-05T00:51:52.13-0500 [APP/PROC/WEB/0] OUT 5 Dec 05:51:52 - Loading application settings
   2020-12-05T00:51:52.28-0500 [APP/PROC/WEB/0] OUT Failed to initialize storage module
   2020-12-05T00:51:52.28-0500 [APP/PROC/WEB/0] OUT Failed to create database: Error: one of _admin, server_admin is required for this request
   2020-12-05T00:51:52.31-0500 [APP/PROC/WEB/0] OUT Exit status 0
   2020-12-05T00:51:52.34-0500 [CELL/SSHD/0] OUT Exit status 0

[johnwalicki]

I started over with a new Node-RED application deployed to IBM Cloud Foundry. It failed in exactly the same place.

Failed to create database: Error: one of _admin, server_admin is required for this request

In both attempts, I was trying to bind the Node-RED application to a Cloudant Standard on Transaction Engine instance. It only provides IAM credentials.
I deleted all of my apps again and I deleted the Cloudant Standard on Transaction Engine instance.
I created a new Cloudant Standard instance with both IAM and Legacy credentials. Which is different than the Cloudant Standard on Transaction Engine.
I then created a Node-RED application using the Cloudant Standard instance. It worked.

I think the errors are related to Cloudant Standard on Transaction Enging and/or IAM only credentials.

[youming-lin]

@johnwalicki This repo is not meant to be used outside of the Developer Experience flow because it requires custom service binding/mapping that is done via generation.

To get it working by hand, you'll have manually update the code to load in credentials you have exposed to your runtime environment (Cloud Foundry, Kubernetes, Code Engine, etc.). In the Developer Experience flow, we generate a mappings.json file that, together with the ibm-cloud-env library, will load the credentials automatically based on the service binding we create at the time of toolchain creation. This is also the reason we switched to the ibm-cloud-env; it allows us to expand support for this starter kit beyond just Cloud Foundry.

The generated server/config/mappings.json looks like this:

{
  "cloudant_apikey": {
    "searchPatterns": [
      "cloudfoundry:$['cloudantNoSQLDB'][0].credentials.apikey",
      "file:/server/localdev-config.json:$.cloudant_apikey"
    ]
  },
  "cloudant_host": {
    "searchPatterns": [
      "cloudfoundry:$['cloudantNoSQLDB'][0].credentials.host",
      "file:/server/localdev-config.json:$.cloudant_host"
    ]
  },
  "cloudant_iam_apikey_description": {
    "searchPatterns": [
      "cloudfoundry:$['cloudantNoSQLDB'][0].credentials.iam_apikey_description",
      "file:/server/localdev-config.json:$.cloudant_iam_apikey_description"
    ]
  },
  "cloudant_iam_apikey_name": {
    "searchPatterns": [
      "cloudfoundry:$['cloudantNoSQLDB'][0].credentials.iam_apikey_name",
      "file:/server/localdev-config.json:$.cloudant_iam_apikey_name"
    ]
  },
  "cloudant_iam_role_crn": {
    "searchPatterns": [
      "cloudfoundry:$['cloudantNoSQLDB'][0].credentials.iam_role_crn",
      "file:/server/localdev-config.json:$.cloudant_iam_role_crn"
    ]
  },
  "cloudant_iam_serviceid_crn": {
    "searchPatterns": [
      "cloudfoundry:$['cloudantNoSQLDB'][0].credentials.iam_serviceid_crn",
      "file:/server/localdev-config.json:$.cloudant_iam_serviceid_crn"
    ]
  },
  "cloudant_password": {
    "searchPatterns": [
      "cloudfoundry:$['cloudantNoSQLDB'][0].credentials.password",
      "file:/server/localdev-config.json:$.cloudant_password"
    ]
  },
  "cloudant_port": {
    "searchPatterns": [
      "cloudfoundry:$['cloudantNoSQLDB'][0].credentials.port",
      "file:/server/localdev-config.json:$.cloudant_port"
    ]
  },
  "cloudant_url": {
    "searchPatterns": [
      "cloudfoundry:$['cloudantNoSQLDB'][0].credentials.url",
      "file:/server/localdev-config.json:$.cloudant_url"
    ]
  },
  "cloudant_username": {
    "searchPatterns": [
      "cloudfoundry:$['cloudantNoSQLDB'][0].credentials.username",
      "file:/server/localdev-config.json:$.cloudant_username"
    ]
  },
  "cloudant_serviceInfo": {
    "searchPatterns": [
      "cloudfoundry:$['cloudantNoSQLDB'][0].credentials.serviceInfo",
      "file:/server/localdev-config.json:$.cloudant_serviceInfo"
    ]
  }
}

With this file, the code in this repo will work on Cloud Foundry with the regular CF service binding.

[johnwalicki]

Yes, then this is a regression from prior versions - which allowed me to host the repository on GitHub.

While that is important, there's a separate bug related to IAM credentials, Cloudant Standard on Transaction Engine.

[youming-lin]

Do you have more info about the bug? Is it a bug in the code with this skit, or can you get it working using the mappings.json approach I mentioned above?

[johnwalicki]

I didn't change mappings.json. I just created a Cloudant Standard with Transaction Engine instance. Bound that to the Node-RED app. The deploy works but the cf log shows the error above

   2020-12-05T00:51:51.82-0500 [APP/PROC/WEB/0] OUT 5 Dec 05:51:51 - Starting Node-RED on IBM Cloud bootstrap
   2020-12-05T00:51:51.82-0500 [APP/PROC/WEB/0] OUT 5 Dec 05:51:51 - Loading bluemix-settings.js
   2020-12-05T00:51:51.88-0500 [APP/PROC/WEB/0] OUT 5 Dec 05:51:51 - Using Cloudant service: Cloudant-openeew-93573 db:openeewdevicemgmt prefix:openeewdevicemgmt
   2020-12-05T00:51:52.13-0500 [APP/PROC/WEB/0] OUT 5 Dec 05:51:52 - Loading application settings
   2020-12-05T00:51:52.28-0500 [APP/PROC/WEB/0] OUT Failed to initialize storage module
   2020-12-05T00:51:52.28-0500 [APP/PROC/WEB/0] OUT Failed to create database: Error: one of _admin, server_admin is required for this request
   2020-12-05T00:51:52.31-0500 [APP/PROC/WEB/0] OUT Exit status 0

That error handling console logging is from bluemix-settings.js

[johnwalicki]

I was able to reproduce the error so it shouldn't be too hard to recreate in an IBM Cloud Standard account.

[johnwalicki]

Interesting, if you look at this repo file list, the last commit to cloudantStorage.js is
Revert to cloudant legacy auth
which implies (without me reading / understanding the code) that the Cloudant Standard on Transaction Engine service, which only offers IAM credentials, won't work.

[johnwalicki]

If cloudantStorage.js can be fixed to support IAM credentials instead of legacy Cloudant auth, that would resolve the problem.

[youming-lin]

Is it due to the change in the way the Cloudant object is instantiated?

[johnwalicki]

Follow the commit link above. Scroll down to the cloudantStorage.js diff. It includes a comment and one line of code that is causing the problem. Nick definitely saw the error and wrote the illuminary comment. We need more eyes on how to fix it.

[knolleary]

I seem to remember this was an issue with the default IAM roles cloundant set not being allowed to write to the database, and the pipeline wasn't smart enough to set the roles up properly. But that was a year ago, so can't recall the full details.

Yes the cloudant storage plugin needs updating to use IAM - and maybe with this starter kit it'll be possible to get cloudant setup properly.

[youming-lin]

I did. I think I understand what's going on now.

At first I thought this is a regression based on the original issue description, but the code comment indicates that this was a known issue.

This probably affects our nodejs-cloudant starter kit as well.