Use a version of mongodb-queue with built in ttl support for messages #22
Conversation
david-martin
force-pushed
the
add-mongodb-queue-with-ttl
branch
from
0dfae15 to
10a7068
Compare
lib/mongodbQueue.js
Outdated
| if (!existingIndex) { | ||
| needCreate = true; | ||
| return callback(null, false); | ||
| } else if (existingIndex.expireAfterSeconds !== self.queueTTL) { |
There was a problem hiding this comment.
Does self.queueTTL exist anymore?
There was a problem hiding this comment.
No. Good spot. this might be related to the failing tests.
This should probably be this.queueOptions.ttl
This updates the version fo the fh-sync module to include a fix for the ttl of queue messages. See feedhenry/fh-sync#22 for more info. The dependencies in package.json have also been updated to remove unused dependencies. These were discovered when looking at where the mongodb-queue module was being required. It was originally required in fh-mbaas-api, but was moved out with the sync implementation to the fh-sync module. The majority of the unused modules were from fh-sync, with a few extra ones discovered using this command: ``` dependency-check ./package.json --no-dev --unused ``` The npm-shrinkwrap.json file was regenerated as a result
david-martin
force-pushed
the
add-mongodb-queue-with-ttl
branch
from
10a7068 to
bdd6660
Compare
This updates the version fo the fh-sync module to include a fix for the ttl of queue messages. See feedhenry/fh-sync#22 for more info. The dependencies in package.json have also been updated to remove unused dependencies. These were discovered when looking at where the mongodb-queue module was being required. It was originally required in fh-mbaas-api, but was moved out with the sync implementation to the fh-sync module. The majority of the unused modules were from fh-sync, with a few extra ones discovered using this command: ``` dependency-check ./package.json --no-dev --unused ``` The npm-shrinkwrap.json file was regenerated as a result
david-martin
force-pushed
the
add-mongodb-queue-with-ttl
branch
from
bdd6660 to
0dd2f24
Compare
This updates the version fo the fh-sync module to include a fix for the ttl of queue messages. See feedhenry/fh-sync#22 for more info. The dependencies in package.json have also been updated to remove unused dependencies. These were discovered when looking at where the mongodb-queue module was being required. It was originally required in fh-mbaas-api, but was moved out with the sync implementation to the fh-sync module. The majority of the unused modules were from fh-sync, with a few extra ones discovered using this command: ``` dependency-check ./package.json --no-dev --unused ``` The npm-shrinkwrap.json file was regenerated as a result
david-martin
force-pushed
the
add-mongodb-queue-with-ttl
branch
from
0dd2f24 to
76ef90b
Compare
This updates the version fo the fh-sync module to include a fix for the ttl of queue messages. See feedhenry/fh-sync#22 for more info. The dependencies in package.json have also been updated to remove unused dependencies. These were discovered when looking at where the mongodb-queue module was being required. It was originally required in fh-mbaas-api, but was moved out with the sync implementation to the fh-sync module. The majority of the unused modules were from fh-sync, with a few extra ones discovered using this command: ``` dependency-check ./package.json --no-dev --unused ``` The npm-shrinkwrap.json file was regenerated as a result
aidenkeating
left a comment
aidenkeating
left a comment
There was a problem hiding this comment.
LGTM 👍 Don't know if we want to wait until the mongodb-queue change is released before merging this and the fh-mbaas-api changes?
|
@aidenkeating I'm going to merge this now and revisit to reference the upstream version if/when the change is merged. |
This updates the version fo the fh-sync module to include a fix for the ttl of queue messages. See feedhenry/fh-sync#22 for more info. The dependencies in package.json have also been updated to remove unused dependencies. These were discovered when looking at where the mongodb-queue module was being required. It was originally required in fh-mbaas-api, but was moved out with the sync implementation to the fh-sync module. The majority of the unused modules were from fh-sync, with a few extra ones discovered using this command: ``` dependency-check ./package.json --no-dev --unused ``` The npm-shrinkwrap.json file was regenerated as a result
|
@david-martin It looks like this change broke community version. |
|
@wtrocki Apologies for breaking something with this change. I wonder if existing indexes or records causes the problem when migrating? Could you get a reproducable test case or example? |
|
@david-martin Things like that happening all the time so no worries, but I really stuck on debugging this so just wanted to point it out here. Code is breaking for Raincatcher but basically it will be possible to see the same problem when running example application in this repository. I'm going to use unreleased version for the moment and we can have chat about how this can be fixed without removing this change. |
|
What I noticed is that initially sync works for 2-3 requests but then entire processing stops. |
|
@david-martin @wtrocki I think this is because of this section here Line 89 in 76ef90b |
|
Mentioned PR: #29 |
@wei-lee @aidenkeating Would you mind reviewing this change?
The mongodb-queue dependency has been updated to include a change for ttl support chilts/mongodb-queue#22