Replace shell API with NodeJS api?

[taylordowns2000]

@stuartc , i'm interested in your thoughts on this (https://github.com/revelrylabs/elixir-nodejs) for use in dispatcher: NodeJS.call(module, args \\ [], opts \\ [])

Options
:name - (optional) The name used for supervisor registration. Defaults to Elixir.NodeJS.Supervisor.
:path - (required) The path to your Node.js code's root directory.
:pool_size - (optional) The number of workers. Defaults to 4.

I wonder if we could save on the CPU and memory usage by having each microservice manage its own pool of NodeJS workers. If core will be available both with and without the CLI, does this make more sense going forward?

[stuartc]

@taylordowns2000 this is a great idea - and something I've considered for a very long time. We won't benefit much in low load scenarios, but when highly parallel I think the difference will be noticeable.

Two things that occupy memory consistently, the node process itself and the VM we start inside the process (and the modules/code it requires in - like adaptors).
However depending on how we do this, say an expression changes - we would need to either replace a cached expression or send the expression in every time (and if the required language packs might change) we would need to re-init the VM every time.

That said, we should reinit the VM every time for security and stability reasons. I'm gonna assume Node will cache the requires and creating VMs with expressions etc will be faster and they will share memory.

One thing we must test really thoroughly is what happens if you modify a language pack inside an expression, does that show in subsequent expressions. Thats the main reason we avoided this early on.

[taylordowns2000]

Brilliant. I'll just hold on to this for now, but when you've got the new pattern down for core, I'll make the change in microservice.

…

On Fri, Oct 30, 2020 at 7:50 AM Stuart Corbishley ***@***.***> wrote: @taylordowns2000 <https://github.com/taylordowns2000> this is a great idea - and something I've considered for a very long time. We won't benefit much in low load scenarios, but when highly parallel I think the difference will be noticeable. Two things that occupy memory consistently, the node process itself and the VM we start inside the process (and the modules/code it requires in - like adaptors). However depending on how we do this, say an expression changes - we would need to either replace a cached expression or send the expression in every time (and if the required language packs might change) we would need to re-init the VM every time. That said, we *should* reinit the VM every time for security and stability reasons. I'm gonna assume Node will cache the requires and creating VMs with expressions etc will be faster and they will share memory. One thing we *must* test really thoroughly is what happens if you modify a language pack inside an expression, does that show in subsequent expressions. Thats the main reason we avoided this early on. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#6 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACCUBLNKHKBBXKK3HP6B2CLSNJV2RANCNFSM4TEUZ4GQ> .