When the lightning instance that was executing an oban job is killed abruptly, say OOMKill, the job will get stuck in executing state.
This issue allows lightning to recover these stuck jobs. For now, we only know of data_retention jobs.
Oban has a plugin for doing this exactly, https://hexdocs.pm/oban/Oban.Plugins.Lifeline.html, but it has a bug disclaimer:
This plugin may transition jobs that are genuinely executing and cause duplicate execution. For more accurate rescuing or to rescue jobs that have exhausted retry attempts see the DynamicLifeline plugin in Oban Pro
https://hexdocs.pm/oban/ready_for_production.html#rescuing-jobs
When the lightning instance that was executing an oban job is killed abruptly, say
OOMKill, the job will get stuck inexecutingstate.This issue allows lightning to recover these stuck jobs. For now, we only know of
data_retentionjobs.Oban has a plugin for doing this exactly, https://hexdocs.pm/oban/Oban.Plugins.Lifeline.html, but it has a bug disclaimer:
https://hexdocs.pm/oban/ready_for_production.html#rescuing-jobs