add zbus client on the node

[ashraffouda]

we need a zbus client on the node to be able to debug zbus running services

[RawanMostafa08]

Investigation and Solution

• After investigations and discussions, we found that zbusdebug does the job we're looking for, it prints service status if they'r running, and returns an error if they aren't

• So the solution steps now are:

  • add test: zbusdebug --module <module> to Zinit services yaml files for Zinit to run this command periodically until it succeeds (exit code 0), at which point the service is considered running.
  • add missing zinit services yaml files
  • test that this behavior is correct

[RawanMostafa08]

In progress

• Was trying to fix the local zos node not running correctly issue, after discussions it was due to the releases happening in mycelium, mycelium binary was not found on the node
• Added mycelium binary manually using scp to /usr/bin on the node, or added it in zos/qemu/overlay/usr/bin as it's mounted as rootfs (as a temp fix to continue the trials)
• Issue is fixed on main later
• I added the test line to the yaml files again but something was getting stuck and services were not starting, maybe adding the periodic test to some services messed up the dependency between the services
• Tried them one by one to figure out what was causing the node to get stuck, turned out to be storaged is the one causing the problem, still not sure why