[FIX] Correctly handle empty channel

[Firestone82]

Currently when everyone leaves from channel bot should disconnect. But that works also when bot is moved to different channel. Fix this so it works correctly.