How to cancel a bounty?

How to cancel a bounty if I added Billy to watch the issue, but then changed my mind? Or maybe I decided to change the amount of the bounty.
If I delete this issue comment for Billy, my issue still displayed on the dashboard.