If I do Census Things, from wherever, the layer turns on. Then I go do Other Things. I'm over the Census Thing. Just want it gone. Many non intuitive clicks later, I've found Census panel. Clear the dropdown. Sigh of relief and frustration.
Ideally it could be turned off (aka cleared) in the same manner as the other polygon layers, via the layers popout menu in the map.