How dangerous is this exactly? What is the primary bad thing that could happen? (If its going to kill someone, how?) What should I worry about if I am near such a device that is not within my control?
If everything else is wired and working correctly, the ground wire does nothing. That's why you can remove the ground connection, and you don't notice any changes.
Now, the ground connection does a couple of things. First of all, its there to protect you in the event that something goes wrong in a piece of gear. If a hot wire inside a piece of gear gets frayed and touches the chassis, on ungrounded gear, the chassis becomes hot. Touch that and complete a circuit and you can be dead. If the gear is grounded, the result would be a blown breaker and you'd still be alive to complain about that piece of equipment that's broken.
The second thing that the ground connection does is make sure that all devices have the same ground potential. Without a ground connection, even properly wired gear can float the zero point of the voltage. This could result in dangerous voltage potential between pieces of gear. Tying all the gear to the same ground evens this out. Now, this is probably the reason that the ground lift switch is on the box. If you're tying from multiple power sources, they may not all be tied to the same ground point. The result is a ground loop between the devices. Lifting ground from some of the gear breaks the ground loop, but exposes the potential voltage difference. The difference in potential doesn't need to be large to cause problems in audio, so under most cases, it wouldn't kill you, but if the difference is great enough, even properly wired gear can now expose you to electrical shock. The correct solution is to make sure the grounds are identical for all power sources, or break the ground loops in the signal chain versus on the power chain.
The last thing the ground is supposed to protect against is if there is an unexpected surge on the power line. With a lightning strike, the surge is going to be looking for it's easiest path to ground. If the surge gets to a piece of gear with the chassis properly grounded, the surge will then travel down the ground wire and into the earth. If there's no path to ground, that surge is going to look for any path that it can find to get to ground. And surges are persistent little devils. They don't give up easily. If they see you as the shortest path to ground, you can be sure they're going to take that path. You want to make sure to give them every other opportunity to get to ground without involving your body as a path.
