We use an ECAT line transient tester at work (made by Thermo Fisher Scientific). It's more than just a big cap; there's some form of LC pulse-shaping network in there. There's a specific test sequence that we use (if I recall, 3 pulses 30 seconds apart at +/-6kV 200A every multiple of 30 degrees from zero crossing of the AC line). That's for indoor stuff; when we did a device that mounts to street lighting poles, we had to send it out to a site that could do much higher power.
As for failure modes, MOVs tend to fail shorted, and since they are across the AC line, then tend to go bang... So yes, the blast shield is needed :-)