ProSoundWeb Community

Sound Reinforcement - Forums for Live Sound Professionals - Your Displayed Name Must Be Your Real Full Name To Post In The Live Sound Forums => AC Power and Grounding => Topic started by: Thomas Lamb on May 11, 2014, 12:23:19 PM

Title: Power problem?
Post by: Thomas Lamb on May 11, 2014, 12:23:19 PM
So I'm pretty sure we have a power issue. However, I can't track it down so I solicit the help of those much wiser than I.

Here's the deal. We have four almost identical systems from gear to power distribution. They are all portable (it's portable churches) so they are all at different locations. This one is at a movie theatre. Here's the layout. Each location has 3 phase 120/208 power and we have had 30 amp breakers installed with L21/30 connections. It runs about 100' on 8/5soow to a stinger BREAKERED breakout (L21/30 in and thru) that feeds lights off of 2 legs and FOH off the other (GLD and ETC smartfade). From there it feeds through the breakout downstage to the rack with a racpac in it
L21/30 in. That racpac feeds a NEXO NX4x4, IEM's, wireless, and computers.

Last week I got a emergency call that the amp had an error (error PS CH 2 2 V) pulled the amp took it back to the shop (thought then maybe we had a issue on a leg but it metered good) put it on the bench same error. Power supply? Probably so sent it out for repair and sent out a brand new one (BRAND NEW!) got a call this morning they kicked the amp on and it immediately let the smoke out! Not happy. Went out metered everything each leg 119-121 to neutral and to ground 208 phase to phase 0v ground to neutral. 3 weeks ago they lost the power supply to the lightng console it runs off the same leg as the audio console no problem with it though. Back around Christmas we lost a keyboard a few months before that we lost the power supply to the wireless distro. Every time something happens we always go back meter everything and nothing stands out. So... Where do I go from here? FYI there are projectors that the theatre owns running out of the same panel.

HELP!!

PS.
Never had a issue like this at any other campus (same systems).
Title: Re: Power problem?
Post by: Brian Jojade on May 11, 2014, 01:18:51 PM
My first guess would be that you have a loose neutral connection somewhere.  When you're metering with no load, it looks just dandy fine.  However, when you put a load on it, the connection somewhere starts to arc and disconnect.  When this happens, if you've got an unbalanced load on the 3 phase system, you're going to see voltage swings, with the lowest load devices getting the highest voltage across them.

Title: Re: Power problem?
Post by: TJ (Tom) Cornish on May 11, 2014, 01:23:31 PM
My first guess would be that you have a loose neutral connection somewhere.  When you're metering with no load, it looks just dandy fine.  However, when you put a load on it, the connection somewhere starts to arc and disconnect.  When this happens, if you've got an unbalanced load on the 3 phase system, you're going to see voltage swings, with the lowest load devices getting the highest voltage across them.
yep. Get some large par lamps or heaters and see what the voltage does under load.
Title: Re: Power problem?
Post by: Raul Suarez on May 11, 2014, 04:47:03 PM
Is it possible that the lights on just 2 legs are making the load unbalanced, and drawing down the neutral, thus causing the 3rd leg with a lighter load to go very high?  You don't say what the draw of your lighting system is.

Raul Suarez
Third Ear Sound
Title: Re: Power problem?
Post by: Matthew Knischewsky on May 11, 2014, 05:17:59 PM
I agree with the others. Load down the power with heaters or par cans or whatever you have that can be used as a constant load - I've used heat guns, coffee makers... and then meter the power and see what's happening. If you load down one leg the other legs should remain close to the same (nominally 120V). The danger of a loose neutral is that it turns the power distribution system into a series circuit instead of a parallel circuit and produces the kinds of failures you're seeing.

Having the lights on 2 circuits shouldn't cause problems if everything is working normally. Best practice would be to load all phases as evenly as possible but it's not a big deal on a 30 amp circuit.

I'd open up all your cables and distribution boxes and tighten the connections as a matter of course. At the same time inspect all of the connections and make sure the wire is being clamped down on and not the insulation, and a general inspection that all is as it should be. Standard disclaimer, always do this work un-energized.

Matt

Title: Re: Power problem?
Post by: Mike Sokol on May 11, 2014, 06:49:09 PM
Yup, sounds like a loose or undersized neutral somewhere. I once did a gig at an old mill where my 50 KW of PAR lights (25 KW per leg) could cause the neutral to go high by 10 volts or more on the opposite leg. So if we bumped all the lights on one phase, anything on the other unloaded leg would see over 130+ volts. Many older buildings with 3-phase power regularly undersized the neutral feeding the service panel since 3-phase motor loads never draw any neutral current. But throw some power amps and lights on that same 3-phase circuit, and you'll see the neutral dragging up and down which could be reflecting into the other legs.

This probably has nothing to do with your own distro wiring, but is a PoCo and building problem. However, don't expect them to find this out for you. 

So as others have suggested, meter an unloaded leg of your distro then throw as much steady-state load as you can get on another leg. If its voltage jumps up by more than a few volts, then you've got a problem with the neutral being undersized or loose. Also be aware that if you're using tungsten lights that they draw a huge current spike when cold, typically several times (or more) of the steady state amperage draw. So if you're doing flashing tungsten lighting on one leg, then you could be dragging the neutral around on that leg by dozens of volts, and possibly spiking the H-N voltage on the other two legs by 10 or 20 volts or even more. That could be what's killing your gear.

If you could set up your lighting and get a digital storage scope on one of the unloaded legs, I'll bet you could figure this out rather quickly. 
Title: Re: Power problem?
Post by: Thomas Lamb on May 11, 2014, 07:48:47 PM
Guys thank for the help so far. The lighting load is moderate 2 source4 eliosodals with 750 watt bulbs in them per leg (x2).

So are we saying the problem may not be in my gear? I'm pulling that rig and will go through all the power distros this week. I can definately load test the phases independently and together. We have brought in portable power before and are considering doing it again until we get this figured out.

Thanks again guys.
Title: Re: Power problem?
Post by: Mike Sokol on May 11, 2014, 08:13:51 PM
Guys thank for the help so far. The lighting load is moderate 2 source4 eliosodals with 750 watt bulbs in them per leg (x2).

So are we saying the problem may not be in my gear? I'm pulling that rig and will go through all the power distros this week. I can definately load test the phases independently and together. We have brought in portable power before and are considering doing it again until we get this figured out.

Thanks again guys.

I could certainly be in the PoCo feed between the power transformers on the street to your service panel. But it would be prudent to go through all your power distros and check for loose screws or anything goofy on the neutral connections.

Also, others on this forum can speak to this better than me, but you should get a look at the 3-phase transformers feeding your building. If there's not 3 transformers identical in size, then there could some variation of standard 3-phase Wye power. Also, get an electrician to open up your service panel and see how big the neutral is sized. If it's smaller than the three hot legs, that could be an issue with your single-phase loads. Don't do this yourself unless you're a qualified electrician or engineer as that's a good way to get yourself killed.

Just how old is this building and the power feeding it? That will give us a hint as where else to look.
Title: Re: Power problem?
Post by: Thomas Lamb on May 11, 2014, 10:11:48 PM
Biulding is maybe 6-8 yrs old. It is 120/208v wye. 16 theaters all digital projection. We run out of the same panel as some of the projectors each projector I presume are 208. However they have racks of QSC CX series amps 120v. Each projector has it's own sub panel mounted in the base of the projector stand.
Title: Re: Power problem?
Post by: Matthew Knischewsky on May 11, 2014, 11:07:29 PM
Biulding is maybe 6-8 yrs old. It is 120/208v wye. 16 theaters all digital projection. We run out of the same panel as some of the projectors each projector I presume are 208. However they have racks of QSC CX series amps 120v. Each projector has it's own sub panel mounted in the base of the projector stand.

Did you ask anyone at the theatre if they're been experiencing any unexplained failures of equipment in the projection booth?
Title: Re: Power problem?
Post by: Thomas Lamb on May 12, 2014, 10:24:58 AM
Did you ask anyone at the theatre if they're been experiencing any unexplained failures of equipment in the projection booth?
That's the question of the day as well as have they had any air conditioning work done or any othe service calls. Ice machines,etc.....
Title: Re: Power problem?
Post by: Mike Sokol on May 12, 2014, 11:05:02 AM
Biulding is maybe 6-8 yrs old. It is 120/208v wye. 16 theaters all digital projection. We run out of the same panel as some of the projectors each projector I presume are 208. However they have racks of QSC CX series amps 120v. Each projector has it's own sub panel mounted in the base of the projector stand.

How many amps are each service panel? 100, 200, ???

I can't imagine that any projector is going to be 3-phase/208, but I've never seen one up close and personal. Anything's possible for 3-D theaters. So does each theater have it's own HVAC powered by that particular service panel?
Title: Re: Power problem?
Post by: Tim McCulloch on May 12, 2014, 11:58:11 AM
How many amps are each service panel? 100, 200, ???

I can't imagine that any projector is going to be 3-phase/208, but I've never seen one up close and personal. Anything's possible for 3-D theaters. So does each theater have it's own HVAC powered by that particular service panel?

It's been a couple of years since I've been in a projection booth, but I recall big Christie projectors and a sub-panel on the wall for each projection station.  Each booth in the complex had its own main panel that handled curtain motors, curtain warmers, auditorium lighting, sound system racks and feeds for projectors.  This was a recent (10 years or newer) build.  I believe the service was 120/208 3 phase wye to each main panel, with  primary distribution of 480v 3 phase delta in the building.

I don't recall auditorium HVAC powered from the panels in the projection booths, but it's possible that the smaller booths had local air handlers; the big booth that served 8 auditoriums had its own HVAC sub system that was independent of the auditorium air.
Title: Re: Power problem?
Post by: Greg_Cameron on May 12, 2014, 03:19:52 PM
I can't imagine that any projector is going to be 3-phase/208, but I've never seen one up close and personal.

We've got NEC, Christie, and Barco cinema projectors at my day job. They all use use 30A 3∅ wye PSUs, though at lease one has an option for single phase 240.
Title: Re: Power problem?
Post by: Mike Sokol on May 12, 2014, 09:00:14 PM
We've got NEC, Christie, and Barco cinema projectors at my day job. They all use use 30A 3∅ wye PSUs, though at lease one has an option for single phase 240.
Wow, that's a lotta juice. Is the power supply itself 3-phase, or are the phases split up for fan, bulb, whatever?
Title: Re: Power problem?
Post by: Greg_Cameron on May 13, 2014, 01:32:51 AM
Wow, that's a lotta juice. Is the power supply itself 3-phase, or are the phases split up for fan, bulb, whatever?

I haven't torn one down to see. We've got one parked in the hallway near my cubicle. I'll take a look tomorrow to see If I can figure it out.
Title: Re: Power problem?
Post by: Thomas Lamb on May 15, 2014, 08:00:11 AM
Well I went through all of my PDs last night and everything is fine. Tested it with the shop power source no deviation in voltage. Pulled all the PDs and cables apart and everything was good. So I'm getting into the theatre today hoping for a resolution.
Title: Re: Power problem?
Post by: Thomas Lamb on May 18, 2014, 10:32:16 AM
Well I went through all of my PDs last night and everything is fine. Tested it with the shop power source no deviation in voltage. Pulled all the PDs and cables apart and everything was good. So I'm getting into the theatre today hoping for a resolution.

Ok so when we tested it back at the office when I would flip on the breakers that had the pars hooked up the voltage would jump 2 10ths at the most. I was at the theatre this morning and when untested it there it jumped 3 maybe 4 10ths. So what do I do know? I'm kinda at a loss. I'm thinking ok maybe I did have a bad amp out of the box for the second one but.... It just seems weird. If it was a craptastic amp vs a voltage problem can the repair shop tell that? What do I do know? We are running off a generator this week. Don't really want to do that forever.
Title: Re: Power problem?
Post by: Mike Sokol on May 18, 2014, 11:27:34 AM
Ok so when we tested it back at the office when I would flip on the breakers that had the pars hooked up the voltage would jump 2 10ths at the most. I was at the theatre this morning and when untested it there it jumped 3 maybe 4 10ths. So what do I do know? I'm kinda at a loss. I'm thinking ok maybe I did have a bad amp out of the box for the second one but.... It just seems weird. If it was a craptastic amp vs a voltage problem can the repair shop tell that? What do I do know? We are running off a generator this week. Don't really want to do that forever.

So it sounds like your neutrals are stable, which is to be expected with a modern installation. I don't think that any repair shop is going to be able to determine the cause of the amp failure beyond a WAG, so there's more to think about. Let's boil this down to the basics.

There's really three principal failure mechanisms as I see it.

#1) There's some voltage spike incident that's trashing the power amps.
#2) The amplifiers have a design or build flaw.
#3) You're loading the outputs of the amplifier with too low impedance.

You can rule out #1 by actual testing or historical review. Historically, if other pieces of gear on this power feed have experienced recent failures, then that's the smoking gun. However, perhaps the other gear is more tolerant of voltage spikes (but I doubt it). See if you can rent a power quality monitor for a week or so. Fluke makes some nice ones that will report all voltage over and under conditions as well as log any spike incidents. Something like http://www.fluke.com/Fluke/r0en/Electrical-Test-Tools/Power-Quality-Tools/Fluke-43B-Series.htm?PID=56081 (http://www.fluke.com/Fluke/r0en/Electrical-Test-Tools/Power-Quality-Tools/Fluke-43B-Series.htm?PID=56081) should do the trick.

You can rule out #2 by contacting the amplifier manufacturer or their user forum. Some amplifiers can be a bit sensitive to silly things. For instance, IIRC there was a pro amp design 20 years ago (Carver?) that wouldn't tolerate speakers being connected while the amp was powered up and passing signal. That would trash the output transistors a high percentage of the time.     

You can rule out #3 by double checking all speaker connections and confirming you're not driving too low of a load. And sometimes the load isn't obvious.  I was at a church in Texas a few years back that had a rack full of Crown amps, some of which had blown output channels. Everything appeared to be wired properly and none of the amps was driving under 4 ohms. However, I found one NL4 cable that was improperly wired and shorting between the 1+ and 2+ terminals on one end. This was a bi-amped monitor system so this short in the cable produced a direct connection between the high and low freq outputs of the amplifier. Now how did this problem travel across multiple amps in the rack? Well this cable was used to connect floor monitor speakers every Sunday, so it was wrapped after every gig and hung on the wall during the week. The following Sunday it would be used to connect another floor monitor randomly, thus it eventually found it's way on a bunch of different amplifiers and it would burn out an amp output with any sustained music.

Most apparently random events are not truly random if you can locate what's really changing. 
Title: Re: Power problem?
Post by: Stephen Swaffer on May 18, 2014, 06:06:34 PM
If running on a genny fixes the problem, it would appear that both 2 and 3 are ruled out?  The only difference is the power supply at that point.

Depending on when "this morning" and what else was happening it could be an issue in some other part of the building.  For example, I know that at at our church on a summer Sunday morning, the AC will be running hard for several hours.  If you tested when nothing much was going on, there still might be an issue with harmonics on a neutral once all lighting in other areas is powered up.

Three power supplies in 6 months is certainly suspicious-but consider how many hours you have run without a problem, and how much time you have actually spent with a meter connected.  How likely are you to actually catch an anomaly "in the act"? 
Title: Re: Power problem?
Post by: Mike Sokol on May 19, 2014, 12:58:57 AM
Three power supplies in 6 months is certainly suspicious-but consider how many hours you have run without a problem, and how much time you have actually spent with a meter connected.  How likely are you to actually catch an anomaly "in the act"?

That's exactly why I suggested some sort of data logger. Even if there was a voltage spike happening several times a day, you might never see it if you blink at the wrong time. And one other thing comes to mind. I know that big water pump motors can create a big emf spike when the contactor opens up, but what about HVAC compressors? Since there's no zero-crossover switch like you get with a Triac switch, there's got to be a spike which will vary in size depending on what part of the sine wave it occurs. So perhaps this is the time to invest in some SurgeX voltage protection. 
Title: Re: Power problem?
Post by: Jonathan Johnson on May 19, 2014, 04:21:14 PM
That's exactly why I suggested some sort of data logger. Even if there was a voltage spike happening several times a day, you might never see it if you blink at the wrong time. And one other thing comes to mind. I know that big water pump motors can create a big emf spike when the contactor opens up, but what about HVAC compressors? Since there's no zero-crossover switch like you get with a Triac switch, there's got to be a spike which will vary in size depending on what part of the sine wave it occurs. So perhaps this is the time to invest in some SurgeX voltage protection.

I had a customer whose network would completely lock up for about a minute at fairly regular intervals. We determined that there was a power quality issue, but were never able to find the root cause. There was, however, a large welding/fabricating shop a couple of doors down that could have had something to do with it. We ended up putting in an online UPS on the network gear to overcome the issue.

I'm not proposing a solution, but simply trying to point out that power events can be sporadic, difficult to identify, and sometimes impossible to find the root cause. If you can determine that a voltage sag or spike is occurring on the incoming power feed, you'll probably need to get the power company involved. They do have a financial interest in delivering clean power, as in some states they can be held liable for equipment damage caused by events on their distribution system.
Title: Re: Power problem?
Post by: Stephen Swaffer on May 19, 2014, 10:47:30 PM
POCOs usually also have the ability to require their customers to comply with certain conditions (ie soft starts/VFDs on  large motors, etc) to keep from interfering with other customers.  If there is a possibility that incoming power is an issue, I have seen them supply and install data loggers (usually a lot better than you can afford)in an attempt to find the issue-just be diplomatic-you want them to want to help you!