There is a wiring detail for the inserts in the User Manual (page 24 in the one I'm looking at), verify that you have the correct wiring. Or describe more details of how it is wired.
For one thing, make sure the +4dBu/-10dBv switches on the rear panel are set correctly. If they are in the -10dBv position they apparently boost the input level and then attenuate the output level which could affect how the compressor operates.
Have you verified gain structure? You should then be able to use the metering on the dbx to look at the input and output levels. Bypassing the 1046 apparently just direct connects the input to output so I'm not sure if the gain settings, etc. are engaged. You might want to set it to a 1:1 ratios, very high threshold and turn off the OverEasy, this should let you see basically just the device gain and levels with no processing.
Keep in mind that compressing does let you increase the overall average level so you may be compressing on one thing and then turning up the overall level thus increasing the frequency content causing the feedback. For example, you may be compressing based on some very dynamic baritones, letting you bring up the overall mix including the alto frequencies that cause feedback.