The XLR input to you camera may be mic level not line level.
Check in the camera software if you can set the XLR input from "mic" to "line" or if there is a pad available.
The camera he has will accept either mic or line inputs. The switch is a physical one on the camera and is clearly labeled. I feed to this camera often.
The hum is a balanced/unbalanced problem coming from trying to simply hardwire from the RCA outs to the camera. The Ebtech unit I linked in the first response will solve this problem.
Edit for the OP:
The transformer iso (Ebtech) unit is fed from the board using RCA > 1/4" TS cables, then run your long cables to the camera using XLR (mic) cables. This should eliminate the noise you're hearing now.
As I said before, this will likely not yield a sonically balanced program to your camera, neither will it enable you to set the output level from the board to a stable level. Any changes in the masters on the board will affect the RCA outs and you may well need to have separate level control for the camera feed which does NOT follow the main mix fader levels.
It's easy to get a quiet feed to the camera. But to get a decent mix at a usable level is a whole 'nother thing.