100mb interfaces can become saturated in a Dante network if there is multicast traffic and no IGMP snooping. This is probably why you saw the problem worsen when you tried multicasting everything.
In your situation my guess is either the PLMs are being saturated or the multicast clock data is not getting through to them. Best to wait for the controller screenshots / log.
In the meantime are you running anything AES67 or do you have internet connection sharing on the machine running DVS?
I did not realize that there was a newer version of DVS than the one I was using. I updated that and eliminated all of the unnecessary multicasting and it is working fine now. FWIW, the frequency offset for the PLMs was bouncing around all over the place, from ~+175 to ~-150 or so. Everything else on the network was pretty steady, occasionally fluctuating a point or two. Once I removed the multicasting and updated DVS, the PLMs are holding at a steady ~26ppm offset.
Thanks for the tip on snooping, it's something I hadn't looked into. In the future if I need to set up multicasting I will employ IGMP snooping.