ProSoundWeb Community

Sound Reinforcement - Forums for Live Sound Professionals - Your Displayed Name Must Be Your Real Full Name To Post In The Live Sound Forums => LAB: The Classic Live Audio Board => Topic started by: Andrew Broughton on April 16, 2017, 03:28:54 PM

Title: Help troubleshooting new DVS recording issues
Post by: Andrew Broughton on April 16, 2017, 03:28:54 PM
So, for years before this, I've had no real problems recording multi-track (32 tracks) of 16 or 24 bit/48k audio using DVS on my Lenovo ThinkPad from RIOs

This year, I cannot get a clean recording. I know there's been Dante and Firmware updates between this year and last, but I don't know if they are the cause.
I thought maybe something had happened to my computer (a piece of software or driver update) that had caused me to not be able to cleanly record DVS multitracks, so I did a clean install of Win10 on the computer. No dice, same issue. Wiped that partition and did a clean install of Win7. (The computer's a few years old and Win7 has always worked best on this computer) Tried another recording, still having issues.

I get random skips and glitches in the recordings. In the past, I would have chalked that up to too slow a hard drive, but this machine (even when it had spinners) has never had an issue. It now has SSDs all around, so that can't be an issue. I've always had the ASIO and DVS settings at maximum latency and buffer sizes and my computer still has the same RAM it always had - 8gb, so I'm at a loss as to what is causing the problems.

It's skipping and glitching. I see no clocking errors in DC, the only warnings are Fanout detections, but I've always had those, and I'm still well under the max flows limitation.

The RIOs are on firmware 3.11 which I know is not the latest, but has been fine for the past years.
DVS is 3.10.1.1
Computer is a Lenovo ThinkPad W520 with 8gb ram, 3 Samsung 500gb EVO 850 SSDs (1 mSATA, 2 SATA), running Win 7 with all updates.

I need some help to figure out how to troubleshoot this issue. I'm pretty sure it's my computer as another person has been able to record clean multi-tracks on this system using the same version of DVS.

Thanks for any and all help!
Title: Re: Help troubleshooting new DVS recording issues
Post by: boburtz on April 16, 2017, 03:35:39 PM
I need some help to figure out how to troubleshoot this issue. I'm pretty sure it's my computer as another person has been able to record clean multi-tracks on this system using the same version of DVS.

Thanks for any and all help!
Do you have the dpc latency checker? I would start there, it's an invaluable tool for rooting out  computer problems as they relate to DVS stability.
Title: Re: Help troubleshooting new DVS recording issues
Post by: David Sturzenbecher on April 16, 2017, 03:45:57 PM
Have you followed all the suggestions here?

https://www.audinate.com/faq/how-can-i-tune-windows-pc-best-audio-performance

I had issues that were resolved with a LAN driver update.


Sent from my iPhone using Tapatalk Pro
Title: Re: Help troubleshooting new DVS recording issues
Post by: Andrew Broughton on April 16, 2017, 04:42:18 PM
Do you have the dpc latency checker? I would start there, it's an invaluable tool for rooting out  computer problems as they relate to DVS stability.
Great tip. I forgot about that program.
Ran it and even though I'm not getting any Red bars, disabling my WiFi Driver definitely lowers the latency. My computer has a hardware switch to disable the WiFi and I thought that was enough (certainly helps when using DC) but even with the hardware switch off, disabling the driver in Device Manager lowers the latency further.
Even though I have the MS approved driver, I'll see if there's a later version from Intel.

Thanks!
Title: Re: Help troubleshooting new DVS recording issues
Post by: Andrew Broughton on April 16, 2017, 04:43:28 PM
Have you followed all the suggestions here?

https://www.audinate.com/faq/how-can-i-tune-windows-pc-best-audio-performance

I had issues that were resolved with a LAN driver update.


Sent from my iPhone using Tapatalk Pro
Thanks for that link. Looks like I already had done all the suggestions there except of course for running DPC.
Title: Re: Help troubleshooting new DVS recording issues
Post by: jabney (john abney) on April 17, 2017, 03:03:01 PM
...recording multi-track (32 tracks) of 16 or 24 bit/48k audio using DVS...

This year, I cannot get a clean recording...

I need some help to figure out how to troubleshoot this issue. ...
We had some problems with Dante DVS, and it seems that one or more of the following work ... so far:

. 32 Dante channel limit (we had been using 64 Dante channel limit)
. Multicast for everything (wish Dante would make Multicast the default)
. Hardware Dante at 'both ends' (we added an Ashly digiMIX24 with a Brooklyn II card for monitoring the Dante signal, etc - we already had the Allen&Heath iLive 48 with its Brooklyn card)

best, john




Title: Re: Help troubleshooting new DVS recording issues
Post by: Ade Stuart on April 18, 2017, 06:10:54 PM
If disabling WiFi hasn't worked have you checked the latency and clock histograms for each device.

https://www.youtube.com/watch?v=FLQDwDXXoxc
https://www.youtube.com/watch?v=Sgnx0-TwDjE

What is the full network setup?
Title: Re: Help troubleshooting new DVS recording issues
Post by: Andrew Broughton on April 18, 2017, 06:24:07 PM
Thanks, Ade. My recording last night was better, but still not perfect. I'll try these logs while recording to see if they are showing errors.
Title: Re: Help troubleshooting new DVS recording issues
Post by: Andrew Broughton on April 18, 2017, 08:48:44 PM
Ok, looks like the clock is fine, but latency is the problem. It's fine for up to a few minutes, then glitches with a 30-60ms delay. Sometimes a few in a row.
1st Latency pic is when not recording, 2nd is while recording.
Title: Re: Help troubleshooting new DVS recording issues
Post by: Ade Stuart on April 19, 2017, 04:48:57 PM
Ok, looks like the clock is fine, but latency is the problem. It's fine for up to a few minutes, then glitches with a 30-60ms delay. Sometimes a few in a row.

A few thoughts

Is dpclat always in the green, if not have you tried running dpclat whilst recording and monitoring latency and if so do you see any simultaneous spikes?

Are the hard drives RAID or JBOD?






Title: Re: Help troubleshooting new DVS recording issues
Post by: Andrew Broughton on April 19, 2017, 05:03:43 PM
A few thoughts

Is dpclat always in the green, if not have you tried running dpclat whilst recording and monitoring latency and if so do you see any simultaneous spikes?
Yes, always green once I disable the WiFi driver, but I haven't run it while recording. As you can see, I get latency errors even when not recording showing up in DC.
Quote
Are the hard drives RAID or JBOD?
No, just independent drives, and as I mentioned, very high speed SSDs.

Title: Re: Help troubleshooting new DVS recording issues
Post by: Ade Stuart on April 20, 2017, 04:28:01 PM
It may be worth getting in touch with Audinate, I found them very helpful with a DVS update problem I once had.

In the meantime, is dpclat green with DVS running?

These additional things may not help if dpclat is green but they've helped me to minimise latency;

In the BIOS disable C state control, speedstep and any peripherals that you do not need eg memory card readers, UMTS modems, serial ports etc.

In control panel disable all instances of Batteries - microsoft ACPI-Compliant Control Method Battery.

Double check that the Ethernet adaptor is running at 1Gb/s and change the ethernet cable... sorry a bit obvious I know but worth mentioning.

If you are running managed switches check the port statistics - if you have errors here can you post the network configuration and may be someone else can chime in.

Finally, Audinate used to advise against using Intel 82579 network adapter. I'm not sure if this is still the case, I've certainly used them successfully. Is this what your machine uses? You could try using a USB3 to GbE adapter.

Good Luck!
Title: Re: Help troubleshooting new DVS recording issues
Post by: Andrew Broughton on April 20, 2017, 07:17:51 PM
Thanks Ade.
What's frustrating is this same machine had no issues with DVS until this year.


Sent from my iPhone using Tapatalk Pro
Title: Re: Help troubleshooting new DVS recording issues
Post by: Riley Casey on April 20, 2017, 07:31:41 PM
My personal MacBook suddenly developed an aversion to running anything Dante related throwing up an error message when launching DVS  or controller. Apparently some process I now can't remember the name of was failing to launch even after multiple installs and restarts. Audinate support was not interested in pursuing any help beyond repeating the usual FAQ type solutions. I also had a second partition with a different OS version and the Dante apps ran fine on that.  Not sure if that sort of thing is an option on Winwdows but it seems like some bit of newly changed software on your machine is not feeling cooperative.
Title: Re: Help troubleshooting new DVS recording issues
Post by: Justice C. Bigler on April 22, 2017, 03:05:24 PM
I'm convinced that there is a fundamental problem with DVS.

I have had various things regarding latency happen with DVS that cause it to have recording issues that either produce glitches, or cause the CPU utilization to spike so high that it is unusable, or it just won't connect at all. This has happened across multiple platforms with multiple different operating systems over several years and different versions of DVS. The only solutions that I have found work thus far is to get a new computer. It seems that once DVS stops working on a system, it won't ever go back to working again.

Anything connected to the network through Dante hard interfaces works fine. DVS seems to be rather flaky.
Title: Re: Help troubleshooting new DVS recording issues
Post by: Andrew Broughton on April 22, 2017, 09:18:10 PM
I would have disagreed up until this year. Before this I never really had any issues.
Now with the latest DVS (and even the previous version) I cannot get a glitch-free recording. I've got a clean Win 7 install, and using dpc I have tried disabling various devices. I thought at first it was the wifi driver, but in the end it doesn't make a difference. It still seems to be kinda random. I'll disable a device and test and it seems ok, then a few minutes into the recording, huge latency and another glitch. It's very slow going to diagnose this problem.

At the moment, DPC shows acceptable latency until I enable DVS. So now I'm trying various drivers for my network card. Pretty slow progress with driver changes, reboot, test, etc...

Currently waiting for more activation licenses since each activation uses up a license. Luckily Audinate have been great and quick with helping with the activations.

Not giving up just yet, but in the end I'll probably have to buy a new computer, as Justice says. Frustrating!
Title: Re: Help troubleshooting new DVS recording issues
Post by: Justice C. Bigler on April 22, 2017, 10:04:44 PM
I've had issues with Windows 7, Windows 10, Max OSX 10.6.8 and now the newest 10.11.12 on my newest Mac Pro. Haven't had a chance to sit down and trouble shoot the Mac Pro yet. But it just stopped working. So far my Lenovo P50 laptop with Windows 10 Pro has stayed solid and continues to work.

My systems stay locked down with no system updates or software updates unless absolutely needed.

It is really frustrating as one day DVS will just up and decide to not work.
Title: Re: Help troubleshooting new DVS recording issues
Post by: boburtz on April 22, 2017, 11:54:57 PM
At the moment, DPC shows acceptable latency until I enable DVS.

From what I understand, DPC latency checker's benefits can not be used while running programs of any kind, it's more for finding out what background processes are taking place while your machine is idle. 
Title: Re: Help troubleshooting new DVS recording issues
Post by: Andrew Broughton on April 23, 2017, 12:37:16 AM
From what I understand, DPC latency checker's benefits can not be used while running programs of any kind, it's more for finding out what background processes are taking place while your machine is idle.
Oh really? Well there's no issues at all in DPC until I run DVS, so how should I proceed to troubleshoot this?


Sent from my iPhone using Tapatalk Pro
Title: Re: Help troubleshooting new DVS recording issues
Post by: boburtz on April 23, 2017, 02:08:21 PM
Oh really? Well there's no issues at all in DPC until I run DVS, so how should I proceed to troubleshoot this?


Sent from my iPhone using Tapatalk Pro
From the DPC user guide: "Note that the Deferred Procedure Call concept exists in kernel mode only. Any user-mode code (Windows applications)
runs in the context of a thread. Threads are managed and scheduled for execution by the dispatcher"
If you are not seeing DPC latency spikes at idle, I would suggest that your dropout issue is not caused by DPC, but something else. I don't know which program you are using, but the one from Resplendence has a bit more detailed information than the one from Thesycon, which is very basic.
Title: Re: Help troubleshooting new DVS recording issues
Post by: Mike Ottinger on April 23, 2017, 02:47:50 PM
Yes, always green once I disable the WiFi driver, but I haven't run it while recording. As you can see, I get latency errors even when not recording showing up in DC.No, just independent drives, and as I mentioned, very high speed SSDs.
I have been having the same issue, Just wandering if your laptop is a quad core i7? 
I have a T520 dual core with 16 gig of ram and 256 G SSD.  Also 7200 750 g drive in bay.  I have given up trying to record with DVS.  Same problem all is well then occasional glitches.

Mike
Pyramid Sound Works
Title: Re: Help troubleshooting new DVS recording issues
Post by: Mike Ottinger on April 29, 2017, 02:47:11 PM
Did anyone discover any magic to make DVS work without glitches?
I have Lenovo 520 i5 16 g Ram, 256 SSD, and 750 GB 7200 speed drive. 
Thanks for any help !  Do I need to switch to Quad core Processor?

Mike


Mike
Pyramid Sound Works
[/quote]
Title: Re: Help troubleshooting new DVS recording issues
Post by: Robert Lofgren on April 29, 2017, 03:41:47 PM
I don't see anywhere the connection patth between your dante device and dvs?

Check the network interface tunable parameters and disable flow-control if possible. Make sure that none of your network devices are using energy star powersaving features.
Title: Re: Help troubleshooting new DVS recording issues
Post by: Andrew Broughton on April 29, 2017, 05:45:40 PM
I don't see anywhere the connection patth between your dante device and dvs?

Check the network interface tunable parameters and disable flow-control if possible. Make sure that none of your network devices are using energy star powersaving features.
That's all pretty basic Dante stuff.
Title: Re: Help troubleshooting new DVS recording issues
Post by: Tim McCulloch on April 29, 2017, 06:44:42 PM
So at this point I have to ask "what did Audinate have to say about this?"

They had a guy who posted here a year or so previous and I was hoping he'd respond.  Perhaps it's time to light the fire under them?
Title: Re: Help troubleshooting new DVS recording issues
Post by: Andrew Broughton on April 29, 2017, 06:50:48 PM
They said to connect directly to the device as a test. Unfortunately, we had a redundant system that didn't allow this and I don't have time to do the serious troubleshooting independently from the connected rig.
Now I'm away from that rig for quite a while so I expect I'll not get to the bottom of this until next year, and by then I'll probably have purchased a new computer. File it under Murphy.
Title: Re: Help troubleshooting new DVS recording issues
Post by: Justice C. Bigler on April 29, 2017, 07:02:34 PM
They said to connect directly to the device as a test. Unfortunately, we had a redundant system that didn't allow this and I don't have time to do the serious troubleshooting independently from the connected rig.
Now I'm away from that rig for quite a while so I expect I'll not get to the bottom of this until next year, and by then I'll probably have purchased a new computer. File it under Murphy.
This has been my experience also. I start troubleshooting, and then get swamped with back to back shows and can't troubleshoot it for awhile and then have to come back and start all over.


In troubleshooting my last round of problems, I updated the firmware on my CL and RIOs and made sure that all software was up to date. Things still didn't work. So I shut everything down, and rebooted, starting with the computer first. Let the computer boot up all the way first, then I booted up the console and RIOs and the network switches. Suddenly it all start working.  :-\
Title: Re: Help troubleshooting new DVS recording issues
Post by: brian maddox on April 30, 2017, 12:02:15 AM
This has been my experience also. I start troubleshooting, and then get swamped with back to back shows and can't troubleshoot it for awhile and then have to come back and start all over.


In troubleshooting my last round of problems, I updated the firmware on my CL and RIOs and made sure that all software was up to date. Things still didn't work. So I shut everything down, and rebooted, starting with the computer first. Let the computer boot up all the way first, then I booted up the console and RIOs and the network switches. Suddenly it all start working.  :-\

The number of times i have had this happen with Yamaha Dante products...  And i am in the Yamaha Fanboy Hall Of Fame, so it pains me to say it.

That being said, i have never been completely unable to make a Dante Network work, at least now that i know kinda what i'm doing in setting one up.  But that doesn't mean i don't have to kick the tires a couple of times to make it work....
Title: Re: Help troubleshooting new DVS recording issues
Post by: Robert Lofgren on April 30, 2017, 04:42:32 AM
If configuring the computer network interface tunable parameters like network traffic flow-control is basic stuff for dante (I'm not talking about dante flows or router/switch configuration) then why can't I easily find any information about it and must be well hidden? If you have a link to such information I'd be happy to read more about it.

Your issue sounds just like one of my test cases with dvs/via and dante where the computer network interface was only working at about 25% of its full speed due to flow control.

That's all pretty basic Dante stuff.
Title: Re: Help troubleshooting new DVS recording issues
Post by: Andrew Broughton on April 30, 2017, 03:11:52 PM
If configuring the computer network interface tunable parameters like network traffic flow-control is basic stuff for dante (I'm not talking about dante flows or router/switch configuration) then why can't I easily find any information about it and must be well hidden? If you have a link to such information I'd be happy to read more about it.
It's all over the Audinate web site.
https://www.audinate.com/resources/networks-switches

Quote
Your issue sounds just like one of my test cases with dvs/via and dante where the computer network interface was only working at about 25% of its full speed due to flow control.
It wasn't the switch(es) because other computers worked without issue.
Title: Re: Help troubleshooting new DVS recording issues
Post by: Justice C. Bigler on April 30, 2017, 03:38:21 PM
Your issue sounds just like one of my test cases with dvs/via and dante where the computer network interface was only working at about 25% of its full speed due to flow control.
Tell me you are not trying to use DVS and VIA on the same computer on the same network at the same time. From what I have heard the two pieces of software are nearly incompatible. I'm not really sure what the point of VIA is, other than to use a USB/Firewire/Thunderbolt/PCI audio interface on a network without other Dante hardware. But from what I have heard, even just having the two pieces of software installed at the same time is a show stopper.
Title: Re: Help troubleshooting new DVS recording issues
Post by: Robert Lofgren on April 30, 2017, 03:43:51 PM
Again, I am not talking about network switches. I am talking about your computer network interface, the NIC. I am talking about a case where even a direct connection between the computer/dvs and your dante device (no switch) still give you problems at times unless you tweak your NIC tunable parameters.

It's all over the Audinate web site.
https://www.audinate.com/resources/networks-switches
It wasn't the switch(es) because other computers worked without issue.
Title: Re: Help troubleshooting new DVS recording issues
Post by: Robert Lofgren on April 30, 2017, 03:50:24 PM
I have done a lot of beta testing regarding dante/ravenna/aes67. This has included both hardware and software in different configurations.

Dvs and via can be installed on the same computer without any problem, at least on my test computers running osx and windows. Dvs and via have a blocking mechanism that only allow you to run either one but not both at the same time. If you have heard anything different then this has not showed during my tests.

Tell me you are not trying to use DVS and VIA on the same computer on the same network at the same time. From what I have heard the two pieces of software are nearly incompatible. I'm not really sure what the point of VIA is, other than to use a USB/Firewire/Thunderbolt/PCI audio interface on a network without other Dante hardware. But from what I have heard, even just having the two pieces of software installed at the same time is a show stopper.