Troubleshooting in Neuroarchiver

[Kevan Hashemi]

A customer writes:

"We are having some issues with the new Neuroarchiver tool and we feel it's not running smoothly. The issue with running the neuroplayer together with the receiver is confusing. If a student loads a file to check a recording it will not only cause a loss of data but will cause some troubles when we will convert the NDFs to another format. This has already caused a loss of several hours of recording. A couple of days ago I went back to a previous version and I think it worked better. However, we had another issue: the system is still running (Record is on), but the files are empty! And this happened twice spontaneously already. Attached are 2 ndfs, 5155 is the same ndfs when this happened, 8881 is the following one with no data in it."

I am sorry to hear that you are losing data.

I have your two NDF files. Both are recorded with Neuroarchiver 143. You say that the two files were recorded with different versions of the Neuroarchiver. What am I missing?

File 8881 contains no telemetry signals. It is a perfect recording from a receiver that is receiving no signals. File 5155 contains twenty robust recordings up until time 1400 s, and then those recordings disappear. They stop suddenly, as if all the transmitters were turned off simultaneously, which is of course impossible. I'm looking at the raw data in the NDF file now, and there is no corruption of the timing, so the recording is from the same Octal Data Receiver. The Octal Data Receiver has an obsolete feature that allows you to tell it to record only from transmitters in the range 64-79, or 160-175.

Neuroarchiver 143 was bundled with LWDAQ_10.2.2. That's the version of LWDAQ we released in May, 2021. In April 2022, with the release of LWDAQ_10.4.1, we split the Neuroarchiver into two independent parts: the Neuroplayer and the Neurorecorder. In your version of the software, it is possible to re-configure an ODR to ignore your transmitters. In the new version, these obsolete features of the ODR are hidden.

In the past two years, we have done a lot of work making the Neurorecorder and Neuroplayer more robust and easy to use, as you can see for yourself in the revision histories below.

https://www.opensourceinstruments.com/Electronics/A3018/Neuroarchiver.html
https://www.opensourceinstruments.com/Electronics/A3018/Neuroplayer.html#Version%20Changes
https://www.opensourceinstruments.com/Electronics/A3018/Neurorecorder.html#Version%20Changes

The current release of LWDAQ is 10.5.5 with Neuroplayer 165 and Neurorecorder 162. You are welcome to download and try out the latest version using this link:

http://www.bndhep.net/Software/Download/LWDAQ_10.5.5.zip

Or you can clone our GitHub repository and pull the pre-release of 10.6.0:

https://github.com/OSI-INC/LWDAQ

The Neuroplayer and Neurorecorder manuals are here:

https://www.opensourceinstruments.com/Electronics/A3018/Neuroplayer.html
https://www.opensourceinstruments.com/Electronics/A3018/Neurorecorder.html

In the new software, Neurorecorder runs independently of all other processes. You can have multiple Neurorecorders running on the same machine, recording from different data receivers, writing their NDF files to different directories. Each Neurorecorder has its own Receiver button that you can press to view the live data.

I suggest you try out the new software and let me know what happens. Any problems you encounter, we can fix.

Best Wishes, Kevan

[Kevan Hashemi]

Dear Tawfeeq,

Receive more files from, recorded with LWDAQ 10.5.5, Neurorecorder 162. Looking at file 2968, we see clock messages from an Octal Data Receiver (A3027E, FV=13) for the first 184 s. At time 185 s, something resets the ODR. The ODR clock goes to zero and begins counting up again. All transmitters reappear in the recording.

Conclusion: There are two processes communicating with the ODR. One is configuring the ODR to ignore all existing telemetry signals. This process must be an older version of LWDAQ, one that provides an entry box for "set selection". The other is configuring the ODR to record all existing telemetry signals. This is most likely a newer version of LWDAQ.

Suggestion: Reboot your data acquisition computer. During the reboot, do you see the red and orange lights flashing on your ODR? If so, some other computer is communicating with your ODR. You must find that computer and stop the competing process.

Suggestion: Once the red and orange lights have stopped flashing on the ODR, start LWDAQ 10.5.5. Open the Neurorecorder. In the Neurorecorder window, press Configure. The Configuration Panel opens. Press "Unsave Settings". Close the Configuration Panel. In the Neurorecorder window, press Receiver. The Receiver Panel opens. Press Configure and "Unsave Settings". Close the Receiver Panel. Now you have removed old settings from your LWDAQ.

Enter your LWDAQ Driver's IP address in the Neurorecorder. Select your recording directory. Press Start. Recording should begin. Press the Receiver button in the Neurorecorder. You should see the live signals being recorded. The red and yellow lights on the ODR should be flashing.

Best Wishes, Kevan

[Kevan Hashemi]

Suggestion: You might have two Octal Data Receivers (ODRs) and two LWDAQ Drivers connected to the same subnet. Suppose you have two LWDAQ Drivers sitting at IP address 10.0.0.37. When you try to record from that IP address, it is your network switch that decides which LWDAQ Driver will respond. If you have more than one LWDAQ Driver, each driver must receive its own IP address. Instructions for changing the IP address using the Configurator Tool are at link below.

http://www.bndhep.net/Electronics/LWDAQ/Manual.html#Configurator

[Kevan Hashemi]

Tawfeeq writes, "Just wanted to update you that after replacing the ODR and resetting the system according to your suggestion, we have now over 50 hours of continuous recording with no troubles. I'll try to run the other ODR (the one we had an issue with) in another system and see if we will have this issue again."

[Kevan Hashemi]

Thank you for your report. I predict that the first ODR will work fine. You had two LWDAQ processes downloading from the ODR at the same time. When you launch LWDAQ, you create LWDAQ No1. When you launch the Neurorecorder, you create LWDAQ No2. You launch a Neuroplayer and you get LWDAQ No2. The No2 is downloading from the ODR. You can press the Receiver button in No2 and look at the raw telemetry data. You can select the Receiver Instrument in No1 and try to look at the raw data. But now you are going to mess things up because No1 and No2 are both trying to get data from the same ODR. I hope that makes sense. Best Wishes, Kevan

[Kevan Hashemi]

UPDATE: There were two problems in the Hebrew University system.

First problem was the laptop running the LWDAQ software was crashing. This appears to be solved by stopping the laptop from uploading files to an external server.

Second problem was one of the Octal Data Receivers would stop receiving telemetry signals after a few hours of running. The institute returned this receiver to us. We observed the same problem, replaced a faulty part in the receiver's local oscillator circuit, and now the receiver is running well.