Neurorecorder ToolNote: This manual applies to Neurorecorder 162+ with LWDAQ 10.5.1+.
Warning: The Neurorecorder assumes there are no spaces in any directory or file names.
[28-DEC-22] The Neurorecorder is a component of our LWDAQ Software. It downloads SCT signals from data receivers and writes the signals to disk. To play and analyze these recordings, we us the separate Neuroplayer tool. For a video introduction the Neurorecorder and Neuroplayer, see here.
When we select Neurorecorder in the Tool Menu, the Neurorecorder runs in a new and separate process. It will continue recording even when we quit our original LWDAQ process. Each time we select Neurorecorder in the Tool Menu, we create a new and independent Neurorecorder. We can record simultaneously from any number of data receivers on the same computer, and process the recordings as they are written to disk using several Neuroplayers, without worrying about the failure of a processor halting recording.
To prepare for recording, we specify the internet protocol (IP) address of the recording hardware. If our receiver is connected to a LWDAQ Driver, we must also specify the driver socket to which the receiver is connected. If our receiver supports the pre-selection of signal channels for recording, we list the channels in the Select field. We choose a directory for recording with PickDir. We specify how long we want each NDF archive to be, the default being one hour.
To begin recording we press Start. The state label says "Start" with a red background. The Neurorecorder resets the data receiver at the beginning of the next full clock second. It creates a new disk archive named after the UNIX time. It starts downloading telemetry signals and storing them immediately to the archive. The name of the archive is to the right of the Archive label. During recording, the state label says "Record". The background flashing yellow. A yellow background means the Neurorecorder is waiting for data from the data receiver. Press Receiver to examine the signals as they arrive. The Receiver Instrument will open, presenting plots of the signals and providing summaries of reception quality. The Receiver Instrument's graphical user interface consumes some of your computer's computational resources, so we recommend you close the Receiver Instrument window when you are done looking at the signals.
Press Stop to stop recording. Press Resume to continue recording from the same data receiver to the same archive. So long as we press Resume within a few seconds of stopping we can be confident no data will be lost. We refer to the files written by the Neurorecorder as archives. The archives are written in our NDF format. A Neuroplayer can read, display, and analyze an archive as it is being written by the Neurorecorder, or it can read the archive any time thereafter.
The Configure button opens the a Configuration Panel, which contains configuration parameters for the Neurorecorder. When you press Save Configuration, the Neurorecorder saves all its configuration parameters in the LWDAQ/Tools/Data folder. When you next open the Neurorecorder, all these settings will once again be loaded into the tool. All Neurorecorders share the same settings file, so we cannot save and recall the distinct settings of multiple Neurorecorders. When you press Unsave Configuration, the Neurorecorder deletes any existing settings file.
Follow our Subcutaneous Transmitter (SCT) set-up instructions to get your data receiver and computer communicating with one another. As part of the set-up you will download and install the LWDAQ Software. To record signals to disk, select the Neurorecorder from the Tool Menu. To play back and process existing archives, select the Neuroplayer in the Tool Menu.
To configure the Neurorecorder, set ip_addr to the internet protocol (IP) address of your LWDAQ Driver, Animal Location Tracker, or Telemetry Control Box. If you are using a LWDAQ Driver, set driver_sckt to the socket on the driver into which you have plugged your data receiver. If your receiver supports pre-recording channel selection, enter the channels you wish to record, or use a "*" to record all available channels. Press PickDir in the Neurorecorder to select a directory for recording archives.
Press Start. The Neurorecorder state indicator will turn red as it detects the data receiver type, resets the data receiver, and creats a new archive file. Several lines of text will appear in the Neurorecorder's text window, giving the data receiver type, the features it supports, and the name of the archive. The name of the archive will be of the form Mx.ndf, where x is a ten-digit UNIX timestamp. If you want to use a prefix other than "M", change ndf_prefix in the Configuration panel.
Once the reset is complete, the Neurorecorder starts recording to disk. The Neurorecorder state indicator should start flashing yellow, indicating that it is waiting for data to become available. To stop the recording press Stop. Otherwise, leave the Neurorecorder running. It is designed to run continuously.
Warning: Do not use white spaces in your directory names or file names. Use underscores or dashes instead.
The Neurorecorder stores transmitter messages in NDF (Neuroscience Data Format) files. It performs no processing upon the messages as it stores them to disk. What appears in the NDF file is exactly the same sequence of messages that the Data Receiver stored it its memory. Thus we have the raw data on disk, and no information is lost in the storage process. An NDF file contains a header, metadata string, and a data block to which we can append data at any time without writing to the header or the data string. We define the NDF format in the Images section of the LWDAQ Manual. We describe how SCT messages are stored in the data section of NDF files in Reading NDF Files. The Neurorecorder manipulates NDF files with a NDF-handling routines provided by LWDAQ. These routines are declared in LWDAQ's Utils.tcl script. You will find them described in the LWDAQ Command Reference. Their names begin with LWDAQ_ndf_.
All archives created by the Neurorecorder receive a name of the form Mx.ndf, where M is the prefix string specified in ndf_prefix and x is a ten-digit number giving the time of the start of the recording. By default, the prefix is itself the letter "M". The ten-digit number specifies standard Unix Time. The ten-digit number is the number of seconds since time 00:00 hours on 1st January 1970, GMT. We get the Unix time in a Tcl script with command clock seconds. From the name of each file, we can determine the time, to within a second, at which its first clock message occurred. From there we can count clock messages and determine the time at which any other part of the data occurred. The Neuroplayer's Clock Pane uses the timestamps in NDF file names to find intervals corresponding to specified absolute times.
The Neurorecorder stores data in NDF archives and the Player reads the NDF archives to extract voltages and calculate spectra. When the Player reaches the end of an archive, it looks for a newer archive in the same directory and starts playing that one immediately afterwards. Thus if we are playing the archive that is being recorded, the Player will play the fresh data from the expanding archive until the Neurorecorder starts a new archive, at which time the Player will switch to the new archive automatically. If we are playing old archives, the Player will still move from the end of one to the start of the next, even if the next is unrelated to the first. Thus we can go through a collection of archives that are from different experiments and different times, and apply processing to extract characteristics from all the archives.
The Neurorecorder provides a Header button that opens a text window and allows you to enter a comment or document describing the recordings. This header string will be added to the metadata of every archive created by the Neurorecorder.
When recording generates a warning or an error, these appear in the text window in blue and red respectively. If we set log_warnings to 1, the Neurorecorder will write all warnings and errors to a log file. The name of the log file is stored in log_file. By default, the log file is the /Tools/Data directory and is named Neurorecorder_log.txt. We can change the name of the log file and so place it somewhere else. The warnings and error messages all include the current time as a suffix, which is the time the Neurorecorder discovered a problem.
The data acquired by the Neurorecorder takes the form of a list of data receiver messages, as we describe in the Receiver Instrument manual. In general, the data will contain values from one or more channel numbers. The Neurorecorder selects which channels to record with its pre-recording select string, available in the Neurorecorder's Select entry box. Pre-recording selection is supported only by newer receivers such as the Animal Location Tracker (A3038) and Telemetry Control Box (A3042). Pre-recording selection is a simple list of channel numbers, or a "*" character to indicated all channels should be recorded as they are available.
The Neurorecorder uses the Receiver Instrument to download signals from data receivers. We can open the Receiver Instrument with the Signals button and view the signals as they arrive. We can close the same window at any time without affecting recording.
The start time of each NDF archive is encoded in its file name. The file name begins with "M" by default, but you can change the file prefix with the ndf_prefix parameter. After the prefix is a ten-digit number giving the archive start time according to the clock on the computer running the Neurorecorder. This number is the UNIX time in seconds. When the Neurorecorder resets the data receiver, it waits until the start of a new computer clock second before resetting the data receiver. We say the data receiver clock and the computer clock have been synchronized. The delay between the start of the new second and the first clock message in the NDF file is the synchronization delay. The delay between the start of the new second and the moment when the Neurorecorder receivers confirmation of the reset command is the reset delay. The synchronization delay is what we are interested in, but we cannot measure it directly from the Neurorecorder. We can measure only the reset delay. The reset delay is greater than the synchronization delay, usuallly twice the synchronization delay. Here are repeated reports of wait period and reset delay for an Octal Data Receiver (A3027, ODR).
Detected: Octal Data Receiver (A3027), applying driver socket, ignoring channel selection. Synchronization: Waiting period 773 ms, reset delay 53 ms. Created: Synchronized archive M1674839047.ndf at 27-Jan-2023 12:04:07. Detected: Octal Data Receiver (A3027), applying driver socket, ignoring channel selection. Synchronization: Waiting period 665 ms, reset delay 44 ms. Created: Synchronized archive M1674839058.ndf at 27-Jan-2023 12:04:18.
Here is a similar report from a Telemetry Control Box (A3042, TCB). The TCB is faster than the ODR at responding to commands.
Detected: Telemetry Control Box (A3042), applying channel selection, ignoring driver socket. Synchronization: Waiting period 906 ms, reset delay 26 ms. Created: Synchronized archive M1674840015.ndf at 27-Jan-2023 12:20:15. Detected: Telemetry Control Box (A3042), applying channel selection, ignoring driver socket. Synchronization: Waiting period 889 ms, reset delay 31 ms. Created: Synchronized archive M1674840026.ndf at 27-Jan-2023 12:20:26.
Without periodic re-synchronization, our initial synchronization will eventually be lost, as the data receiver clock and the computer clock drift apart from one another. The autocreate parameter in the Neurorecorder is the number of seconds of data we want each NDF file to contain. The default value of autocreate is 3600, and the Neurorecorder will terminate the existing NDF file and start a new NDF file after one hour. The data receiver contains its own clock, accurate to ±4 ms/hr. The Neurorecorder terminates an NDF file when the NDF file contains the correct number of seconds of data as measured by the data receiver clock.
The computers we use to run the Neurorecorder in animal laboratories are usually isolated from the internet. When we isolate computers, their clocks cannot be maintained by a network time server, and when left to run on their own, they can drift by up to ±40 ms/hr. After a hundred hours, the computer clock and the data receiver clock might disagree by several seconds. By default, the Neurorecorder re-synchronizes the data receiver and computer clocks at the start of each newly-created NDF file, as controlled by the synchronize flag. Before the Neurorecorder begins the next file, it waits until the start of the next second on the computer clock, and resets the data receiver, just as it did when it created the first NDF file in when we pressed the Record button. If the computer clock is slower than the data receiver clock, a new file will be created every autocreate seconds in computer time. If the computer clock is faster than the data receiver clock, however, a new file will be created every autocreate + 1 seconds in computer time. The re-synchronization requires that we discard a fraction of a second of data every time a new file is started. But the re-synchronization is essential when we want video and NDF recordings to be synchronous to ±50 ms.
With the synchronize flag unset, the Neurorecorder does not reset the data receiver. When one NDF file is complete, the Neurorecorder creates a new file using the current computer clock time and starts writing data to the new file. No data is lost. The NDF recordings are synchronous with the computer clock to within ±500 ms. The Header button opens a text window into which you can enter a comment or paste a text document describing the contents of the archive you are recording. This comment will be written to the metadata of every archive created by the recorder.
The Neurorecorder's customization string allows us to override default values for parameters such as the coordinates of animal location tracker coils. We open the customization string editor with the Customize button in the Neurorecorder, edit the string, and save with the Save button. To override tracker coil positions, we enter them as a string of numbers enclosed in an alt tag, like this:
<alt>100 100 100 100 100 200 100 200 100 100 200 200 200 100 100 200 100 200 200 200 100 200 200 200 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1</alt>
In the above example, we have eight loop antennas with coaxial cables connected to the first eight detector modules of an A3038 animal location tracker. We are are not using the remaining eight detector modules. We give the x, y, and z coordinates of the eight antennas in centimeters. We choose our coordinate origin so all coordinates are positive. We use negative values for the coordinates of the remaining coils, which instructs the Neurotracker to ignore these coils in the calculation of power centroid. We enter the string as above, with the alt tags, and the Neurorecorder will use these coordinates in place of the default A3038 array locations. We can also specify tracker background powers with the alt_bg tag. We give one integer background power for each of the available detectors. For the A3038C we would enter sixteen values.
The NDF format contains a header, a metadata string, and a data block. Transmitter messages and clock messages are stored by the Neurorecorder in the data block. New data is appended to the data block without any alteration of existing data. The metadata string has a fixed space allocated to it in the file, but is itself of variable length, being a null-terminated string of characters. We can edit the metadata of an archive with the Metadata button in the Neuroplayer. At the top of the metadata there is a metadata header, which the Neurorecorder writes into the metadata when it creates the recording archive. The metadata header contains one or two comment fields, where one comment is a string delimited by an xml "c" tags, like this:
<c> Date Created: 17-Jun-2021 14:29:09. Creator: Neurorecorder 145, LWDAQ_10.2.10. </c> <payload>16</payload> <coordinates>0 0 0 12 0 24 12 0 12 12 12 24 24 0 24 12 24 24 36 0 36 12 36 24 48 0 48 12 48 24</coordinates>
The Neurorecorder always generates a header comment like the one shown above. It also writes the message payload and tracker coil locations to the metadata so that subsequent playback of archives recorded from various data receivers will be configured by the metadata automatically. The Neurorecorder will also add another header comment defined by the user to every file it creates. We press the Header button in the Neurorecorder and we get the Header Panel, in which we can create, edit, and save a header string. This string might describe the apparatus from which we are recording, so that every archive we create contains a record of where the archive came from, and what it contains. We don't have to include xml "c" tags around the text in the Header Window. When the Neurorecorder writes the header to the metadata, it adds the "c" itself.
Here we list changes in recent versions that will be most noticeable to the user. Earlier changes are here. The Neurorecorder source code is the file Neurorecorder.tcl bundled with the LWDAQ distribution.