Adding Support for Neuralynx Header using ``Date''

[fccaa]

In the data set I am working with, I found some nev file having a header like this:

######## Neuralynx Data File Header
## File Name: V:\not_to_show.nev
## Date Opened: (mm/dd/yyy): 07/04/2015 At Time: 17:01:39
## Date Closed: (mm/dd/yyy): 07/04/2015 At Time: 17:01:39

However, the version in the official repository supports only "## Time Opened". After making the change, I can open the directory without the error reported in https://github.com/SpikeInterface/spikeextractors/issues/480 .

[JuliaSprenger]

Hi @fccaa
thanks for providing a solution for this new header. Do you know what application generated these files and what is the application version? If this information would be available a more targeted usage of the use_date header would be more robust.
Also would it be possible for you to provide a small test file with this kind of header to be added to the Neo test file repository here: https://gin.g-node.org/NeuralEnsemble/ephy_testing_data ?

[apdavison]

hi @fccaa - so we can move forward with this, please could you reply to the questions from @JuliaSprenger ?

[PeterNSteinmetz]

Agreed it would be best to see the actual header structure here and try to figure out how it fits into the history of the various headers we are trying to parse in NlxHeader.

[fccaa]

Hi @fccaa
thanks for providing a solution for this new header. Do you know what application generated these files and what is the application version? If this information would be available a more targeted usage of the use_date header would be more robust.
Also would it be possible for you to provide a small test file with this kind of header to be added to the Neo test file repository here: https://gin.g-node.org/NeuralEnsemble/ephy_testing_data ?

Deeply sorry for my late reply. Here is the sample of the data.
sample.zip

[PeterNSteinmetz]

Just had a look at this sample data. There is one .Nev file. It appears to be produced by NeuraView. That may be a Neuralynx product available at https://neuralynx.com/software/category/data-analysis. Curious if that Neuralynx program is producing yet another header format, but it won't be the first time.

This will require the addition of another format type being parsed in the NlxHeader class.

[samuelgarcia]

bravery!
It is not the first manufacturers that accumulate versions and variations around the same file format over years.
Maintaining all of the is painfull but having then centralised here is really helpfull.

[PeterNSteinmetz]

@fccaa Alan, can you please provide text you would like included with the test data to describe its source? Usually this states the laboratory, investigator and contributor. I am preparing a commit to the ephy_testing_data project to include your test file. That is normally the first step to make the test data available prior to committing the code that handles that data format to the project here.

[PeterNSteinmetz]

Also would like to see #913 and #957 merged into master before adding another branch for dealing with this issue.

[JuliaSprenger]

@PeterNSteinmetz Done, #913 and #957 are merged. Feel free to code ahead :)

[PeterNSteinmetz]

Thanks. Appears there are now some changes with the testing data that I will need merge in locally.

[PeterNSteinmetz]

@JuliaSprenger This is now failing on my end on new line test_neuralynxrawio.py:157 because of the comparison to the number 58, versus sigHdrs[0][1] containing the string '58'.

It appears the other headers are also coded this way as a string for this RawIO. Should we change the RawIO on this? It is a bit odd as the dtype from baserawio is specifying a U64 for this item, but neuralynxrawio:150 explicitly sets it to a str. Both the name and id fields are set as 'U64' but the name in particular seems to need to be a string.

The quick fix here would be to change the testing code to compare against the string '58' rather than the number 58, and this is what is done just below on line test_neuralynxrawio.py:172.

[JuliaSprenger]

Hi @PeterNSteinmetz this sounds like the fixed I implemented in #968 to update the neuralynx tests to the latest version of the rawio specifications as these had a slight change in the intermediate time. I guess if you rebase to the Neuralensemble/master this should be fixed already.

[PeterNSteinmetz]

Hi @JuliaSprenger . Thanks, that does look like it is now fixed in #968. Further work on this request (#944), is now waiting on @fccaa indicating how he would like to have the test data credited in its README file before I push a set of test data onto the ephy_testing_data repository.

[fccaa]

@fccaa Alan, can you please provide text you would like included with the test data to describe its source? Usually this states the laboratory, investigator and contributor. I am preparing a commit to the ephy_testing_data project to include your test file. That is normally the first step to make the test data available prior to committing the code that handles that data format to the project here.

Here it is.
"
The data file was created by D. Miyamoto at Murayama's Lab at RIKEN. This data file sample is submitted by C. Fung.
"
Please check if this is appropriate. Thank you.

[PeterNSteinmetz]

Test files now incorporated in pull request for ephy_testing_data at its PR #43 https://gin.g-node.org/NeuralEnsemble/ephy_testing_data/pulls/43.

[JuliaSprenger]

solved via #1005