Time-series data

This section covers pre-processed fMRI time-series data and other measures that exist at the level of the time-series data, which include motion parameter estimates, design matrix information (i.e. which stimulus was shown when), physiological data, and eyetracking data.

Pre-processing of the functional data involved two operations. First, a temporal resampling was performed using a cubic interpolation. The time-series for each voxel was upsampled to either 1 s (high-resolution version) or 1.333 s (standard-resolution version) and in doing so, slice-time differences were corrected. Note that the first time point (after pre-processing) is coincident with the start of the acquisition of the very first volume (i.e. the time of the first RF pulse). Second, a spatial resampling was performed using a cubic interpolation. Each volume was sampled at either 1 mm (high-resolution version) or 1.8 mm (standard-resolution version). This operation corrects for head motion, EPI distortion, gradient nonlinearities, and across-scan-session alignment. Note that no high-pass filtering, nuisance regression, nor units conversion are performed for the pre-processed functional data.

Pre-processed time-series data

 
nsddata_timeseries/ppdata/subjAA/func*/timeseries/timeseries_BB_runCC.nii.gz
 
These are the pre-processed fMRI volumes. The only processing that has been performed for these data is a temporal resampling and a spatial resampling. To save space, a liberal brain mask has been used to zero-out the data for non-brain voxels (same mask for all data from a given subject). "BB" is either prffloc (referring to the scan session in which the prf and floc experiments were conducted) or sessionNN (where NN is the number of the core NSD scan session). Note that scan sessions involving resting-state acquisition consist of 14 runs (as opposed to the typical 12 runs), so in these cases CC ranges from 01 to 14.
 
For the high-resolution (1-mm) preparation, the data are sampled at 1-s and contain 301 volumes in each run (for the core NSD experiment). For the standard-resolution (1.8-mm) preparation, the data are sampled at 1.333-s and contain 226 volumes in each run (for the core NSD experiment). In both cases, the time associated with the first volume corresponds to the start of the acquisition of the first volume (first RF pulse).

For the prffloc scan session, there are 12 runs in the following order: prfbar, prfwedge, floc, floc, prfbar, prfwedge, floc, floc, prfbar, prfwedge, floc, floc.

Motion parameter estimates


nsddata_timeseries/ppdata/subjAA/func*/motion/motion_BB_runCC.tsv
 
Motion parameter estimates (SPM style). These reflect rigid-body transformations that indicate how each given fMRI volume is aligned to the reference fMRI volume (which is taken to be the first volume acquired in each scan session).

Note that each fMRI volume is spatially undistorted before estimating the rigid-body motion. Also, note that the motion parameter estimation is done with the first volume as the reference. However, the full pre-processing involves also estimating an affine transformation that aligns the data from each given scan session to the master space defined for each subject; this affine transformation is concatenated with the rigid-body transformations in order to generate the final pre-processed fMRI data.

In the .tsv files, the first 3 columns correspond to translation parameters (mm) and the second 3 columns correspond to rotation parameters (radians). The number of rows matches the number of volumes in the pre-processed time-series data. Positive on the first column means the brain is displaced towards the posterior direction; positive on the second column means the brain is displaced towards the subject’s right; positive on the third column means the brain is displaced towards the inferior direction; positive on the fourth column (roll) means the head is twisted such that the nose is fixed and the top of the head goes towards the subject’s right; positive on the fifth column (pitch) means the ears are fixed and the head nods up; positive on the sixth column (yaw) means the top of the head is fixed and the head twists such that the nose goes to the subject’s left. 

Design matrix information


Below, we document design matrix files for the NSD and floc experiments. Note that the pre-preprocessed fMRI data (and motion files) extend for one volume beyond the number of elements contained in the .tsv design files. This is expected behavior (due to how the pre-processing is performed); to achieve correspondence to the .tsv design files, one can simply trim (drop) the trailing volumes of the fMRI (and motion) data.

nsddata_timeseries/ppdata/subjAA/func*/design/design_sessionBB_runCC.tsv
 
This is a specification of the design of the NSD experiment. Each file is a column vector of integers, and the number of elements corresponds to the number of volumes in the functional data preparation for a given run. Each element is either N where N is a 73k ID (1-indexed), marking the onset of a presentation of that image, or 0 for all other elements. Note that in order to achieve correspondence to the motion and fMRI time-series data files, the run number CC is 1-12 for scan sessions that contained only NSD runs but is 1-14 for scan sessions that included resting-state runs (in this case, the first (1) and last (14) runs are resting-state runs and the middle 12 runs are the NSD runs). Also, note that in the case of resting-state runs, the .tsv file consists simply of all 0s. Finally, note that the information contained in these .tsv files is redundant with respect to the nsd_expdesign.mat file (see  Experiments⁠ ), but is provided in this .tsv format for your convenience.

nsddata_timeseries/ppdata/subjAA/func*/design/design_floc_runCC.tsv

This is a specification of the design of the floc experiment. Each file is a column vector of integers, and the number of elements corresponds to the number of volumes in the functional data preparation for a given run. Each element is either N where N is between 1 and 10 marking the onset of one of the 10 categories in the floc experiment, or 0 for all other elements. Note that CC ranges from 1 through 6 (even though the 6 floc runs were acquired chronologically as runs 3, 4, 7, 8, 11, and 12 in the prffloc scan session).

Physiological data

 
Pulse and respiratory data were collected in NSD scan sessions 21-30 (same as when the primary set of resting-state data are acquired).
 
nsddata_timeseries/ppdata/subjAA/physio/physio_BB_runCC_DDDD.tsv
 
CC ranges from 1 to 14 (chronological acquisition order), and DDDD is either ‘puls’ or ‘resp’, indicating pulse and respiratory data, respectively. Each file consists of a column of numbers (typically numbering 15040 or 15041). The numbers in the .tsv file contain the actual physiological data samples extracted from the Siemens files. It appears that they can be interpreted as close to exactly 50-Hz sampling (more on this below). The data samples start immediately after the AcquisitionTime of the first DICOM volume and end immediately after the completion of the last DICOM volume. Note that no actual analysis of the physiological data has been performed (aside from the timing extraction).
 
Notes on how we handled the synchronization of the physiological data and the fMRI data: Our strategy was to assume the accuracy of the LogStartMDHTime and LogStopMDHTime values stored in the Siemens files. We assume that these times correspond to the absolute time of the first and last physiological data samples. We also assume that the data samples come in equally spaced in time. In order to synchronize with the fMRI data, we extracted the AcquisitionTime stored in the DICOM headers of the first volume of each run, and used that time accordingly along with an empirical measurement of the average DICOM duration as recorded by the scanner internal clock.
 
To interpret the timing of a .tsv file, the following is suggested. Since the TR is 1600 ms and since we acquired 188 volumes in each run, we expect the fMRI acquisition to last from time 0 s through time 300.8 s. Thus, if there are say, 15040 samples in a given .tsv file, we can assume that the time points corresponding to these samples is something like linspace(0,300.8,15040). Moreover, the acquisition times for each of the raw 188 fMRI volumes would correspond to 0, 1.6, 3.2, and so on. In pre-processing, we correct for slice time differences and also upsample the data to either 0.999878 s (for the func1mm preparation) or 0.999878*(4/3) = 1.333171 s (for the func1pt8mm preparation). Thus, the times corresponding to the pre-processed fMRI time-series volumes would be 0, 0.999878, 1.999756, and so on (for func1mm) or 0, 1.333171, 2.666342, and so on (for func1pt8mm).
 
Occasionally, a physio .tsv file will have a different number of samples (e.g. 15000). It is not clear what the cause of this is (perhaps dropped frames?). We suggest to proceed as described above and assume that the first and last frames still correspond to 0 s and 300.8 s.

Eyetracking data


Note that only the NSD runs (and not the resting-state runs) have associated eyetracking video and data. For this reason, the files from a given scan session may start with run02 and this is correct behavior (since sessions with resting-state data have resting-state runs as run01 and run14).

Note that the "CC" in the runCC filename is in chronological acquisition order. If you are matching these to the raw BIDS data, please see the  Raw data⁠  page for how the naming scheme is designed.

nsddata_timeseries/ppdata/subjAA/eyevideo/eyevideo_BB_runCC.mp4

This is a video capture of the eyetracker computer's display (via a cell phone). This may be a useful complement to the actual eyetracking data (e.g., for informal inspection of the subject's eye and/or for when the eyetracker failed to lock onto the subject's pupil).

All of the .mp4 clips have been cropped to exactly match the fMRI data acquisition duration (i.e., from the start of the very first fMRI volume through the acquisition of the very last fMRI volume in a given run). This cropping was done manually by a human on basis of the audio cues from the video recording; the approximate accuracy of this manual procedure is estimated to be about +/- 1 s. For example, the expected duration of an .mp4 file corresponding to 1 NSD run is 188*1.6 s = 300.8 s.

To protect privacy, the .mp4 files have had the audio stripped (only video is present). The .mp4 files often begin with a few seconds of a black screen — this is correct behavior and is due to video codec issues. When interpreting the timecodes from these video files, be careful to ensure that whatever software you are using is using precise timecodes as opposed to approximate estimates.

https://slite.com/api/files/Db_SY7ChmB/image.png?apiToken=eyJhbGciOiJIUzI1NiIsImtpZCI6IjIwMjMtMDUtMDQifQ.eyJzY29wZSI6Im5vdGUtZXhwb3J0IiwibmlkIjoidmpXVGdoUFRiMyIsImlhdCI6MTczMDg0MjEwOSwiaXNzIjoiaHR0cHM6Ly9zbGl0ZS5jb20iLCJqdGkiOiJWRVJYdXo3dWRaanNRMCIsImV4cCI6MTczMzQzNDEwOX0.eDaplaeDhQ8I4u3r8SSpgA0bAjT5JvJOIJdaEpTCcTk

nsddata_timeseries/ppdata/subjAA/eyedata/eyedata_BB_runCC.edf

This is the raw eyetracking data file obtained from the EyeLink device. The eyetracker was run at 2000 Hz. The BOLDscreen was run at 1920 x 1080. Note that 8.4° of visual angle (the size of the NSD stimuli) corresponds to 714 pixels on the BOLDscreen.

The utility edf2asc can be used to convert the .edf file to ASCII format. The edf2asc utility is available from  SR Research .

Keep in mind that eyetracking data acquisition starts well before actual fMRI data acquisition (approximately 30-90 seconds before). To determine precise synchronization between the eyetracking data and other measures (e.g. the fMRI data), the stimulus computer issues a synchronization message (using PsychToolbox) to the eyetracker computer:
Eyelink('Message','SYNCTIME')
and this is done right before the actual experiment starts (i.e. right before the display of the very first stimulus frame) and right after the experiment ends (i.e. right after the display of the very last stimulus frame). For example, in a sample .edf file for an NSD run, we find that there is a SYNCTIME message that occurs at timestamp 12829505 and timestamp 13129473. Notice that 13129473-12829505 = 299968, which is interpreted as 299.968 s. The experiment conducted in NSD runs is indeed intended to be 300 s long. If we use the precise time estimates (see  Technical notes⁠ ), we find that 0.999878 s * 300 = 299.9634 s, which is quite close to the duration indicated by the eyetracking timestamps. (However, keep in mind that the fMRI data acquisition extends a little bit longer than the actual experiment duration (e.g., 1 NSD run consists of 188 volumes * 1.6 s = 300.8 s). See  Technical notes⁠  for more details.)

nsddata/inspections/eyetrackinginspections/pupil_subjAA_BB_runCC.jpg

This shows the pupil area over time before (top panel) and after preprocessing (bottom panel). Detected blinks and noise shown in orange. Each file shows the data of a single scanning run and subject.

nsddata/inspections/eyetrackinginspections/XY_subjAA_BB_runCC.jpg

This shows the preprocessed gaze positions as 2D scatter plot (top left) and as line plots for horizontal (X, top right) and vertical gaze coordinates (Y, bottom right panel). It further shows a histogram of the Euclidean distances between each recorded gaze position and the median gaze position (bottom left panel). Removed blinks and noise marked in orange.

nsddata_timeseries/ppdata/subjAA/eyedata_preprocessed.mat

This contains the pre-processed eyetracking data. The data is stored in a cell array named “data”. Each cell represents one scanning run. Following fields are included.

  • samples: Raw data cut to imaging session
  • samples_clean: Preprocessed data (no blinks & noise)
  • samples_blinks: Blinks & noise removed from samples_clean
  • filename: Name of the imported raw data file (after edf-to-ascii conversion)
  • euclDist: Euclidean distance to median gaze position over time
  • messages: EDF-file header and recorded messages 
  • saccs: Saccade on-/offsets detected by the Eyelink
  • blinks: Blink on-/offsets detected by the Eyelink
  • valid_ratio: Percent valid samples after preprocessing

Note that “samples”, “samples_clean” and “samples_blinks” all contain 2D matrices with time stamps (column 1), horizontal gaze position (column 2), vertical gaze position (column 3) as well as pupil area (column 4) over time (rows).