Note that the full set of 40 NSD scan sessions were collected for four of the eight subjects but that only the first 30 or 32 NSD scan sessions were collected for the other four subjects. Hence, for exact numbers you must take this into account. Also, note that the numbers of images for which responses are available depend on whether you have access to all collected NSD scan sessions or not. (Remember that the last 3 NSD scan sessions from each subject are held-out from public release due to the Algonauts challenge. For example, there were actually 40 NSD scan sessions collected for subject 1, but only the first 37 scan sessions are publicly downloadable.)
Here is a simple example showing how to determine which of the shared 1,000 images were seen all 3 times by all 8 subjects (assuming that the full dataset is downloaded). Note the subjects who had the fewest scan sessions had 30 NSD scan sessions. Hence, we use "30" in the code snippet below. (To figure out which of the shared 1,000 images were seen all 3 times by all 8 subjects within the currently downloadable data, you would simply substitute "27" for "30" in the code snippet below.)
temp =masterordering(1:750*30);%750 trials per session; all 8 subjects participated in at least the first 30NSD scan sessions
shared515 =[];%1 x 515 vector of1-indices.these indices are between 1-1000.
for q=1:1000
ifsum(temp==q)==3
shared515 =[shared515 q];
end
end
"Can I average prf time-series data across runs? Do they have different stimulation protocols?"
As described in the NSD data paper, the prf experiment involved six runs acquired as BWBWBW (where B and W refer to multibar and wedgering run types). The spatial aperture pattern was identical within each run type, and hence averaging is reasonable (e.g. average the Bs together; average the Ws together). (However, note that the specific colorful texture shown at a given point in time and the precise fixation dot behavior are stochastic across runs and are therefore not exactly identical across runs.)
"I am getting "access denied" errors and/or I am getting 403 errors from AWS ("fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden"). Can you help?"
A subset of the NSD data files (e.g. nsdsynthetic, nsdimagery) is forbidden from being downloaded at this point (see 'Held-out data' on the Overview of the data page).
"I noticed that there are some voxels/signals outside of the brain. Is there something wrong?"There are two factors at play here. (1) In our pre-processing strategy, we intentionally erred on the side of being conservative, in the sense of not aggressively using a brain mask and potentially removing signals of interest. In other words, the brain mask was fairly liberal. (2) In the correction of EPI spatial distortion, noise in the fieldmap estimates outside of the brain can often "pull" signals from brain voxels into locations that are outside of the brain. In a sense, things are correct insofar that the voxels located in actual brain locations should be getting accurate and correct signals. However, it is true that voxels outside of actual brain locations may have funny/weird/bogus signals. One strategy is to simply ignore voxels that are outside of actual brain locations, and one way to do this is to apply your own (stringent) brain mask, if you want to. (Alternatively, you can just restrict your interest to surface-prepared data, which are guaranteed to come from gray matter.)There is yet one more factor to consider. Outside of the brain, the raw signal intensity is quite low, and random noise can often manifest as large %BOLD signal changes (i.e. dividing by a small number (e.g. the mean signal intensity) will tend to inflate the resulting %BOLD numbers). This is known behavior, and one approach is to simply ignore these seemingly large %BOLD values as unreproducible and meaningless noise.
"What is the breakdown of image databases NSD pulls from and what resources/annotation already exist for those?"
All NSD images come from the Microsoft COCO image database. As for resources, there are a number of online 'computer vision' resources that provide a wealth of annotations on the COCO images (see http://cocodataset.org/#external). In addition, note that the externally contributed nsd_access toolbox (see General Information) provides a convenient Python interface for understanding how the images selected for use in NSD are mapped onto the COCO images.
"How do I load in the NSD-generated ROI files, like lh.floc-faces.mgz, into FreeSurfer's freeview? It won't load in freeview as a surface annotation."
If you use the "Overlay → Load generic..." option, freeview should be able to load and interpret the surface data in the .mgz files.
"How do I map from MNI to fsaverage and/or vice versa?"
Since MNI and fsaverage are fundamentally different in nature (volume vs. surface-based), the mapping is, in general, a bit ill-defined. But given that we have lots of information on the 8 NSD subjects, you could use nsd_mapdata (volume-to-nativesurface option) to go from MNI to the native subject surfaces (e.g. [lh,rh].layerB2), and then use nsd_mapdata (nativesurface-to-fsaverage option) to go from the native subject surface space (e.g. [lh,rh].white) to fsaverage. You could repeat this process for each of the NSD subjects and then you could average the results in fsaverage space. For additional ideas and background, see here.
"How do I map a specific MNI coordinate using nsd_mapdata?"
The easiest approach would be to copy the MNI 1mm NIFTI template (MNI152_T1_1mm.nii.gz), modify the image data inside the template to specifically label the MNI coordinate that is desired (ITK-SNAP reports the MNI coordinate as "World units (NIFTI)") (e.g., create a binary volume with a "1" at the location of interest), load the image data, and then use nsd_mapdata to map the image data to some other space. Note that the motivation for building off of the MNI template is to ensure that the headers and the RPI ordering is all preserved and handled correctly.
"I want to use some of the FreeSurfer outputs, but I am having trouble getting the outputs to work well with nsd_mapdata."There are tricky issues in terms of how the volume data stored in, e.g., the .mgz files are oriented (e.g., NIFTI header issues). The best bet is to see how we handled this in the code: https://github.com/cvnlab/nsddatapaper/blob/main/main/analysis_transforms.m
"How can I access the gradient nonlinearity information?"
The pre-processed files provided with NSD involved correcting for gradient nonlinearities (these are fairly negligible for the 3T data, but are somewhat substantial for the 7T data). We cannot publicly supply the gradient coefficient files. The following information was taken from the Human Connectome Project, and it is assumed to apply equally to NSD: "The gradient field coefficients are considered proprietary and need to be obtained from your institution's Siemens collaboration manager. Your institution must have a research agreement or be willing to sign a non-disclosure agreement with Siemens. Contact Yulin Chang (yulin.chang@siemens-healthineers.com) (USA) or Martin Stoltnow (martin.stoltnow@siemens-healthineers.com) (rest of world)."
Common pitfalls and things to watch out for
Please note that the noise ceiling metrics provided with NSD assume that voxel-wise beta weights are z-scored within each session and then aggregated across sessions. Analyses that wish to use the noise ceiling metrics must mimic these operations. It is possible to apply the general theory of noise ceiling to the case where z-scoring is not performed, but this is up to the user and is not currently provided with NSD.
Some data files and/or results can involve NaN values (for example, NaN might indicate when data are missing), and this may cause problems with various software tools. When processing files, it is recommended to check for these cases and resolve them appropriately (e.g., possibly setting NaNs to 0).
In the nsd_mapdata utility (and associated transform files), the 'native surface' to 'fsaverage' transform is accomplished using the arbitrary naming convention of 'lh.white' and 'rh.white', even though the concept of the fsaverage transform is not actually specific to any cortical depth (i.e. it would be equally applicable to the mid-gray or pial surfaces). Thus, do not let the naming convention cause any confusion. For example, using nsd_mapdata, it would be reasonable to map data from the 'func1pt0' space to 'lh.layerB2' (mid-gray surface), and then map from 'lh.white' (which is just the arbitrary naming convention for data on the subject's surface) to 'fsaverage'.
Note that no intensity normalization, detrending, or noise removal has been applied to the pre-processed fMRI time-series data that are provided with NSD. In particular, note that the mean signal intensity present at a given voxel may drift to some degree over the course of a run, and might be somewhat variable across runs and scan sessions. One should keep these observations in mind when designing an approach that starts with the time-series data.
Some of the files that contain betas in percent signal change units are actually multiplied by 300 and stored as integer format (to save space), and thus need to be casted to decimal format and divided by 300 upon loading. Be careful.
The category-selective ROIs that are provided are intentionally defined with a liberal threshold (t > 0). This has the consequence that some ROIs may overlap with other ROIs (e.g., the floc-faces ROI collection may label some of the same voxels/vertices as the floc-bodies ROI collection). If you require more stringent ROIs, you can further whittle them down based on the provided t-values and/or winner-take-all operations, etc.
In the floc experiment, the 'body' category is distinct from the 'bodies' domain. The latter pools over body the 'body' and 'limb' categories.
The MNI formatting conventions (especially regarding left vs. right) are tricky. Please see Technical notes for details.
If you plan to try to use the transform files provided with NSD, keep in mind that these files have very specific meanings and conventions, so do not necessarily assume that they will work "out of the box" with some specific software. If possible, we recommend using nsd_mapdata.
Note that the number of volumes in the pre-processed time-series data may be slightly larger than expected. This is correct behavior (some "excess" volumes are at the end) and has to do with how the pre-processing is performed. For example, the duration of the experiment conducted in each prf run is 300 s (or more precisely, 300 x 0.999878 s = 299.9634 s). The TR is 1.6 s. We acquired 188 volumes for a given prf run. Notice that 188 x 1.6 = 300.8 s, which therefore extends a little beyond the end of the actual experiment duration. In pre-processing for the standard resolution version, we resample to a rate of 4/3 s (or more precisely, 4/3 * 0.999878 = 1.33317 s). To ensure that we accommodate the full duration of the acquired data, the pre-processing is designed to produce 300.8/(1.33317) = 225.63 volumes. But of course, fractional volumes are non-sensical; hence, what we actually do is to round up to produce 226 volumes. (Note that there is a little bit of extrapolation involved to compute the very final volume.) Thus, 226 volumes is obtained in the pre-processing, even though there is a sense in which 225 volumes should have been obtained (since 300 s / (4/3 s) = 225). Nonetheless, everything is correct, and you can simply strip the 226th volume from the end of the pre-processed data, and you can interpret the first 225 volumes as coinciding with, say, the prf stimulus design information that we have provided.
The flattened surfaces provided with NSD may have a rotation that may be unexpected in certain software packages. If that is the case, you may wish to read in the flattened surfaces (e.g. read_patch.m) and apply your own rotation to the vertices and then re-save the files.
NSD provides a manually flattened version of the fsaverage surface (?h.full.flat.patch.3d) that is distinct from the one that comes with FreeSurfer (?h.cortex.patch.flat). The former is a bit less jaggedy than the latter. Also, the same general cutting strategy used for the manually flattened fsaverage was used to generate the manually flattened version of each native-subject surface.