The nsdsynthetic experiment took place in one 7T scanning session (8 runs). The pre-processing of the data from this experiment is essentially the same as that used for the NSD core experiment. The total number of volumes, after pre-processing, is 429 volumes in each run for the func1mm preparation (TR 1 s) and 322 volumes in each run for the func1pt8mm preparation (TR 1.333 s).
In brief, the nsdsynthetic experiment involved 8 runs, alternating between fixation runs and one-back runs. During the fixation runs, the subject performed a fixation task on a central dot; during the memory runs, the subject performed a 1-back task on the presented images. Each image was presented for 2 s and was followed by a 2-s gap before the next trial. There are a total of 284 distinct images. The stimuli and trial ordering in the 8 runs was exactly the same for all subjects (including the 1-back trials). In each run, there were a total of 93 stimulus trials (as well as blank trials). The 93 stimulus trials consisted of 83 regular stimulus trials plus 10 special 1-back trials (in which the presented stimulus was identical to the previously presented stimulus). There were a total of 93 x 8 = 744 stimulus trials conducted in the scan session.
The 284 images are described in detail in the NSD synthetic data paper. In brief, the following general description can be given to the images: white noise, white noise with a large block size, pink noise, natural scenes, upside-down versions of these scenes, Mooney versions of these scenes, line-drawing versions of these scenes, contrast-modulated natural scenes (4 scenes x 5 contrast levels (100%, 50%, 10%, 6%, 4%) = 20 images), phase-coherence-modulated natural scenes (4 scenes × 4 coherence levels (75%, 50%, 25%, 0%) = 16 images), single words (2 word lengths × 5 positions × 4 words = 40 images), spiral gratings varying in orientation and spatial frequency (112 images), and chromatic noise varying in hue (68 images).
For the GLM analysis, we provide “fithrf” and “fithrf_GLMdenoise_RR” versions of single-trial betas. All beta weights reflect the response to a stimulus duration of 2 s, which is slightly different from the NSD core experiment (in which each stimulus was 3-s long). Each scan session generates a total of 744 beta weights.
For the “fithrf_GLMdenoise_RR” analysis, it is necessary to deem some of the trials as “repeats” of other trials (so that cross-validation can be performed). For this analysis, we kept the different tasks (fixation, memory) separate --- we did not assume that the response to an image was the same across the tasks. There are some instances in which the presented image is repeated exactly (and under the same task). These contributed to what was used for cross-validation. In addition, we presumed that for some of the images, we could group the images into groups of 4. Specifically, we made groups of 4 from images 1:12 and 105:284. For example, the spiral images naturally come in groups of 4, as the only difference in each group of 4 is just a phase shift. Finally, we treated the special 1-back trials as distinct and used separate one-shot regressors for those trials: in other words, the 1-back trials did not enter the cross-validation process. Ultimately, the designation of certain trials as repeats is used only to determine the number of GLMdenoise regressors to use and to determine the regularization parameter in the ridge-regression procedure.
Contents:
- <masterordering> is 1 x 744 with the sequence of trials (indices relative to the 284 images)
- <stimpattern> is 1 session x 8 runs x 107 trials. Elements are 0/1 indicating when stimulus trials actually occur.
Note: masterordering is 1 x 744 indicating the temporal sequence of 284-ids shown to each subject. This sequence refers only to the stimulus trials (ignoring the blank trials and the rest periods at the beginning and end of each run).
Note: All of these indices are 1-based indices.
Note that the number of repetitions of each image is complex. Users interested in, e.g., computing noise ceilings, should carefully take into account the differing number of trials of each image.
This .csv file provides convenient names and groupings for the NSD synthetic images. The first column (Image number) indicates 1-index numbering for the 284 images. The second column (Image) provides a short unique name for each image. The third column (Image subclass number) provides 1-index numbering for a first-level grouping of the images. The fourth column (Image subclass) is a text label associated with each image subclass. The fifth column (Image class number) provides 1-index numbering for a second-level grouping of the images (multiple image subclasses make a single image class). The sixth column (Image class) is a text label associated with each image class.
This text file provides the 40 distinct words used for the word stimuli in the NSDsynthetic image set. The words are provided in the order that they appear in the image set.
This is an .hdf5 file that contains the first 220 images (1 through 220) prepared for the nsdsynthetic experiment. <imgBrick> is 3 channels x 1360 pixels x 714 pixels x 220 images and is in uint8 format. These images were shown on a gray background with RGB value (126,110,108). There are 714 rows and 1360 columns. The reason for the non-square shape of the image is that a few of the word images extend far to the left and far to the right.
The images in the .hdf5 file (combined with the “colorstimuli” files described below) constitute the official list of the 284 images. When we use the term ‘284-ID’, this refers to an index into this list of 284 images (1-indexed).
This is the same format as the nsdsynthetic_stimuli.hdf5 file. The difference is that there are 64 images. After concatenating the 220 images with the 64 images, the result is 284 images. Hence, this file contains images 221 through 284.
There is a separate .hdf5 file created for each of the NSD subjects (based on color calibration that was tailored to each subject). We act as if there are 284 distinct images, even though in reality the last 64 images are slightly different for each subject (based on a flicker photometry calibration).
In these folders, there are standard RGB .png files (uint8, 714 pixels x 1360 pixels x 3 channels). Each file is named "nsdsyntheticBBB.png" where BBB indicates the 284-ID (1-indexed). The “nsdsynthetic” folder contains 220 images, while the “nsdsynthetic_subjAA” folders contain the remaining 64 images tailored to subject AA.
NOTE: These .png files for the nsdsynthetic experiment have been prepared by taking the square-root of the values before writing to disk. The motivation is that when viewing these .png files on a standard display, the images will resemble what the subjects actually saw. The .png files should not be treated as quantitative; for quantitative versions of the stimuli, one should use the .hdf5 files.
The contents of these files are the same as described elsewhere. Note that MNI versions of the betas are provided only for the “fithrf” preparation.
- Bear in mind that the main NSD experiment uses standard computer RGB images. When these were shown on the BOLDscreen display (which is linearized), we specifically used a squaring gamma response such that the images are delivered as they are intended.
- In contrast, the stimuli in the NSD synthetic experiment are prepared as "vision science" images and are not intended to be shown with a squaring gamma response. Thus, they were delivered as-is on the BOLDscreen display (which is linearized).
- For some analyses, it may be important to correctly handle the luminance of these two different sets of stimuli. One approach is to convert the RGB images of the main NSD experiment by squaring them; doing so will ensure that the result is comparable to the images of the NSD synthetic experiment (which should be used as-is without squaring). Another approach is to use the RGB images of the main NSD experiment as-is, and apply a square-root transformation ot the images of the NSD synthetic experiment.