How to get the data

Before accessing the data, you must agree to the  Terms and Conditions⁠  and fill out the  NSD Data Access  form. After doing so, you are granted full access to the NSD dataset.

AWS

The NSD data are available for download via Amazon Web Services (AWS)'s Simple Storage Service (S3). Thanks to the  Public Dataset Program , access to files (request, egress, and transfer costs) is free of charge.

There are several ways to access the data:
  • For a light-weight experience (no AWS account necessary), you can directly browse the NSD data files via a simple web interface at  https://natural-scenes-dataset.s3.amazonaws.com/index.html 
  • Alternatively, you can use AWS and browse the NSD data files at  https://s3.console.aws.amazon.com/s3/buckets/natural-scenes-dataset 
  • Note that you can directly download individual files from AWS via a URL, like:  https://natural-scenes-dataset.s3-us-east-2.amazonaws.com/nsddata/experiments/nsd/nsd_screencapture.mp4 
  • You can use a graphical S3 client (e.g. Cyberduck) to browse and download the data. If using a client, connect to  natural-scenes-dataset.s3-us-east-2.amazonaws.com . (Note that in order to connect, you have to supply an access key ID and secret that is associated with your own personal AWS account.)
  • For large-scale data downloading, the best bet is probably to use the AWS CLI (command-line interface) which is "rsync"-like.

Note that as an alternative to downloading the data and analyzing on local machines, AWS also provides access to cloud computing resources in the form of EC2 instances.

For your convenience, here is a text listing of  all files in the AWS bucket (natural-scenes-dataset) .

AWS CLI

The  AWS CLI  provides convenient programmatic access to the data.

Consider the following example:
aws s3 ls s3://natural-scenes-dataset
This command simply lists the buckets (folders) available.

As another example:
aws s3 cp s3://natural-scenes-dataset/nsddata/experiments/nsd/nsd_screencapture.mp4 /path/to/local/dir/
This command downloads the .mp4 file and places it inside the local directory "dir".

As another example:
aws s3 sync --dryrun s3://natural-scenes-dataset/nsddata_betas /path/to/local/nsddata_betas --exclude "*func1mm*" --exclude "*MNI*" --exclude "*betas_assumehrf*" --exclude "*betas_fithrf_GLMdenoise_RR*" --exclude "*betas*session*nii.gz"
This command synchronizes the "nsddata_betas" directory from the server to the local "nsddata_betas" directory (located under /path/to/local/). Note that we include the --dryrun flag for cautionary purposes; you should remove the --dryrun flag once you are ready to actually perform the download. Also, note that the command includes several --exclude flags in order to reduce the amount of data downloaded. Specifically, the command excludes the 1-mm preparation of the functional data, the MNI version of the data, beta version 1 ("betas_assumehrf") and beta version 3 ("betas_fithrf_GLMdenoise_RR"), and the NIFTI version of the very large beta files.

As another example:
aws s3 sync --dryrun --exclude "*" --include "*eyedata*" s3://natural-scenes-dataset/nsddata_timeseries /path/to/local/nsddata_timeseries
This command synchronizes the "nsddata_timeseries" directory, excluding ALL files except for the "eyedata" files (using a wildcard mechanism). Remove the --dryrun flag if all looks good.

The AWS CLI includes many customizable flags. Some flags that may be useful include --size-only, --exact-timestamps, and --delete.