MRI data quality

MRI data quality

Introduction

  • The topic here is commonly encountered issues/problems that often come up when collecting, analyzing, and visualizing MRI and fMRI data, as well as potential solutions.
  • Bottom line: Yes, there are a million things that can go wrong and will go wrong unless you think about it and prevent it and monitor it.
  • Here is a  video walkthrough where Kendrick talks through an inspection of a sample fMRI scan session .

Types of errors/noise/artifacts:

  • Head motion - Basically, transient head motion (e.g. hiccups, jerks, deliberate shift of one's head position, swallowing) is going to cause irreparable intensity fluctuations ("noise") in your time-series data
  • For anatomical imaging, even slow drifts in head position are really bad (since your images basically will lose sharpness/detail).
  • For fMRI, slow drifts are less catastrophic.
  • Thermal noise - So-called "unstructured" noise that can be averaged out by, e.g., spatial smoothing
  • Physiological noise - related to breathing and/or heart rate → generally these are "structured" effects in the data.
  • Eyeblinks (for vision studies)
  • Fixation ability (for vision studies)
  • Display issues (e.g. blurry mirrors, not the whole screen visible, reflections from the bore)
  • Scanner acoustic noise (for auditory studies)
  • Auditory calibration issues (e.g. volume variations, ear bud insertion variations, etc.)
  • Timing errors (e.g. synchronization isn't perfect, accumulation of timing errors over time)
  • Cognitive compliance (e.g. falling asleep, not doing the task you asked them to)
  • Really poorly designed stimuli (or experimental conditions) and/or poorly sequenced experimental trials/runs.
  • MRI imaging artifacts
  • Things that should NOT exist: stripes in your images, some darkening, some holes, overly bright regions, ghosts, wavy patterns, RF coil element failures, scanner failed to reconstruct images for you, missing brain volumes in a long scan, temporal instabilities (e.g., a few brain volumes look "weird" and then it goes away)
  • Things that are inevitable (for GE-EPI fMRI sequences): signal dropout, EPI distortions, some amount of intensity inhomogeneity, some amount of noise
  • Errors in acquisition and/or pre-processing:
  • Too aggressive skull stripping
  • Tissue (or image artifacts) surrounding the brain causing errors in segmentation and/or surface reconstruction
  • High levels of thermal noise (can confuse segmentation)
  • Inhomogeneties across the image (can confuse segmentation)
  • Image artifacts (can confuse segmentation)
  • Run-to-run stability can be low, which adds unnecessary noise to fMRI data

How can we prevent bad data in terms of data collection?

  • Instruct your subjects really really well and forcefully. Show them the effect of head motion in terms of "bad brain images" vs. "good brain images"
  • Recruit experienced and compliant subjects
  • Use mock scanners (that include head cameras) that can show subjects how much they are actually moving.
  • Actually do serious training of subjects? You can think of this in terms of head motion, or in terms of fixation monitoring, etc.
  • Have your subjects practice on your experiment!
  • Incentive good performance? Give verbal feedback? Tell the subject how well they did in the run. If you notice them falling asleep, politely tell them wake up (or give them a mini-break to micro-nap).
  • Instruct your subjects to eat well, sleep well, drink well, go to the bathroom beforehand, drink caffeine?, do jumping jacks?, etc.??
  • Set up foam padding well
  • Use headcases?
  • Practice your data collection procedures on a phantom. Get it down smoothly. Have your data collection protocol streamlined so your subjects don't get mad and antsy.
  • Actually carefully look at the images as they come in. Don't just look once and assume everything is fine for the rest of the session.
  • If you notice a problem, then you can collect another run (e.g. for an anatomical scan).
  • Maybe have a distracting movie for subjects to watch (e.g. during anatomy) to help reduce head motion.
  • Play it conservative with your pulse sequence (use tried-and-true protocols).
  • Be careful about trying to do too much in your experiment (too many different novel directions, trying to collect too much data).
  • Be aware that if your experiment is too long or too hard, subjects will get tired or mad or even leave.
  • For calibration and timing stuff, test and retest and retest and measure your experiment; or, if you don't know much about these things, talk to experts. In your stimulus presentation software, RECORD everything that you can with as much precision as you can bear.
  • For experimental / cognitive design stuff, talk to someone who knows about these things! For fMRI there is a lot of a domain-specific knowledge that is necessary.
  • Do real, full-fledged tests of your ENTIRE experiment (not just "make sure it runs for the first run").
  • Actually watch/listen to your experiment in the actual scanner. Maybe actually do your own experiment (completely).
  • If you aren't an expert in MRI data, ask someone to confirm that your scan plan is good and complete (e.g. fieldmaps, T1, pulse sequence parameters, etc.).
  • Log everything and have a game plan. Do you know all of your pulse sequence parameters? Do you know every single trial/stimulus/run/acquisition order?
  • Consider making a checklist for the data collection procedures, and follow it to make sure you do everything as intended.
  • Consider marking a few sessions as "pilots" and analyze those data to assess them before pulling the trigger and collecting many sessions.
  • DO NOT blindly record data and think that you will just "analyze it" later.

How can you confirm that data are "good"?

  • Behavior. Analyze the behavioral data and actually see if subjects performed well.
  • MRI image quality/pre-processing.
  • Look at it. (Of course, this requires expertise, and of course, doing this at scale requires technical prowess.)
  • Use some automated quantitative tool to quantify quality and compare these numbers across your subjects / runs. Even if you don't what value is "good", you can at least flag bad/suspicious subjects / runs.
  • A common metric for fMRI data is tSNR (mean of a voxel divided by its standard deviation over time). But note that it tells you something but not everything.
  • Head motion. You can just plot head motion traces. And/or you can compute framewise displacement (FD) or DVARS and these are highly straightforward to see what's good and what's bad. You can also generate carpet plots.



  • Timing/experiment design issues. Assuming you log everything that happened, run some simple checks of what occurred. For example, if you have empirical timing results, you can check that empirically the experiment was for exactly as long as you intended (within some tolerance).
  • BOLD SNR issues / statistical power. Typically, we just compute a metric that is sensitive to "experimentally-driven" BOLD changes (variance explained by a model of the time-series; test-retest correlation of identical runs; computing statistical metrics (e.g. t-values) for response changes that you expect to see; etc.), and you look at it across voxels, runs, subjects, sessions.
  • Be aware of the difference between strong BOLD responses in general (i.e. your voxel reliably goes up in its BOLD response to all of your experimental conditions) vs. actual differences in the level of BOLD responses across your conditions. These are distinct concepts.
  • Analysis ideas. Shuffle your data and see how things look. Create synthetic "noise" datasets to see what that looks like.

Convenient tools/packages/ideas/methods to assess quality

  • MRIQC
  • fmriprep generates figures
  • Generate your own.
  • Idea: to assess quality, ask someone else with expertise to look at your data/figures
  • QA checks for neuroimaging data:  https://youtu.be/fvv2dr3pT7I 

Visual inspection

  • Visual inspection is probably the best and most effective way to diagnose and assess MRI data quality (assuming you have sufficient experience doing this).
  • Here is a recording of a special-topics session where we look at and inspect various MRI volumes:  20230223 [General visual inspection of MRI/fMRI data quality]