Transferring data from XNAT and Converting to BIDS format

In today’s post, I will cover transferring data from XNAT (storage server for MRI data) and converting it to BIDS format. This is an important precursor step to running fMRIPrep. Unfortunately, the steps I’ll walk through are specific to the Brown community

Before I get too deep into the weeds, I want to give a HUGE shoutout to the Behavioral Neuroimaging Core (BNC), who have provided the bulwark of these tools to the neuroimaging community. I wouldn’t have been able to implement half these tools without their support and heroic efforts to set up this infrastructure. Also, FYI for those following at home outside of the Brown community, all of these tools are available on Github and can be implemented using your own system.

Github Repo for XNAT Tools: https://github.com/brown-bnc/xnat-tools
Brown-Specific XNAT Portal: https://bnc.brown.edu/xnat/

Okay, so let’s get started.

First, our group has set up an XNAT, and neuroinformatics platform to store MRI data developed at Washington University in St. Louis. If you’ve worked with any of the Human Connectome Project Datasets or larger neuroimaging datasets generally, you may have used this interface before (e.g., IntraDB, CNDA, etc). It’s a great interface that allows the user to click buttons to look at what data they’ve stored after each scan. As someone who has spent a lot of time with this interface, I highly recommend it and think it’s super easy to use.

Once you’ve set up your project and your data are all stored, you should be all set to run the ‘xnat2bids’ function from the xnat tools singularity container.  The way it was set up for us, we need to extract the subject number and the accession number.  (See image below).

As a side note, there may be MRI sessions in which you have extra runs you don’t want to use, and there are numerous ways you can deal with that. I tend to prefer to not delete raw data and just remove data at a post-processing step. So for our study, we’re just specifying which blocks to use and not use (relevant for scripts later).

Finally, you want to set up your scripts. Here, I’ve created two loops – one that is more general and calls ‘xnat2bids’ for the “perfect” runs (i.e., the number of BOLD runs collected is as expected), and the “imperfect” runs (i.e. the number of BOLD runs collected is more than expected). For the imperfect/irregular runs, I’ve used the “skiplist” argument to skip the scan numbers that we do not want to use (see image above).

To run the script, you’ll want to navigate to the folder where the script is, and then rn it using sbatch (if you’re on a supercomputer) or bash (if you’re running locally or don’t have access to cluster). If the latter, make sure you set your computer to rocket launch for a period of time.

*NOTE*: By default, this script will prompt you to login your password for every subject. you can circumvent this by setting your XNAT_USER and XNAT_PASSWORD in your .bashrc file. Some more detailed instructions are here. Here’s a bit of code below.

# Navigate to home directory
cd 

# Edit the .bashrc file
gedit .bashrc

# Run the bashrc file to implemenet changes in current Termminal window
. ~/.bashrc

Here is what my .bashrc file looks like (without my password):

When your scripts are done, you should have your BIDS formatted data. You can check that your data are formatted corrected. In my dataset, I have T1mprage anatomical images (in anat folder), 8 functional task runs (in func folder), fieldmap images (in fmap folder).

A quick note on file naming conventions

What if my DICOM files are named differently across runs or from what I want to name them in after conversion?

Here, we’ve created a bidsmaps file to edit the filenames so they are all the same. If the naming convention is correct/consistent across all studies, you won’t need this file.

Well, that’s it for now. Please feel free to let me know if there’s anything else that should be added in terms of BIDS naming convention.

 

 

Advertisement

Running fMRIPrep on your BIDS-Compatible MRI Dataset

In this post, I’ll describe the steps for how I run fMRIPrep. (sorry that this is coming before some of the earlier steps, I’m currently in the middle of this so I figured I’d document this now before I forget. I’ll get to the earlier steps later, when I work with some additional data). As an aside, through some googling, I found this github repo  by a doctoral student to be very helpful for articulating some of these more details.

I tend to prefer to edit my scripts locally, so I will mount the server onto my computer so I have access to the files locally. Here is an example script below. Note that I’m using ‘singularity’ instead of ‘docker’, since I have access to fMRIPrep installed on a cluster. So if you decide to run fMRIPrep locally, you can user the docker command instead.

In my script, I plan to run both fMRIPrep and Freesurfer, with fieldmap correction (see also here) and ica-aroma.

singularity run --cleanenv \

   --bind ${bids_root_dir}/${investigator}/study-${study_label}:/data \

   --bind /gpfs/scratch/dyee7:/scratch \

   --bind /gpfs/data/bnc/licenses:/licenses \

   /gpfs/data/bnc/simgs/fmriprep/fmriprep-${fmriprep_version}.sif \

   /data/bids /data/bids/derivatives/fmriprep-${fmriprep_version}-nofs \

   participant \

   --participant-label ${pid} \

   --fs-license-file /licenses/freesurfer-license.txt \

   -w /scratch/fmriprep \

   --stop-on-first-crash \

   --nthreads 64 \

   --write-graph \

   --use-aroma \

Here are some arguments that are useful for consideration, especially if you don’t have fieldmaps and want to apply some signal distortion correction.

# If you don't want to run freesurfer
   --fs-no-reconall

# If you don't have field maps, or want to do fieldmap-less correction
   --use-syn-sdc
   --force-syn

 

If you want to do fieldmap correction, you’re going to need to add an “IntendedFor” in your .json files for your fieldmaps. Here, because I am applying gradient echo field map, I want to apply the “phasediff” compressed nifti to all of my functional runs, so I added this argument below in alphabetical order, because I’m only a little neurotic. You’ll want to check that this file also has two Echo Times, EchoTime1 and EchoTime2.

(*It is worth noting that if you have spinecho field maps, ideally would have collected both A-P and P-A direction, if you are alternating AP and PA directions in your functional acquisition, and need to apply the opposite direction of the fieldmap to your function run. This may be more relevant if you analyze any of the connection datasets like HCP and ABCD, which default collect spinecho field maps, as well as other files like SB-refs.)

(**It is also worth noting that while there has been discussion about whether to automate this process in fMRIPrep, because there is so much variability in how fieldmaps are applied, they are deciding (as of right now) not to implement some general purpose tool (See Forum Here). There are some folks who are creating custom scripts that crawl through their data to automate this process, but it seems reasonable to me that because of the diversity of how fieldmaps are used in preprocessing, having some type of automation be project-specific or lab-specific make more sense.

When you log on the VNC, you can open a terminal window and navigate to the directory where your scripts are located. I like to use the ‘gedit‘ function to check that script is correct.

gedit TCB-fmriprep_fieldmap2_ica.sh

This is what the text file should look like, and should match the script you were editing outside of the VNC. Don’t worry if the spacing is a bit off, though it may be aesthetically unpleasant, it still works.

To look at your available scripts before running, I always like to use an ‘ls’ function.

Hooray! you are ready to run your script. You can run the script using sbatch function (or bash if you are not working on a supercomputer).

sbatch TCB-fmriprep_fieldmap2_ica.sh

If you are submitting a job on a cluster, you will also want to check that your script submitted the job and check the status of your job. More details on the Oscar website can be found here.

# check the status or your queued jobs
myq

# check the status of ongoing or recently completed jobs
sacct

Occasionally you will run into ERRORS, in which the state will say “FAILED” or “INCOMPLETE”. When it says failed, it usually means something in your scripts (or the data) prevented the script from completing without issues. However, when you see this, don’t panic! You can easily investigate what happened by looking at your log and error files. These files live in a folder in the directory where your scripts are located. As you can see at the top of my script, I included n argument to output errors and logs by the job ID (%J), which you can see easily in your status table above. (As you can see, I have a lot of logs and errors lately…)

Navigate to the folder, and you can use “ls -lt” to order your files by time and date. I used the ‘head’ argument to list only the top 20 of these recent files, since I don’t want to look at a million in my terminal window.

You can use ‘gedit’ to open the log and error files, or you can just click them open if you have the server mounted onto your desktop.

Example Log File

Example Error File

Assuming that you don’t have any major errors and that your job is completed, then your ‘derivatives folder within your BIDS folder will have the preprocessed data.

There’s a nice HTML file that gives a summary of all of the preprocessing steps, as well as some nice text about how to include these steps in your manuscript.

Happy scripting and debugging! Let me know if there are any key steps that I’ve missed here. 

 

 

Getting started with fMRIPrep on a computer cluster

It’s currently day n of shelter-at-home of covid-19 pandemic times, and I’ve started to dig myself into a hole with fMRI preprocessing using some (relatively) fancy new tools on a fancy supercomputer cluster, which I, fortunately, have access to at Brown. One of the blessings (and/or curse, depending on how you view it) of being a postdoc is that despite these pandemic times, you realize you really have no serious responsibilities and much freedom. In some ways, you can potentially be disposable, but also potentially indispensable depending on the circumstances. That being said, one of the true perks of being a postdoc is that your primary job is to do a lot of data crunching and analysis (and writing), so as long as you have data, it’s not too bad of a gig, and as long as you have a generous PI who is willing to pay for you to analyze the data.

Okay, so given that abundance of time I have to dig into the weeds of fMRI preprocessing, I’ve decided it’s not a terrible time to start documenting some of these analyses adventures, in case it may potentially be useful to others. Also, a colleague of mine who blogs her work regularly (check out her blog on mvpa here) encouraged me to do this a while back when I was fidgeting with using syringe pumps in the scanner. (I highly discourage any PhD candidate from doing this, unless you really enjoy tinkering with equipment). Basically, I’ve spend the last decade of my life tinkering around with different fMRI analyses Softwares, and recently have decided to make the full switch to fMRIPrep to make my life easier — so I suppose that these blogposts in upcoming weeks (months? lets hope not…) will be directly most useful folks who know a think or two (or many) about fMRI, and may want to get their feet wet with using this cool innovative wrapper that makes preprocessing less of a headache. That, and I have a terrible memory, so this will hopefully helpful for me 5 to 10 years down the road. Anyways, since here goes nothing.

So, you want to get started with fMRIprep (on a computer cluster)? 

Some basics to get familiar with, if you have limited computer programming background:

• Docker: https://www.docker.com/
• Linux Command Basics: https://help.ubuntu.com/community/UsingTheTerminal
  (scroll down to File & Directory commands, this will help you navigate on the Terminal on your server)
• Bash Scripting: https://www.shellscript.sh/
• Some Text Editors for editing code (without gouging your eyes out):
  • Visual Studio: https://code.visualstudio.com/
  • Sublime: https://www.sublimetext.com/

Some Excellent Books on getting started with fMRI analyses:

• Handbook of Functional MRI (Poldrack): http://www.fmri-data-analysis.org/
• fMRI (Bandettini): https://mitpress.mit.edu/books/fmri
  (I’ve really loved reading this book, it provides an excellent historical overview of the field in the last 30 years. definitely an enjoyable read to novices and veterans alike!)
• Functional Magnetic Resonance Imaging (Huettel): http://sites.sinauer.com/fmri3e/index.html
  (This is a classic textbook, but incredibly dense! If you’re into the physics of fMRI, this is definitely the book for you)
• Statistical Analysis of fMRI Data (Ashby): https://mitpress.mit.edu/books/statistical-analysis-fmri-data-second-edition

Here are some more Brown-specific resources:

• XNAT: https://bnc.brown.edu/xnat/ (MRI Data Storage)
• Oscar: https://docs.ccv.brown.edu/oscar/ (Brown’s supercomputer, Note that if you are a Brown user, you will need to request an account)
• Behavioral Neuroimaging Core (BNC)User Manual: https://docs.ccv.brown.edu/bnc-user-manual/ 
• BNC Github Repository: https://github.com/brown-bnc/bnctools

fMRIPrep Resources

• fMRIprep Docs: https://fmriprep.readthedocs.io/en/stable/
  (I highly recommend reading this paper if you are interested in looking under the hood to understand which functions are being implemented)
• fMRIprep Github Repository: https://github.com/poldracklab/fmriprep
• Neurostars Forum: https://neurostars.org/( a great place to ask questions and/or see if your questions have already been answered by experts)

Since I’ll be working on our university’s supercomputer, I’ll likely provide more detailed instructions regarding those types of analyses, but if you plan to run some of these analyses from a local computer, you can easily apply much of the scripts using “bash” instead of “sbatch” commands, since you won’t have parallel computing available. The only downside is that your computer may sound like a rocket when you’re running such analyses, and it may take a longer time for your scripts to finish. I will probably have separate blog posts that address each of these steps as I do them, so stay tuned if your appetite for fMRI preprocessing is still growing. Nevertheless, hopefully, this will be useful to some of your hypothetical readers *fingers crossed.*

Some general high-level steps to consider

Step 1: Exporting Data and Converting to BIDS

• Brain Imaging Data Structure (BIDS): https://bids.neuroimaging.io/
  (basically a fancy way to implement a standard for naming files)
• Heudiconv: https://github.com/nipy/heudiconv
  (this is useful tool to incorporate to get your data into BIDS format)

Step 2: Validating that your MRI dataset is BIDS-compatible

• Getting Started with the BIDS Validator (easy):
  • https://github.com/bids-standard/bids-starter-kit/wiki/bids-validator-info
  • Online: https://bids-standard.github.io/bids-validator/https://bids-standard.github.io/bids-validator/
• Installing the Validator and running on your computer (medium/hard): 
  • BIDs Validator Github Repository: https://github.com/bids-standard/bids-validator

(Optional) Step 2a: Quality Control checks for your MRI dataset

• Run mriqc: https://mriqc.readthedocs.io/en/stable/
  (While this step is not required to run fMRIprep, it can be a helpful way to visualize and check that your data are of high quality!)

Step 4: Run fMRIprep on your MRI dataset using singularity or docker containers

• Singularity: https://fmriprep.readthedocs.io/en/stable/singularity.html
• Docker: https://fmriprep.readthedocs.io/en/stable/docker.html

It is worth noting that there are still a few quirks to this seemingly magical tool. Although fMRIPprep is pretty good automating most of the preprocessing steps, here are some things that it does NOT do automatically. It is still possible to implement these steps, it just requires a bit of tweaking, and such things would be good to consider in the data processing steam in the future.

• Spatial Smoothing
  • A useful reference: http://jpeelle.net/mri/image_processing/smoothing.html
    (This is just a fancy of way of saying we’re going to blur your data with a gaussian kernal to reduce false-positive results and improve signal to noise).

• Fieldmap correction
  • https://fmriprep.readthedocs.io/en/stable/sdc.html
    (These will differ depending on whether you have a gradient or spinecho fieldmap)

 

I think I’ve inundated you with enough links for now. I will probably update this post as I think of more resources. Feel free to leave a comment below for suggestions on resources or anything I may have missed!