Installing and Using SPM Preprocessing Batch Script Templates
This document contains instructions for installing and using the SPM fMRI preprocessing template batch scripts. These scripts are designed to perform initial preprocessing on an fMRI image and a matched T1 anatomical image. The two images should be roughly in alignment before putting them into this preprocessing (if they were acquired during the same scan session the initial alignment should be okay). This preprocessing will produce a version of the fMRI data and a version of the T1 data spatially normalized to MNI-152 standard space ready to go into first-level analysis. These scripts will also produce a quality assurance (QA) HTML file for initial evaluation of the fMRI data.
The general flow of the preprocessing is as follows: install the necessary scripts and SPM modules, edit the template batch script to match the specifics of the data at hand, run one subject to make sure the SPM batch works, edit the template multi-subject processing script to match the specifics of the data at hand, run the multi-subject processing script.
NOTE: You’ll need to be able to log into and transfer files to/from the BIAC servers in order to have any access to your imaging data. If you have not set this up on your local computer, instructions for doing so can be found on the wiki here.
a. “Unzip”: The zip files associated with the batch script templates all contain directories of Matlab scripts and SPM modules. To unzip each of them, right click on the .zip file to be unzipped; select 7-Zip -> Extract Here. This will create an unzipped version of the directory in the same directory in which the .zip file is sitting.
b. “Add to Matlab path”: The “Matlab path” is simply a list of directories that Matlab will search to find functions the user attempts to call. In order for a new function or script to be run, it must either be saved into a directory that is already in this list, or the directory in which it resides must be added to the list. In most cases the cleanest way to handle new modules and scripts is the second option: keep them in a new directory and add that directory to the Matlab path. In the main Matlab window, the current directory is displayed along the top, just under the grey panel, in the thin white bar. The contents of this directory are displayed along the left-hand side of the window. The current directory can be changed by either double-clicking on directories in the left-hand panel or by clicking on parts of the directory path in the white bar. To add a directory to the Matlab path, right click on it in the left-hand panel and select Add to Path -> Selected Folders and Subfolders. This will add the directory to the Matlab path until Matlab is closed. When re-opening Matlab the directory will need to be added to the path again. Changes to the path can be saved between Matlab sessions using the Set Path button to bring up the path editing window and pushing the Save button or typing “savepath” into the Matlab command line. Either method will require administrative access on the computer for changes to actually be saved.
c. “SPM8 Install Directory”: The SPM install directory is the directory containing the actual spm.m script file that gets called when you type “spm” into the Matlab command line. It also contains sub-directories housing various SPM toolboxes and modules. Typing “which spm” into the Matlab command line should display the location of this directory (if instead it produces an error, odds are the SPM install directory has not been added to your Matlab path).
a. All the parts necessary to use the SPM fMRI Preprocessing Batch Templates are saved on Keoki in: …/Graner/Data/SPM_Preprocessing_Things
b. Copy Graner_Batch_Modules.zip to your local drive and unzip it. The unzipped directory contains Matlab scripts for implementing a quality-assurance module in SPM. Move the unzipped directory to a place on your hard drive in which you want such scripts to reside (if nowhere else comes to mind, this can go in the MATLAB directory). Add, with subfolders, the Graner_Batch_Modules directory to your Matlab path.
c. Copy SPM_Batch_Templates.zip to your local drive and unzip it. This directory contains templates of SPM batch scripts for running preprocessing of fMRI data. The directory should be moved somewhere near wherever you are planning on saving the output results (this is just a “good practice” suggestion; the directory can actually go anywhere, but it is often helpful to have analysis scripts near their associated outputs to keep a clear data/analysis trail).
d. Copy vbm8.zip to your local drive and unzip it. The unzipped directory contains the VBM8 module for SPM created by Christian Gaser (http://www.neuro.uni-jena.de/vbm/). It will be used to segment the anatomical T1 image and does a better job than the built-in SPM module. This directory must go in a specific place. Move it into the \toolbox directory that resides in the SPM8 install directory. (After the unzip process make sure there is only a single "vbm8" directory rather than a pair of nested "vbm8" directories.) After moving the vbm8 directory into \toolbox, you will need to either restart SPM or add, with subfolders, the vbm8 directory to your Matlab path.
a. Open Matlab, add the Graner_Batch_Modules directory to the Matlab path, and type “spm fmri” into the command line.
b. Click the “Batch” button on the bottom of the SPM window.
c. The drop-down menus across the top of the Batch Editor window should include one called “LocalMods” with “QA” and “ReadHdr” sub-options. If this menu is not there the Graner_Batch_Modules directory may not have been added to the Matlab path. There may also have been an error when trying to load the modules. If this were the case there should be some red error text in the main Matlab window.
d. The VBM8 module should show up in the SPM -> Tools dropdown menu. If “VBM8” is not an option in this sub-menu the vbm8 directory may not be in the correct place.
e. If the checks above all went smoothly, select the File -> Load Batch option in the batch editor window. Navigate to the SPM_batch_templates\SPM8\ directory, select fmri_batch_job.m and click Done. If everything is good-to-go the following list of modules should be listed in the left-hand portion of the window in this order:
3D to 4D File Conversion
Expand image frames
Realign: Estimate & Reslice
VBM8: Estimate & Write
In order for the multi-subject processing script to work properly, the subjects’ data for a given study must be inside a directory structure named such that the path to each subject’s fMRI and T1 image files is distinguished from that of other subjects only by that subject’s unique study ID. In other words, if one replaced all occurrences of the subject ID in a subject’s data path with the subject ID of another subject, the new path would be a valid path to the second subject’s data. An example of such a directory structure is:
In this example, replacing all occurrences of the first subject’s subject ID (“0001”) in that subject’s file paths and names produces the paths and names of the second subject’s (“0002”) files.
When data are initially saved on the BIAC server, they will be in a slightly different directory structure. However, the biac_create_preproc_dir.py Python script packaged in Graner_Batch_Modules.zip will create the directory structure required by the multi-subject processing script and copy the imaging data into it.
a. Create a copy of the biac_create_preproc_dir.py script in the BIAC study directory containing all of your imaging data (the directory on the BIAC server that also contains the directory called “Data”). This can be done by copying the script over to the BIAC server from your local computer.
b. Log into the BIAC server and enter interactive mode (see the wiki page here). Navigate to the study directory and type “python –m biac_create_preproc_dir.py” into the command line.
c. If anything goes wrong with the script it should print a message to the terminal window. The script will create a log file that starts with “biac_create_preproc_dir_log” and ends in a long time-stamp and “.txt”. This log file should be written to the study directory and will contain much more output about what the script did. Open this log file (type “nedit ./filename” into the terminal, where “filename” is replaced by the name of the log file) and look for any WARNINGs. If there are, some trouble-shooting may be necessary. If all else fails, ask John G. about it.
d. The default version of the script has a flag set so that it does not actually copy anything, but only writes to the log file what it WOULD HAVE done. This is so you can check to make sure it will do what it is supposed to and prevent anything dire from occurring. If the log file from part c looks good, edit the script so that it actually makes copies of the data. Type "nedit ./biac_create_preproc_dir.py", find the line (probably around line #36) that reads "actually_copy = 0", change the "0" to a "1", save the file, and close it. Now rerun the file using “python –m biac_create_preproc_dir.py”. Again, look for any WARNINGs in the new log file.
e. Look for a new “Preprocessing” directory in the study directory. This new directory should contain one sub-directory per subject, containing all of that subject’s imaging data. Briefly look through some of these sub-directories to make sure it looks like this is the case.
f. If everything looks good in the new Preprocessing directory, prepare to copy it and all of its sub-directories to your local computer. First, make sure there is enough room on your local computer to hold a copy of the data: type “du –h ./Preprocessing” on the command line. The first number that appears in the output from this command is the total disk space used by the Preprocessing directory. Check your local drive to make sure there is at least this much space plus 1 Gb free. If not, see if there is anything you can clean out…
g. Copy the Preprocessing directory to your local computer using either WinSCP (Windows) or the scp command with the “-r” option in a new terminal window (OSX). You can now carry out preprocessing/analysis on the local copy of the data. If something goes wrong and the local copy gets partially deleted or otherwise compromised, you can simply re-copy the data from the BIAC server.
The image files to be processed must be in NIFTI format. The functional data may be a single 4D NIFTI file or a series of 3D NIFTI files. The anatomical T1 data must be a single 3D NIFTI file.
The SPM preprocessing batch templates have many processing parameters set already but also require some information specific to the images to be processed. The following is a list of parameters you will need to enter into the template and information on how to extract them from images in NIFTI format (NOTE: The “private” field items listed below are not necessarily standard and may be missing or named differently. If they are absent, you may need to go back to the BIAC header (.bxh) file or, as a last resort, ask the MRI technologist who acquired the images about these parameters). You may already know what these are without needing to get them from the headers (but it never hurts to double-check).
In order to read the necessary information from the files, you’ll first need to load the headers into Matlab using some SPM functions. These functions are called from the Matlab command line (not via the SPM graphical interface). Navigate, in Matlab, to the directory containing the functional image to be processed. Enter “vol = spm_vol(FILENAME)” into the Matlab command line, with FILENAME replaced by the name of the NIFTI functional image file. This will load the image header information into the variable “vol.”
a. Slice Acquisition Order – Type “vol(1).private.diminfo.slice_time” into the Matlab command line. This should list four fields related to the order and timing of slice acquisition for the functional sequence. The “code” field should contain a string describing how the slices were acquired. It will most likely include either “alternating” or “sequential” and either “increasing” or “decreasing”. The first pair of words refers to the general order in which the slices were acquired. “Alternating” means half the slices of the brain were acquired in an “every-other” fashion (i.e. slice 1, then slice3, then slice 5, etc.) until the edge of the image was reached and then the other half were acquired, again in an every-other fashion. “Sequential” means all the slices were simply acquired in order across the image (NOTE: the word “order” in this sentence and the slice numbers in the parenthetical phrase of the previous sentence are referencing the spatial order of the slices, NOT the temporal order). The second pair of words refers to which slice was acquired first. “Increasing” means slice 1 was acquired first. “Decreasing” means slice N was acquired first, where N is the total number of slices in the image. Thus, if an image had 6 slices and was acquired “alternating_increasing” the slice order would be [1, 3, 5, 2, 4, 6]. If the same image had been acquired “sequential_decreasing” the slice order would be [6, 5, 4, 3, 2, 1]. To add to the fun, different imaging centers or different scanners may use different words for these slice acquisition patterns. For example, “alternating_increasing” may be referred to as “odd_up” or similar.
The “start” and “end” fields simply contain the first and last slices acquired.
The “duration” field contains the acquisition time of each slice, in seconds. This multiplied by the number of slices should equal the acquisition TR.
b. Number of Slices – Type “vol(1).dim(3)” into the Matlab command line. The displayed number should be the number of slices in the image. As a double-check, type “vol(1).dims” into the command line. This will display all three dimension sizes of the image. The number of slices will almost always be the one number that is smaller and different than the other two.
c. TR – Type “vol(1).private.timing.tspace” into the Matlab command line. The displayed number should be the TR of the image acquisition, in seconds.
d. TA – This is actually a value calculated from the TR and the number of slices and is equal to (TR – (TR/#slices)). The help box in the Slice Timing SPM module (where you’ll need to enter the TA) also has this equation listed.
e. Number of Initial Image Volumes Discarded from Functional Image (“disdaqs”) – During the first few volume acquisitions the fMRI BOLD signal will change rapidly because the longitudinal component of the collective magnetic moment of the hydrogen atoms within each voxel has not yet reached a steady state. Some imaging data have already had these volumes discarded while others have not. You will need to know if the functional data to be processed still contain these initial volumes or if they have been removed.
This section goes through each of the modules included in the “nophysio_nob0” batch template for SPM8 and contains information on which options need to be updated to carry out preprocessing on the specific data at hand. Modules with no fields needing updating are excluded from this list. The goal is to set up the batch script for the first subject to be run. Once the edited batch script has been set up for one subject, the multiple-subject-analysis script included in the SPM8_Batch_Templates directory can be used to run it on all the subjects of a study (see below).
NOTE: You will want to save the batch script with a different name in order to not overwrite the original template file!!! It is recommended that this new name somehow reflect the study or analysis in which the data will be used. When saving an updated version of the batch, use the File -> Save Batch and Script option; this will create both the batch script file and the associated “_job” file.
a. 3D to 4D File Conversion
i. 3D Volumes – This should be changed to include the volumes of fMRI data to be processed (any volumes acquired before steady-state should NOT be included here). If the fMRI data are in a single 4D file, individual volumes can be displayed in the file selection window by changing the “1” in the white box on the right-hand side of the window to “1:400” and hitting the tab key.
ii. Output Filename – This string should begin as a number followed by “_TASK_short.nii”. TASK should be replaced with some word by which you can identify the data (e.g. "EmoReg", "MemRecon", "DisgVigns"). Change the number in this string to the subject ID of the first subject you wish to run.
b. Slice Timing
i. Number of Slices – This should be changed to the number of slices per volume of the functional data.
ii. TR – This should be changed to the TR of the functional data.
iii. TA – As mentioned in the batch editor window help box, this value should be set to (TR – (TR/#slices)).
iv. Slice Order – This needs to be set to a list of integers representing the temporal order in which the slices were acquired. There are helpful instructions in the bottom box of the batch editor window on how to enter long lists easily.
v. Reference Slice – Unless you know you want to set it otherwise, this should be changed to the number of the slice that was acquired half-way through each TR. If the functional slices were acquired in “alternating_increasing” and there were 36 slices, this would be slice 35.
c. VBM8: Estimate and Write
i. Volumes – This should be set to the T1 anatomical image acquired in the same imaging session as the functional data.
ii. Tissue Probability Map – This needs to point to an image template sitting in a sub-directory of the SPM8 Install Directory (see section 0 above). Specifically, select the file: [SPM8 Install Dir]\toolbox\Seg\TPM.nii.
iii. Dartel Template – Similar to the Tissue Probability Map, this needs to point to an image template that exists in the VBM8 toolbox: [SPM8 Install Dir]\toolbox\vbm8\Template_1_IXI550_MNI152.nii.
d. Image Calculator (the first one)
i. Output Directory – Set this to the directory in which you want the preprocessed results written for the first subject of the study.
e. Image Calculator (the second one)
i. Output Directory – Set this, again, to the directory in which you want results written.
f. Deformations (the first one)
i. Output Directory – Set this to the same directory as in the Image Calculator modules.
g. Deformations (the second one)
i. Image to base ID on – When the functional data are spatially warped to standard space, the output will be matched to have the same voxel size and dimensions as the image entered here. If you do not have such an image, there is one called “EPI_resamp.nii” in the SPM_Batch_Templates directory. This image has a voxel size of 3.75 x 3.75 x 3.75mm3 and dimensions of 49 x 58 x 49. It is a resampled version of the MNI-152 EPI template that comes with SPM8.
ii. Output Directory – This should be set to match the output directory listed in the previous modules.
i. Orig_anat – This should be set to the same T1 anatomical image file as the “Volumes” item in the VBM8: Estimate and Write module.
ii. Orig_fmri – If the functional data are in a single 4D file, that file should be selected for this item. If the functional data are in multiple 3D files, all of those files should be selected here.
Once the above edits have been made to the batch script, it is time to test it. Since the script is already set up for the first subject in the study, simply hit the green arrow near the top of the Batch Editor window to start the script. It will probably take on the order of 20-30 minutes to complete, depending on your computer hardware. Throughout the processing, various pieces of information and progress notes will get displayed in the main Matlab window. Matlab may also pop up several dynamic figures of plots and progress bars. The processing also takes up a lot of Matlab’s resources, so opening and saving files in the editor may be very sluggish while it is running.
If everything went as planned, the last thing displayed in the Matlab window should include:
HTML file written: [FILE]
Running ‘Move/Delete Files’
Done ‘Move/Delete Files’
The final output directory will contain roughly 450Mb of files, depending on the sizes of the original image files.
The QArunall module creates a series of quality-assurance pictures, plots, and movies, and ties them together into a single HTML file. This file is located in the [OUTPUT_DIR]\QA_summary directory and is called “QA_summary_display.html”. Double-clicking the file should open it in your default web browser. More details on what’s included in this summary and information on determining if the subject has usable fMRI data can be found in the documentation file “QA_Output_notes.docx”. For now, if all the pictures look brain-shaped the batch script successfully ran to completion for this subject.
Unfortunately, the SPM graphical interface cannot run the processing batch script on multiple subjects. However, finding the occurrences of the subject ID in the “_job.m” file associated with the batch script, replacing them with the subject ID of the next subject, and rerunning the batch script is relatively straight-forward. This is basically what the “run_multi_spm8_template.m” script is written to do. As with the batch script processing, there are a few changes that will need to be made to the multi-subject script in order to make it work with the data at hand.
Open the run_multi_spm8_template.m file in the Matlab editor (in the main Matlab window, navigate to the …\SPM_batch_templates\SPM8 directory and double-click on the file). Save a copy of the file with a new name pertaining to the study/analysis at hand using the Save As option. You may want to save this new copy in the directory containing the edited batch script file.
The three things that need to be changed in the multi-subject script file are:
The function name (line 1) – This must be changed from “run_multi_spm8_template” to whatever the name of your copy of the file is called (minus the “.m”)
Subject_list (line 6) – This should be changed to include the subject IDs of all the subjects to be run. Each ID should be encased in single parentheses and followed by a semi-colon (except for the last subject ID, which should NOT have a semi-colon after it).
Base_dir (line 9) – This should be changed to the path/directory containing all the subject directories (see section above on how to best format the data directory structure).
File_to_edit (line 15) – This should be changed to the path/filename of the “_job.m” file associated with the edited version of the SPM batch script created in section 4 above.
There is one important “link” between the multi-subject script file and the batch script file that must be maintained for the multi-subject script to run properly. The batch script file must be set up to run the subject whose ID appears first in the batch script subject_list. There are a couple ways to handle this: 1) If the batch script was set up and run on the first subject in sections 4 and 5 above and no changes were made to the files in the resulting output, simply include the first subject in the multi-subject script’s subject_list. This will rerun the processing of that first subject, but this will not cause any issues. 2) If you do not want to rerun the processing of the first subject, open the “_job.m” file in the file_to_edit variable above and run a find/replace (Ctrl+f) replacing all instances of the subject ID of the subject who has already been run with the subject ID of the first subject in the batch script subject_list.
Once the multi-subject script has been edited, it can be run by simply navigating, in Matlab, to the directory containing the script file and typing the name of the file (excluding the “.m”) into the Matlab command line. The Matlab window should then display the first subject’s ID followed by the beginning of the processing batch output.
If all goes well, the script will run the processing for each subject once the previous subject is complete (thus, the entire script will take about 20-30 minutes per subject to complete; this is a good task for your computer to do overnight). If the processing fails for any subject for any reason, it will make note of it and move on to the next subject. When the script has attempted to run the script (successfully or not) for every subject in the list it will display two lists to the Matlab window: “Good Subjects” is a list of the subjects for which the processing did not encounter any programmatic errors (success here makes no comment on the quality of the data); “Bad Subjects” is a list of subjects for which the processing did not complete due to an error in attempting to run one of the processing modules.