... | ... | @@ -3,45 +3,48 @@ |
|
|
[[_TOC_]]
|
|
|
|
|
|
# Directories and Structure
|
|
|
The BIDS Lossless EEG pipeline is unsurprisingly, BIDS compliant. As such, data is to be stored in a subject-wise manor and should follow the standards introduced by BIDS. See below for examples, and [here](http://bids.neuroimaging.io/) for full BIDS documentation.
|
|
|
The BIDS Lossless EEG pipeline is unsurprisingly, BIDS-compliant. As such, data is to be stored in a subject-wise manner and should follow the standards introduced by BIDS. See below for examples, and [here](http://bids.neuroimaging.io/) for full BIDS documentation.
|
|
|
|
|
|
The goal of this page is to provide a working understanding of how the *pipeline* is configured. As such, when folder contain external dependencies please refer to their respective documentation.
|
|
|
The goal of this page is to provide a working understanding of how the *pipeline* is configured. As such, when folders contain external dependencies please refer to their respective documentation.
|
|
|
|
|
|
## Main Pipeline Directory
|
|
|
Baseline initialization scripts and remote configuration scripts can be found at the highest level directory.
|
|
|
The main pipeline directory initially contains the `code` and `derivatives` folders. Initialization scripts and remote configuration setup scripts can be found in the `code` folder. The files that have been processed by the *pipeline* should end up in the `derivatives/lossless/` folder in a BIDS-compliant subject-wise folder structure (**e.g.** `sub-01/ses-01/eeg/sub-01_ses-01_eeg_ll.set`). The initialized BIDS-compliant files should be in an identical subject-wise folder structure in the top-level directory along with the `code` and `derivatives` folders.
|
|
|
|
|
|
|**Filename**|**Explanation**|
|
|
|
|**Directory**|**Explanation**|
|
|
|
|-------------|-------------|
|
|
|
|code| BIDS Definition: This folder should contain the source code of scripts that prepared the dataset for running on the pipeline.
|
|
|
|derivatives|BIDS Definition: Derived data storage location. Can contain multiple derivatives from one set of subjects. The pipeline is one such derivative, and is contained with. See below for pipeline information.|
|
|
|
*Subject files are omitted even though this is the directory they would be found as subjects are independent of pipeline structure.*
|
|
|
|code|This folder should contain the source code of scripts that prepared the dataset for running on the pipeline.|
|
|
|
|derivatives|Derived data storage location. Can contain multiple derivatives from one set of subjects. The pipeline is one such derivative (*lossless*), and is contained within. See below for pipeline information.|
|
|
|
|sourcedata|This folder can be created by the user and should contain all the raw data files and accompanying articles, scripts, and other documentation related to acquisition. Currently, there is no standard for the folder structure within this directory.|
|
|
|
|sub-*|Subject folders containing the initialized BIDS-compliant data files.|
|
|
|
|
|
|
## bids/derivatives/lossless
|
|
|
|**Filename**|**Explanation**|
|
|
|
## derivatives/lossless
|
|
|
|**Directory**|**Explanation**|
|
|
|
|-------------|-------------|
|
|
|
|sub-??|Subject folders containing the final output of the pipeline. Refer to bash config scripts to determine which .set file is which.|
|
|
|
|code|Contains all necessary scripts and configuration files required to run the *lossless pipeline*. See section following the log section.|
|
|
|
|log|Contains useful debugging information and necessary intermediate pipeline files. See below section for more details.|
|
|
|
|code|See section following the log section.|
|
|
|
|sub-*|Subject folders containing the final output of the pipeline. Refer to bash config scripts to determine which .set file is which.|
|
|
|
|
|
|
## bids/derivatives/lossless/log
|
|
|
## derivatives/lossless/log
|
|
|
Log is automatically populated with scripts when you execute the pipeline. Folders are generated for every script identified by the script that was run followed by the date and time of the scripts execution. These folders contain:
|
|
|
* A specific **.m** file for each subject data file.
|
|
|
* A specific **.err** file for each subject if an error occurred.
|
|
|
* A specific **.log** file for each subject that records the remote system command window.
|
|
|
* A single **.sh** submit script that contains the job commands that were sent to the remote servers. (Remote Jobs Only)
|
|
|
**Note:** When debugging errors this is the most important folder as you can see what is happening by using these four files.
|
|
|
* A single **.sh** submit script that contains the job commands that were sent to the remote servers. (Remote Jobs Only)
|
|
|
|
|
|
## bids/derivatives/lossless/code
|
|
|
**Note:** When debugging errors, this is the most important folder as you can see what is happening by using these four files.
|
|
|
|
|
|
## derivatives/lossless/code
|
|
|
This folder contains the body of the pipeline and houses all scripts and configurations.
|
|
|
|
|
|
|**Filename**|**Explanation**|
|
|
|
|**Directory**|**Explanation**|
|
|
|
|-------------|-------------|
|
|
|
|config|This folder contains all of the configuration files and init files that are needed for the EEGLAB plugins. Configuration files will have a **.cfg** or **.txt** extension. The naming of each configuration file should mirror the script that it goes with.|
|
|
|
|dependencies|Contains packages that the pipeline are dependent on. All folders within should be their own git repository, with their own documentation.|
|
|
|
|misc|Miscellaneous support pieces such amica parameter files, EEG location models, and event renaming files are stored here.|
|
|
|
|config|This folder contains all of the configuration files and remote initialization files that are needed for the EEGLAB plugins. Configuration files will have a **.cfg** or **.txt** extension. The naming of each configuration file should mirror the script that it goes with.|
|
|
|
|dependencies|Contains packages that the *pipeline* is dependent on. All folders within should be their own git repository, with their own documentation.|
|
|
|
|misc|Miscellaneous support pieces such amica parameter files, EEG montage files, and event renaming files are stored here.|
|
|
|
|scripts|Folder containing all of the script files that make up processes behind the pipeline. These have the extension **.htb** and are called **H**istory **T**emplate **B**atch as they are executed in the *batch_context* extensions.|
|
|
|
|tools|Various script files to be used as in-house optimizers or cleanup scripts for remotes.|
|
|
|
*Note: Excluded files at this level are git repository specific and have no actual effect on the pipeline*
|
|
|
|tools|Various script files to be used as in-house optimizers, diagnostics, or cleanup scripts for remotes.|
|
|
|
|
|
|
**Note:** Excluded files at this level are git repository specific and have no actual effect on the pipeline.
|
|
|
|
|
|
|
|
|
***
|
... | ... | |