... | ... | @@ -5,28 +5,28 @@ This EEGLAB extension provides an easy to use interface for generating script pi |
|
|
|
|
|
# Pipeline Basics
|
|
|
|
|
|
* **History Batch File (.htb)** is a matlab code skeleton that contains all of the function framework but all of the specifics are [swap_strings]. Once created they have only their specifics modified by adjusting a batch configuration file.
|
|
|
* **History Batch File (.htb)** is a matlab code framework that contains all of the function framework but all of the specifics are [swap_strings]. Once created they have only their specifics modified by adjusting a batch configuration file.
|
|
|
* **Pipeline** is a series of *History Batch File* scripts that are linked up so that the output of one is the input for the next. They can streamline what would be multiple processes into into one single batch execution.
|
|
|
* **Script (.m)** is also a matlab script but has a .m file type. It looks like a *History Batch File* but has been successfully string swapped according to the configuration file so that its specifics are unique for each data file. These contain the actual code that matlab will be preforming.
|
|
|
|
|
|
***
|
|
|
|
|
|
**Example:**
|
|
|
:b: The pipeline below is from the tutorial pipeline **eeglab_pipe_asr_amica**
|
|
|
The pipeline below is an example pipeline based off of the **eeglab_pipe_asr_amica** tutorial pipeline.
|
|
|
![mfilecreate2](/uploads/2dd1115ad018f519d6d1cdb381e4d5a3/mfilecreate2.png)
|
|
|
This pipeline has 7 .htb scripts (yellow and red).
|
|
|
If we were to run 3 data files through this pipeline we would automatically generate 3 x 7 = 21 unique .m scripts and run them each in succession.
|
|
|
If we were to run 3 data files through this pipeline we would automatically generate 3 x 7 = 21 unique .m scripts and run them each in succession. Locally these would be executed in matlab one at a time, but if you are using remote resources then each of the 3 pipelines will be independently scheduled.
|
|
|
|
|
|
# Swap String
|
|
|
String swapping is an essential aspect of creating a well functioning pipeline. It works by writing the scripts filled with strings in [ ] brackets rather than values. The script will not be able to run on its own, but when using it in conjunction with the configuration file and the batch context plugin you can rapidly create unique .m scripts for each of your dataset files. You can also adjust the the script functions without permanently changing the code.
|
|
|
Below we have a .htb pipeline script that is looking for 5 swap string inputs 4 from the config file, and 1 from the dataset file name. The configuration swap string section should look like this:
|
|
|
String swapping is an essential aspect of creating an efficient pipeline. It works by writing the .htb scripts filled with strings in [ ] brackets rather than values. The script will not be able to run on its own, but when used in conjunction with a configuration file in the batch context plugin you can rapidly create unique .m scripts for each of your data .set files. You can also adjust the the script function parameters without permanently changing the code.
|
|
|
Below we have a .htb pipeline script that is looking for 5 swap string inputs 4 from the custom configuration file, and 1 from the dataset file name. The configuration swap string should look like this:
|
|
|
```
|
|
|
[in_path],analysis/data/1_init
|
|
|
[low_bound],.1
|
|
|
[high_bound],30
|
|
|
[out_path],analysis/data/2_preproc
|
|
|
```
|
|
|
The swap string [batch_dfn] is special as it automatically takes its information from the loaded dataset files.
|
|
|
The swap string [batch_dfn] is one of a few special strings as it automatically takes its information from the loaded dataset files.
|
|
|
![swapstrtry3](/uploads/74dba0c532071b81d202a0c3aca7e3ef/swapstrtry3.png)
|
|
|
When these .m files are executed they will apply the script filters too each of the dataset files.
|
|
|
|
... | ... | @@ -38,29 +38,11 @@ You can have any number of swap strings and for any type of variable, as the str |
|
|
|
|
|
***
|
|
|
|
|
|
:b: The swap strings for *s1_scalpart.htb*, the first script in the pipeline.
|
|
|
|
|
|
```matlab
|
|
|
%replace_string comments:
|
|
|
% in_fname_key = [in_fname_key] - key string for the input file..[batch_dfn,_,-1]_mrg.set
|
|
|
% in_path = [in_path] - Relative path to input data files assuming cd = work_path
|
|
|
% recur_sec = [init_recur_sec] - Recurrence (sec) for the original artifact detection epoching (e.g. .5)
|
|
|
% limit_sec = [init_limit_sec] - Limits (sec) for the original artifact detection epoching (e.g. [-.5 0])
|
|
|
% out_path = [out_path] - Relative path to output data files assuming cd = work_path
|
|
|
% ref_loc_file = [ref_loc_file] - name of the file containing the reference channel locations (including the relative path)
|
|
|
% low_bound_hz = [low_bound_hz] - Low bound of the filter bass-band
|
|
|
% high_bound_hz = [high_bound_hz] - Upper bound of the filter bass-band
|
|
|
% recur_sec = [recur_sec] - Recurrence (sec) for the current artifact detection epoching (e.g. .5)
|
|
|
% limit_sec = [limit_sec] - Limits (sec) for the current artifact detection epoching (e.g. [-.5 .5])
|
|
|
% epoch_z = [epoch_z] - z threshold for flagging channel during icsd criteria (e.g. 3)
|
|
|
% epoch_p = [epoch_p] - Percentage of flagged components required to flag an epoch during icsd criteria (e.g. .1)
|
|
|
% epoch_trim = [epoch_trim] - Percentage trim for confidence intervals during channel neighbour r criteria (10 = 5% top and bottom)
|
|
|
% min_gap_ms = [min_gap_ms] - Minimum time (ms) to allow between periods marked for rejection
|
|
|
```
|
|
|
## Looking into the Example Further
|
|
|
This is the code taken from the graphic above. As you can see the [strings] have been replace with the correct 'strings' or values.
|
|
|
Below is the code representing the graphic above. As you can see the [strings] have been replace with the correct 'strings' or values. The first example is switching out for a string and the second is when switching out for a numeric.
|
|
|
**Loading Data Files:**
|
|
|
**Before string swap:** The strings [batch_dfn] and [in_path] are waiting to be replaced.
|
|
|
* **Before string swap:** The strings [batch_dfn] and [in_path] are waiting to be replaced.
|
|
|
|
|
|
```matlab
|
|
|
%%LOAD DATASET...
|
|
|
disp('Loading set file: [batch_dfn]...');
|
... | ... | @@ -70,7 +52,8 @@ EEG = eeg_checkset( EEG ); |
|
|
disp('TIME TO: LOAD DATASET...');
|
|
|
toc
|
|
|
```
|
|
|
**After string swap:** You can see that the previous [batch_dfn] and [in_path] have been exchanged out for the correct unique strings.
|
|
|
* **After string swap:** You can see that the previous [batch_dfn] and [in_path] have been exchanged out for the correct unique strings.
|
|
|
|
|
|
```matlab
|
|
|
%%LOAD DATASET...
|
|
|
disp('Loading set file: datafile1.set...');
|
... | ... | @@ -82,7 +65,8 @@ toc |
|
|
```
|
|
|
|
|
|
**Filling in Variables:**
|
|
|
**Before string swap:** The string [low_bound] is waiting to be replaced with a number.
|
|
|
* **Before string swap:** The string [low_bound] is waiting to be replaced with a number.
|
|
|
|
|
|
```matlab
|
|
|
%%FILTER HIGH PASS...
|
|
|
disp('High pass filtering the data...');
|
... | ... | @@ -93,7 +77,8 @@ EEG = eeg_checkset( EEG ); |
|
|
disp('TIME TO: FILTER HIGH PASS...');
|
|
|
toc
|
|
|
```
|
|
|
**After string swap:** You can see that the previous [low_bound] has been replaced with a number .
|
|
|
* **After string swap:** You can see that the previous [low_bound] has been replaced with a number .
|
|
|
|
|
|
```matlab
|
|
|
%%FILTER HIGH PASS...
|
|
|
disp('High pass filtering the data...');
|
... | ... | |