... | ... | @@ -73,7 +73,7 @@ A detailed description of each of the fields can be found below: |
|
|
* text
|
|
|
|
|
|
## Saving
|
|
|
Once you have filled out all of the fields click on the |`Save As`| button to save the file. It is recommended that this file's name includes the project and the user. (ie. *adlofts_eyespilot2016_contextconfig.cfg* ) Make sure to place it in support/config.
|
|
|
Once you have filled out all of the fields click on the |`Save As`| button to save the file. It is recommended that this file's name includes the project and the user. (ie. *adlofts_eyespilot2016_contextconfig.cfg* ) Make sure to place it in support/config directory.
|
|
|
Now when running a job you can simply load this file and it will populate the required fields for you. You can also use the text file created as a template to quickly edit and create new context config files in a text editor.
|
|
|
|
|
|
# Job configuration files (*.cfg)
|
... | ... | @@ -89,7 +89,8 @@ You can create a new or edit and old batch config in this user interface. To loa |
|
|
A detailed description of each of the fields can be found below:
|
|
|
***
|
|
|
|
|
|
**1. Execute Function**
|
|
|
**1. Execute Function**
|
|
|
This is a drop down menu where you can select one of the following:
|
|
|
* Current Base
|
|
|
This signifies for the job to be completed locally.
|
|
|
* Sq Sub
|
... | ... | @@ -98,7 +99,10 @@ This signifies for the job to be completed on the ____________server. |
|
|
**2. Replace String**
|
|
|
* Replace strings is one of the most important aspects of batch_context. Replace string is the communication between the config file, the script and the data files. You will need to create a string similar to the one below, initialising variables to be used in each of the loaded data files.
|
|
|
For Example, the BUCANL pipeline *scalpart.htb* script looks for the variable *[in_path]* which you will have created in the swap string field like so:
|
|
|
![swapstringpopupgoodquality](/uploads/004e1ce8d684ed5c9315bc2ca34b3d41/swapstringpopupgoodquality.png)
|
|
|
![swapstringpopupgoodquality](/uploads/004e1ce8d684ed5c9315bc2ca34b3d41/swapstringpopupgoodquality.png)
|
|
|
|
|
|
***
|
|
|
|
|
|
The script can now load the correct files as it has the in path directory. To find out which variables the config file must set, look in the comments of the scripts.
|
|
|
![swpstrpart2](/uploads/5d92049f629fcc35bdcbbdf2a4193fa1/swpstrpart2.png)
|
|
|
Swap string is also helpful as you can change variables like filter strengths by simply editing the text config files, rather then changing the script code.
|
... | ... | @@ -108,11 +112,16 @@ Swap string is also helpful as you can change variables like filter strengths by |
|
|
* The order of the script is an important part of creating a script pipeline.
|
|
|
When running a job locally your computer will focus on one task at a time so your script order will be:
|
|
|
* [1] [2] [3] [4] and so on...
|
|
|
When running a job remotely you have the option to sub jobs in parallel in order to increase the jobs efficiency. The field input will be an array, where the first value is an arbitrary number, that helps denotes the location of the script in the pipeline. Where number 1 is first and the largest number is last. Two scripts can have the same identifier. The next values in the array denote what other scripts must be complete before this one starts.
|
|
|
|
|
|
* When running a job remotely you have the option to sub jobs in parallel in order to increase the jobs efficiency. The field input will be an array, where the first value is an arbitrary number, that helps denotes the location of the script in the pipeline. Where number 1 is first and the largest number is last. Two scripts can have the same identifier. The next values in the array denote what other scripts must be complete before this one starts.
|
|
|
In the example below the first script is number [1]. After it is complete both script [2,1] and [5,1] start. Once [2,1] is finished [3,2] starts. Once [3,2] finishes both parallel processes [4,3] start. By this time our longer script [5,1] will finish. Script [6,4,5] will wait until both parrallel [4,3]'s have been completed and [5,1] has been completed.
|
|
|
![smallerorderchart](/uploads/9d8f415d657d92c96647a31efe17245c/smallerorderchart.png)
|
|
|
***
|
|
|
This order structure allows for infinite combinations and layouts of pipelines.
|
|
|
|
|
|
![smallerorderchart](/uploads/9d8f415d657d92c96647a31efe17245c/smallerorderchart.png)
|
|
|
|
|
|
***
|
|
|
|
|
|
* This order structure allows for infinite combinations and layouts of pipelines.
|
|
|
|
|
|
**4. Job Name**
|
|
|
* text
|
|
|
|
... | ... | @@ -126,17 +135,22 @@ This order structure allows for infinite combinations and layouts of pipelines. |
|
|
* text
|
|
|
|
|
|
**8. Software**
|
|
|
* text
|
|
|
This is a drop down menu where you can select one of the following:
|
|
|
* *Matlab* : select matlab for every script when running a job locally
|
|
|
* *Octave* : select octave when you are running matlab scripts remotely, octave is a free software than can be downloaded onto the remote system.
|
|
|
* *None* : select none when you are running a binary process remotely such as AMICA.
|
|
|
|
|
|
**8. Program Options**
|
|
|
* text
|
|
|
|
|
|
* Use the text *--traditional* when running Octave on a remote cluster. This will help octave run the same as matlab. Otherwise this field can be left blank.
|
|
|
|
|
|
***
|
|
|
|
|
|
These fields once filled out play a part in creating a *submit string* that is used by the remote servers to correctly process the scripts. These strings are created for each data file for each scrip, and stored in the log folder.
|
|
|
![sshsubmitline](/uploads/9775d9429b7326bf1c937e3a578eb121/sshsubmitline.png)
|
|
|
***
|
|
|
|
|
|
## Saving
|
|
|
Once you have filled out all of the fields click on the |`Save As`| button to save the file. It is recommended that the files are named to mirror their corresponding scripts as they used hand in hand. (ie. *S1_Script1.htb* and *C1_Config_for_Script1.cfg* )
|
|
|
Once you have filled out all of the fields click on the |`Save As`| button to save. If you have multiple files open *Ctrl - Click* to select as many as you would like to save. These will **not** be combined into one file. It is recommended that the files are named to mirror their corresponding scripts as they used hand in hand. (ie. *S1_Script1.htb* and *C1_Config_for_Script1.cfg* )
|
|
|
Now when running a job you can simply load this file and it will populate the required fields for you. You can also use the text file created as a template to quickly edit and create new batch config files in a text editor.
|
|
|
|
|
|
# Best Practices |
|
|
\ No newline at end of file |