... | ... | @@ -96,14 +96,23 @@ This signifies for the job to be completed locally. |
|
|
This signifies for the job to be completed on the ____________server.
|
|
|
|
|
|
**2. Replace String**
|
|
|
* text
|
|
|
|
|
|
* Replace strings is one of the most important aspects of batch_context. Replace string is the communication between the config file, the script and the data files. You will need to create a string similar to the one below, initialising variables to be used in each of the loaded data files.
|
|
|
For Example, the BUCANL pipeline *scalpart.htb* script looks for the variable *[in_path]* which you will have created in the swap string field like so:
|
|
|
![swapstringpopupgoodquality](/uploads/004e1ce8d684ed5c9315bc2ca34b3d41/swapstringpopupgoodquality.png)
|
|
|
The script can now load the correct files as it has the in path directory. To find out which variables the config file must set, look in the comments of the scripts.
|
|
|
![swpstrpart2](/uploads/5d92049f629fcc35bdcbbdf2a4193fa1/swpstrpart2.png)
|
|
|
Swap string is also helpful as you can change variables like filter strengths by simply editing the text config files, rather then changing the script code.
|
|
|
* **[batch_dfn]** is a special swap string as it is not collected from the config file. Instead it is the name of the current data file the script is analysing.
|
|
|
|
|
|
**3. Order**
|
|
|
* text
|
|
|
|
|
|
* The order of the script is an important part of creating a script pipeline.
|
|
|
When running a job locally your computer will focus on one task at a time so your script order will be:
|
|
|
* [1] [2] [3] [4] and so on...
|
|
|
When running a job remotely you have the option to sub jobs in parallel in order to increase the jobs efficiency. The field input will be an array, where the first value is an arbitrary number, that helps denotes the location of the script in the pipeline. Where number 1 is first and the largest number is last. Two scripts can have the same identifier. The next values in the array denote what other scripts must be complete before this one starts.
|
|
|
In the example below the first script is number [1]. After it is complete both script [2,1] and [5,1] start. Once [2,1] is finished [3,2] starts. Once [3,2] finishes both parallel processes [4,3] start. By this time our longer script [5,1] will finish. Script [6,4,5] will wait until both parrallel [4,3]'s have been completed and [5,1] has been completed.
|
|
|
![smallerorderchart](/uploads/9d8f415d657d92c96647a31efe17245c/smallerorderchart.png)
|
|
|
***
|
|
|
|
|
|
This order structure allows for infinite combinations and layouts of pipelines.
|
|
|
**4. Job Name**
|
|
|
* text
|
|
|
|
... | ... | |