|
|
|
[[_TOC_]]
|
|
|
|
|
|
|
|
# Diagnostics
|
|
|
|
Throughout the pipeline the scripts make many decisions that result in marks on both channels and time segments. The criteria for these decisions are laid out in the configuration files, and the results and the intermediate stages are saved in the EEG structure. These matrices can be easily access and plotted to get a visual representation of what is happening.
|
|
|
|
Throughout the pipeline the scripts make many decisions that result in marks on both channels and time segments. The criteria for these decisions are laid out in the configuration files, and the results and the intermediate stages are saved in the EEG structure. These matrices can be easily access and plotted to get a visual representation of what is happening behind the scenes.
|
|
|
|
|
|
|
|
To visualise some of the choices that were made in the pipeline either create your own plotting functions, or try running the diagnotics script.
|
|
|
|
To visualise some of the choices that were made in the pipeline either create your own plotting functions, or try running the diagnostics script. To learn more about how data structures and calculations see the end of this section.
|
|
|
|
|
|
|
|
**Using the Diagnostics Script**
|
|
|
|
Open up a EEG dataset that has complete the dipfit stage of the pipeline to get all of the available figures. Most typically these will have the ending *_dip.set.* Make sure that the diagnostics.m file is on the MATLAB path. In the matlab editor call the diagnostics function in the MATLAB editor:
|
|
|
|
```
|
|
|
|
diagnostics
|
|
|
|
```
|
|
|
|
By typing the function name you are calling the diagnostics script. This function does not contain any input values but directly starts accessing your loaded dataset. Figures will be generated and you can browse them as you would like. The figures contain some visual representations of the calculations that are being preformed based on the quality of the data file. Follow along below for a more in depth view of some of the figures.
|
|
|
|
By typing the function name you are calling the diagnostics script. This function does not need any input values but directly starts accessing your loaded dataset. Figures will be generated and you can browse them as you would like. The figures contain some visual representations of the calculations that are being preformed based on the quality of the data file. Follow along below for a more in depth view of some of the figures.
|
|
|
|
|
|
|
|
# Data Quality
|
|
|
|
The first plot is a general overview of how much good data has been found in your data set. If the % of time and the % of channels of your data is very high you might want to reconsider your experiment modality. From the figure below:
|
|
|
|
The first plot is a general overview of how much good data has been found in your data set. If the % of time and the % of channels removed from your data is very high you might want to reconsider your experiment modality. From the figure below:
|
|
|
|
|
|
|
|
|
|
|
|
|
| ... | ... | @@ -21,7 +21,7 @@ Looking into this example further you can see from the editor printout that ther |
|
|
|
* 41 bad channels out of 135
|
|
|
|
* ~21% of data time flagged as bad
|
|
|
|
|
|
|
|
The designation of *Bad* is based off a few simple calculations which parameters are controlled by the configuration files. We will look at these calculations in more detail below, and go through which configs parameters change each of the criteria.
|
|
|
|
The designation of *Bad* is based off a few simple calculations based on parameters in the configuration files. We will look at these calculations in more detail below, and go through which configs parameters change each of the criteria.
|
|
|
|
|
|
|
|
|
|
|
|
|
| ... | ... | @@ -30,7 +30,7 @@ The designation of *Bad* is based off a few simple calculations which parameters |
|
|
|
|
|
|
|
# Channel Flags
|
|
|
|
Channel flags are the way that channels are identified and staged for removal. A few examples of channel flag calculations are:
|
|
|
|
* Standard Deviation (Comicaly Bad)
|
|
|
|
* Standard Deviation (Comically Bad)
|
|
|
|
* High correlation (Bridging)
|
|
|
|
* Low Correlation
|
|
|
|
|
| ... | ... | @@ -40,13 +40,13 @@ Diagnostics will present the data for the Standard Deviation and the Low correla |
|
|
|
The figures below are separated into 3 sub figures to show how the calculations are done.
|
|
|
|
For example lets review the Standard deviation plot (Figure: 4) subplots from left to right.
|
|
|
|
* Sub Figure 1
|
|
|
|
This figure is the raw standard deviation values for each channels for every epoch of time. The colour represents the magnitude of the deviation there is at each of these time points. In this particular case the dark red colour means there is a high deviation and the blue means lower, but the exact colouring will depend on your matlab plot settings.
|
|
|
|
This figure is the raw standard deviation values for each channels for every epoch of time. The colour represents the magnitude of the standard deviation there is for each of these epochs. In this particular case the dark red colour means there is a high deviation and the blue means lower, but the exact colouring will depend on your Matlab plot settings.
|
|
|
|
* Sub Figure 2
|
|
|
|
This plot expands on the values from the first sub figure. In this plot only the very highest values are selected from plot 1 and plotted as a 0 or 1. In this particular case blue means that the designated epoch was not over the specified criteria. You can adjust the criteria of the flagging by adjusting variables:
|
|
|
|
* ```[sd_chan_z] ``` for standard deviation. Default is 2.326 deviation, a higher number will produce less marks, lower will flag more.
|
|
|
|
This plot expands on the values from the first sub figure. In this plot only the very highest values that pass the channel **z** criteria are selected and plotted as a 1. In this particular case blue means that the designated epoch was not over the specified z criteria. You can adjust the z criteria of the flagging by adjusting variables:
|
|
|
|
* ```[sd_chan_z] ``` for standard deviation. Default is 2.326 deviations, a higher number will produce less marks, lower will flag more.
|
|
|
|
* ```[r_chan_z]``` for correlations. Default is 2.326, a higher number will produce less marks, lower will flag more.
|
|
|
|
* Sub Figure 3
|
|
|
|
The third plot uses the results from the second plot. It adds up the marks for each of the channels and finds the % of each channel that was flagged. Looking at the example below the red segment had a high deviation, and produced a high % of the channel being covered in flags. This causes the waveform on the right to peak at this channel. This peak crosses the threshold vertical line, meaning the channel has to much time in which it is comically bad. The criteria line can be adjusted left and right to allow for more or less acceptable deviation. You can adjust the criteria of the flagging by adjusting variables like:
|
|
|
|
The third plot uses the results from the second plot. It adds up the marks for each of the channels and finds the % of each channel that was flagged. Looking at the example below the red segment had a high deviation, and produced a high % of the channel being covered in flags. This causes the waveform on the right to peak at this channel. This peak crosses the z threshold vertical line, meaning the channel has to much time in which it is comically bad. The criteria line can be adjusted left and right to allow for more or less acceptable deviation. You can adjust the criteria of the flagging by adjusting variables like:
|
|
|
|
* ```[sd_chan_p] ``` for standard deviation. Default is 0.1(10%) of the channel has to be flagged with the ```[sd_chan_z] ``` mark, a higher number will make the % required larger and thus less channels will be flagged.
|
|
|
|
* ```[r_chan_p]``` for correlations. Default is 0.1(10%)of the channel has to be flagged with the ```[r_chan_z] ``` mark, a higher number will make the % required larger and thus less channels will be flagged.
|
|
|
|
|
| ... | ... | @@ -55,12 +55,12 @@ The third plot uses the results from the second plot. It adds up the marks for e |
|
|
|
* Example 1:
|
|
|
|
* ```[sd_chan_z],2.236 ``` to ```[sd_chan_z],2 ```
|
|
|
|
* ```[sd_chan_p],0.1 ``` to ```[sd_chan_p],0.5 ```
|
|
|
|
This would lower the criteria for making the flags, and lower the amount that each channel needs to be flagged in order to marked as comically bad. This would flag more channels, and do **more removals** - potentially cleaning the data more.
|
|
|
|
This would **lower the criteria** for making the flags, and lower the amount that each channel needs to be flagged in order to marked as comically bad. This would flag more channels, and do **more removals** - potentially cleaning the data more, but reducing the amount of data.
|
|
|
|
|
|
|
|
* Example 2:
|
|
|
|
* ```[sd_chan_z],2.236 ``` to ```[sd_chan_z],3 ```
|
|
|
|
* ```[sd_chan_p],0.1 ``` to ```[sd_chan_p],0.2```
|
|
|
|
This would increase the criteria for making the flags, and increase the amount that each channel needs to be flagged in order to marked as comically bad. This would flag less channels, and do **less removals** - potentially reducing the data quality.
|
|
|
|
This would **increase the criteria** for making the flags, and increase the amount that each channel needs to be flagged in order to marked as comically bad. This would flag less channels, and do **less removals** - potentially reducing the data quality, but increasing the amount of data.
|
|
|
|
|
|
|
|
|
|
|
|
|
| ... | ... | @@ -69,10 +69,13 @@ This would increase the criteria for making the flags, and increase the amount t |
|
|
|
**Yellow:** Little deviation is visible, no flags are made, channels are under the % criteria and not marked.
|
|
|
|
**Green:** The % criteria threshold. Any channel that peaks past this line will be marked for removal. As indicated this line can be moved based on your goals.
|
|
|
|
|
|
|
|
For more information on these calculations see the [Pipeline Scripts Page](https://git.sharcnet.ca/bucanl_pipelines/eeg_pipe_asr_amica/wikis/pipeline-scripts#scalpart).
|
|
|
|
|
|
|
|
# Time Flags
|
|
|
|
Time flags work in a very similar way as the channels, but marks are place on time segments rather than channels. Since time is important to look at the figures have been rotated starting with 1 at the top and 3 at the bottom. A few examples of time flag calculations are:
|
|
|
|
Time flags work in a very similar way as the channels, but marks are placed on time segments rather than channels. Since time dimension is important to look at now, the figures have been rotated starting with 1 at the top and 3 at the bottom. A few examples of time flag calculations are:
|
|
|
|
* Low Correlation
|
|
|
|
* Independent Component Standard Deviation (ICSD) (usually several rounds)
|
|
|
|
* Alpha and Beta Power (Not in Manual Removals)
|
|
|
|
|
|
|
|
|
|
|
|
|
| ... | ... | @@ -80,14 +83,16 @@ Time flags work in a very similar way as the channels, but marks are place on ti |
|
|
|
This figure works the same as before with channel marks, containing the raw calculated values of the test preformed.
|
|
|
|
* Sub Figure 2
|
|
|
|
This plot expands on the values from the first sub figure, and works the same as with channels. Any raw calculated value that is over the z criteria will be flagged a 1, anything below will be a 0. To adjust this criteria change the following in the config files:
|
|
|
|
* ```[epoch_z] ``` in ```c01_scalpart.cfg``` for correlations . Default is 2.326 deviation, a higher number will produce less flags, lower will flag more.
|
|
|
|
* ```[epoch_z]``` in ```c03_compart.cfg``` (or c6,c13,c14) for ICSD. Default is 2.326, a higher number will produce less flags, lower will flag more.
|
|
|
|
* ```[epoch_z] ``` in ```c01_scalpart.cfg``` for correlations . Default is 2.326, a higher number will produce less flags, lower will flag more.
|
|
|
|
* ```[epoch_z]``` in ```c03_compart.cfg``` (or c6,c13,c14) for ICSD. Default is 2.326 deviations, a higher number will produce less flags, lower will flag more.
|
|
|
|
* Sub Figure 3
|
|
|
|
The third plot uses the results from the second plot. It adds up the marks for each column of time and finds the % of each
|
|
|
|
time segment that was flagged between all of the channels combined. The percentage is displayed as a waveform travelling through time. When the wave spikes high it means there is a time segment where many channels are all detecting bad data, and if it passes the horizontal line it is passed your designated criteria. That time segment will then be flagged and marked for removal. This segment of time will be gone for all of the channels. The criteria line can be adjusted up and down to allow for more or less acceptable deviation. You can adjust the criteria of the flagging by adjusting variables like:
|
|
|
|
* ```[epoch_p] ``` in ```c01_scalpart.cfg``` for correlations. Default is 0.1(10%) of the channels in this time frame need to be flagged with the ```[epoch_z] ``` mark, a higher number will make the % required larger and thus less time segments will be flagged.
|
|
|
|
* ```[epoch_p]``` in ```c03_compart.cfg``` (or c6,c13,c14) for ICSD. Default is 0.1(10%) of the channels in this time frame need to be flagged with the ```[epoch_z] ``` mark, a higher number will make the % required larger and thus less time segments will be flagged.
|
|
|
|
|
|
|
|
For more information on these calculations see the [Pipeline Scripts Page](https://git.sharcnet.ca/bucanl_pipelines/eeg_pipe_asr_amica/wikis/pipeline-scripts#scalpart).
|
|
|
|
|
|
|
|
# Component Classification
|
|
|
|
Component classification can only be done on files that have successfully run through the complete dipfit script. It this script we make use of a plugin tool box developed by Laura Frolich called IC_MARC (classification of Independent Components of EEG into Multiple ARtifact Classes). This plugin will look at the components generated by AMICA, and attempt to classify them by their characteristics. It then saves the probability of the component belonging to one of the following categories.
|
|
|
|
* Neural
|
| ... | ... | |
