Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions docs/source/behavior_table.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,16 +17,17 @@ Setting the Behavior
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Only one field can be set as the Behavior to test against. This is done by highlighting (clicking on)
the desired column and pressing the green button (marked "Set Behavior"). When clicked away from or another
column is clicked, the column should stay green. *Note: If a column is selected as a behavior or covariate(s),
it must have a nuermical value. No blanks or NaNs*
the desired column and pressing the green button (marked ``Set Behavior``). When clicked away from or another
column is clicked, the column should stay green.

**Note**: If a column is selected as a behavior or covariate(s), it must have a nuermical value. No blanks or NaNs

Setting Covariate(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Setting covariates is the same as setting the Behavior except that more than column may be selected.
Covariates can also be unselected. These actions are done by highlighting a column and pressing either of the two pink buttons
(marked: "Add Covariate" and "Remove Covariate"). These columns should be pink afterwards.
(marked: ``Add Covariate`` and ``Remove Covariate``). These columns should be pink afterwards.

Setting Permutation Groups
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Expand All @@ -38,4 +39,4 @@ To do this a column should be added to the behavior file. This column should be
per group. For example, one group would be 0, all of the subjects in this group should have a 0 in this column. The next group
would be 1, each subject in this group should have a 1. Continue this for the number of groups desired.

This column can then be selected by pushing the blue buttons (marked: "Add Permutation Group Level" and "Remove Permutation Group Level").
This column can then be selected by pushing the blue buttons (marked: ``Add Permutation Group Level`` and ``Remove Permutation Group Level``).
35 changes: 19 additions & 16 deletions docs/source/edge_level_tests.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,13 @@ Methods
-------------------------

The non-permuted method calculates the correlation of each Region Of Interest (ROI) to all other
ROIs via the given test. These results are stored as a correlation coefficient, ``coeff``, a p-value, ``prob``,
and a thresholded p-value, ``prob_sig``. The permuted method is identical except the variables have a ``_perm`` suffix.
ROIs via the given test. These results are stored as a correlation coefficient, ``coeff``, a *p*-value, ``prob``,
and a thresholded *p*-value, ``prob_sig``. The permuted method is identical except the variables have a ``_perm`` suffix.

Common Inputs
--------------------------

:P: Edge-level p-value threshold
:P: Edge-level *p*-value threshold
:Network Atlas: :doc:`Network Atlas </network_atlases>`
:Functional Connectivity: Initial coorelation matrix (r-values or Fisher z-transformed r-values) of size N\ :sub:`ROIs`\ x N\ :sub:`ROIs`\ x N\ :sub:`scans`\
:Behavior: MATLAB table (``.mat``) or tab seperated text file (``.txt``)
Expand All @@ -30,28 +30,29 @@ Common Inputs
After loading this file, the table should display in the GUI.
The user may mark one column as 'Behavior' for the score of interest.
Other columns may be marked as 'Covariates' which are partialed prior to running statistics.
(Note: Network Level Analysis cannot handle missing values for behavior or covariates. If there are ``NaNs`` or missing values, do not select this columns)

**Note**: Network Level Analysis cannot handle missing values for behavior or covariates. If there are ``NaNs`` or missing values in a column, using this column will result in errors

Provided Tests
--------------------------------

* **Pearson's r**

* MATLAB `corr <https://www.mathworks.com/help/stats/corr.html>` function with ``type``, ``Pearson``
* **Spearman's** :math:`\rho`\
* MATLAB `corr <https://www.mathworks.com/help/stats/corr.html>_` function with ``type``, ``Pearson``
* **Spearman's** :math:`\rho`

* MATLAB `corr <https://www.mathworks.com/help/stats/corr.html>` function with ``type``, ``Spearman``
* **Spearman's** :math:`\rho`\ **estimator**
* MATLAB `corr <https://www.mathworks.com/help/stats/corr.html>_` function with ``type``, ``Spearman``
* **Spearman's** :math:`\rho` **estimator**

* Faster approximation of the Spearman's rho function at the cost of slightly less accurate result.
* Based on developer testing, rho values may differ by :math:`10^{-4}` and p-values by :math:`10^{-5}`.
* This error is passed on to the network-level tests, and can cause p-value difference by :math:`10^{-4}`
* Based on developer testing, rho values may differ by :math:`10^{-4}` and *p*-values by :math:`10^{-5}`.
* This error is passed on to the network-level tests, and can cause *p*-value difference by :math:`10^{-4}`
* These differences were found with 10,000 permutations. Less permutations results in higher error in a less evenly distributed fashion.
* This is recommended for exploratory research with the Spearman's rho function for publications
* **Kendall's** :math:`\tau`\ **-b**
* **Kendall's** :math:`\tau` **-b**

* Implements Kendall's :math:`\tau`\ -b using C code in a MATLAB MEX file (``+mex/+src/kendallTauB.c``)
* Faster implementation that stardard MATLAB code providing identical :math:`\tau`\ and p-values.
* Implements Kendall's :math:`\tau` -b using C code in a MATLAB MEX file (``+mex/+src/kendallTauB.c``)
* Faster implementation that stardard MATLAB code providing identical :math:`\tau` and *p*-values.
* Run-time difference from *O*\ (*n*\ :sup:`2`) to *O*\ (*n* log *n*)
* This is done with a red-black tree.
* **Welch's t-test**
Expand All @@ -68,12 +69,12 @@ Provided Tests

* Allows loading of observed and permuted edge-level data the user has pre-calculated outside the NLA.
* Four ``.mat`` files needed as inputs
* p-values should be thresholded
* *p*-values should be thresholded

:Observed p: ``.mat`` file containing N\ :sub:`ROI_pairs`\ x 1 matrix of logical values, the observed, thresholded edge-level p-values.
:Observed p: ``.mat`` file containing N\ :sub:`ROI_pairs`\ x 1 matrix of logical values, the observed, thresholded edge-level *p*-values.
N\ :sub:`ROI_pairs`\ are the lower triangle values of a N\ :sub:`ROIs`\ x N\ :sub:`ROIs`\ matrix.
:Observed coeff: ``.mat`` file containing N\ :sub:`ROI_pairs`\ x 1 matrix of observed edge-level coefficients.
:Permuted p: ``.mat`` file containing N\ :sub:`ROI_pairs`\ x N\ :sub:`permutations`\ of logical values. Observed, thresholded, permuted p-values.
:Permuted p: ``.mat`` file containing N\ :sub:`ROI_pairs`\ x N\ :sub:`permutations`\ of logical values. Observed, thresholded, permuted *p*-values.
:Permuted coeff: ``.mat`` file containing N\ :sub:`ROI_pairs`\ x N\ :sub:`permutations`\ of permuted edge-level coefficients.

Creating additional edge-level tests
Expand All @@ -82,6 +83,7 @@ Creating additional edge-level tests
To create an edge-level test, a test class must be added to the codebase. Refer to current tests in ``+nla/+edge/+test`` for examples. Guidelines are listed below

* **Test objects**

All test objects must inherit from ``nla.edge.test.Base`` and be in the ``+nla/+edge/+test`` directory. There are also a few methods and
properties that must be included

Expand Down Expand Up @@ -122,6 +124,7 @@ To create an edge-level test, a test class must be added to the codebase. Refer
fulfilled. These values are all stored in the ``input_struct`` object.

* **Result object**

A result object must be defined for the test edge-level results. If no custom data fields are needed, then the object in ``+nla/+edge/+test/Base.m``
may be used and this step can be skipped.

Expand Down
31 changes: 16 additions & 15 deletions docs/source/getting_started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,48 +4,49 @@ Getting Started
Running with example data
--------------------------------------------------

First, open the NLA software (as described in :doc:`setup`). Select 'Pearson's r' as the edge-level
First, open the NLA software (as described in :doc:`setup`). Select ``Pearson's r`` as the edge-level
test from the edge-level test dropdown.

Click 'Select' to choose a network atlas, navigating to the ``support_files`` folder withing your
'NetworkLevelAnalysis' installation and selecting ``Wheelock_2020_CerebralCortex_15nets_288ROI_on_MNI.mat``.
Click ``Select`` to choose a network atlas, navigating to the ``support_files`` folder withing your
NetworkLevelAnalysis installation and selecting ``Wheelock_2020_CerebralCortex_15nets_288ROI_on_MNI.mat``.
This file is used to parcellate the data.

Then, select the functional connectivity, located in the ``examples/fc_and_behavior`` folder under the name
``sample_func_conn.mat``. Click 'Yes' to Fisher z-transform the data. Take a moment to visualize the functional
connectivity (FC) average by clicking 'View'. Note that the FC appears to match the parcellation, (effects
connectivity (FC) average by clicking ``View``. Note that the FC appears to match the parcellation, (effects
generally line up with network boundaries) - this can be a useful diagnostic tool if you are having issues
with parcellations not matching data.

Finally, load the behavior ``sample_behavior.mat`` from the ``examples/fc_and_behavior`` folder (The 'file type' drop-down
will need to be changed from 'Text' to 'MATLAB table' in the file browser). Set the behavioral variable to 'Flanker_AgeAdj' by
clicking on that column in the table and then the 'Set Behavior' button.
will need to be changed from ``Text`` to ``MATLAB table`` in the file browser). Set the behavioral variable to 'Flanker_AgeAdj' by
clicking on that column in the table and then the ``Set Behavior`` button.

Having finished our edge-level inputs, we now move over to the network-level panel on the right side. Select all the tests by clicking
the top one, and then shift+clicking the bottom one.

_running_network_tests:
Run the tests using the 'Run' button on the bottom-right. The number of permutations can be changed with the input field
to the left of the 'Run' button. After pushing the 'Run' button, a result window will open. The edge-level test will be run
and the results can be visualized by pressing 'View' in the upper-left of the result window. To run the network-level tests,
push the 'Run' button in the results window. This will take longer, a progress window will show up displaying the progress.
To visualize the results, expand the lists in the reloaded (automatically) panel, and highlight a test. Press the 'View figures'
.. _running_network_tests:

Run the tests using the ``Run`` button on the bottom-right. The number of permutations can be changed with the input field
to the left of the ``Run`` button. After pushing the ``Run`` button, a result window will open. The edge-level test will be run
and the results can be visualized by pressing ``View`` in the upper-left of the result window. To run the network-level tests,
push the ``Run`` button in the results window. This will take longer, a progress window will show up displaying the progress.
To visualize the results, expand the lists in the reloaded (automatically) panel, and highlight a test. Press the ``View figures``
button. Other visualization options, such as chord plots and convergence maps, can also be shown. The results can be saved using the
'File' menu in the top-left. These results can be loaded into MATLAB or opened in the NLA main window also using the 'File' menu on that
``File`` menu in the top-left. These results can be loaded into MATLAB or opened in the NLA main window also using the ``File`` menu on that
window.

Running with example pre-calculated data
----------------------------------------------------------

Similarly to the previous example, open the NLA window and load the ``Wheelock_2020_CerebralCortex_15nets_288ROI_on_MNI.mat`` parcellation. This
time, select the 'Precalculated data' edge-level test. Load the four input matrices in the ``examples/precalculated`` folder.
time, select the ``Precalculated data`` edge-level test. Load the four input matrices in the ``examples/precalculated`` folder.

* Observed coefficients: ``SIM_obs_coeff.mat``
* Observed, thresholded p-values: ``SIM_obs_p.mat``
* Permuted coefficients: ``SIM_perm_coeff.mat``
* Permuted, thresholded p-values: ``SIM_perm_p.mat``

Set the lower and upper coefficient bounds to the range of the coefficients. For this case, the range is [-2, 2]. These bounds can be checked
with the 'View' button for the edge-level results button. In the bottom right corner, set the ``perm_count`` to the desired amount of
with the ``View`` button for the edge-level results button. In the bottom right corner, set the ``perm_count`` to the desired amount of
permutations. The example data provided has a maximums of 600 permutations. Run the tests using the procedure described in the
:ref:`previous section <running_network_tests>`.
8 changes: 4 additions & 4 deletions docs/source/methodology.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,14 @@ General Linear Model / Edge-wise Statistical Model Selection
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

NLA also requires the user to specify the desired statistical model for testing associations between
behavioral data and edge-wiseor ROI-pair connectivityconnectome data. The analysis pipeline within
behavioral data and edge-wise or ROI-pair connectivity connectome data. The analysis pipeline within
the NLA toolbox offers both parametric and non-parametric correlation.

Connectivity Matrices
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Other software packages are used to create the connectivity matrices that are provided as input into the
NLA toolbox. One useful option for mapping functional connectivity matrices is CONN - MATLAB-based
NLA toolbox. One useful option for mapping functional connectivity matrices is `CONN <https://web.conn-toolbox.org/>_` - a MATLAB-based
software with the ability to compute, display, and analyze functional connectivity in fMRI.

The NLA Method
Expand All @@ -40,9 +40,9 @@ Both p-value and test-statistic binarization are offered in the current NLA pipe
supported the incorporation of a proportional edge density threshold, given that uneven edge density
thresholds have been shown to unfairly bias results :cite:p:`vandenHeuvelM`.
For enrichment statistic calculation, NLA offers a number of statistical tests. Prior research has relied on
chi-square and Fisher's Exact test, as well as a Kolmogorov-Smirnov (KS) test and non-parametric tests
chi-squared and Fisher's Exact tests. As well as a Kolmogorov-Smirnov (KS) test and non-parametric tests
based on ranks, which compare the distribution of test values within a region to other regions :cite:p:`WheelockM:2018,RudolphM,MoothaV,ZahnJ`. In
addition, KS alternatives such as averaging or minmax have also shown promise in connectome
addition, KS alternatives such as averaging or min-max have also shown promise in connectome
applications :cite:p:`ChenJ,NewtonM,YaariG,EfronB`.

NLA then conducts data-driven permutation testing to establish significance. In the NLA toolbox, network
Expand Down
22 changes: 13 additions & 9 deletions docs/source/network_level_tests.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ the given statistical test.

The full connectome method ranks the non-permuted (observed) significance of each network against the
significance of the same network calculated over many permutations using the same test.
I DON"T KNOW HOW TO EXPLAIN THE PROBABILITY BEING CALCULATED - Jim
**I DON"T KNOW HOW TO EXPLAIN THE PROBABILITY BEING CALCULATED** - Jim

The within network-pair method measures how significant each network is compared to all permutations of
only the selected network.
Expand All @@ -25,27 +25,29 @@ Provided Tests

* **Hypergeomtric**

* MATLAB's `hypercdf <https://www.mathworks.com/help/stats/hygecdf.html>` used to find the probablity
* MATLAB's `hypercdf <https://www.mathworks.com/help/stats/hygecdf.html>_` used to find the probablity
* **Chi-squred**

* Runs a :math:`\chi`\ :sup:`2` test.
* Runs a :math:`\chi` :sup:`2` test.

.. math::

\chi^2 = \sum_{n=1}^n \frac{O_i - E_i)^2}{E_i}

* **Kolmogorov-Smirnov**

* MATLAB `kstest2 <https://www.mathworks.com/help/stats/kstest2.html>`_ function.
* MATLAB `kstest2 <https://www.mathworks.com/help/stats/kstest2.html>_` function.

* **Wilcoxon rank-sum test**

* MATLAB `ranksum <https://www.mathworks.com/help/stats/ranksum.html>`_ function.
* MATLAB `ranksum <https://www.mathworks.com/help/stats/ranksum.html>_` function.

* MATLAB's `kstest2 <https://www.mathworks.com/help/stats/kstest2.html>_`

* MATLAB's `kstest2 <https://www.mathworks.com/help/stats/kstest2.html>`
* **Wilcoxon**

* MATLAB's `ranksum <https://www.mathworks.com/help/stats/ranksum.html>`
* MATLAB's `ranksum <https://www.mathworks.com/help/stats/ranksum.html>_`

* **Welch's t-test**

* Implements an optomized Welch's t-test comparing the functional connectivity of two groups.
Expand All @@ -56,14 +58,15 @@ Provided Tests

* **Student's t-test**

* MATLAB `ttest2 <https://www.mathworks.com/help/stats/ttest2.html>`_ function.
* MATLAB `ttest2 <https://www.mathworks.com/help/stats/ttest2.html>_` function.

Creating additional network-level tests
-----------------------------------------------------

To create a network-level test, a test class must be added to the codebase. Refer to the current tests in ``+nla/+net/+test``

* **Test objects**

All test objects must inherit from ``nla.net.test.Base`` and be in the ``+nla/+net/+test`` directory. There are also properties and methods
that must be included.

Expand All @@ -83,7 +86,8 @@ To create a network-level test, a test class must be added to the codebase. Refe
:statistics: All statistics that will be generated by the test. No special characters
:ranking_statistic: The statistic used for ranking and calculating p-values. Note: if there is a single sample version of the statisticin addition to a two sample statistic, the GUI will automatically add "single_sample\_" during rankings for non-permuted and within network pair ranking.

* ``run``
* A ``run`` method

::

result = run(obj, test_options, edge_test_results, network_atlas, permutations)
Expand Down
Loading
Loading