documentation

docs/Makefile

@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = source
+BUILDDIR      = build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/make.bat

@@ -0,0 +1,35 @@
+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+	set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=source
+set BUILDDIR=build
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+	echo.
+	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+	echo.installed, then set the SPHINXBUILD environment variable to point
+	echo.to the full path of the 'sphinx-build' executable. Alternatively you
+	echo.may add the Sphinx directory to PATH.
+	echo.
+	echo.If you don't have Sphinx installed, grab it from
+	echo.https://www.sphinx-doc.org/
+	exit /b 1
+)
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+
+:end
+popd

docs/source/bibliography.rst

@@ -0,0 +1,3 @@
+.. rubric:: References
+
+.. bibliography::

docs/source/conf.py

@@ -0,0 +1,51 @@
+import os
+# Configuration file for the Sphinx documentation builder.
+#
+# For the full list of built-in configuration values, see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+html_baseurl = os.environ.get("READTHEDOCS_CANONICAL_URL", "/")
+
+# -- Project information -----------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
+
+project = 'Network Level Analysis Toolbox'
+copyright = '2024, Muriah Wheelock'
+author = 'Muriah Wheelock'
+
+# -- General configuration ---------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
+
+extensions = [
+    'sphinx.ext.intersphinx', 
+    'sphinxcontrib.matlab', 
+    'sphinx_rtd_theme',
+    'sphinxcontrib.bibtex',
+    'sphinx.ext.autodoc'
+]
+this_dir = os.path.dirname(os.path.abspath(__file__))
+matlab_src_dir = os.path.abspath(os.path.join(this_dir, '../../+nla'))
+
+templates_path = ['_templates']
+exclude_patterns = []
+
+
+
+# -- Options for HTML output -------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
+
+html_theme = 'sphinx_rtd_theme'
+html_static_path = ['_static']
+
+
+# -- Options for Intersphinx ------------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/extensions/intersphinx.html#module-sphinx.ext.intersphinx
+
+intersphinx_mapping = {}
+
+# -- Options for bibtex ----------------------------
+# https://sphinxcontrib-bibtex.readthedocs.io/en/latest/quickstart.html#installation
+
+bibtex_bibfiles = ['refs.bib']
+bibtex_default_style = 'plain'
+bibtex_reference_style = 'super'

docs/source/edge_level_tests.rst

@@ -0,0 +1,78 @@
+Edge-level Statistical Tests
+==========================================
+
+Methods
+-------------------------
+
+The non-permuted method calculates the correlation of each Region Of Interest (ROI) to all other
+ROIs via the given test. These results are stored as a correlation coefficient, ``coeff``, a p-value, ``prob``,
+and a thresholded p-value, ``prob_sig``. The permuted method is identical except the variables have a ``_perm`` suffix.
+
+Common Inputs
+--------------------------
+
+:P: Edge-level p-value threshold
+:Network Atlas: :doc:`/network_atlases`
+:Functional Connectivity: Initial coorelation matrix if size N\ :sub:`ROIs`\ x N\ :sub:`ROIs`\ x N\ :sub:`scans`\. 
+  r-values or Fisher z-transformed r-values.
+:Behavior: MATLAB table (``.mat``) or tab seperated text file (``.txt``)
+
+  ============== =================== ================
+  Variable Name  Next Variable Name  More Variable...
+  ============== =================== ================
+  1              5                   1.5
+  0              1                   3
+  1              7                   2.6
+  ...            ...                 ...
+  ============== =================== ================
+
+  Each column header is a name of a variable.
+  Each column contains N\ :sub:`scans`\ entries.
+  After loading this file, the table should display in the GUI.
+  The user may mark one column as 'Behavior' for the score of interest.
+  Other columns may be marked as 'Covariates' which are partialed prior to running statistics.
+  (Note: Network Level Analysis cannot handle missing values for behavior or covariates. If there are ``NaNs`` or missing values, do not select this columns)
+
+Provided Tests
+--------------------------------
+
+* **Pearson's r**
+
+  * MATLAB `corr <https://www.mathworks.com/help/stats/corr.html>` function with ``type``, ``Pearson``
+* **Spearman's** :math:`\rho`\
+
+  * MATLAB `corr <https://www.mathworks.com/help/stats/corr.html>` function with ``type``, ``Spearman``
+* **Spearman's** :math:`\rho`\ **estimator**
+
+  * Faster approximation of the Spearman's rho function at the cost of slightly less accurate result.
+  * Based on developer testing, rho values may differ by :math:`10^{-4}` and p-values by :math:`10^{-5}`.
+  * This error is passed on to the network-level tests, and can cause p-value difference by :math:`10^{-4}` 
+  * These differences were found with 10,000 permutations. Less permutations results in higher error in a less evenly distributed fashion. 
+  * This is recommended for exploratory research with the Spearman's rho function for publications
+* **Kendall's** :math:`\tau`\ **-b**
+
+  * Implements Kendall's :math:`\tau`\ -b using C code in a MATLAB MEX file (``+mex/+src/kendallTauB.c``)
+  * Faster implementation that stardard MATLAB code providing identical :math:`\tau`\ and p-values.
+  * Run-time difference from *O*\ (*n*\ :sup:`2`) to *O*\ (*n* log *n*)
+  * This is done with a red-black tree.
+* **Welch's t-test``
+
+  * Implements an optomized Welch's t-test comparing the functional connectivity of two groups.
+  * Extra imports compared to other edge level tests
+  :Group name(s): Names associated with each group. (For example, 'Male' and 'Female')
+  :Group val(s): Behavioral value associated with each group. If 'Female' is donated as '0', and 'Male' as '1', set the vals to the numerical values.
+
+* **Pre-calculated data loader**
+
+  * Allows loading of observed and permuted edge-level data the user has pre-calculated outside the NLA.
+  * Four ``.mat`` files needed as inputs
+  * p-values should be thresholded
+  :Observed p: ``.mat`` file containing N\ :sub:`ROI_pairs`\ x 1 matrix of logical values, the observed, thresholded edge-level p-values.
+    N\ :sub:`ROI_pairs`\ are the lower triangle values of a N\ :sub:`ROIs`\ x N\ :sub:`ROIs`\ matrix.
+  :Observed coeff: ``.mat`` file containing N\ :sub:`ROI_pairs`\ x 1 matrix of observed edge-level coefficients.
+  :Permuted p: ``.mat`` file containing N\ :sub:`ROI_pairs`\ x N\ :sub:`permutations`\ of logical values. Observed, thresholded, permuted p-values.
+  :Permuted coeff: ``.mat`` file containing N\ :sub:`ROI_pairs`\ x N\ :sub:`permutations`\ of permuted edge-level coefficients.
+
+Creating additional edge-level tests
+-----------------------------------------------
+

docs/source/getting_started.rst

@@ -0,0 +1,51 @@
+Getting Started
+================================================
+
+Running with example data
+--------------------------------------------------
+
+First, open the NLA software (as described in :doc:`setup`). Select 'Pearson's r' as the edge-level
+test from the edge-level test dropdown.
+
+Click 'Select' to choose a network atlas, navigating to the ``support_files`` folder withing your
+'NetworkLevelAnalysis' installation and selecting ``Wheelock_2020_CerebralCortex_15nets_288ROI_on_MNI.mat``.
+This file is used to parcellate the data.
+
+Then, select the functional connectivity, located in the ``examples/fc_and_behavior`` folder under the name
+``sample_func_conn.mat``. Click 'Yes' to Fisher z-transform the data. Take a moment to visualize the functional
+connectivity (FC) average by clicking 'View'. Note that the FC appears to match the parcellation, (effects
+generally line up with network boundaries) - this can be a useful diagnostic tool if you are having issues
+with parcellations not matching data.
+
+Finally, load the behavior ``sample_behavior.mat`` from the ``examples/fc_and_behavior`` folder (The 'file type' drop-down
+will need to be changed from 'Text' to 'MATLAB table' in the file browser). Set the behavioral variable to 'Flanker_AgeAdj' by
+clicking on that column in the table and then the 'Set Behavior' button.
+
+Having finished our edge-level inputs, we now move over to the network-level panel on the right side. Select all the tests by clicking
+the top one, and then shift+clicking the bottom one.
+
+_running_network_tests:
+Run the tests using the 'Run' button on the bottom-right. The number of permutations can be changed with the input field
+to the left of the 'Run' button. After pushing the 'Run' button, a result window will open. The edge-level test will be run 
+and the results can be visualized by pressing 'View' in the upper-left of the result window. To run the network-level tests, 
+push the 'Run' button in the results window. This will take longer, a progress window will show up displaying the progress.
+To visualize the results, expand the lists in the reloaded (automatically) panel, and highlight a test. Press the 'View figures'
+button. Other visualization options, such as chord plots and convergence maps, can also be shown. The results can be saved using the 
+'File' menu in the top-left. These results can be loaded into MATLAB or opened in the NLA main window also using the 'File' menu on that
+window. 
+
+Running with example pre-calculated data
+----------------------------------------------------------
+
+Similarly to the previous example, open the NLA window and load the ``Wheelock_2020_CerebralCortex_15nets_288ROI_on_MNI.mat`` parcellation. This
+time, select the 'Precalculated data' edge-level test. Load the four input matrices in the ``examples/precalculated`` folder.
+
+* Observed coefficients: ``SIM_obs_coeff.mat``
+* Observed, thresholded p-values: ``SIM_obs_p.mat``
+* Permuted coefficients: ``SIM_perm_coeff.mat``
+* Permuted, thresholded p-values: ``SIM_perm_p.mat``
+
+Set the lower and upper coefficient bounds to the range of the coefficients. For this case, the range is [-2, 2]. These bounds can be checked
+with the 'View' button for the edge-level results button. In the bottom right corner, set the ``perm_count`` to the desired amount of 
+permutations. The example data provided has a maximums of 600 permutations. Run the tests using the procedure described in the 
+:ref:`previous section <running_network_tests>`. 

docs/source/index.rst

@@ -0,0 +1,28 @@
+.. NetworkLevelAnalysis documentation master file, created by
+   sphinx-quickstart on Mon Nov 18 13:00:09 2024.
+   You can adapt this file completely to your liking, but it should at least
+   contain the root `toctree` directive.
+
+Welcome to NetworkLevelAnalysis's documentation!
+================================================
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Table of Contents:
+
+   preface
+   overview
+   methodology
+   setup
+   getting_started
+   network_atlases
+   edge_level_tests
+   bibliography
+
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`

docs/source/methodology.rst

@@ -0,0 +1,79 @@
+Methodology
+================================
+
+Brain Network Map Selection
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+NLA requires the user to specify the network map that will be used to depict the known architecture of the
+human connectome, which is crucial given that the network map selection affects both statistical
+significance testing and interpretation :cite:p:`BellecP`. The current pipeline uses network maps that are generated with
+Infomap, due to its greater congruence with networks derived from task-activation and seed-based
+connectivity studies than alternative modularity algorithms :cite:p:`PowerJ,RosvallM`. Network maps can be generated using
+one's preferred algorithm or one of several published ROI and corresponding network map options that
+will be included in the NLA toolbox :cite:p:`GordonE,PowerJ,ThomasY,GlasserM,ShenX,CraddockR`. The use of standardized ROI and network maps creates a
+common, reproducible framework for testing brain-behavior associations across connectome research
+
+General Linear Model / Edge-wise Statistical Model Selection
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+NLA also requires the user to specify the desired statistical model for testing associations between
+behavioral data and edge-wise�or ROI-pair connectivity�connectome data. The analysis pipeline within
+the NLA toolbox offers both parametric and non-parametric correlation.
+
+Connectivity Matrices
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Other software packages are used to create the connectivity matrices that are provided as input into the
+NLA toolbox. One useful option for mapping functional connectivity matrices is CONN - MATLAB-based
+software with the ability to compute, display, and analyze functional connectivity in fMRI.
+
+The NLA Method
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+First, connectome-wide associations are calculated between ROI-pair connectivity and behavioral data,
+resulting in a set of standardized regression coefficients that specify the brain-behavior association at
+each ROI-pair of the connectome matrix. Next, network level analysis-consisting of transformation of the
+edge-wise test statistics and enrichment statistic calculation :cite:p:`AckermanM` - is done to determine which networks are
+strongly associated with the behavior of interest.
+
+Both p-value and test-statistic binarization are offered in the current NLA pipeline :cite:p:`EggebrechtA,WheelockM:2018`. Prior research has
+supported the incorporation of a proportional edge density threshold, given that uneven edge density
+thresholds have been shown to unfairly bias results :cite:p:`vandenHeuvelM`.
+For enrichment statistic calculation, NLA offers a number of statistical tests. Prior research has relied on
+chi-square and Fisher's Exact test, as well as a Kolmogorov-Smirnov (KS) test and non-parametric tests
+based on ranks, which compare the distribution of test values within a region to other regions :cite:p:`WheelockM:2018,RudolphM,MoothaV,ZahnJ`. In
+addition, KS alternatives such as averaging or minmax have also shown promise in connectome
+applications :cite:p:`ChenJ,NewtonM,YaariG,EfronB`.
+
+NLA then conducts data-driven permutation testing to establish significance. In the NLA toolbox, network
+level significance is determined by comparing each measured enrichment statistic to permuted
+enrichment p-values which are calculated by randomly shuffling behavior vector labels and computing
+the enrichment statistic many times to produce a null distribution for each network. The FPR is controlled
+at the network level using Bonferroni correction. Therefore, NLA is able to retain edge-wise correlations
+within each network module, but network communities are used to reduce the number of comparisons
+and control the FPR at the network level. After significance is determined, the pipeline allows users to
+create publication quality images to visualize network level findings both in connectome format and on
+the surface of the brain.
+
+**Note**: While the behavior vector labels are shuffled to conduct permutations in the enrichment pipeline,
+functional connectivity data are not shuffled in order to preserve the inherent covariant structure of the
+data across permutations
+
+How Should the Test Statistic Threshold Be Chosen?
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+A nominal threshold is used for the thresholding and binarization step of the edge-level tests. The
+nominal threshold is uncorrected and is typically set at 0.05 or 0.01 in the edge-level prob_max field. In
+contrast, a network-level corrected threshold using the Bonferroni method is used in the net-level
+statistics, where the nominal threshold is divided by the number of tests being done to correct for
+multiple comparisons.
+
+How Should the Networks Be Chosen?
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+There are many canonical ROI sets and there are many network definitions. Some of these network
+definitions include ROI that are not consistently assigned to any network. These ROI are typically removed
+prior to network level analysis, as is the case in the ``Seitzman_15nets_288ROI_on_TT`` and the
+``Gordon_12nets_286parcels_on_MNI`` network atlases included in this version of the toolbox. Network
+atlases that are not included in this package may also be used, but they must first be formatted into the 
+correct structure

docs/source/network_atlases.rst

@@ -0,0 +1,40 @@
+Network Atlases
+==================================
+.. mat:module:: .
+
+Overview
+------------------------------------
+
+A network atlas is a data file describing networks of the brain, each containing a number of related
+regions of interest. It also contains metadata such as network colors and names, ROI spatial coordinates
+(with associtated mesh/space), and optionally, a surface parcellation.
+
+.. mat:autoclass:: NetworkAtlas
+
+    .. mat:automethod:: numNets
+
+    .. mat:automethod:: numNetPairs
+
+    .. mat:automethod:: numROIs
+
+    .. mat:automethod:: numROIPairs
+
+Provided Network Atlases
+--------------------------------
+
+A number of network atlases are provided with the NLA software package in the ``support_files`` directory.
+Only NLA-specific details will be provided about them, if you wish to go into more depth on a particular atlas
+you should follow the link provided in its ``source`` field.
+
+* ``Gordon_13nets_333parcels_on_MNI``
+  * Surface space.
+  * Consists of 333 parcels and corresponding 13 networks :cite:p:`GordonE`. Contains both the MNI centrois and surface parcels on a ``MNI_32k`` mesh.
+* ``Gordon_12nets_286parcels_on_MNI``
+  * Surface space
+  * Same as ``Gordon_13nets_333parcels_on_MNI`` with 'None' network and its ROIs removed :cite:p:`GordonE`.
+* ``Seitzman_17nets_300ROI_on_TT``
+  * Volume space
+  * 300 ROIs in 17 networks :cite:p:`SeitzmanB`. Contains TT centroids
+* ``Seitzman_15nets_288ROI_on_TT``
+  * Volume space
+  * Same as ``Seitzman_17nets_300ROI_on_TT`` with 12 ROI and 2 networks removed due inconsistent placement in a network :cite:p:`SeitzmanB`.