Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
*~
.DS_Store
.jekyll-cache
.sass-cache
_site
__pycache__
Expand Down
18 changes: 17 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@ How to use this template:
choose either "ReproNim" (if that's an option) or your own user account and
then enter the name of the lesson/repository that you wish to create.

2. Change the following variables in the `_config.yml` file:
2. Change the following variables in the `_confi
.yml` file:
- `title`
- `repo`
- `root`
Expand All @@ -24,6 +25,21 @@ then enter the name of the lesson/repository that you wish to create.
- Add coffee breaks into the lesson. This keeps the timing of each section
accurate.

# Running with Docker locally for development

Instead of installing ruby and jekyll, you can use docker to run a live version of your repo locally.
As you make changes, it will recompile the changes and all you will need to do is refresh the browser.

The following command it assumes that you are inside your cloned github repo. Adapted from [this blog](https://dev.to/michael/compile-a-jekyll-project-without-installing-jekyll-or-ruby-by-using-docker-4184):

```
docker run --rm -it --volume="$PWD:/srv/jekyll" --volume="$PWD/vendor/bundle:/usr/local/bundle" \
--env JEKYLL_ENV=development --platform linux/amd64 -p 4000:4000 jekyll/jekyll:4.0.1 \
jekyll serve --config _config.yml,_config_dev.yml
```

You should then be able to open the live page at http://localhost:4000/

# Acknowledgment

Please see [LICENSE.md](LICENSE.md) for copyright, license, and how to acknowledge information.
56 changes: 45 additions & 11 deletions _episodes/01-lesson-statistical-basis.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ questions:
- "Sampling, notion of estimation : estimates of mean and variances"
- "Distributions, relation to frequency, PDF, CDF, SF, ISF "
- "Hypothesis testing: the basics H0 versus H1"
- "The Multiple Comparisons Problem"
- "Confidence intervals "
- "Notion of model comparison : BIC/Akaike"
- "Notion of bayesian statistics "
Expand Down Expand Up @@ -64,7 +65,9 @@ There is also this [link](http://mathworld.wolfram.com/StatisticalDistribution.h


Next, we need to be able to use the [Cumulative distribution function (CDF)](https://en.wikipedia.org/wiki/Cumulative_distribution_function). For a given value $$x$$, the CDF will give us the probability that the random variable will be less (or equal) than $$x$$.
The CDF is related to its Complementary cumulative distribution function (also called tail distribution, or survival function : $$ SF(x) = 1 - CDF(x) $$.
The CDF is related to its Complementary cumulative distribution function (also called tail distribution, or survival function:

$$SF(x) = 1 - CDF(x)$$

See the [survival function](https://en.wikipedia.org/wiki/Survival_function) in wikipedia.

Expand Down Expand Up @@ -100,19 +103,40 @@ is less than 5%. See our [lesson on p-values and caveats](http://www.repronim.or

### Risk of errors

There are at least two risks of errors (as introduced by J. Neyman and G. Pearson): 1) Deciding that the two groups are drawn from different populations when in fact that is not the case, (type I error) and 2) deciding that the two groups are drawn from one homogeneous population while this is not the case (type II error). There are also errors of higher types see [the wikipedia article](https://en.wikipedia.org/wiki/Type_III_error) on this.
There are at least two risks of errors (as introduced by J. Neyman and G. Pearson): 1) Deciding that the two groups are drawn from different populations when in fact that is not the case, (type I error) and 2) deciding that the two groups are drawn from one homogeneous population while this is not the case (type II error).

> ## External reading
>
> There are also errors of higher types see [the wikipedia article](https://en.wikipedia.org/wiki/Type_III_error) on this.
>
{: .callout}


### The multiple comparison problem

### Introduction to the multiple comparison problem
The multiple comparison problems occurs when many hypotheses are made. If we control for the risk of Type I error for each test, after many tests, some will be "significant" just by chance.

The issue occurs when many hypotheses are made. If we control for the risk of error of type I for each test, after many tests, some will be "significant" just by chance.
Typically, a functional scan will separate the brain into about 150,000 voxels. Generalised linear model is run one time for each voxel, meaning that about 150,000 hypotheses are tested, one for each voxel.

Supposes that one selects a decision threshold (e.g., *p* < 0.05) to reject the null hypothesis. If only conducting one test, it's quite straightforward to find the threshold by looking up a statistical table (e.g. *Z* = 1.65). However, if 150,000 tests are performed, then we would expect $$150,000 \times 0.05 = 7,500$$ false positives if none of the voxel is affected by the task.

## Bayesian statistics

Finding the right level of introduction for this topic is not easy. We propose to start with [this blog](https://www.analyticsvidhya.com/blog/2016/06/bayesian-statistics-beginners-simple-english/) which should give you a good introduction.
Finding the right level of introduction for this topic is not easy. We propose to start with these external materials:

There is also [this one](http://faculty.washington.edu/kenrice/BayesIntroClassEpi515.pdf). Exercises on this still need to be developed [TBD].
> ## External materials:
>
> * [Analytics Vidhya: Power of Bayesian Statistics & Probability](https://www.analyticsvidhya.com/blog/2016/06/bayesian-statistics-beginners-simple-english/) (Updated 2023)
>
> * [Bayesian Statistics by Prof Ken Rice](https://faculty.washington.edu/kenrice/teaching.html)
>
{: .callout}

> ## Exercise
>
> Exercises on this still need to be developed [TBD].
>
{: .challenge}


Bayesian statistics are *rarely* used, because researchers are often unsure on how to use them, and because they sometimes can be used to include subjective knowledge. However, they have a better theoretical ground and in many cases they should be used.
Expand All @@ -125,12 +149,22 @@ The inertia of the community to adopt the bayesian framework is clear. One reas
* What are the key advantage of Bayesian statistics ?
* What do Bayesian statistics require before you can apply them - do we have this

> ## Additional reading
>
> * [Practical Bayesian Inference in Neuroscience: Or How I Learned To Stop Worrying and Embrace the Distribution](https://doi.org/10.1523/ENEURO.0484-23.2024)
>
{: .callout}

## Notion of model comparison : BIC/Akaike

Model comparison is **fundamental**. Here are a few links on this topic:

* [Wikipedia AIC](https://en.wikipedia.org/wiki/Akaike_information_criterion)

* [Wikipedia BIC](https://en.wikipedia.org/wiki/Bayesian_information_criterion)

* And a comparison of these two [here](https://methodology.psu.edu/AIC-vs-BIC)
> ## External materials:
>
> * [Wikipedia AIC](https://en.wikipedia.org/wiki/Akaike_information_criterion)
>
> * [Wikipedia BIC](https://en.wikipedia.org/wiki/Bayesian_information_criterion)
>
> * And a comparison of these two [here](https://methodology.psu.edu/AIC-vs-BIC)
>
{: .callout}
4 changes: 4 additions & 0 deletions _includes/javascript.html
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@
<script src="{{ root }}/assets/js/bootstrap.min.js"></script>
<script src="{{ root }}/assets/js/lesson.js"></script>
<script src="{{ root }}/assets/js/katex.min.js"></script>
<!-- https://github.com/mmistakes/minimal-mistakes/issues/735 -->
<script type="text/javascript" async
src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-MML-AM_CHTML">
</script>

<script>
var scripts = document.getElementsByTagName("script");
Expand Down
12 changes: 6 additions & 6 deletions pdf/talk.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
---
title: ReproNim workshop: The use of p-values in brain imaging and statistical aspects in reproducibility
title: ReproNim workshop - The use of p-values in brain imaging and statistical aspects in reproducibility
shortitle: Evil p-values
subtitle: A reproducibility perspective
date: Aug 2nd 2018
author: JB Poline \newline
institute: McGill University, UC Berkeley

header-include:
- \usefonttheme{professionalfonts}
- \usepackage{fontspec}
- \setsansfont[BoldFont = {Fira Sans Medium}]{Fira Sans}
- \setmonofont{Fira Mono}
- \usepackage{amsmath}
- \usefonttheme{professionalfonts}
- \usepackage{fontspec}
- \setsansfont[BoldFont = {Fira Sans Medium}]{Fira Sans}
- \setmonofont{Fira Mono}
- \usepackage{amsmath}
---

Plan
Expand Down
12 changes: 6 additions & 6 deletions pdfs/intro-to-stats.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
---
title: ReproNim workshop: The use of p-values in brain imaging and statistical aspects in reproducibility
title: ReproNim workshop - The use of p-values in brain imaging and statistical aspects in reproducibility
shortitle: Evil p-values
subtitle: A reproducibility perspective
date: Aug 2nd 2018
author: JB Poline \newline
institute: McGill University, UC Berkeley

header-include:
- \usefonttheme{professionalfonts}
- \usepackage{fontspec}
- \setsansfont[BoldFont = {Fira Sans Medium}]{Fira Sans}
- \setmonofont{Fira Mono}
- \usepackage{amsmath}
- \usefonttheme{professionalfonts}
- \usepackage{fontspec}
- \setsansfont[BoldFont = {Fira Sans Medium}]{Fira Sans}
- \setmonofont{Fira Mono}
- \usepackage{amsmath}
---

Plan
Expand Down