Skip to content

Commit bd8d154

Browse files
Remi-GauRemi-Gau
authored
Merge pull request #2 from cpp-lln-lab/marco_create-tsv
add code to create raw tsv and make a markdown table + add submodules
2 parents 4569556 + abc025e commit bd8d154

File tree

12 files changed

+333
-4
lines changed

12 files changed

+333
-4
lines changed

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
__pycache__

‎.gitmodules‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
[submodule "cpp-lln-lab_raw"]
2+
path = cpp-lln-lab_raw
3+
url = https://github.com/cpp-lln-lab/cpp-lln-lab_raw.git
4+
datalad-url = https://github.com/cpp-lln-lab/cpp-lln-lab_raw.git
5+
[submodule "cpp-lln-lab_derivatives"]
6+
path = cpp-lln-lab_derivatives
7+
url = https://github.com/cpp-lln-lab/cpp-lln-lab_derivatives.git
8+
datalad-url = https://github.com/cpp-lln-lab/cpp-lln-lab_derivatives.git

‎.pre-commit-config.yaml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# See https://pre-commit.com for more information
2+
# See https://pre-commit.com/hooks.html for more hooks
3+
repos:
4+
- repo: https://github.com/pre-commit/pre-commit-hooks
5+
rev: v4.5.0
6+
hooks:
7+
- id: trailing-whitespace
8+
- id: check-yaml
9+
- id: check-added-large-files
10+
- id: end-of-file-fixer
11+
12+
- repo: https://github.com/psf/black
13+
rev: 23.10.1
14+
hooks:
15+
- id: black

‎cpp-lln-lab_derivatives‎

Submodule cpp-lln-lab_derivatives added at 629c65d

‎cpp-lln-lab_raw‎

Submodule cpp-lln-lab_raw added at 54fdad8

‎mkdocs.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,9 @@ site_name: CPP Lab datasets
33
repo_url: https://github.com/cpp-lln-lab/Datasets.git
44

55
nav:
6-
- CPP Lab raw data:
7-
- Home: README.md
8-
- Contributing: CONTRIBUTING.md
6+
- Home: README.md
7+
- Contributing: CONTRIBUTING.md
8+
- Raw datasets table: datasets_raw.md
99

1010
theme:
1111
name: material

‎src/CONTRIBUTING.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,4 +25,4 @@ Serving on http://127.0.0.1:8000/
2525
```
2626

2727
- push your changes to your forked repository
28-
- open a pull request
28+
- open a pull request

‎src/datasets_raw.md‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
| name | nb_subjects | modalities | sessions | tasks | fmriprep | freesurfer | mriqc |
2+
|:----------------------------------------|--------------:|:-------------------------|:-------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------|:-------------|:--------|
3+
| 2008_Montreal_BlindRestingState_OC_raw | 50 | ['anat', 'func'] | [] | ['rest'] | n/a | n/a | n/a |
4+
| 2012_Montreal_AudVisMotion_GD_raw | 35 | ['anat', 'func'] | [] | ['audMotion', 'visMotion'] | n/a | n/a | n/a |
5+
| 2014_Toronto_CataractsDiffusion_Gao_raw | 26 | ['anat', 'dwi'] | ['1'] | n/a | n/a | n/a | n/a |
6+
| 2015_Trento_Diffusion_SM_raw | 30 | ['anat', 'dwi', 'func'] | [] | ['unknown'] | n/a | n/a | n/a |
7+
| 2015_Trento_MultimConn_SM_raw | 29 | ['anat', 'dwi', 'func'] | ['01'] | ['Catgs'] | n/a | n/a | n/a |
8+
| 2015_Trento_categs_SM_raw | 53 | ['anat', 'dwi', 'func'] | [] | ['Catgs'] | n/a | n/a | n/a |
9+
| 2016_Trento_Categs_MR_raw | 29 | ['anat', 'dwi', 'func'] | ['01'] | ['Catgs'] | n/a | n/a | n/a |
10+
| 2016_Trento_MultimodalMotion_MR_raw | 23 | ['anat', 'func'] | ['01'] | ['audMotion', 'motDecoding', 'visMotion'] | n/a | n/a | n/a |
11+
| 2016_Trento_SpatiotopyDir_CB_raw | 33 | ['anat', 'func'] | [] | ['pRF'] | n/a | n/a | n/a |
12+
| 2018_LLN_FPAS_FB_raw | 16 | ['eeg'] | [] | ['FPAS'] | n/a | n/a | n/a |
13+
| 2018_Toronto_Cataract_SM_raw | 46 | ['anat', 'func'] | [] | ['categ', 'categBlur1', 'categBlur2'] | n/a | n/a | n/a |
14+
| 2019_Nancy_iEEG_FB_raw | 1 | ['ieeg'] | ['1', '2', '3', '4'] | ['ERPaudmot', 'ERPbraille', 'ERPcategory', 'FPSaudmot', 'FPSemotion', 'FPSface', 'FPSlexical', 'FPSvismot', 'FPSvoice', 'Readingbraille', 'Restingstate', 'Speechtracking', 'Stimulation'] | n/a | n/a | n/a |
15+
| 2021_Dijon_BabyFPAS_RPC_raw | 23 | ['eeg'] | [] | ['FPAS'] | n/a | n/a | n/a |
16+
| 2021_LLN_FVDE_FB_raw | 20 | ['eeg'] | ['001'] | ['fvde'] | n/a | n/a | n/a |
17+
| 2021_LLN_emotion_ST_raw | 14 | ['eeg'] | [] | ['emotion'] | n/a | n/a | n/a |
18+
| 2021_SaintLuc_TmsMT_FB_raw | 16 | ['anat', 'beh', 'func'] | ['mri', 'sham', 'tms'] | ['audInstrumentDiscrimination', 'audMotionDirDiscrimination', 'auditoryLocalizer', 'visColourDiscrimination', 'visMotionDirDiscrimination', 'visualLocalizer'] | n/a | n/a | n/a |
19+
| 2021_SaintLuc_combiEmo_FF_raw | 24 | ['anat', 'func'] | ['01', '02'] | ['correction', 'eventrelatedCombiemoAuditory', 'eventrelatedCombiemoBimdal', 'eventrelatedCombiemoBimodal', 'eventrelatedCombiemoVisual', 'facelocalizerCombiemo', 'facelocalizerCombiemoCombiemo', 'voicelocalizer', 'voicelocalizerCombiemo'] | n/a | n/a | n/a |
20+
| 2022_SaintLuc_lipSpeech_AA_raw | 2 | ['anat', 'func'] | ['01', '02', '03'] | ['MVPAAud', 'MVPAVis', 'PhonoLoc', 'VisLoc'] | n/a | n/a | n/a |
21+
| 2023_Liege_BLAM_MB_raw | 14 | ['anat', 'func'] | ['01', '02', '03', '04'] | ['BimodalMotionAud', 'BimodalMotionVis', 'audioV1', 'auditoryLocalizer', 'bimodalMotionAud', 'bimodalMotionVis', 'mtMstLocalizer', 'rdkBimodalMotion', 'restingState', 'visualLocalizer'] | n/a | n/a | n/a |
22+
| 2023_SaintLuc_VisTacMotionFoR_IS_raw | 21 | ['anat', 'func'] | ['001', '002'] | ['handDown', 'handUp', 'mtMstLocalizer', 'tactileLocalizer2', 'visual', 'visualLocalizer2'] | n/a | n/a | n/a |
23+
| 2023_Trento_plosBiology_YX_raw | 48 | ['anat', 'fmap', 'func'] | [] | ['judgement', 'resting'] | n/a | n/a | n/a |
24+
| Toronto_VisMotionLocalizer_MR_raw | 3 | ['anat', 'func'] | ['01'] | ['visMotion'] | n/a | n/a | n/a |
25+
| olf_blind_raw | 35 | ['anat', 'dwi', 'func'] | [] | ['olfid', 'olfloc', 'rest'] | n/a | n/a | n/a |

‎tools/datasets_raw.tsv‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
name nb_subjects has_participant_tsv has_participant_json participant_columns has_phenotype_dir modalities sessions tasks fmriprep freesurfer mriqc
2+
2008_Montreal_BlindRestingState_OC_raw 50 True True ['participant_id', 'group', 'match', 'gender', 'age', 'handedness', 'onset_blindness', 'years_of_blindness', 'total_blindness', 'years_of_total_blindness'] False ['anat', 'func'] [] ['rest'] n/a n/a n/a
3+
2012_Montreal_AudVisMotion_GD_raw 35 False False n/a False ['anat', 'func'] [] ['audMotion', 'visMotion'] n/a n/a n/a
4+
2014_Toronto_CataractsDiffusion_Gao_raw 26 True True ['/annex/objects/MD5E-s436--a3ab24b813254f34bc5773ec7427c33c.tsv'] False ['anat', 'dwi'] ['1'] n/a n/a n/a n/a
5+
2015_Trento_Diffusion_SM_raw 30 True True ['participant_id', 'group', 'group_counter', 'initials', 'recording'] False ['anat', 'dwi', 'func'] [] ['unknown'] n/a n/a n/a
6+
2015_Trento_MultimConn_SM_raw 29 True True ['/annex/objects/MD5E-s567--adb7ca1aa64cfe8680a842000ec06a43.tsv'] False ['anat', 'dwi', 'func'] ['01'] ['Catgs'] n/a n/a n/a
7+
2015_Trento_categs_SM_raw 53 False False n/a False ['anat', 'dwi', 'func'] [] ['Catgs'] n/a n/a n/a
8+
2016_Trento_Categs_MR_raw 29 False False n/a False ['anat', 'dwi', 'func'] ['01'] ['Catgs'] n/a n/a n/a
9+
2016_Trento_MultimodalMotion_MR_raw 23 False False n/a False ['anat', 'func'] ['01'] ['audMotion', 'motDecoding', 'visMotion'] n/a n/a n/a
10+
2016_Trento_SpatiotopyDir_CB_raw 33 True True ['/annex/objects/MD5E-s724--cfd24d6ce2366dd29831a737b3f48752.tsv'] False ['anat', 'func'] [] ['pRF'] n/a n/a n/a
11+
2018_LLN_FPAS_FB_raw 16 False False n/a False ['eeg'] [] ['FPAS'] n/a n/a n/a
12+
2018_Toronto_Cataract_SM_raw 46 False False n/a False ['anat', 'func'] [] ['categ', 'categBlur1', 'categBlur2'] n/a n/a n/a
13+
2019_Nancy_iEEG_FB_raw 1 True False ['participant_id'] False ['ieeg'] ['1', '2', '3', '4'] ['ERPaudmot', 'ERPbraille', 'ERPcategory', 'FPSaudmot', 'FPSemotion', 'FPSface', 'FPSlexical', 'FPSvismot', 'FPSvoice', 'Readingbraille', 'Restingstate', 'Speechtracking', 'Stimulation'] n/a n/a n/a
14+
2021_Dijon_BabyFPAS_RPC_raw 23 False False n/a False ['eeg'] [] ['FPAS'] n/a n/a n/a
15+
2021_LLN_FVDE_FB_raw 20 False False n/a False ['eeg'] ['001'] ['fvde'] n/a n/a n/a
16+
2021_LLN_emotion_ST_raw 14 False False n/a False ['eeg'] [] ['emotion'] n/a n/a n/a
17+
2021_SaintLuc_TmsMT_FB_raw 16 True True ['participant_id'] False ['anat', 'beh', 'func'] ['mri', 'sham', 'tms'] ['audInstrumentDiscrimination', 'audMotionDirDiscrimination', 'auditoryLocalizer', 'visColourDiscrimination', 'visMotionDirDiscrimination', 'visualLocalizer'] n/a n/a n/a
18+
2021_SaintLuc_combiEmo_FF_raw 24 True True ['participant_id', 'education', 'bmi'] False ['anat', 'func'] ['01', '02'] ['correction', 'eventrelatedCombiemoAuditory', 'eventrelatedCombiemoBimdal', 'eventrelatedCombiemoBimodal', 'eventrelatedCombiemoVisual', 'facelocalizerCombiemo', 'facelocalizerCombiemoCombiemo', 'voicelocalizer', 'voicelocalizerCombiemo'] n/a n/a n/a
19+
2022_SaintLuc_lipSpeech_AA_raw 2 False False n/a False ['anat', 'func'] ['01', '02', '03'] ['MVPAAud', 'MVPAVis', 'PhonoLoc', 'VisLoc'] n/a n/a n/a
20+
2023_Liege_BLAM_MB_raw 14 True True ['participant_id', 'age', 'sex', 'group'] False ['anat', 'func'] ['01', '02', '03', '04'] ['BimodalMotionAud', 'BimodalMotionVis', 'audioV1', 'auditoryLocalizer', 'bimodalMotionAud', 'bimodalMotionVis', 'mtMstLocalizer', 'rdkBimodalMotion', 'restingState', 'visualLocalizer'] n/a n/a n/a
21+
2023_SaintLuc_VisTacMotionFoR_IS_raw 21 True True ['participant_id', 'codename', 'date', 'age', 'gender'] False ['anat', 'func'] ['001', '002'] ['handDown', 'handUp', 'mtMstLocalizer', 'tactileLocalizer2', 'visual', 'visualLocalizer2'] n/a n/a n/a
22+
2023_Trento_plosBiology_YX_raw 48 True False ['participant_id'] False ['anat', 'fmap', 'func'] [] ['judgement', 'resting'] n/a n/a n/a
23+
Toronto_VisMotionLocalizer_MR_raw 3 True False ['participant_id'] False ['anat', 'func'] ['01'] ['visMotion'] n/a n/a n/a
24+
olf_blind_raw 35 True True ['participant_id', 'Group', 'Sex', 'Age', 'Educational level', 'Smoker', 'Medication', 'Vision level', 'Use of guide dog', 'Use of white cane', 'Musical practice', 'Braille reading', 'Braille reading hand used', 'Age of total blindness onset', 'Blindness Reason', 'Handedness', 'DK_C1_Letter_Fluency', 'DK_C2_Category_Fluency', 'DK_C3_Category_SwitchingTC', 'DK_C3_Category_SwitchingACC', 'TEA_C2', 'TEA_C3', 'CVLT_T1_T5', 'CVLT_ImmediateFreeRecall', 'CVLT_ImmediateCuedRecall', 'CVLT_DelayedFreeRecall', 'CVLT_DelayedCuedRecall', 'CVLT_Recognition', 'SS_Iden_O1', 'SS_Iden_O2', 'SS_Iden_O3', 'SS_Iden_O4', 'SS_Iden_O5', 'SS_Iden_O6', 'SS_Iden_O7', 'SS_Iden_O8', 'SS_Iden_O9', 'SS_Iden_O10', 'SS_Iden_O11', 'SS_Iden_O12', 'SS_Iden_O13', 'SS_Iden_O14', 'SS_Iden_O15', 'SS_Iden_O16', 'SS_Pls_O1', 'SS_Pls_O2', 'SS_Pls_O3', 'SS_Pls_O4', 'SS_Pls_O5', 'SS_Pls_O6', 'SS_Pls_O7', 'SS_Pls_O8', 'SS_Pls_O9', 'SS_Pls_O10', 'SS_Pls_O11', 'SS_Pls_O12', 'SS_Pls_O13', 'SS_Pls_O14', 'SS_Pls_O15', 'SS_Pls_O16', 'SS_Int_O1', 'SS_Int_O2', 'SS_Int_O3', 'SS_Int_O4', 'SS_Int_O5', 'SS_Int_O6', 'SS_Int_O7', 'SS_Int_O8', 'SS_Int_O9', 'SS_Int_O10', 'SS_Int_O11', 'SS_Int_O12', 'SS_Int_O13', 'SS_Int_O14', 'SS_Int_O15', 'SS_Int_O16', 'SS_Threshold_rightN', 'SS_Threshold_leftN', 'SS_Discrimination_Total', 'SS_OM_Total', 'SS_OM_Hits', 'SS_OM_FalseAlarms', 'SS_OM_Miss', 'SS_OM_Correct_Rejection', 'SS_OM_O1', 'SS_OM_O2', 'SS_OM_O3', 'SS_OM_O4', 'SS_OM_O5', 'SS_OM_O6', 'SS_OM_O7', 'SS_OM_O8', 'SS_OM_O9', 'SS_OM_O10', 'SS_OM_O11', 'SS_OM_O12', 'SS_OM_O13', 'SS_OM_O14', 'SS_OM_O15', 'SS_OM_O16', 'IRM_Pleasantness_Eucalyptus', 'IRM_Pleasantness_Almond', 'IRM_Intensity_Eucalyptus', 'IRM_Intensity_Almond'] False ['anat', 'dwi', 'func'] [] ['olfid', 'olfloc', 'rest'] n/a n/a n/a

‎tools/list_raw.py‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
"""List datasets contents on cpp-lln-lab_raw and write the results in a tsv file.
2+
3+
to do:
4+
5+
- [ ] Also checks for derivatives folders for mriqc, frmiprep and freesurfer.
6+
"""
7+
8+
from pathlib import Path
9+
10+
import pandas as pd
11+
12+
from utils import init_dataset
13+
from utils import list_datasets_in_dir
14+
15+
cpp_raw = Path(__file__).parent.parent / "cpp-lln-lab_raw"
16+
17+
18+
# Overwrite the tsv file with the current raw datasets
19+
20+
DEBUG = False
21+
22+
datasets = init_dataset()
23+
input_dir = cpp_raw
24+
datasets = list_datasets_in_dir(datasets, input_dir, debug=DEBUG)
25+
26+
datasets_df = pd.DataFrame.from_dict(datasets)
27+
28+
datasets_df = datasets_df.sort_values("name")
29+
30+
root_dir = Path(__file__).parent.parent
31+
32+
output_file = Path(__file__).parent / "datasets_raw.tsv"
33+
34+
datasets_df.to_csv(output_file, index=False, sep="\t")
35+
36+
mk_file = Path(__file__).parent.parent / "src" / "datasets_raw.md"
37+
38+
datasets_df = datasets_df.drop(
39+
columns=[
40+
"has_participant_tsv",
41+
"has_participant_json",
42+
"has_phenotype_dir",
43+
"participant_columns",
44+
],
45+
)
46+
47+
with open(mk_file, "w") as f:
48+
datasets_df.to_markdown(f, index=False, mode="a")

0 commit comments

Comments
 (0)