Merge pull request #13 from Integrative-Transcriptomics/dev

Merge dev into main.

Author: s-t-h

README.md

@@ -4,92 +4,102 @@
 
 # MUSIAL
 
-**MUSIAL** (MUlti Sample varIant AnaLysis) is a Java command-line tool designed to analyze and summarize single nucleotide variants (SNVs) and insertions/deletions (indels) across multiple prokaryotic samples.
-The software aggregates and analyzes variant calls from multiple samples of a prokaryotic species and provides an interface to generate comprehensive statistics and alignments at the genome, gene and protein level.
-MUSIAL enables a comprehensive assessment of variability within a species at the genome, gene and protein level, providing insights into, for example, conserved and variable regions, diversity at the gene level and common proteoforms among samples.
-
-## ✨ Features
+**MUSIAL** (MUlti Sample varIant AnaLysis) is a Java command-line tool to analyze large sets of VCF files with prokaryotic single nucleotide variants (SNVs) and insertions/deletions (indels). It provides an interface for generating comprehensive statistics and alignments, as well as assessing variability at genome, gene and protein levels.
 
 - **Integrates SnpEff and other Sequence Ontology compliant annotations** to help interpret variants.
 - **Projection to genomic features (genes) facilitates allele- and proteoform-specific information** that supports the characterization of individual samples.
 - **VCF based sequence reconstruction** at nucleotide and protein sequence level and tabular reports on sample, feature and variant statistics.
 
-## 📖 Usage
+### 📖 Usage
 
 An executable `jar` file (`Java 21`) is available from the [Releases](https://github.com/Integrative-Transcriptomics/MUSIAL/releases) section.
 MUSIAL operates on a modular, task-based architecture that is primarily initiated by the `build` task, which creates a JSON file (_storage_) as its primary output; this is then used as input for all other tasks.
 
-The general CLI usage is `java -jar MUSIAL-v2.4.0.jar <task>`, whereby the following tasks are available:
+Details on the use of the software and tutorials can be found in the repository [Wiki](https://github.com/Integrative-Transcriptomics/MUSIAL/wiki). The general CLI usage is `java -jar MUSIAL-v2.4.2.jar <task>`, whereby the following tasks are available:
 
 <details>
 <summary><code>build</code> - Build a local database file (storage) in JSON format from variant calls; the mandatory input for other tasks.</summary>
 
 ```
 Command line arguments of task build
 
- -C,--configuration <arg>   Path to a JSON file specifying the build task parameter configuration for MUSIAL.
+ -C,--configuration <arg>   Path to a JSON file specifying the build task parameter configuration for MUSIAL. Visit the documentation for details.
 ```
 </details>
 
 <details>
-<summary><code>expand</code> - Expand an existing storage file from variant call format (VCF) files.</summary>
+<summary><code>expand</code> - Expand an existing storage file from variant call files and/or meta data.</summary>
 
 ```
 Command line arguments of task expand
 
+ -d,--dry-run          Only report on novel entries without writing the updated storage.
  -I,--storage <arg>    Path to a .json(.gz) file generated with the build task of MUSIAL.
  -m,--vcfMeta <arg>    Path to a .tsv or .csv file specifying sample annotations.
  -o,--output <arg>     Path to write the output file (default: overwrite input file).
- -p,--preview          Only report on novel entries without writing the updated storage.
- -V,--vcfInput <arg>   List of file or directory paths. All files must be in VCF format.
+ -V,--vcfFiles <arg>   List of file or directory paths. All files must be in VCF format.
 ```
 </details>
 
 <details>
-<summary><code>view</code> - View the content (features, samples or variants) and their attributes, of a MUSIAL storage file.</summary>
+<summary><code>view</code> - View the content (features, samples or variants; and their attributes) of a MUSIAL storage file.</summary>
 
 ```
 Command line arguments of task view
 
- -C,--content <arg>   One of sample, allele, call, variant, type, feature.
- -f,--filter <arg>    List of feature-, sample names, and/or positions for which the output is to be filtered (default: no filters). Entries may be
-                      ignored depending on the content.
+ -C,--content <arg>   The content to view. One of FEATURES, SAMPLES, VARIANTS (case-insensitive).
  -I,--storage <arg>   Path to a .json(.gz) file generated with the build task of MUSIAL.
- -o,--output <arg>    Path to directory or file to write the output to (default: stdout).
+ -o,--output <arg>    Path to write the output file. If not provided, a default file will be created based on the input file (default). If `print` or
+                      `stdout` is specified, the output will be printed to the console.
+ -q,--query <arg>     One or multiple identifiers or genomic ranges (contig:start-end) to query.
 ```
 </details>
 
 <details>
-<summary><code>sequence</code> - Export FASTA format sequences of features from a MUSIAL storage file.</summary>
+<summary><code>profile</code> - Profile samples with respect to variants, alleles, or proteoforms.</summary>
 
 ```
-Command line arguments of task sequence
+Command line arguments of task profile
 
- -c,--content <arg>    One of `nt` or `aa` (default: `nt`).
- -F,--features <arg>   List of feature names to export data for. Non-coding features are skipped if `content` is `aa`.
- -I,--input <arg>      Path to a .json(.gz) file generated with the build task of MUSIAL.
- -k,--conserved        Export conserved sites.
- -m,--merge            Export sequences per allele or proteoform instead of per sample.
- -o,--output <arg>     Path to a directory to write the output files to (default: parent of input).
- -r,--reference        Include the reference sequence within the export.
- -s,--samples <arg>    List of sample names to restrict the sequence export to.
- -x,--strip            Strip all gap characters from the exported sequences.
+ -C,--content <arg>   The content to view. One of VARIANTS, ALLELES, PROTEOFORMS (case-insensitive).
+ -I,--storage <arg>   Path to a .json(.gz) file generated with the build task of MUSIAL.
+ -o,--output <arg>    Path to write the output file. If not provided, a default file will be created based on the input file (default). If `print` or
+                      `stdout` is specified, the output will be printed to the console.
+ -q,--query <arg>     One or multiple identifiers or genomic ranges (contig:start-end) to consider.
+ -x,--reduced         Represent entries in a reduced format, i.e., sequence types as numbers with 0 as the reference or synonymous sequence and
+                      variants without detailed call information.
 ```
 </details>
 
----
+<details>
+<summary><code>sequence</code> - Generate and write sequence data.</summary>
 
-Further details on the use of the software and internal workflows can be found in the repository [Wiki](https://github.com/Integrative-Transcriptomics/MUSIAL/wiki).
+```
+Command line arguments of task sequence
+
+ -a,--align             Whether to align sequences (optional, default: false).
+ -c,--content <arg>     Whether to generate NUCLEOTIDE or AMINOACID sequences (optional, case-insensitive, default: NUCLEOTIDE).
+ -f,--split <arg>       Whether to split output files by FEATURE, SAMPLE, BOTH, or NONE (optional, case-insensitive, default: FEATURE).
+ -I,--storage <arg>     Path to a .json(.gz) file generated with the build task of MUSIAL.
+ -l,--locations <arg>   One or multiple feature identifiers or genomic ranges (contig:start-end) to generate sequence data of. If none are provided,
+                        all features or full contig ranges will be considered.
+ -m,--merge             Whether to merge identical sequences (optional, default: false).
+ -o,--output <arg>      Path to write the output. If not provided, the directory of the input storage is used. If a directory is provided, files are
+                        created there. If a file is provided, its parent directory is used.
+ -s,--samples <arg>     One or multiple sample identifiers to retrieve sequences for (optional).
+ -v,--variable          Whether to only consider variable positions (optional, default: false).
+```
+</details>
 
-## 🌐 Web Interface
+### 🌐 Web Interface
 
-To provide user-friendly access to its functionalities, MUSIAL is available via a web interface at https://musial-tuevis.cs.uni-tuebingen.de/ currently running version `v2.3.10`. The code is deposited in the `web` branch.
+MUSIAL is also available via a web interface at https://musial-tuevis.cs.uni-tuebingen.de/ currently running version `v2.3.10`.
 
-## 🔨 Build
+### Build
 
-MUSIAL `v2.4` is built with `JDK 21.0.6` and `Gradle 8.2.1`. If you want to compile the source code, run `gradle clean build` in the root directory of the project. The JavaDoc of the software is available at [https://integrative-transcriptomics.github.io/MUSIAL/javadoc/](https://integrative-transcriptomics.github.io/MUSIAL/javadoc/).
+MUSIAL `v2.4` is built with `JDK 21.0.6` and `Gradle 9.1.0`. If you want to compile the source code, run `gradle clean build` in the root directory of the project. The JavaDoc of the software is available at [https://integrative-transcriptomics.github.io/MUSIAL/javadoc/](https://integrative-transcriptomics.github.io/MUSIAL/javadoc/).
 
-## 🙋 Need Help?
+### Need Help?
 
-- 🎓 Detailed information about the software can be found in the repository's [Wiki](https://github.com/Integrative-Transcriptomics/MUSIAL/wiki). 
-- 🐛 Found an issue or have a feature request? Feel free to [Open a GitHub issue](https://github.com/Integrative-Transcriptomics/MUSIAL/issues/new).
+- Detailed information about the software can be found in the repository's [Wiki](https://github.com/Integrative-Transcriptomics/MUSIAL/wiki). 
+- Found an issue or have a feature request? Feel free to [Open a GitHub issue](https://github.com/Integrative-Transcriptomics/MUSIAL/issues/new).

build.gradle

@@ -1,88 +1,23 @@
-/*
-Set meta-information for the project build.
-*/
-version 'v2.4.2'
-group 'de.tue.cs.ibmi.it'
-
-println "Name: $name"
-println "Project directory: $projectDir"
-println "Build directory: $buildDir"
-println "Version: $version"
-println "Group: $project.group"
-println "AntBuilder: $ant"
-
-/*
-The `buildscript` block defines properties (repositories, plugins, ...)
-used within the Gradle build process.
-*/
-buildscript {
-    repositories {
-        mavenCentral()
-    }
-}
-
-/*
-Set JUnit platform for testing.
- */
-tasks.withType(Test) {
-    useJUnitPlatform()
+/* -------------------- Plugins and Toolchain -------------------- */
+plugins {
+    id 'java'
 }
 
-/*
-Import plugins used during the build process.
-*/
-/*
-To prevent duplicate access to output directories a dependency-hierarchy is
-established on different tasks.
-*/
-tasks.whenTaskAdded { task ->
-    if (task.name == 'jar' || task.name == 'processResources') {
-        task.dependsOn unpackSnpEff
+java {
+    toolchain {
+        languageVersion = JavaLanguageVersion.of(21)
     }
 }
 
-/*
-This task unzips SnpEff.
-*/
-task unpackSnpEff(type: Copy) {
-    from zipTree('src/main/resources/snpEff.zip')
-    into 'src/main/resources/'
-}
-
-/*
-After building, the Jar is copied into the `releases` directory.
-*/
-task copyJarToReleases(type: Copy) {
-    mkdir 'releases'
-    def jarName = "build/libs/" + rootProject.name + "-" + version + ".jar"
-    from jarName
-    into "releases"
-}
-
-/*
-Defines the repositories to look up dependencies.
-*/
+/* -------------------- Repositories -------------------- */
 repositories {
     mavenCentral()
-    maven {
-        url "https://repository.jboss.org/nexus/content/repositories/thirdparty-releases/"
-    }
-    maven {
-        url "https://bio.informatik.uni-jena.de/repository/libs-release-oss/"
-    }
-    maven {
-        url "https://artifactory.cronapp.io/public-release/"
-    }
+    maven { url "https://repository.jboss.org/nexus/content/repositories/thirdparty-releases/" }
+    maven { url "https://bio.informatik.uni-jena.de/repository/libs-release-oss/" }
+    maven { url "https://artifactory.cronapp.io/public-release/" }
 }
 
-/*
-The following plugins are used during the gradle build process.
-*/
-apply plugin: 'java'
-
-/*
-All project dependencies are defined in the following block.
-*/
+/* -------------------- Dependencies -------------------- */
 dependencies {
     implementation 'commons-cli:commons-cli:1.5.0'
     implementation 'commons-io:commons-io:2.14.0'
@@ -99,24 +34,52 @@ dependencies {
     implementation 'org.ehcache:ehcache:3.11.1'
     implementation 'uk.co.omega-prime:btreemap:1.2.0'
     implementation 'tech.tablesaw:tablesaw-core:0.44.4'
-    testImplementation 'org.junit.jupiter:junit-jupiter-api:5.8.1'
-    testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.8.1'
+    testImplementation 'org.junit.jupiter:junit-jupiter-api:6.0.0'
+    testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:6.0.0'
+    testRuntimeOnly 'org.junit.platform:junit-platform-launcher:6.0.0'
+}
+
+dependencyLocking {
+    lockAllConfigurations()
+}
+
+/* -------------------- Project Metadata -------------------- */
+version 'v2.4.2'
+group 'de.tue.cs.ibmi.it'
+
+println "Name: $name"
+println "Project directory: $projectDir"
+println "Build directory: $buildDir"
+println "Version: $version"
+println "Group: $project.group"
+println "AntBuilder: $ant"
+
+/* -------------------- Task Definitions -------------------- */
+tasks.withType(Test) {
+    useJUnitPlatform()
+}
+
+tasks.named('processResources') {
+    dependsOn tasks.named('unpackSnpEff')
+}
+
+tasks.named('jar') {
+    dependsOn tasks.named('unpackSnpEff')
 }
 
-/*
-In order to build a final FatJar, all entries from configurations.implementation (the above defined
-dependencies) are copied into configurations.includeJars.
-*/
-configurations {
-    includeJars.extendsFrom implementationjava
+tasks.register('unpackSnpEff', Copy) {
+    from zipTree('src/main/resources/snpEff.zip')
+    into 'src/main/resources/'
 }
 
-sourceCompatibility = JavaVersion.VERSION_21
-targetCompatibility = JavaVersion.VERSION_21
+tasks.register('copyJarToReleases', Copy) {
+    mkdir 'releases'
+    def jarName = "build/libs/" + rootProject.name + "-" + version + ".jar"
+    from jarName
+    into "releases"
+}
 
-/*
-Defines the source directories for the tasks executed by the `java` plugin.
-*/
+/* -------------------- Source Sets and Resource Processing -------------------- */
 sourceSets {
     main {
         java {
@@ -128,11 +91,8 @@ sourceSets {
     }
 }
 
-/*
-Copies the title and version of the project into resources files.
-*/
 processResources {
-    duplicatesStrategy 'exclude' // If duplicated files exist, they are excluded.
+    duplicatesStrategy 'exclude'
     exclude('*.zip')
     filesMatching('version.properties') {
         expand projectVersion: version
@@ -142,29 +102,24 @@ processResources {
     }
 }
 
+/* -------------------- Jar Configuration -------------------- */
 jar {
-    duplicatesStrategy 'exclude' // If duplicated files exist, they are excluded.
+    duplicatesStrategy 'exclude'
     manifest {
         attributes("Implementation-Title": rootProject.name,
                 "Implementation-Version": archiveVersion,
                 "Main-Class": "main.Musial")
     }
     doFirst {
         from { configurations.runtimeClasspath.collect { it.isDirectory() ? it : zipTree(it) } }
-        // Collects all Jars from dependencies and builds a FatJar.
     }
 }
 
-/*
-Defines steps executed by calling `gradle clean`.
-*/
+/* -------------------- Build Finalization and Cleanup -------------------- */
 clean.doFirst {
     delete "${rootDir}/releases/"
     delete "${rootDir}/build/"
     delete "${rootDir}/src/main/resources/snpEff"
 }
 
-/*
-After building the FatJar, it is copied to the releases directory.
-*/
 build.finalizedBy(copyJarToReleases)

settings.gradle

@@ -1,19 +1 @@
-/*
- * This settings file was auto generated by the Gradle buildInit task
- * by 'seitza' at '9/23/16 8:26 AM' with Gradle 2.13
- *
- * The settings file is used to specify which projects to include in your build.
- * In a single project build this file can be empty or even removed.
- *
- * Detailed information about configuring a multi-project build in Gradle can be found
- * in the user guide at https://docs.gradle.org/2.13/userguide/multi_project_builds.html
- */
-
-/*
-// To declare projects as part of a multi-project build use the 'include' method
-include 'shared'
-include 'api'
-include 'services:webservice'
-*/
-
 rootProject.name = 'MUSIAL'

src/main/java/cli/CLIProfile.java

@@ -74,11 +74,17 @@ public static Options options() {
      * The content type to profile.
      */
     public enum Content {
-        // Per sample variants.
+        /**
+         * Per sample variants.
+         */
         VARIANTS,
-        // Per sample alleles of features.
+        /**
+         * Per sample alleles.
+         */
         ALLELES,
-        // Per sample proteoforms of features.
+        /**
+         * Per sample proteoforms.
+         */
         PROTEOFORMS
     }