v5.7.5

Self-synteny improvement docs

Author: cariann09

README.md

@@ -1,12 +1,10 @@
 ### SyMAP (Synteny Mapping and Analysis Program) ###
 
-**Description**: SyMAP is a software package for detecting, displaying, and querying syntenic relationships between a fully sequenced genome and   
-(i) another sequenced genome, (ii) draft sequence, and (iii) self-synteny.  It is designed for divergent eukaryotic genomes (not bacteria).
+**Description**: SyMAP is a software package for detecting, displaying, and querying syntenic relationships between a fully sequenced genome and (i) another sequenced genome, (ii) draft sequence, and (iii) self-synteny.  It is designed for divergent eukaryotic genomes (not bacteria).
 
 **Download SyMAP tarball**: https://github.com/csoderlund/SyMAP/releases. The SyMAP tarball contains the executable jar file, all necessary external software, and demo files.
 
-**Requirements**: Java v17 or later and MySQL (or MariaDB). SyMAP uses [MUMmer](https://mummer4.github.io) for the sequence to sequence alignment, which requires Perl.   
-SyMAP has been tested on Linux and MacOS.
+**Requirements**: Java v17 or later and MySQL (or MariaDB). SyMAP uses [MUMmer](https://mummer4.github.io) for the sequence to sequence alignment, which requires Perl. SyMAP has been tested on Linux and MacOS.
 
 **To use**: 
 1. Download the latest SyMAP tarball.

docs/3D.html

@@ -154,9 +154,10 @@ <h3>Draft: Demo-Draft to Demo-Seq2</h3>
 <h3>Self-synteny: Demo-Seq</h3>
 
 <table class="tx">
-<tr><td colspan="2"><tt>Demo-Seq</tt> has been run against itself, resulting in the <ttb>Dot Plot</ttb> shown on the lower right.
-There are a few off-diagonal blocks, which may indicate repetitiveness (but these blocks are probably
-too small to be meaningful).
+<tr><td colspan="2"><tt>Demo-Seq</tt> has been run against itself.
+There are a few blocks, but these blocks are probably too small to be meaningful.
+The blocks below the diagonal are mirrored above
+the diagonal, e.g. 3.5.1 and 5.3.1 are the same.
 <tr> <td style="vertical-align: text-top; width: 670px">
 <a href="img/demoSumSelf.png">
     <img src="img/demoSumSelf.png" alt="demo Sum Self" style="border: 1px solid black; width: 340px"></a>
@@ -217,7 +218,9 @@ <h2>symap.log</h2>
 <b>Algo1 verbose</b>
 
 <pre><small>
-Loading annotations
+Start calculating cluster hits for demo_seq and demo_seq2
+   Alignment files in data/seq_results/demo_seq_to_demo_seq2/align/
+   Loading annotations
     10,108 demo_seq genes
     10,137 demo_seq2 genes
    Scan files to create candidate genes from hits

docs/Demo.html

@@ -77,8 +77,12 @@ <h2>Collinear sets</h2>
 In the image on the right, two different sets are shown where the pink highlighted hits is the 1st set,
 green is the 2nd; these are <a href="UserGuide.html#wheel" class="ext" target="_blank">Color Wheel</a>
 <ttx>Highlight1</ttx> and <ttx>Highlight2</ttx> colors. The sets were broken by an extra gene on Human Chr01.
-<p>A collinear set either has all genes to the same strand (+/+, -/-) or different (+/-, -/+).
+<p>A collinear set either has all hits/genes to the same strand (+/+, -/-) or different (+/-, -/+).
 In the image on the right, all genes in both sets are on different strands.
+<p><b>Note</b>: if <ttl>Algo2</ttl>
+was used, this guarantees that the strands for the two homologous genes are the same as their hit; this is not
+the case for <ttl>Algo1</ttl> (see <a href="UserGuide.html#term" target="_blank" class="ext">Terminology</a>).
+
 
 <p>&nbsp;
 <p>The collinear set algorithm does not consider the amount of overlap of the hit to a gene, or the similarity

docs/Details.html

@@ -80,7 +80,7 @@
 <ul>
 <li><a href=SystemHelp.html class="ext" target="_blank">Parameters and Interface</a>
 <li><a href=MUMmer.html class="ext" target="_blank">MUMmer with SyMAP</a>
-<li><a href=Demo.html class="ext" target="_blank">Demo Results; Synteny Results</a>
+<li><a href=Demo.html class="ext" target="_blank">Results: Demo and Synteny</a>
 <li><a href=peach.html class="ext" target="_blank">Ordering wild Prunus yedoensis against Prunus persica</a>
 <li><a href=TroubleShoot.html class="ext" target="_blank">Trouble Shooting</a>
 
@@ -122,9 +122,9 @@ <h4>History of SyMAP</h4>
 was written to build physical maps from fingerprinted clones, but is obsolete now.
 SyMAP was originally written for FPC maps-to-sequence synteny
 (<a href="./fpc/UserGuideFPC.html" class="ext" target="_blank">FPC User Guide</a>),
-then was upgraded for sequence-to-sequence synteny. FPC was removed from SyMAP in v5.0.8,
-but is still available from Github downloads and
-<a href="http://www.agcol.arizona.edu/software/symap" class="ext" target="_blank">www.agcol.arizona.edu</a>.
+then was upgraded for sequence-to-sequence synteny. FPC was removed from SyMAP in v5.0.9,
+but v5.0.8 is still available from Github downloads; if it does not work with latest Java and MySQL,
+email me and I will update it for you.
 </ttt>
 <!---- END BODY -->
 	</td></tr>

docs/Docs.html

@@ -634,7 +634,7 @@ <h4>2.2. Single genes columns</h4>
 <ttl>Gene Info</ttl> and <ttl>Annotation</ttl> columns, with one additional column, as follows.
 <table class="tx">
 	<tr><td style="vertical-align: text-top;"><ttx>NumHits</ttx>
-	<td>This is the number of hits to the gene in the ENTIRE databases, except for SELF synteny.
+	<td>This is the number of hits to the gene in the ENTIRE databases, including SELF synteny.
 </table>
 For example:
 <table><tr>
@@ -652,7 +652,7 @@ <h4>2.2. Single genes columns</h4>
 	<td>Gene #1.2 is an orphan for Arab-Cabb but has a count of 1 because it is not an orphan in Arab-Brap.
 	<br>It is not listed in the Arab-Brap-Cabb table because Arab-Brap has a hit to it.
 </table>
-
+Note: For self-synteny, the second row of columns will be empty.
 <a id="save"></a>
 <h4>2.3 Columns and order shown</h4>
 

docs/Query.html

@@ -55,19 +55,82 @@
 
 	<tr><td style='vertical-align: top; text-align: left; padding:15px;'>
 	<!---- START BODY -->
-For existing databases, there are no database or build changes since <a href="#v560">v5.6.0</a>.
-<br>You must have Java v17.0.11 or later, starting with SyMAP v5.5.0.
+For existing databases, there are no database or build changes since <a href="#v575">v5.7.5</a> for self-synteny and  <a href="#v560">v5.6.0</a> for all other inputs.
+You must have Java v17.0.11 or later, starting with SyMAP v5.5.0.
 
 <h3>Summary of recent major releases</h3>
 <table>
-<tr><td><a href="#v572">v5.7.2</a><td>(2-Sept-25)<td>Improved synteny algorithm using <ttx>Strict</ttx> option.
-<tr><td><a href="#v560">v5.6.0</a><td>(23-Jan-25)<td>Improve <ttx>Cluster Hit</ttx> algorithm <ttx>Algo2</ttx>,
+<tr><td><a href="#v572">v5.7.5</a><td>(15-Oct-25)<td>Self-synteny has almost complete support.
+<tr><td><a href="#v572">v5.7.2</a><td>(02-Sep-25)<td>Improve synteny algorithm using new <ttx>Strict</ttx> option.
+<tr><td><a href="#v560">v5.6.0</a><td>(23-Jan-25)<td>New <ttx>Cluster Hit</ttx> algorithm <ttx>Algo2</ttx>,
 which is aware of the exon-intron structure.
-<tr><td><a href="#v557">v5.5.7</a><td>(8-Aug-24)<td>Add <ttl>xToSymap</ttl> and improve ease of input.
+<tr><td><a href="#v557">v5.5.7</a><td>(08-Aug-24)<td>Add <ttl>xToSymap</ttl> and improve ease of input.
 <tr><td><a href="#v556">v5.5.6</a><td>(14-July-24)<td>Improved the collinear set algorithm.
-<tr><td><a href="#v540">v5.4.0</a><td>(6-Apr-23)<td>Improved <ttx>Cluster Hit</ttx> algorithm <ttx>Algo1</ttx> for the detection of gene homology.
+<tr><td><a href="#v540">v5.4.0</a><td>(06-Apr-23)<td>Improved <ttx>Cluster Hit</ttx> algorithm <ttx>Algo1</ttx> for the detection of gene homology.
 </table>
 
+<a id="v575"></a>
+<h3>Release: v5.7.5 (25-Oct-25)</h3>
+<i>Self-synteny</i>: This release has updated algorithms and <ttp>Queries</ttp> works for self-synteny (see <a href="UserGuide.html#self" target="_blank" class="ext">Self-synteny</a>).
+<p>For existing self-synteny databases: Re-run the full <ttp>A&S</ttp> algorithm.
+<p>
+<ul>
+<li>Self-synteny algorithm and display. In summary, the updates are:
+<ttp>Queries</ttp> works for self-synteny, the summaries are all correct now, the <ttx>Block#</ttx> and <ttx>Hit#</ttx>
+are consistent now, and a few tiny display bug fixes. For more detail:
+<ul>
+<li>Algorithm:
+<ul>
+<li>Due to internal changes, the results may not be exactly the same but only on small 'iffy' blocks, e.g.
+the demo self-synteny is different, but all the blocks were iffy to start with.
+<li>The numbering of blocks and hits will be different.
+<li>The self-chromsomes no longer have hits representing the diagonal which reduces the number of hits. However,
+the diagonal is still shown on the <ttp>Dotplot</ttp>.
+<li><ttx>NumHits</ttx> (<ttp>Query</ttp> <ttl>Single</ttl> column) now includes self-synteny hits.
+<li><ttx>Collinear sets</ttx> results were not very useable because it did not mirror sets or work for
+self-chromosomes; both of these issues have been fixed.
+</ul>
+<li><ttx>Block#</ttx> of mirrored blocks (same above and below the diagonal) now have the same number;
+this is most obvious in <ttp>Dotplot</ttp>.
+<li><ttx>Hit#</ttx> two mirrored hits now have the same number; this is most obvious in <ttp>2D</ttp>.
+<li><ttp>2D</ttp>: The following are fixed: For the <ttl>Sequence Filter</ttl>, the <ttx>Gene# Hit</ttx> did not work.
+For the hit popup, when the two chromsomes were the same, a 2g hit would show the same
+gene for both ends of hit.
+<li><ttp>Reports...</ttp>: both <ttl>Summary</ttl> and <ttl>Block Report</ttl> only report
+blocks and hits for one side of the diagonal (they no longer double the blocks and hits).
+<li><ttp>Query</ttp>: Now works for self-synteny with a few exceptions (see <a href="UserGuide.html#self" target="_blank" class="ext">Self-synteny</a>).
+</ul>
+
+<li><ttp>Reports:</ttp>
+<ul><li>The <tt>Species</tt> columns have been removed and the name is now
+over species specific columns. See <a href="UserGuide.html#breport" target="_blank" class="ext">Report</a>.
+</ul>
+<li>Small things:
+<ul>
+<li><ttp>Project Manager</ttp>:
+<ul>
+<li>All computations that read files sorts them before processing; this is not necessary, but more logical for the output.
+For <ttx>Cluster Algo1</ttx>, this can slightly change the hits found.
+<li>If only one project is selected, the <ttx>Selected Pair</ttx> cell will be shown as selected.
+<li>The lower <ttl>All Pairs</ttl> is now <ttl>All &#10003; Pairs</ttl>.
+<li><ttl>Verbose</ttl> did not work for the synteny algorithm if not set the first time
+it was run.
+<li><ttl>Pseudo Genes</ttl> uses <ttx>Directory</ttx> name in comment instead of <ttx>Abbreviation</ttx> since it does not change.
+<li><ttx>Align PROmer</ttx> or <ttx>Align NUCmer</ttx> have been added to the <ttp>Summary</ttp>.
+</ul>
+<li><ttp>2D</ttp>: <ttl>Sequence Filter</ttl> has a few labels changed and re-ordered.
+<li><ttp>Queries</ttp>: The number of blocks and collinear sets was wrong, plus unclear what the numbers were.
+Also, it did not work right if there was two abbreviations the same; now it warns the user.
+</ul>
+<li><small>Developers:</small>
+<ul>
+<li><small>AnchorMain1 self-synteny save to database methods have been re-written to not be
+dependent on the exact Schema. The diagonal hits are no longer loaded in database as they are only used
+in the Dotplot, which creates the diagonal on the fly. The <tt>stddev</tt> database column was removed and some columns moved around.
+Remove <tt>AnchorMain1.BinStats.java</tt></small>.
+</ul>
+</ul>
+
 <a id="v574"></a>
 <h3>Release: v5.7.4 (30-Sept-25)</h3>
 Small <ttp>Synteny</ttp> <ttx>Strict</ttx> and interface improvements.
@@ -310,7 +373,6 @@ <h3>Release: v5.6.7 (22-May-25)</h3>
 Any java file I looked at, I took out old CASddd comments.</small>
 </ul>
 
-
 <a id="v566"></a>
 <h3>Release: v5.6.6 (6-May-25)</h3>
 The MacOS M4 silicon executables have been added to the <tt>/ext</tt> directory.
@@ -2132,8 +2194,7 @@ <h3>Release: v5.1.1 (31July22)</h3>
 <li>Scripts:
 <ul>
 <li>The <tt>symap</tt> and <tt>viewSymap</tt> scripts have been changed from Perl to shell.
-<li>The <tt>viewSymap3D</tt> script has been removed, but the <a href="./3D.html" class="ext" target="_blank">3D</a> documentation explains
-how you could recreate the 3D view.
+<li>The <tt>viewSymap3D</tt> script has been removed; 3D is no longer available.
 </ul>
 <li>Data directory
 <ul>