Support run mode providing FASTQ pair and TSV with allowable cell/umi barcodes

[bbimber]

As is, nimble accepts either a pair of FASTQs as input (for bulk RNA-seq), or a 10x BAM. The only reason we need this BAM is b/c it has the cellbarcode+UMIs parsed from the raw reads. This is somewhat limiting, because this means we need to re-create that BAM every time before running nimble. We should support a third mode:

1. I created a tool that will iterate the 10x BAM and write out two TSVs, one for each CR->CB and UR->UB mapping (https://github.com/BimberLab/DISCVRSeq/blob/c456a336cbb7bed2c7c17a7b72781554f3004554/src/main/java/com/github/discvrseq/walkers/Save10xBarcodes.java#L62). The idea is to provide a map of raw->corrected sequence for all valid combination that CellRanger found in the data.

2. We should make a python-based entrypoint (maybe just a different argument in the existing 'align' task). This would accept a N pairs of FASTQs as input, along with TSVs for raw->corrected cellbarcode and raw->corrected UMIs. We also need to supply the cellbarcode pattern and UMI pattern.

3. The python code should read those TSVs into memory as hash maps. It should then iterate each pair of FASTQs (Ideally we with some kind of dual iterator to read both forward/reverse at once). For each read pair, parse R1 to grab the raw CB and raw UMI. Use the in-memory lookup to translate that to corrected CB/UB.

4. Strip the barcode portion from R1 and then write both forward and reverse reads to a new BAM file (as unaligned reads). If the R1 has no bases beyond the barcode information, skip it. Write UB/CB tags in that BAM.

5. This new BAM should be able to feed directly into the existing codepath. We still need to sort it, but all downstream code is exactly the same as for a regular 10x BAM.

[bbimber]

We've been generating these per-lane barcode TSVs here: https://prime-seq.ohsu.edu/query/Labs/Bimber/executeQuery.view?queryName=outputfiles&query.category~eq=10x%20CellBarcode%20Map&schemaName=sequenceanalysis

[bbimber]

@hextraza: this is largely done, but I want to confirm a few things:

• We discussed walking back our last change to the fastq-to-bam entrypoint, meaning that tool would accept one TSV with two columns, containing the raw and corrected CBs. The python tool should read this map and do CB correction. It should the UMI as-is from the FASTQs.

• I noticed that the CB tags sometimes have a "-1" suffix. Is the tool handling this in some form? If not, can you please add detection/parsing for CBs ending with the regex "-[0-9]+$", and strip that off? I will fix the DISCVRseq tool to drop them upstream too; however, we have existing tables like this (see below).

An example TSV (prior to fixing the -1) is:

RawCellBarcode	CorrectedCellBarcode
CCTCTAAAGAATGCAA	GCTCTAAAGAATGCAA-1
ANTAATCAGCTGCAGT	ACTAATCAGCTGCAGT-1
ACAATTGGTAGGCGAT	ACAATTGGTAGGGGAT-1

[bbimber]

After further review, I think we might want to implement 10x's algorithm for parsing/correcting cell barocdes in python. I dont think it needs to be that bad. We need to do this because there isnt a consistent mapping between uncorrected and corrected CB. This example illustrates way

AC-A-C-CCAAGAGAGGTT: This is a raw cell barcode, encountered in more than one read
AC-A-T-CCAAGAGAGGTT: In one instance, it was corrected to this
AC-G-C-CCAAGAGAGGTT: In another instance, it was corrected to this

10x does the correction by comparing the raw barcode to a list of allowable CBs. If there's a perfect match, it uses that. If not, it identifies any candidates with hamming distance = 1. Within those candidates, 10x identifies the base that's different and looks at the quality score in the raw read. It prioritizes within candidates based on that score. This means it picks the candidate where the changed base has the lowest quality (i.e., it's the least confident residue).

Specific changes mean:

• nimble's fastq-to-bam needs to accept a barcode inclusion list (https://kb.10xgenomics.com/s/article/115004506263-What-is-a-barcode-inclusion-list-formerly-barcode-whitelist). this is always the same for a given chemistry, not data-specific.
• stop requiring any sort of barcode map beyond that.
• read that inclusion list into memory, compare raw CBs against it
• implement the hamming distance check (doing the same in-memory caching we talked about)
• if there is more than one hit with HD==1, use quality score to prioritize the correction