Missing fits files

[samaloney]

Some TM has not been converted into fits files for various reasons this issue to track the overall progress on the analysis and resolution of these issues.

• Produce fits files for some incomplete packet sequences #370 Incomplete Packets (reshape error in CPD)
• Add try except around RID lookup #397 RID lookup error
• Incomplete packet reporting #354
• Investigate number of incomplete science data requests #351
• TM Processing 21,6,21 packet not parsable #273
• Disseapearing packets (? below)
• 2x AttributeError: 'PacketData' object has no attribute 'NIX00268'

This table covers a list of currently missing L0 files based of of LB files created using the following script modified for each type

def missing_lb_l0(num):
    lb = Path(f'LB/21/6/{num}').glob("*.fits")
    l0 = Path(f'L0/21/6/{num}').glob("*.fits")
    lb_uid = [f.name.split('_')[-1] for f in lb] # extract UID-SEQ part
    l0_uid = [f.name.split('_')[-1] for f in l0]
    return set(lb_uid).difference(set(l0_uid))

Filename	Incomplete	DC Missing list	Miss Type	UID	Error/reason
solo_LB_stix-21-6-21_0000000000-9999999999_V02_2112090002-51143.fits	FALSE	TRUE		2112090002	ValueError: cannot reshape array of size 154752 into shape (7,31,32,12)
solo_LB_stix-21-6-21_0000000000-9999999999_V02_2203151617-62310.fits	FALSE	TRUE		2203151617	bitstring.ReadError: Reading off the end of the data. Tried to read 8 bits when only 0 available.
solo_LB_stix-21-6-21_0000000000-9999999999_V02_2204028914-63370.fits	FALSE	TRUE		2204028914	?
solo_LB_stix-21-6-21_0000000000-9999999999_V02_2209280787-63265.fits	FALSE	TRUE		2209280787	?
solo_LB_stix-21-6-21_0000000000-9999999999_V02_0074661904-00000.fits	FALSE	FALSE		74661904	?
solo_LB_stix-21-6-21_0000000000-9999999999_V02_2304131472-49403.fits	FALSE	TRUE		2304131472	?
solo_LB_stix-21-6-21_0000000000-9999999999_V02_2305117159-57036.fits	TRUE	TRUE	Cont	2305117159	ValueError: cannot reshape array of size 1974528 into shape (191,27,32,12)
solo_LB_stix-21-6-21_0000000000-9999999999_V02_2302120212-60797.fits	FALSE	TRUE		2302120212	?
solo_LB_stix-21-6-21_0000000000-9999999999_V02_2302257105-61109.fits	FALSE	TRUE		2302257105	?
solo_LB_stix-21-6-21_0000000000-9999999999_V02_2402287253-57269.fits	TRUE	TRUE	Cont	2402287253	ValueError: cannot reshape array of size 4308096 into shape (661,17,32,12)
solo_LB_stix-21-6-21_0000000000-9999999999_V02_2403296400-62046.fits	TRUE	TRUE	Cont	2403296400	ValueError: cannot reshape array of size 2324736 into shape (357,17,32,12)
solo_LB_stix-21-6-21_0000000000-9999999999_V02_2404042006-63069.fits	FALSE	TRUE		2404042006	?
solo_LB_stix-21-6-21_0000000000-9999999999_V02_2404213657-64616.fits	FALSE	TRUE		2404213657	RID lookup: KeyError: 'No matches found for key 2404213657'
solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405017696-64617.fits	FALSE	TRUE		2405017696	RID lookup: KeyError: 'No matches found for key 2405017696'
solo_LB_stix-21-6-21_0000000000-9999999999_V02_2404206759-64619.fits	FALSE	TRUE		2404206759	RID lookup: KeyError: 'No matches found for key 2404206759'
solo_LB_stix-21-6-21_0000000000-9999999999_V02_2405066983-64618.fits	FALSE	TRUE		2405066983	RID lookup: KeyError: 'No matches found for key 2405066983'
solo_LB_stix-21-6-21_0000000000-9999999999_V02_2404189403-64620.fits	FALSE	TRUE		2404189403	RID lookup: KeyError: 'No matches found for key 2405066983'
solo_LB_stix-21-6-21_0000000000-9999999999_V02_2404132170-64621.fits	FALSE	TRUE		2404132170	RID lookup: KeyError: 'No matches found for key 2404132170'
solo_LB_stix-21-6-21_0000000000-9999999999_V02_2407080920-55959.fits	FALSE	FALSE		2407080920	ValueError: cannot reshape array of size 43008 into shape (3,28,32,12)
solo_LB_stix-21-6-24_0000000000-9999999999_V02_2201270033-59178.fits	TRUE	FALSE	First	2201270033	ValueError: cannot reshape array of size 43008 into shape (3,28,32,12)
solo_LB_stix-21-6-24_0000000000-9999999999_V02_2201260047-59172.fits	FALSE	FALSE		2201260047	?
solo_LB_stix-21-6-24_0000000000-9999999999_V02_2201270032-59177.fits	TRUE	FALSE	First	2201270032	?
solo_LB_stix-21-6-24_0000000000-9999999999_V02_2307113758-59050.fits	FALSE	TRUE		2307113758	?
solo_LB_stix-21-6-24_0000000000-9999999999_V02_2303248707-53119.fits	FALSE	TRUE		2303248707	ValueError: cannot reshape array of size 29 into shape (64,newaxis)
solo_LB_stix-21-6-24_0000000000-9999999999_V02_2303258858-53121.fits	FALSE	TRUE		2303258858	ValueError: cannot reshape array of size 29 into shape (64,newaxis)
solo_LB_stix-21-6-24_0000000000-9999999999_V02_2407196281-57017.fits	TRUE	FALSE	Last	2407196281	Incomplete sequence <LevelB LB, 21, 6, 24 778565778:9102-778566616:27036>
solo_LB_stix-21-6-24_0000000000-9999999999_V02_2406115491-52941.fits	FALSE	FALSE		2406115491	_get_compression_scheme: AttributeError: 'PacketData' object has no attribute 'NIX00268'
solo_LB_stix-21-6-24_0000000000-9999999999_V02_2307117985-59051.fits	FALSE	TRUE		2307117985	?
solo_LB_stix-21-6-24_0000000000-9999999999_V02_2209279737-62645.fits	FALSE	TRUE		2209279737	_get_compression_scheme: AttributeError: 'PacketData' object has no attribute 'NIX00268'
solo_LB_stix-21-6-24_0000000000-9999999999_V02_2406111879-52942.fits	FALSE	FALSE		2406111879	RID lookup: KeyError: 'No matches found for key 2406111879'
solo_LB_stix-21-6-24_0000000000-9999999999_V02_2303243039-53120.fits	FALSE	TRUE		2303243039	ValueError: cannot reshape array of size 29 into shape (64,newaxis)
solo_LB_stix-21-6-24_0000000000-9999999999_V02_2303251375-53122.fits	FALSE	TRUE		2303251375	ValueError: cannot reshape array of size 29 into shape (64,newaxis)