-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathdata-parsing.html
More file actions
230 lines (205 loc) · 13.6 KB
/
data-parsing.html
File metadata and controls
230 lines (205 loc) · 13.6 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
<!DOCTYPE HTML>
<html lang="en">
<head>
<title id="title">Jules van Irsel</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1" />
<link rel="stylesheet" href="assets/css/main.css">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.0.0-beta3/css/all.min.css">
<script src="assets/js/jquery.min.js"></script>
</head>
<body>
<div class="page-margins">
<!-- navigation -->
<nav id="nav" class="nav-panel">
<button id="nav-close-btn" onclick=closeNav() type="button" title="closeNav"></button>
<ul>
<li><a href="index.html">Home</a></li>
<li><a href="documents/cv.pdf" target="_">CV</a></li>
<li><a href="javascript:void(0)" onclick=openNavA()>Research</a>
<ul id="navA">
<li><a href="auroral-science.html">Auroral System Science</a></li>
<li><a href="plasma-modeling.html">Ionospheric Plasma Modeling</a></li>
<li><a href="conferences.html">Meetings & Conferences</a></li>
</ul>
</li>
<li class="current"><a href="javascript:void(0)" onclick=openNavB()>Rocket Science</a>
<ul id="navB">
<li><a href="317-lab.html">The Lynch Rocket Lab</a></li>
<li><a href="data-parsing.html">Data Parsing</a></li>
<li><a href="vacuum-testing.html">Vacuum Testing</a></li>
<li><a href="fieldwork.html">Fieldwork Blog</a></li>
<li><a href="javascript:void(0)" onclick=openNavBB() id="rockets">Participated Missions</a>
<ul id="navBB">
<li><a href="https://sites.dartmouth.edu/lynch-rocket-lab/rockets/gneiss/#:~:text=The%20Geophysical%20Non-Equilibrium%20Ionospheric%20System%20Science%20%28GNEISS%29%20rockets%2C,imagery%2C%20to%20the%20study%20of%20non-idealized%2C%20non-sheetlike%20nightsi" target="_">GNEISS</a></li>
<li><a href="https://science.nasa.gov/solar-system/skywatching/nasa-to-launch-sounding-rockets-into-moons-shadow-during-solar-eclipse/" target="_">APEP-2</a></li>
<li><a href="https://www.nasa.gov/missions/sounding-rockets/nasa-rocket-team-to-chase-pulsating-aurora/" target="_">LAMP</a></li>
<li><a href="https://www.nasa.gov/missions/sounding-rockets/science-at-the-cusp-nasa-rocket-to-study-mysterious-area-above-the-north-pole/" target="_">C-REX-2</a></li>
<li><a href="https://www.nasa.gov/missions/sounding-rockets/nasa-wallops-may-rocket-launch-exploring-energy-transport-in-space/" target="_">KiNET-X</a></li>
<li><a href="https://www.nasa.gov/missions/sounding-rockets/visions-2-provides-a-look-at-earths-escaping-atmosphere/" target="_">VISIONS-2</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="javascript:void(0)" onclick=openNavC()>Proposals</a>
<ul id="navC">
<li><a href="documents/finesst-proposal.pdf" target="_">FINESST (NASA Grad. Funding)</a></li>
<li><a href="https://nspires.nasaprs.com/external/solicitations/summary!init.do?solId={10514572-CA34-A48D-7887-425E5A561AD2}&path=open" target="_">GNEISS (NASA HLCAS)</a></li>
<li><a href="https://nspires.nasaprs.com/external/solicitations/summary!init.do?solId={CE94D9F1-858E-F22E-513F-8DF6F389B4AF}&path=open" target="_">ARCS (NASA MIDEX)</a></li>
<li><a href="documents/thesis-proposal.pdf" target="_">Thesis Proposal</a></li>
</ul>
</li>
<li><a href="javascript:void(0)" onclick=openNavD()>Education</a>
<ul id="navD">
<li><a href="grad-courses.html">Physics Ph.D.</a></li>
<li><a href="undergrad-courses.html">Astrophysics B.Sc.</a></li>
<li><a href="diploma-courses.html">Mech. Engineering Diploma</a></li>
<li><a href="other-courses.html">Supplementary Courses</a></li>
</ul>
</li>
<li><a href="javascript:void(0)" onclick=openNavE()>Publications</a>
<ul id="navE">
<li><a href="documents/jgr_vanirsel_etal_2025.pdf">van Irsel, et al. (2025)</a></li>
<li><a href="https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2024JA032722" target="_">van Irsel, et al. (2024)</a></li>
<li><a href="documents/arcs-section-d.pdf" target="_">Lynch, et al. (2024)</a></li>
<li><a href="documents/arcs-section-e.pdf" target="_">Erlandson, et al. (2024)</a></li>
</ul>
</li>
<li><a href="javascript:void(0)" onclick=scrollToForm()>Contact</a></li>
</ul>
</nav>
<!-- dark mode -->
<div id="dark">
<label class="switch">
<input id="darkmode" type="checkbox" title="Dark mode">
<span class="slider round"></span>
</label>
</div>
<div class="hline"></div>
<div class="section-title"><p>Rocket Instrument Data Parsing</p></div>
<div class="hline"></div>
<!-- content -->
<div class="content">
<a href="images/realtime-data.png" target="_" class="img">
<img id="realtime-data" src="images/realtime-data.png" title="realtime-data">
</a>
<div class="caption-right">
<h2>Realtime Data Parsing</h2>
<p>
Rocket instruments demand constant, real-time monitoring to assess hardware health and nominal behavior. This is
especially important when the rocket is on the rail, 60 seconds from launch, awaiting a go/no-go status check.
For this reason, the code or software that parses instrument data streams is a crucial component of rocket
instrument design.
<br><br>
I contributed to the development of such monitoring software for the rocket Mini Plasma Imager
(<a href="https://ucalgary.ca/news/scientists-mini-instruments-help-measure-weather-space" target="_">rMPI</a>)
and overhauled the parsing code for the Petite Ion Probes
(<a href="https://pubs.aip.org/aip/rsi/article/91/9/094503/908508/Auroral-ionospheric-plasma-flow-extraction-using" target="_">PIP</a>),
transitioning it to Python 3 to take advantage of its significantly improved binary data handling. A time
step of the real-time plotting is shown here.
<br><br>
The PIPs communicate using the RS-422 protocol with asynchronous data streams. This necessitates the use of
sentinel values to indicate the start of a specific message. The PIP data can include up to four messages per
data point.
</p>
</div>
</div>
<div class="hline"></div>
<div class="content">
<div class="caption">
<h2>Pass-Through Parsing</h2>
<p>
Sounding rockets collect only a few minutes of data during flight, making it vital to prevent data dropouts or,
worse, parser crashes. To enhance reliability, I divided the parser into two top-level functionalities. This
first script, <a href="documents/code/shield_feed.txt" target="_"><em>shield_feed.py</em></a>, has just two
tasks: read data from the serial port and write it to standard output and a data file. The only event that can
terminate this script is a Keyboard Interrupt, ensuring that, regardless of issues with the parsing and plotting
scripts, the data will not be lost.
<br><br>
The output of this script can be passed through, or <em>piped</em>, into a second script,
<a href="documents/code/shield_realtime.txt" target="_"><em>shield_realtime.py</em></a>, which handles parsing
and plotting. One particularly challenging aspect of this process is synchronization given that the cadence
of the instrument data stream and the speed at which the parsing and plotting happens do not necessarily align.
To address this, the number of bytes parsed per time step adjusts gradually, allowing the parser to function on
different systems with various processing speeds.
<br><br>
The pass-through parser setup has another benefit in that a saved data file can be piped into the real-time
parser script, replaying the data file. Additional scripts I wrote are
<a href="documents/code/shield_plot.txt" target="_"><em>shield_plot.py</em></a>, which parses and plots an
entire data file at once, and
<a href="documents/code/shield_simulator.txt" target="_"><em>shield_simulator.py</em></a>, which simulates the
data stream of an instrument by reading a data file and looping it over a virtual serial port connection.
</p>
</div>
<a href="documents/code/shield_realtime.txt" target="_" class="img">
<img src="images/parser-code.png" title="parser-code">
</a>
</div>
<div class="hline"></div>
</div>
<!-- footer -->
<div class="footer-margins">
<div class="footer" id="footer">
<div id="form">
<h4>Get in Touch</h4>
<form action="https://formspree.io/f/xrgnerza" method="post">
<input type="text" name="name" id="name" placeholder="Name" autocomplete="off" />
<input type="email" name="email" id="email" placeholder="Email" autocomplete="off" />
<textarea name="message" id="message" placeholder="Message" rows="5" autocomplete="off"></textarea>
<input type="submit" class="button" value="Send Message" />
</form>
</div>
<div id="links">
<div>
<h5>Transcripts</h5>
<ul>
<li><a href="documents/transcript-dartmouth.pdf" target="_">Dartmouth College</a></li>
<li><a href="documents/transcript-uofc.pdf" target="_">University of Calgary</a></li>
<li><a href="documents/transcript-sait.pdf" target="_">SAIT Polytechnic</a></li>
</ul>
</div>
<div>
<h5>Selected Publications</h5>
<ul>
<li><a href="documents/jgr_vanirsel_etal_2025.pdf" target="_">van Irsel et al. (2025), <em>JGR</em></a></li>
<li><a href="https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2024JA032722" target="_">van Irsel et al. (2024), <em>JGR</em></a></li>
<li><a href="documents/arcs-section-d.pdf" target="_">Lynch et al. (2024), <em>NASA</em></a></li>
</ul>
</div>
<div>
<h5>Documents</h5>
<ul>
<li><a href="documents/cv.pdf" target="_">Curriculum Vitae</a></li>
<li><a href="documents/thesis.pdf" target="_">Dissertation</a></li>
<li><a href="documents/thesis-proposal.pdf" target="_">Thesis Proposal</a></li>
<li><a href="documents/finesst-proposal.pdf" target="_">NASA FINESST Proposal</a></li>
</ul>
</div>
<div>
<h5>In the news</h5>
<ul>
<li><a href="https://www.bostonglobe.com/2023/10/13/metro/darmouth-professor-kristina-lynch-rockets-eclipse-nasa/" target="_">Rockets Studying the Eclipse</a></li>
<li><a href="https://www.astech.ca/archives/indexofpastwinners/space-physics-group-university-of-calgary" target="_">2018 ASTech Special Award</a></li>
<li><a href="https://home.dartmouth.edu/news/2020/09/dartmouth-proposal-chosen-possible-space-mission" target="_">ARCS Proposal</a></li>
<li><a href="https://www.youtube.com/watch?v=QiX9xL_YhvM" target="_">Surveying the Northern Lights</a></li>
</ul>
</div>
</div>
</div>
<!-- icons -->
<div class="icons">
<a href="https://github.com/julesvanirsel" target="_"><i class="fab fa-github"></i></a>
<a href="https://www.linkedin.com/in/jules-van-irsel-a3413695" target="_"><i class="fab fa-linkedin"></i></a>
<a href="https://www.instagram.com/juicevanirsel" target="_"><i class="fab fa-instagram"></i></a>
</div>
<!-- copyright -->
<div class="copyright" id="copyright">
© 2025 Jules van Irsel
</div>
</div>
</body>
<!-- scripts -->
<script src="assets/js/jquery.dropotron.min.js"></script>
<script src="assets/js/main.js"></script>
<script src="assets/js/darkmode.js"></script>
</html>