Skip to content

GOALS.md

Latest commit

 

History

History
72 lines (62 loc) · 5.44 KB

GOALS.md

File metadata and controls

72 lines (62 loc) · 5.44 KB

Current Implementation Status (December 19 2025)

✅ COMPLETED GOALS:

  • Full BBL binary format parsing using JavaScript blackbox-log-viewer and C blackbox-tools references
  • Complete I, P, S, H, G, E frame parsing with proper predictor implementation
  • Header parsing and field definition extraction with firmware metadata
  • CSV export with field structure following blackbox_decode conventions (exact ordering compatibility not comprehensively validated)
  • Headers CSV export with complete configuration and frame definitions
  • Proper field encoding/decoding (SIGNED_VB, UNSIGNED_VB, NEG_14BIT, TAG8_8SVB, TAG2_3S32, TAG8_4S16)
  • Motor value prediction with accurate P-frame decoding
  • S-frame timestamp inheritance and data merging with main CSV stream
  • Multi-log detection and separate file generation (.01.csv, .02.csv, etc.)
  • Basic unit conversions (voltage, current) with proper scaling
  • Energy calculation (energyCumulative field) integration
  • Time-sorted CSV output with proper chronological ordering
  • Debug mode with comprehensive frame-by-frame analysis and sampling
  • Large file streaming support (tested: 375K+ frames single log in 6.7 seconds; multi-log files with 400K+ combined frames) with memory efficiency
  • G-frame (GPS) parsing and GPX file export with coordinate conversion
  • E-frame (event) parsing and JSONL event export with Betaflight FlightLogEvent enum
  • Betaflight firmware-accurate flag formatting (flightModeFlags, stateFlags, failsafePhase)
  • Official Betaflight event type mapping (sync beep, disarm, flight mode change, log end)
  • Extensive Betaflight/EmuFlight testing with high compatibility across firmware versions
  • Full RUST CRATE for library reusability and modularity with complete API access
  • API documentation and library integration with comprehensive usage examples
  • Library/CLI separation: Parsing duplication removed, parse_single_log exposed in library
  • Configurable export filtering: Heuristics moved to library, accessible via should_skip_export() and has_minimal_gyro_activity()
  • ExportReport type: Structured path tracking for all export operations
  • Public API audit: Zero public functions in CLI, thin wrapper architecture
  • Comprehensive test coverage: 62 unit tests for parsing, filtering, conversions, and exports

🔧 REMAINING WORK: Feature Enhancements

  • Error handling refinement: Some unwrap() calls remain in test/example code; critical paths use proper Result handling
  • Performance validation: Streaming architecture proven effective; tested up to 375K frames in single log (~6.7 seconds for 21MB file)
  • GPS & Event testing: Both formats working and validated (833+ G frames, 5+ E frames; valid GPX and JSON outputs)
  • Unit conversions expansion: Voltage and current conversions complete and tested; missing: time scaling, altitude conversion, speed units, rotation rates, acceleration scaling
  • IMU simulation: Roll/pitch/yaw angle computation from gyro/accel/mag data — not started
  • Current meter simulation: Energy integration improvements — not started
  • GPS data merge: Integrate GPS data into main CSV (currently separate .gps.gpx file) — not started
  • Raw mode output: Export unprocessed sensor values without scaling — not started
  • Enhanced statistics: Loop timing statistics, frame distribution analysis — not started
  • Extended firmware testing: Validates Betaflight 4.0+, EmuFlight, INAV across 6+ years of firmware releases
  • Advanced filtering options: Current implementation: duration + gyro variance heuristics; advanced options not implemented

Implement the actual BBL binary format specification by explicitly replicating the JavaScript code and/or the C code from the Betaflight blackbox-log-viewer and/or blackbox_decode using the following sources: https://github.com/betaflight/blackbox-tools/blob/master/src/blackbox_decode.c https://raw.githubusercontent.com/betaflight/blackbox-log-viewer/master/src/flightlog.js

The goal is to fully read, parse and decode binary BBL files. Do not re-invent, explicitly use the javascript and C as a source to create the RUST project's code.

P, I, S frames:

Every BBL contains headers in plaintext which contain important information about the aircraft's settings, but more importantly, they contain details about the binary data and how to decode them: Field I name Field I signed Field I predictor Field I encoding Field P predictor Field P encoding Field S name Field S signed Field S predictor Field S encoding

Each BBL may or may not contains multiple flights logs. Each flight log starts with it's own set of plaintext headers. Each flight log within a BBL will contain I frames and P frames, and maybe contain E frames and S frames and G frames. G frames are GPS. H frames are GPS home position markers.

Binary utility blackbox_decode outputs useful statistics and creates .csv, .gpx, and .event files that contain the flightlog data. We can use it for data comparison, but do not embed nor call binary tools from within the RUST program. blackbox_decode --limits can be used for any *.BBL file. The --limits is only useful to see the min and max loopIteration and time.

Two RUST projects on github that may help or may hinder, i do not know. I never inspected the first, and the second is betaflight version specific and not up to date for Betaflight 4.6.

  1. https://github.com/ilya-epifanov/fc-blackbox
  2. https://github.com/blackbox-log/blackbox-log