Gyro spectrum analysis fails on logs without gyroUnfilt data

[nerdCopter]

Problem

Many spectrum analysis graphs show "Data Unavailable" for older Betaflight logs (4.0.x, 4.3.x) that contain valid flight data.

Root Cause

The filter delay analysis in src/data_analysis/filter_delay.rs requires BOTH gyroADC (filtered) AND gyroUnfilt (unfiltered) data:

if let (Some(filtered), Some(unfiltered)) = (row.gyro[axis], row.gyro_unfilt[axis]) {
    filtered_data.push(filtered as f32);
    unfiltered_data.push(unfiltered as f32);
}

Note: The renderer already has fallback logic to use debug[0-2] as unfiltered gyro when gyroUnfilt[0-2] is not available (implemented in src/data_input/log_parser.rs). This works for many logs where pilots configured debug mode.

Problem occurs when BOTH sources are missing:

• gyroUnfilt[0-2] - not available in older firmware ✗
• debug[0-2] - pilot did not configure debug mode for this particular flight ✗

Historical Context

Older Betaflight (< 4.5): Unfiltered gyro logging depended on configuration:

• Default: Only gyroADC[0-2] (filtered) was logged
• Debug mode configured: Unfiltered gyro logged to debug[0-2] channels
  • Common debug modes: GYRO_SCALED, GYRO, GYRO_RAW
  • Many pilots configured this when tuning PIDs or filters
  • Some pilots left it off for normal flights (battery/performance/storage reasons)
• Renderer already handles debug fallback - this works for many logs

Modern Betaflight (≥ 4.5): Unfiltered gyro is logged automatically to gyroUnfilt[0-2] fields by default, so this is not an issue for recent logs.

When Does The Problem Occur?

1. Tuning flights: Usually have debug mode → Works fine (fallback to debug)
2. Normal flights with debug disabled: No unfiltered data → Fails (this issue)
3. Storage-constrained flights: Debug disabled to save space → Fails (this issue)

Evidence

Log: 4.0.2_BTFL_AIKON_BLACKBOX_LOG_20190817_222106.06.csv

• CSV fields: Only has gyroADC[0-2]
• Frames: 128,289 (legitimate flight)
• Debug mode: Not configured for this flight
• Renderer fallback: No debug[0-2] to fall back to
• Renderer diagnostic:

  Roll axis: gyro=282354, unfilt=0, both=0, longest_continuous=0
  Axis 0 (Roll) extracted: 0 samples
  

• Result: "Data Unavailable" for gyro spectrum plots

Current Fallback Logic

The renderer already implements debug channel fallback in src/data_input/log_parser.rs:

// Apply Fallback Logic for gyro_unfilt (debug[0-2] --> gyroUnfilt[0-2])
for axis in 0..crate::axis_names::AXIS_NAMES.len() {
    current_row_data.gyro_unfilt[axis] = match parsed_gyro_unfilt[axis] {
        Some(val) => Some(val),
        None => parsed_debug[axis],  // ← Fallback to debug
    };
}

This works well when debug mode was configured. The problem is spectrum analysis still requires unfiltered data even when neither source is available.

Impact

When both gyroUnfilt and debug[0-2] are missing:

• Gyro spectrum analysis graphs fail → "Data Unavailable"
• Gyro PSD (Power Spectral Density) plots fail
• Gyro vs Unfiltered comparison plots fail
• Filter delay analysis skipped

These are valid flight logs with usable gyroADC data, but certain analysis types cannot be performed.

Proposed Solutions

Option 1: Fallback to Filtered-Only Analysis (Recommended)

When both gyroUnfilt and debug[0-2] are unavailable, perform spectrum analysis on gyroADC alone:

• Still useful for identifying resonance frequencies
• Can show motor noise patterns
• Can identify problematic frequency bands
• Just can't show filtering effectiveness or calculate filter delay
• Label plots clearly: "Filtered Gyro Only - Unfiltered data not available"

Option 2: User-Friendly Status Messages

Improve console output to distinguish between cases:

• ✅ "Using debug channels as unfiltered gyro" (fallback working)
• ✅ "Unfiltered gyro available from gyroUnfilt fields" (modern logs)
• ⚠️ "Note: Neither gyroUnfilt nor debug channels available. Spectrum analysis shows filtered data only." (this issue)

Option 3: Configuration Flag

Add command-line flag: --allow-filtered-only-spectrums

• Default: OFF (current behavior - skip plots when unfiltered missing)
• When enabled: Generate filtered-only spectrum plots with clear labeling
• Gives users control over completeness vs. information trade-off

Recommendation

Implement Option 1 (filtered-only analysis with clear labeling) + Option 2 (better status messages):

1. Keep existing debug fallback (already works well)
2. When both sources unavailable: Generate spectrum plots from filtered gyro only
3. Add clear labels:
   • On plots: "[Filtered Gyro Only]" or "[No Unfiltered Reference]"
   • In legend: "(gyroADC - no unfiltered data available)"
4. Improve console messages: Clearly state which data source is being used
5. Skip comparison plots gracefully: Can't compare filtered vs unfiltered without unfiltered data

This approach:

• ✅ Respects existing debug fallback (used by many tuning logs)
• ✅ Maximizes value for logs without debug mode
• ✅ Provides transparency about data limitations
• ✅ Maintains current behavior for logs with debug configured

Files Affected

• src/data_analysis/filter_delay.rs - Add filtered-only fallback mode
• src/plot_functions/plot_gyro_spectrums.rs - Support filtered-only spectrum generation
• src/plot_functions/plot_psd.rs - Support filtered-only PSD analysis
• src/plot_functions/plot_gyro_vs_unfilt.rs - Skip gracefully with informative message
• src/data_input/log_parser.rs - Add status reporting for data source detection

Test Cases

1. Modern log (≥ 4.5): gyroUnfilt[0-2] present → Works (use gyroUnfilt)
2. Tuning log with debug: debug[0-2] present → Works (existing fallback)
3. Normal flight without debug: Only gyroADC[0-2] → Needs fix (filtered-only mode)
   • Example: 4.0.2_BTFL_AIKON_BLACKBOX_LOG_20190817_222106.06.csv (282K frames)
4. Mixed log file: Some logs with debug, some without → Should handle each appropriately

Summary

The renderer already handles the common case (debug mode configured) via debug channel fallback. This issue addresses the remaining case where pilots chose not to use debug mode for normal flights, leaving valuable filtered gyro data underutilized.

[nerdCopter]

@coderabbitai, plan

[coderabbitai]

Coding Plan

Approach

Enable spectrum and PSD analysis for older Betaflight logs that only contain filtered gyro data (gyroADC) by implementing a filtered-only fallback mode when neither gyroUnfilt nor debug channel data is available, with clear labeling to indicate data limitations.

Observations

The project is a Rust-based flight log analysis tool. Flight data flows from CSV parsing (src/data_input/log_parser.rs) through LogRowData structures containing optional gyro fields (gyro for filtered, gyro_unfilt for unfiltered). Header detection flags track which data sources are present. Plot functions in src/plot_functions/ consume this data, with spectrum and PSD plots currently requiring both filtered and unfiltered data. Filter delay analysis is invoked directly by plot functions and requires both data sources for cross-correlation calculations. An existing debug fallback mechanism uses debug[0-2] as unfiltered gyro when gyroUnfilt headers are missing.

Assumptions

Assumption 1: Whether to modify filter_delay.rs directly

Options Considered:

• Add a filtered-only mode to filter_delay.rs that returns partial/empty results
• Leave filter_delay.rs unchanged since it already returns empty results when data is missing, and handle the fallback in plot functions

Chosen Option: 2

Rationale: The analysis fundamentally cannot calculate delay without both signals. Plot functions already receive empty results when data is missing; they just need to handle this gracefully by continuing to render filtered-only spectrums.

💡 User Tips

Regenerate the plan with different choices with @coderabbitai <feedback>.

Implementation Steps

Phase 1: Console Messaging and Data Source Reporting

Enhance the logging output to clearly communicate which gyro data source is being used, helping users understand why certain analyses may be limited.

Task 1: Add Filtered-Only Mode Detection and Console Message

Add detection and reporting when neither gyroUnfilt nor debug fallback data is available, signaling that spectrum analysis will use filtered data only.

• Update src/data_input/log_parser.rs to detect and report the filtered-only condition
• After the existing using_debug_fallback detection logic (around line 426-440), add a check for when NEITHER gyroUnfilt headers NOR debug[0-2] headers are found for any axis
• Print a distinctive console message: "ℹ️ Neither gyroUnfilt nor debug channels available. Spectrum analysis will use filtered gyro data only."
• Keep the existing messages for other cases (gyroUnfilt found, debug fallback used)

Task 2: Add Data Source Status to Main Orchestration

Pass the filtered-only status to plot functions so they can adjust their behavior and labeling accordingly.

• In src/main.rs, derive a filtered_only_mode boolean from the header flags: true when !gyro_unfilt_header_found.iter().any(|&f| f) && !debug_header_found.iter().take(3).any(|&f| f)
• Pass this flag alongside the existing using_debug_fallback flag to relevant plot function calls
• This affects calls to plot_gyro_spectrums, plot_psd, and plot_gyro_vs_unfilt

🤖 Prompt for AI agents

Implement console messaging and data source detection for filtered-only mode.
The goal is to detect when neither gyroUnfilt nor debug data is available and
communicate this to users and downstream plot functions.

In `src/data_input/log_parser.rs`:
- Add detection logic after the existing debug fallback detection to check when
neither gyroUnfilt headers nor debug[0-2] headers are found for any axis
- Print console message indicating spectrum analysis will use filtered gyro data
only
- Maintain existing messages for other data source scenarios

In `src/main.rs`:
- Derive `filtered_only_mode` boolean from header flags
- Pass the filtered-only flag to plot function calls for `plot_gyro_spectrums`,
`plot_psd`, and `plot_gyro_vs_unfilt`

Phase 2: Filtered-Only Mode Support in Plot Functions

Update spectrum and PSD plot functions to render useful visualizations even when unfiltered data is unavailable, with clear labeling of data limitations.

Task 1: Update Gyro Spectrum Plot for Filtered-Only Mode

Modify the gyro spectrum plot to continue rendering when only filtered data is available, rather than skipping the axis entirely.

• Update src/plot_functions/plot_gyro_spectrums.rs to accept a filtered_only_mode: bool parameter
• Modify the data extraction loop (around lines 88-100) to have two paths:
  • When both filtered and unfiltered available: current behavior (extract both)
  • When only filtered available: extract filtered samples only, leave unfiltered vectors empty
• Change the skip condition (lines 97-99) to only skip when filtered data is also empty, not when unfiltered is missing
• When in filtered-only mode for an axis:
  • Still compute and plot the filtered gyro spectrum
  • Skip the unfiltered spectrum series (empty data)
  • Skip filter delay display in the legend (no comparison possible)
  • Add a legend entry indicating data limitation: "⚠ Unfiltered data not available"
• Use the existing infrastructure for legend-only series (empty data vector, descriptive label) to add the warning indicator

Task 2: Update PSD Plot for Filtered-Only Mode

Apply the same filtered-only fallback pattern to the PSD analysis plot.

• Update src/plot_functions/plot_psd.rs to accept a filtered_only_mode: bool parameter
• Apply the same modifications as Task 2.1:
  • Two-path data extraction (both available vs filtered-only)
  • Relaxed skip condition to allow filtered-only plotting
  • Skip delay information when unfiltered unavailable
  • Add warning legend entry for filtered-only axes
• Maintain consistency with spectrum plot labeling conventions

Task 3: Update Gyro vs Unfiltered Comparison Plot

This plot cannot function without unfiltered data; update it to show a clear, informative message rather than a generic failure.

• Update src/plot_functions/plot_gyro_vs_unfilt.rs to accept the filtered_only_mode: bool parameter
• When in filtered-only mode, instead of returning None (which shows generic "Calculation/Data Extraction Failed"):
  • Check if filtered data exists but unfiltered is missing
  • Use the existing draw_unavailable_message() framework function to display: "Unfiltered gyro data not available\nComparison requires gyroUnfilt or debug channel data"
• This provides users clear feedback about why the comparison can't be shown and what data would be needed

Task 4: Update Plot Label Helper for Filtered-Only Indication

Extend the existing label formatting utility to support the filtered-only case.

• Update src/plot_functions/mod.rs to add a new helper function or extend format_debug_suffix() to handle three data source states:
  • Normal: gyroUnfilt available (no suffix needed)
  • Debug fallback: using debug channels (existing "[Debug]" suffix)
  • Filtered-only: no unfiltered reference (new "[Filtered Only]" suffix)
• Use this in spectrum and PSD plots for the filtered gyro series label when in filtered-only mode
• Example output: "Filtered Gyro [Filtered Only - No Unfiltered Reference]"

🤖 Prompt for AI agents

Update plot functions to support filtered-only mode with appropriate
visualizations and labeling. The goal is to render useful spectrum and PSD plots
even when unfiltered data is unavailable, while clearly communicating data
limitations to users.

In `src/plot_functions/plot_gyro_spectrums.rs`:
- Accept `filtered_only_mode: bool` parameter
- Implement two-path data extraction: extract both when available, or only
filtered when in filtered-only mode
- Modify skip condition to only skip when filtered data is empty
- When in filtered-only mode, plot filtered spectrum, skip unfiltered series and
delay display, add warning legend entry

In `src/plot_functions/plot_psd.rs`:
- Accept `filtered_only_mode: bool` parameter
- Apply same pattern as spectrum plot: two-path extraction, relaxed skip
condition, warning legend entry
- Maintain labeling consistency with spectrum plot

In `src/plot_functions/plot_gyro_vs_unfilt.rs`:
- Accept `filtered_only_mode: bool` parameter
- When in filtered-only mode, use `draw_unavailable_message()` to display
informative message about missing unfiltered data

In `src/plot_functions/mod.rs`:
- Add or extend helper function to format data source suffixes for three states:
normal, debug fallback, and filtered-only
- Apply suffix labeling in spectrum and PSD plots when in filtered-only mode

---

🚀 Next Steps

🤖 All AI agent prompts combined

Task: 1

Implement console messaging and data source detection for filtered-only mode.
The goal is to detect when neither gyroUnfilt nor debug data is available and
communicate this to users and downstream plot functions.

In `src/data_input/log_parser.rs`:
- Add detection logic after the existing debug fallback detection to check when
neither gyroUnfilt headers nor debug[0-2] headers are found for any axis
- Print console message indicating spectrum analysis will use filtered gyro data
only
- Maintain existing messages for other data source scenarios

In `src/main.rs`:
- Derive `filtered_only_mode` boolean from header flags
- Pass the filtered-only flag to plot function calls for `plot_gyro_spectrums`,
`plot_psd`, and `plot_gyro_vs_unfilt`
===============================================================================

Task: 2

Update plot functions to support filtered-only mode with appropriate
visualizations and labeling. The goal is to render useful spectrum and PSD plots
even when unfiltered data is unavailable, while clearly communicating data
limitations to users.

In `src/plot_functions/plot_gyro_spectrums.rs`:
- Accept `filtered_only_mode: bool` parameter
- Implement two-path data extraction: extract both when available, or only
filtered when in filtered-only mode
- Modify skip condition to only skip when filtered data is empty
- When in filtered-only mode, plot filtered spectrum, skip unfiltered series and
delay display, add warning legend entry

In `src/plot_functions/plot_psd.rs`:
- Accept `filtered_only_mode: bool` parameter
- Apply same pattern as spectrum plot: two-path extraction, relaxed skip
condition, warning legend entry
- Maintain labeling consistency with spectrum plot

In `src/plot_functions/plot_gyro_vs_unfilt.rs`:
- Accept `filtered_only_mode: bool` parameter
- When in filtered-only mode, use `draw_unavailable_message()` to display
informative message about missing unfiltered data

In `src/plot_functions/mod.rs`:
- Add or extend helper function to format data source suffixes for three states:
normal, debug fallback, and filtered-only
- Apply suffix labeling in spectrum and PSD plots when in filtered-only mode

💡 Iterate on the plan with: @coderabbitai <feedback>

Example Feedback
- `@coderabbitai` You can skip phase 3. Add a simple unit test case for phase 2.
- `@coderabbitai` For assumption 1 go ahead with option 3 and replan.

---

💬 Have feedback or questions? Drop into our discord!