Skip to content

FIX/options.expirations returns an empty DataFrame when filtering columns#27

Open
MarketDataDev03 wants to merge 3 commits into
mainfrom
fix_options_expirations_return_empty_df
Open

FIX/options.expirations returns an empty DataFrame when filtering columns#27
MarketDataDev03 wants to merge 3 commits into
mainfrom
fix_options_expirations_return_empty_df

Conversation

@MarketDataDev03
Copy link
Copy Markdown
Collaborator

Fix: options.expirations returns an empty DataFrame when filtering columns

Closes #23

Summary

options.expirations(..., columns=["expirations"]) returned an empty DataFrame,
and the INTERNAL output format crashed on partial API responses. Both stem from
the same trigger: the columns parameter is applied server-side, so requesting
columns=["expirations"] makes the API respond with only {"s": "ok", "expirations": [...]}
— without the updated field the SDK assumed was always present.

This PR fixes both failure modes and adds regression tests.

Problem

from marketdata import MarketDataClient

df = MarketDataClient(token="...").options.expirations(
    symbol="AAPL", columns=["expirations"]
)
print(df)
# Empty DataFrame
# Columns: []
# Index: [2025-12-05, 2025-12-12, ...]

Two distinct bugs:

  1. Empty DataFrame (primary). The resource always forced
    index_columns=["expirations"]. When expirations is the only column returned,
    the pandas handler promotes it to the index via set_index, leaving the
    DataFrame with zero data columns — so it looks empty even though the data is
    there (in the index).

  2. INTERNAL crash on partial responses (secondary). OptionsExpirations
    required the updated field. A response missing it (e.g. a filtered/partial
    response) raised TypeError, which @handle_exceptions turned into a
    MarketDataClientErrorResult.

Changes

resources/options/expirations.py — primary fix

Stop forcing expirations into the index when the user explicitly filters columns.
The default behavior (no columns filter → expirations as index) is unchanged.

index_columns = [] if user_universal_params.columns else ["expirations"]
return handler(data, output_model, user_universal_params).get_result(
    index_columns=index_columns
)

output_types/options_expirations.py — secondary fix

Make updated optional and skip conversion when it is absent.

updated: datetime.datetime | None = None

def __post_init__(self):
    if self.updated is not None:
        self.updated = format_timestamp(self.updated)
    ...

output_handlers/base.py — required follow-on fix

Changing updated to the PEP 604 union datetime | None broke date-column
detection: _type_includes only recognized typing.Union, but X | None has
origin types.UnionType. Without this, updated stopped being converted to a
datetime and broke two pre-existing DataFrame tests. Now both union forms are
handled.

if origin is Union or origin is types.UnionType:
    ...

Testing

Added 4 tests in src/tests/test_options_expirations.py:

  • test_options_expirations_optional_updated — dataclass accepts updated=None.
  • test_get_options_expirations_columns_filter_dataframe_pandas — filtered DataFrame
    is not empty; compared against a full expected DataFrame via assert_frame_equal.
  • test_get_options_expirations_columns_filter_dataframe_polars — regression guard
    (polars never had the bug).
  • test_get_options_expirations_partial_response_internalINTERNAL parses a
    response missing updated, leaving it None.

Developed with a TDD red → green cycle.

299 passed

Full suite green — no regressions.

Files changed

File Change
src/marketdata/resources/options/expirations.py Don't force index when columns are filtered
src/marketdata/output_types/options_expirations.py updated made optional
src/marketdata/output_handlers/base.py Recognize PEP 604 X | None in date detection
src/tests/test_options_expirations.py 4 new regression tests

@codecov
Copy link
Copy Markdown

codecov Bot commented May 27, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (fb1ea8e) to head (891eda4).

Additional details and impacted files
@@            Coverage Diff            @@
##              main       #27   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           52        52           
  Lines         2281      2284    +3     
=========================================
+ Hits          2281      2284    +3     
Flag Coverage Δ
unittests 100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@MarketDataDev03 MarketDataDev03 marked this pull request as ready for review May 28, 2026 11:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

options.expirations returns an empty DataFrame when columns=["expirations"] is passed

1 participant