Skip to content

feat: Use ORJSONResponse for faster JSON serialization in feature server #5930

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ntkathole wants to merge 1 commit into
base: master
Choose a base branch
from
Open

feat: Use ORJSONResponse for faster JSON serialization in feature server #5930

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ dependencies = [
"psutil",
"bigtree>=0.19.2",
"pyjwt",
"orjson>=3.9.0",
]

[project.optional-dependencies]
Expand Down
14 changes: 7 additions & 7 deletions sdk/python/feast/feature_server.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@
)
from fastapi.concurrency import run_in_threadpool
from fastapi.logger import logger
from fastapi.responses import JSONResponse
from fastapi.responses import JSONResponse, ORJSONResponse
from fastapi.staticfiles import StaticFiles
from google.protobuf.json_format import MessageToDict
from prometheus_client import Gauge, start_http_server
Expand Down Expand Up @@ -324,7 +324,7 @@ async def lifespan(app: FastAPI):
"/get-online-features",
dependencies=[Depends(inject_user_details)],
)
async def get_online_features(request: GetOnlineFeaturesRequest) -> Dict[str, Any]:
async def get_online_features(request: GetOnlineFeaturesRequest) -> ORJSONResponse:
# Initialize parameters for FeatureStore.get_online_features(...) call
features = await _get_features(request, store)

Expand All @@ -341,22 +341,22 @@ async def get_online_features(request: GetOnlineFeaturesRequest) -> Dict[str, An
lambda: store.get_online_features(**read_params) # type: ignore
)

# Convert the Protobuf object to JSON and return it
# Convert Protobuf to dict, then use ORJSONResponse for faster JSON serialization
response_dict = await run_in_threadpool(
MessageToDict,
response.proto,
preserving_proto_field_name=True,
float_precision=18,
)
return response_dict
return ORJSONResponse(content=response_dict)
Copy link
Contributor

@devin-ai-integration devin-ai-integration bot Feb 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 ORJSONResponse serializes NaN/Infinity as null, changing API behavior

Switching from JSONResponse to ORJSONResponse changes how NaN and Infinity float values are serialized in the API response.

Click to expand

Behavior Change

The standard json module (used by JSONResponse) serializes special float values as:

  • float('nan') → NaN
  • float('inf') → Infinity
  • float('-inf') → -Infinity

While orjson (used by ORJSONResponse) serializes them as:

  • float('nan') → null
  • float('inf') → null
  • float('-inf') → null

Impact

The Feast codebase explicitly uses NaN values for missing data. In sdk/python/feast/proto_json.py:99-101:

# Convert each null as NaN.
message.double_list_val.val.extend(
    [item if item is not None else float("nan") for item in value]
)

And tests in sdk/python/tests/unit/test_proto_json.py:65 verify NaN is preserved:

assertpy.assert_that(feature_vector_json["values"][5][3]).is_nan()

With ORJSONResponse, clients that previously received NaN in responses will now receive null, which has different semantics (missing value vs. not-a-number). This is a breaking change for API consumers that rely on distinguishing between null and NaN values.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Copy link
Member Author

@ntkathole ntkathole Feb 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@franciscojavierarceo Thoughts on this ?


@app.post(
"/retrieve-online-documents",
dependencies=[Depends(inject_user_details)],
)
async def retrieve_online_documents(
request: GetOnlineDocumentsRequest,
) -> Dict[str, Any]:
) -> ORJSONResponse:
logger.warning(
"This endpoint is in alpha and will be moved to /get-online-features when stable."
)
Expand All @@ -376,14 +376,14 @@ async def retrieve_online_documents(
lambda: store.retrieve_online_documents(**read_params) # type: ignore
)

# Convert the Protobuf object to JSON and return it
# Convert Protobuf to dict, then use ORJSONResponse for faster JSON serialization
response_dict = await run_in_threadpool(
MessageToDict,
response.proto,
preserving_proto_field_name=True,
float_precision=18,
)
return response_dict
return ORJSONResponse(content=response_dict)

@app.post("/push", dependencies=[Depends(inject_user_details)])
async def push(request: PushFeaturesRequest) -> Response:
Expand Down
1,032 changes: 511 additions & 521 deletions sdk/python/requirements/py3.10-ci-requirements.txt
View file
Open in desktop

Large diffs are not rendered by default.

599 changes: 338 additions & 261 deletions sdk/python/requirements/py3.10-minimal-requirements.txt
View file
Open in desktop

Large diffs are not rendered by default.

26 changes: 13 additions & 13 deletions sdk/python/requirements/py3.10-minimal-sdist-requirements-build.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -520,6 +520,7 @@ maturin==1.11.5 \
--hash=sha256:ffe7418834ff3b4a6c987187b7abb85ba033f4733e089d77d84e2de87057b4e7
# via
# cryptography
# orjson
# pydantic-core
# rpds-py
# watchfiles
Expand Down Expand Up @@ -608,16 +609,16 @@ packaging==26.0 \
# setuptools-git-versioning
# setuptools-scm
# wheel
pathspec==1.0.3 \
--hash=sha256:bac5cf97ae2c2876e2d25ebb15078eb04d76e4b98921ee31c6f85ade8b59444d \
--hash=sha256:e80767021c1cc524aa3fb14bedda9c34406591343cc42797b386ce7b9354fb6c
pathspec==1.0.4 \
--hash=sha256:0210e2ae8a21a9137c0d470578cb0e595af87edaa6ebf12ff176f14a02e0e645 \
--hash=sha256:fb6ae2fd4e7c921a165808a552060e722767cfa526f99ca5156ed2ce45a5c723
# via
# hatchling
# mypy
# scikit-build-core
pdm-backend==2.4.6 \
--hash=sha256:4427c235d9ee84afd23a376a0961e5b9ad7e1db09039127b71b2053d19185784 \
--hash=sha256:5dd9cd581a4f18d57ff506a5b3aad7c8df31e5949b6fd854bbc34c38107e4532
pdm-backend==2.4.7 \
--hash=sha256:1599e3afa6f229b30cb4d3bd9caeca42229c42eb5730abd13e0b5256ec93c9f7 \
--hash=sha256:a509d083850378ce919d41e7a2faddfc57a1764d376913c66731125d6b14110f
# via
# annotated-doc
# fastapi
Expand All @@ -640,9 +641,9 @@ poetry-core==1.9.1 \
# rich
# rsa
# tomlkit
poetry-core==2.3.0 \
--hash=sha256:f6da8f021fe380d8c9716085f4dcc5d26a5120a2452e077196333892af5de307 \
--hash=sha256:fc42f3854e346e4b96fb2b38d29e6873ec2ed25fbd7b8f1afba06613a966eaef
poetry-core==2.3.1 \
--hash=sha256:96f791d5d7d4e040f3983d76779425cf9532690e2756a24fd5ca0f86af19ef82 \
--hash=sha256:db1cf63b782570deb38bfba61e2304a553eef0740dc17959a50cc0f5115ee634
# via aiohappyeyeballs
pybind11-global==3.0.1 \
--hash=sha256:0e8d5a68d084c50bf145ce5efdbdd00704dbe6315035d0b7a255708ddeb9faca \
Expand Down Expand Up @@ -833,14 +834,13 @@ wheel==0.46.3 \
# shellingham
# snowflake-connector-python
# tabulate
# tqdm
# tzdata
# uvloop

# The following packages are considered to be unsafe in a requirements file:
setuptools==80.10.1 \
--hash=sha256:bf2e513eb8144c3298a3bd28ab1a5edb739131ec5c22e045ff93cd7f5319703a \
--hash=sha256:fc30c51cbcb8199a219c12cc9c281b5925a4978d212f84229c909636d9f6984e
setuptools==80.10.2 \
--hash=sha256:8b0e9d10c784bf7d262c4e5ec5d4ec94127ce206e8738f29a437945fbc219b70 \
--hash=sha256:95b30ddfb717250edb492926c92b5221f7ef3fbcc2b07579bcd4a27da21d0173
# via
# aiobotocore
# aiohttp
Expand Down
Loading
Loading