Skip to content

eval_protocol.data_loader#210

Merged
dphuang2 merged 9 commits intomainfrom
data-loader
Sep 24, 2025
Merged

eval_protocol.data_loader#210
dphuang2 merged 9 commits intomainfrom
data-loader

Conversation

@dphuang2
Copy link
Copy Markdown
Collaborator

@dphuang2 dphuang2 commented Sep 22, 2025

I went with a simpler/DRYer design that works with are adapters today. At its core this PR introduces DynamicDataLoader and InlineDataLoader. You use InlineDataLoader for hard-coded/static input rows. You use DynamicDataLoader for all the tracing adapters.

@dphuang2 dphuang2 changed the title Data loader eval_protocol.data_loader Sep 22, 2025
@dphuang2 dphuang2 requested a review from benjibc September 24, 2025 05:45
Comment on lines -24 to -34
try:
from langsmith import Client # type: ignore

LANGSMITH_CLIENT: Optional[Client]
try:
LANGSMITH_CLIENT = Client()
except Exception as exc: # pragma: no cover - surfaced to the caller
print(f"⚠️ LangSmith client unavailable: {exc}")
LANGSMITH_CLIENT = None
except ImportError: # pragma: no cover - optional dependency
LANGSMITH_CLIENT = None
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was causing a warning message in pytest collection so i moved it into the test itself

Comment on lines -8 to +15
# Get API key using the new auth module
api_key = get_fireworks_api_key()
if api_key:
print(f"API key retrieved via auth module: {api_key[:4]}...{api_key[-4:]}")
else:
print("No API key retrieved via auth module.")

# Get account ID using the new auth module
account_id = get_fireworks_account_id()
if account_id:
print(f"Account ID retrieved via auth module: {account_id}")
else:
print("No account ID retrieved via auth module.")
def test_fireworks_api():
# Get API key using the new auth module
api_key = get_fireworks_api_key()
if api_key:
print(f"API key retrieved via auth module: {api_key[:4]}...{api_key[-4:]}")
else:
print("No API key retrieved via auth module.")
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved all of this test under an actual pytest test because this was causing error messages in pytest collection

self.filter = kwargs.get("filter", None)


@pytest.mark.integration
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

non-existent markers warnings

@dphuang2 dphuang2 merged commit f333d81 into main Sep 24, 2025
7 checks passed
@dphuang2 dphuang2 deleted the data-loader branch September 24, 2025 05:59
# ]
input_rows = []

# uncomment when dataloader is fixed
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment still here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants