Skip to content

Evaluate IBM OpenDsStar as replacement for DS-Star implementation #133

@jsboige

Description

@jsboige

Context

The DS-Star / MLE-Star notebook series (Day5-DS-Star, Day6-MLE-Star in ML/DataScienceWithAgents/AgenticDataScience/) currently implements the DS-Star concept based on an independent reimplementation, likely from https://github.com/JulesLscx/DS-Star.

IBM has now released OpenDsStar, an official open-source implementation of the DeepMind DS-Star publication (which was published without code):

Task

  1. Compare both implementations:

    • Architecture quality, code maturity, test coverage
    • Completeness vs the original DeepMind paper
    • API design and extensibility
    • Documentation quality
  2. Evaluate integration opportunity:

    • Can IBM's OpenDsStar replace the current DS-Star code in our notebooks?
    • What would need to change in Lab10, Lab11, Lab12 (Day5-DS-Star)?
    • What about Lab13-Lab16 (Day6-MLE-Star)?
  3. Decision criteria:

    • IBM's implementation is likely more robust (corporate backing, peer review)
    • But verify: does it cover the same scope as our current notebooks?
    • Migration effort vs quality gain

Affected notebooks

  • ML/DataScienceWithAgents/AgenticDataScience/Day5-DS-Star/Lab10-File-Analyzer.ipynb
  • ML/DataScienceWithAgents/AgenticDataScience/Day5-DS-Star/Lab11-Planner-Coder-Loop.ipynb
  • ML/DataScienceWithAgents/AgenticDataScience/Day5-DS-Star/Lab12-DS-Star-Workshop.ipynb
  • ML/DataScienceWithAgents/AgenticDataScience/Day6-MLE-Star/Lab13-Web-Search-SOTA.ipynb
  • ML/DataScienceWithAgents/AgenticDataScience/Day6-MLE-Star/Lab14-Ablation-Refinement.ipynb
  • ML/DataScienceWithAgents/AgenticDataScience/Day6-MLE-Star/Lab15-MLE-Star-Paper.ipynb
  • ML/DataScienceWithAgents/AgenticDataScience/Day6-MLE-Star/Lab16-MLE-Star-Workshop.ipynb

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requeststaleNo activity for 30+ days — needs confirmation or closure

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions