Skip to content

Adds python scripts to support curriculum learning experiments #3

Open
sruan2 wants to merge 14 commits intoace-agent:mainfrom
sruan2:main
Open

Adds python scripts to support curriculum learning experiments #3
sruan2 wants to merge 14 commits intoace-agent:mainfrom
sruan2:main

Conversation

@sruan2
Copy link
Copy Markdown
Collaborator

@sruan2 sruan2 commented Feb 3, 2026

Summary

This PR adds three python scripts to support curriculum learning experiments in AppWorld.

Changes

Added three new scripts in experiments/curriculum/:

analyze_dataset.py - Task difficulty distribution analyzer
Saves analysis results to JSON files in the same dataset folder

data_selector.py - Curriculum dataset builder
Selects subsets of tasks from existing datasets based on size, difficulty, and a range of requirements specified

check_same_data.py - Dataset comparison utility
Compares two dataset files to verify they contain the same tasks

@sruan2
Copy link
Copy Markdown
Collaborator Author

sruan2 commented Feb 19, 2026

Two additional changes:

  1. Fixed SN endpoint timeouts by adding error handling and increasing the timeout
    experiments/code/ace/lite_llm_generator.py

  2. Added an embedding-based clustering method using the OpenAI embeddings endpoint
    experiments/curriculum/similarity_metrics.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant