v0.14.3

v0.14.3 - 2026-04-17

Bugs Fixed

• Fix win computation when uploading benchmark result - Issue #590 by @R-Palazzo
• Latest PyTorch (2.11.0) is not compatible with current GCP GPU benchmark environment - Issue #589 by @R-Palazzo

Internal

• Make BenchmarkConfig.compute support instance configuration in YAML - Issue #586 by @R-Palazzo
• Update the upload benchmark workflow to work with the BenchmarkLauncher - Issue #584 by @R-Palazzo

v0.14.2

v0.14.2 - 2026-04-01

Internal

• Define the get_instance_status and get_job_status() methods of the BenchmarkLauncher - Issue #570 by @R-Palazzo
• Define the terminate() method of the BenchmarkLauncher - Issue #568 by @R-Palazzo
• Define workflows to be able to run from a config file or some given parameters - Issue #547 by @R-Palazzo
• Add a script that launches a benchmark from a yaml file or a set of parameters - Issue #546 by @R-Palazzo
• Move the current benchmark configs to yaml files - Issue #545 by @R-Palazzo

v0.14.1

v0.14.1 - 2026-03-23

Internal

• Update the "wins" computation to include the Pareto front - Issue #572 by @R-Palazzo

v0.14.0

v0.14.0 - 2026-03-09

New Features

The ResultExplorer should look for latest run results by default - Issue #552 by @R-Palazzo
Update load_results to be able to filter on dataset or synthesizer - Issue #551 by @R-Palazzo

Bugs Fixed

OUTPUT_DESTINATION_AWS points to the wrong location - Issue #564 by @R-Palazzo

v0.13.1

v0.13.1 - 2026-02-28

New Features

Include SDV-Enterprise in single-table benchmarks - Issue #549 by @R-Palazzo

Internal

Add extremity data points of the Pareto curve for the Quality–Speed Tradeoff plot - Issue #556 by @R-Palazzo
Internal benchmark results upload crashes if there's no error column in the result table - Issue #544 by @R-Palazzo

Maintenance

Update RELEASE guide to include conda-forge step - Issue #560 by @sarahmish
Support Python 3.14 - Issue #528 by @pvk-developer
Update license information in pyproject.toml to use new format - Issue #527 by @pvk-developer

Miscellaneous

Set the SDGym Slack alert to be posted on the sdgym channel. - Issue #555 by @R-Palazzo

v0.13.0

v0.13.0 - 2026-01-30

New Features

• Add a Dataset Details and a Model Details excel sheets when uploading benchmark results - Issue #532 by @R-Palazzo
• Add workflow to run SDGym multi-table benchmark monthly and publish results - Issue #516 by @R-Palazzo
• Define internal single and multi table methods to run on GCP - Issue #515 by @R-Palazzo
• Add multi table support to ResultsExplorer - Issue #488 by @fealho
• Add benchmark_multi_table_aws - Issue #487 by @R-Palazzo
• Add benchmark_multi_table function - Issue #486 by @pvk-developer
• Add multi-table UniformSynthesizer - Issue #485 by @R-Palazzo

Bugs Fixed

• Private S3 bucket access fails in benchmark_multi_table_aws despite valid credentials - Issue #525 by @R-Palazzo
• RealTabFormer 0.2.4 causes integration to fail - Issue #523 by @R-Palazzo

Internal

• Remove deprecated parameters - Issue #519 by @fealho

Miscellaneous

• Update multi-table dataset list - Issue #535 by @R-Palazzo

v0.12.1

v0.12.1 - 2025-12-05

New Features

• If there are no datasets in the bucket, the DatasetExplorer should show a warning and return an empty table - Issue #475 by @fealho
• Add input validation for the DatasetExplorer class and functions - Issue #474 by @fealho

Bugs Fixed

• Record the train and sample times whenever an error occurs during a benchmark. - Issue #503 by @R-Palazzo

Maintenance

• Workflow fails due to lack of space - Issue #511 by @rwedge

v0.12.0

v0.12.0 - 2025-11-20

New Features

• Rename create_sdv_synthesizer_variant to create_synthesizer_variant - Issue #491 by @R-Palazzo
• SDGym should be able to automatically discover SDV Enterprise synthesizers - Issue #481 by @R-Palazzo
• Incorporate the get_available_datasets functionality into the DatasetExplorer - Issue #473 by @fealho

Bugs Fixed

• Update result aggregation logic in the ResultExplorer to match new naming schema - Issue #494 by @R-Palazzo
• When running a benchmark locally, the additional_datasets_folder path should be the root path - Issue #484 by @fealho

v0.11.1

v0.11.1 - 2025-11-03

Bugs Fixed

• Missing dependency openpyxl - Issue #479 by @rwedge

v0.11.0

v0.11.0 - 2025-10-31

New Features

• Add a DatasetExplorer class that provides a summary of all datasets in a bucket (for a given modality) - Issue #469 by @pvk-developer
• Update SDGym to use the new S3 bucket and bucket structure - Issue #468 by @pvk-developer
• Update Pareto plot data generation to use the Adjusted Time and Quality score - Issue #462 by @R-Palazzo
• The ResultsExplorer should allow programmatic access to all the saved artifacts from benchmarking - Issue #450 by @R-Palazzo
• When performing multiple SDGym runs on the same day, save the artifacts with consistent naming - Issue #448 by @R-Palazzo
• To simulate graceful degradation, fallback to using the results from the UniformSynthesizer - Issue #439 by @rwedge
• Pip install sdgym released version on ec2 machines - Issue #437 by @pvk-developer
• Add a Fallback to UniformSynthesizer when an error occur and improve the time tracker of the synthetic data generation - Issue #436 by @R-Palazzo
• Make the synthesizer names consistent throughout SDGym - Issue #430 by @R-Palazzo
• Simplify the import API for SDGym's results explorer - Issue #429 by @R-Palazzo
• Add workflow to run SDGym monthly and publish results - Issue #425 by @R-Palazzo
• Add benchmark_single_table_aws function - Issue #414 by @R-Palazzo
• Add summarize function to SDGymResultsExplorer class - Issue #412 by @R-Palazzo
• Add SDGymResultsExplorer class - Issue #411 by @R-Palazzo
• Add ability to save synthesizers and data when running benchmark_single_table - Issue #410 by @R-Palazzo
• Update REalTabFormer default parameters so that it runs on benchmarking - Issue #400 by @fealho
• Add DCRBaseline Metric to single table report - Issue #397 by @gsheni

Bugs Fixed

• Update link to s3 results in the Slack Alert message - Issue #464 by @R-Palazzo
• EC2 instance not terminating after timeout - Issue #463 by @R-Palazzo
• Adjusted time and quality score not aggregating correctly on EC2 - Issue #461 by @R-Palazzo
• Update warning message for deprecated parameters - Issue #455 by @R-Palazzo
• The UniformSynthesizer produces multiple UserWarning messages when run on a demo dataset - Issue #449 by @R-Palazzo
• Always include UniformSynthesizer doesn't work on AWS - Issue #446 by @R-Palazzo
• Fix minimum test version due to RealTabFormer and Torch releases - Issue #434 by @R-Palazzo
• Add modality parameter to get_available_datasets function - Issue #403 by @gsheni
• Update the EC2 instance used when run_on_ec2 is enabled - Issue #396 by @R-Palazzo
• All bump-version commands are failing - Issue #391 by @amontanez24

Internal

• To simulate graceful degradation, always run the UniformSynthesizer on all the requested datasets - Issue #438 by @rwedge

Maintenance

• Remove support for Python 3.8 - Issue #457 by @fealho
• Check pyproject for release candidate dependencies - Issue #406 by @rwedge
• Update the library installation script for EC2 machines to install optional dependencies like RealTabFormer - Issue #388 by @R-Palazzo
• Speed up test_benchmark_single_table_realtabformer_no_metrics integration test - Issue #379 by @fealho
• Update python set up step in workflows to use latest python version - Issue #361 by @frances-h
• Support Python 3.13 - Issue #355 by @rwedge

Miscellaneous

• Add workflow to release SDGym on PyPI - Issue #418 by @gsheni