discrete variables

[roussel-ryan]

This pull request adds robust support for discrete variables in the Bayesian optimization workflow. It introduces logic to correctly handle discrete variables throughout candidate generation, optimization, and validation, and ensures that optimizers and generators provide appropriate errors or warnings when discrete variables are not supported. Additionally, it updates dependencies and expands test coverage for these new behaviors.

Support for discrete variables in Bayesian optimization:

• Added logic in bayesian_generator.py to handle discrete variables, including:

  • Detecting discrete variables and generating appropriate bounds [1]], [2]])
  • Creating candidate combinations for discrete and mixed variables, with truncation and warnings if limits are exceeded ([xopt/generators/bayesian/bayesian_generator.pyL888-R993])
  • Snapping candidate values to the nearest allowed discrete value and validating outputs [1]], [2]])
  • Preventing use of unsupported features like max_travel_distances and turbo_controller with discrete variables ([xopt/generators/bayesian/bayesian_generator.pyR1018-R1027])
  • Raising errors when grid optimizer is used with discrete variables [1]], [2]])

• Updated model construction to properly handle discrete variable bounds [1]], [2]])

Enhancements to numerical optimizers:

• Extended LBFGSOptimizer to support mixed and discrete optimization via BoTorch's optimize_acqf_mixed and optimize_acqf_discrete, including configuration limits and warnings [1]], [2]])
• Added new configuration options to control the number of discrete configurations, batch size, and restarts ([xopt/numerical_optimizer.pyR91-R117])

Validation and error handling improvements:

• Updated MGGPOGenerator to explicitly disallow discrete variables, raising a clear error if present [1]], [2]])

Dependency and test updates:

• Bumped botorch dependency to >=0.13.0 for compatibility with new optimization routines ([environment.ymlL11-R11])
• Added comprehensive tests for discrete variable handling, including candidate snapping, validation, and error conditions ([xopt/tests/generators/bayesian/test_bayesian_generator.pyR318-R373])

---

References:
Support for discrete variables in Bayesian optimization: [1]], [2]], [3]], [4]], [5]], [6]], [7]], [8]])
Enhancements to numerical optimizers: [1]], [2]])
Validation and error handling improvements: [1]], [2]])
Dependency and test updates: [1]], [2]])

[codecov]

Codecov Report

❌ Patch coverage is 94.87578% with 33 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
xopt/generators/bayesian/visualize.py	84.94%	9 Missing and 5 partials ⚠️
...s/generators/bayesian/test_discrete_integration.py	94.55%	2 Missing and 6 partials ⚠️
...sts/generators/bayesian/test_bayesian_generator.py	95.18%	2 Missing and 2 partials ⚠️
xopt/generator.py	80.00%	1 Missing and 1 partial ⚠️
xopt/generators/bayesian/bayesian_generator.py	97.40%	1 Missing and 1 partial ⚠️
xopt/vocs.py	96.22%	1 Missing and 1 partial ⚠️
xopt/tests/test_generator.py	90.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[roussel-ryan]

@copilot add tests to this PR that ensures coverage of changes

[Copilot]

@copilot add tests to this PR that ensures coverage of changes

Added targeted coverage tests for the discrete-variable changes in Bayesian generator/model-constructor and numerical optimizer paths, including truncation/warning branches and discrete validation/error cases. Addressed in commit 8ccd393.

[roussel-ryan]

@copilot create an ipynb example demonstrating bayesian optimization in a mixed continuous-discrete space

[Copilot]

@copilot create an ipynb example demonstrating bayesian optimization in a mixed continuous-discrete space

Added the notebook example at docs/examples/single_objective_bayes_opt/mixed_continuous_discrete_bo.ipynb and linked it in mkdocs.yml under Single Objective Bayesian Optimization. Addressed in commit cb4d16f.

[Copilot]

Pull request overview

This PR adds end-to-end support for discrete (and mixed continuous/discrete) variables across Xopt’s Bayesian optimization workflow, including candidate generation, optimizer routing, validation, visualization, and documentation/examples.

Changes:

• Extend bounds/sampling/validation utilities to correctly handle discrete variables (xopt/vocs.py) and enforce generator capability checks (supports_discrete_variables).
• Add mixed/discrete optimization support to LBFGSOptimizer via BoTorch’s optimize_acqf_mixed / optimize_acqf_discrete, and integrate discrete-aware candidate handling in BayesianGenerator.
• Expand tests and docs (including a new mixed discrete tutorial notebook) to cover discrete workflows, plotting, and unsupported-generator errors.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
xopt/vocs.py	Adds discrete-aware variable bounds, random/grid sampling, and input validation.
xopt/tests/test_vocs.py	Adds unit tests for discrete sampling, grid enumeration, and validation membership.
xopt/numerical_optimizer.py	Adds mixed/discrete optimization routes and new configuration limits/logging.
xopt/tests/test_numerical_optimizer.py	Tests optimizer routing, truncation behavior, and argument handling for mixed/discrete paths.
xopt/generator.py	Introduces supports_discrete_variables capability flag + validation hook.
xopt/tests/test_generator.py	Adds tests verifying discrete-support validation behavior for generators.
xopt/generators/random.py	Declares RandomGenerator supports discrete variables.
xopt/generators/bayesian/bayesian_generator.py	Implements discrete optimization kwargs, candidate snapping, and discrete validation; blocks unsupported features.
xopt/tests/generators/bayesian/test_bayesian_generator.py	Adds tests for discrete kwargs generation, snapping/validation, and grid-optimizer error paths.
xopt/generators/bayesian/base_model.py	Ensures model constructors derive bounds correctly for discrete domains.
xopt/generators/bayesian/visualize.py	Adds discrete-aware meshes/axes ticks and 2D plotting adjustments for mixed spaces.
xopt/tests/generators/bayesian/test_visualize.py	Adds tests for mixed discrete visualization mesh generation and tick labeling.
xopt/generators/bayesian/mggpo.py	Explicitly declares MGGPO does not support discrete variables.
xopt/tests/generators/bayesian/test_mggpo.py	Adds test ensuring MGGPO rejects discrete VOCS.
xopt/generators/bayesian/time_dependent.py	Aligns capability flags and preserves inherited VOCS validation behavior.
xopt/generators/bayesian/bax_generator.py	Preserves inherited Bayesian VOCS validation via super().
xopt/generators/init.py	Excludes supports_discrete_variables from generator defaults export.
xopt/tests/generators/bayesian/test_discrete_integration.py	New integration tests exercising discrete candidate flow across multiple generators.
environment.yml	Bumps BoTorch dependency minimum to enable new optimization routines.
mkdocs.yml	Adds nav links for mixed discrete optimization example(s).
docs/examples/single_objective_bayes_opt/mixed_discrete.ipynb	New tutorial notebook demonstrating mixed continuous/discrete BO usage.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[roussel-ryan]

@copilot apply changes based on this feedback

[Copilot]

Implemented the feedback by making discrete configuration generation bounded/lazy in _get_discrete_optimization_kwargs so we no longer materialize the full cartesian product before truncation. Also added a regression test to verify truncation is applied lazily. Addressed in commits 0f356db and 944f7b0.

[nikitakuklev]

A few small things. We can rework numopt later to maybe subclass.

[nikitakuklev]

This scans only _candidate_names, not fixed features.

[roussel-ryan]

candidate names should include fixed features

Xopt/xopt/generators/bayesian/bayesian_generator.py

Line 906 in b736e2f

def _candidate_names(self):

[roussel-ryan]

this is because the generator also returns fixed feature values to Xopt

[nikitakuklev]

Sorry, poor initial wording.

Your link has:

variable_names = self.vocs.variable_names
if self.fixed_features is not None:
    for name in self.fixed_features:
        if name in variable_names:
            variable_names.remove(name)
return variable_names

Then idx, name in enumerate(self._candidate_names): does not contain the discrete variable - max_travel_distances/turbo_controller proceed into _get_max_travel_distances_region which raises AttributeError: 'DiscreteVariable' object has no attribute 'domain'.

Example:

vocs = VOCS(variables={"x1": [0.0, 1.0], "x2": {0.0, 5.0, 10.0}},  # x2 discrete
              objectives={"y1": "MINIMIZE"})
gen = ExpectedImprovementGenerator(vocs=vocs, fixed_features={"x2": 5.0})  # pin x2
gen.max_travel_distances = [0.1, 0.1]
gen.add_data(data)
gen._get_optimization_bounds()   # AttributeError: 'DiscreteVariable' has no attribute 'domain'

[roussel-ryan]

ah I see, ok I will address

[nikitakuklev]

BAX just snaps discrete values at the end - should it be marked as supported?

[roussel-ryan]

I think we can keep supports_discrete_variables to False for now until @dylanmkennedy can investigate further since there may be some issue with GridScanAlgorithm or other algorithm types. This is certainly a topic of interest

[nikitakuklev]

yes, I meant set explicitly false (it inherits right now)

[roussel-ryan]

Thanks for the feedback @nikitakuklev

[nikitakuklev]

this will emit off-grid point now, we should disallow when discrete variables are present

[nikitakuklev]

single discrete value will crash Normalize with a nan