Skip to content

Pyomo.DoE: add advanced run_doe debug config and fix trace/Cholesky initialization consistency#3867

Open
adowling2 wants to merge 11 commits intoPyomo:mainfrom
adowling2:fix-doe-initialization
Open

Pyomo.DoE: add advanced run_doe debug config and fix trace/Cholesky initialization consistency#3867
adowling2 wants to merge 11 commits intoPyomo:mainfrom
adowling2:fix-doe-initialization

Conversation

@adowling2
Copy link
Member

@adowling2 adowling2 commented Mar 4, 2026

Fixes # .

Summary/Motivation:

This PR adds advanced debugging/inspection controls for pyomo.contrib.doe.DesignOfExperiments.run_doe() while preserving default behavior for typical users. It also fixes a trace-objective initialization inconsistency that can produce large residuals in Cholesky-related constraints at the start of the final NLP solve.

The motivating workflow showed:

  • large early infeasibilities in optimize mode,
  • inability to use max_iter=0 for final-model probing because scenario solves reused the same solver options,
  • and inconsistent initialization among fim, L, fim_inv, L_inv, and cov_trace.

Changes proposed in this PR:

  • Added an advanced run_config (ConfigBlock/dict) argument to run_doe(model=None, results_file=None, run_config=None):
    • scenario_solver_options
    • final_solver_options
    • final_solve
    • inspection.enabled
    • inspection.top_constraints
  • Implemented scoped solver option application:
    • scenario-generation and square-initialization solves can use different options from the final optimization solve.
  • Added optional structured residual reporting utility:
    • reports top violated active constraints with fields:
      • constraint_name
      • body
      • lower_bound
      • upper_bound
      • violation
      • constraint_type
  • Added assemble-and-inspect path (final_solve=False) to inspect assembled NLP state without running final optimization.
  • Fixed Cholesky jitter computation and added robust helper usage.
  • Added _initialize_cholesky_from_fim() to re-synchronize L, L_inv, fim_inv, and cov_trace from current FIM immediately before final solve / inspection.
  • Updated docstrings and added advanced usage documentation + example path in reactor_example.py.

Backward compatibility:

  • Default usage (run_doe() with no run_config) remains unchanged.
  • Advanced functionality is available only when run_config is provided.

Testing:

I ran:

  • pytest -q pyomo/contrib/doe/tests/test_doe_debug.py
    • Result: 8 passed
  • pytest -q pyomo/contrib/doe/tests/test_doe_solve.py -k "rooney_biegler_fd_central_solve"
    • Result: 1 passed, 17 deselected

New/updated tests cover:

  • default behavior unchanged,
  • split scenario vs final solver options,
  • inspection mode schema,
  • max_iter=0-style final probing with separate scenario options,
  • Cholesky/FIM inverse consistency after initialization in trace mode.

Outstanding Tasks (before removing WIP):

  • Finalize proposed interface for advanced initialization diagnostics
  • Decide the best way to document the proposed interface. Are examples sufficient or should we build out the Pyomo documentation?
  • Run black
  • @adowling2 carefully reviews proposed changes

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

  1. I agree my contributions are submitted under the BSD license.
  2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

@adowling2 adowling2 marked this pull request as draft March 4, 2026 01:01
@adowling2
Copy link
Member Author

adowling2 commented Mar 4, 2026

@smondal13 @sscini @slilonfe5 @snarasi2 @djlaky I found a small bug in how we initialize the Cholesky factorization when the FIM at the initial point needs "inertia correction". This PR fixes the initialization inconsistency. This PR also proposes an interface to allow for more advanced solver options when diagnosing initialization issues for the Pyomo.DoE (dynamic) optimization formulation.

Edit: This is a work in progress right now. I am not 100% happy with the proposed changes.

@adowling2 adowling2 marked this pull request as ready for review March 4, 2026 23:07
@adowling2 adowling2 changed the title WIP: Pyomo.DoE: add advanced run_doe debug config and fix trace/Cholesky initialization consistency Pyomo.DoE: add advanced run_doe debug config and fix trace/Cholesky initialization consistency Mar 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant