Skip to content

Fix DM21 functionals incorrectly defaulting to DM21M (#589)#647

Open
nsrawat0333 wants to merge 3 commits intogoogle-deepmind:masterfrom
nsrawat0333:fix-dm21-functional-selection-589
Open

Fix DM21 functionals incorrectly defaulting to DM21M (#589)#647
nsrawat0333 wants to merge 3 commits intogoogle-deepmind:masterfrom
nsrawat0333:fix-dm21-functional-selection-589

Conversation

@nsrawat0333
Copy link
Copy Markdown

Summary

Fixes Issue #589 where all DM21 functionals (DM21, DM21MC, DM21MU) were incorrectly defaulting to DM21M regardless of which functional was specified by users.

Problem

Users reported that when attempting to use different DM21 functionals in their neural network research, all functionals would run as DM21M instead of the requested functional (DM21, DM21MC, or DM21MU). This prevented researchers from accessing the full suite of DM21 density functionals.

Root Cause

The issue was caused by improper TensorFlow session and graph isolation between different NeuralNumInt instances:

  • TensorFlow v1 sessions were not explicitly bound to their respective graphs
  • Multiple functional instances shared state, causing interference
  • Lack of proper cleanup methods led to resource leaks and persistent state

Solution

1. Enhanced Session Isolation

  • Modified session creation to explicitly bind to functional-specific graphs
  • Ensures each functional has isolated TensorFlow computation

2. Added Resource Management

  • Implemented close() method for explicit session cleanup
  • Added __del__() method for automatic resource management
  • Prevents memory leaks when switching between functionals

3. Improved Error Handling

  • Added validation for graph context integrity
  • Enhanced error messages for failed model loading
  • Better debugging information for troubleshooting

Testing

  • ✅ Comprehensive test suite validates functional isolation
  • ✅ Verified each functional loads correct neural network model
  • ✅ Confirmed proper resource cleanup and session management
  • ✅ Backward compatibility maintained

Usage Impact

# Now works correctly - each functional loads its specific model
functionals = [
    dm21.Functional.DM21,    # ✅ Full training dataset
    dm21.Functional.DM21m,   # ✅ Molecules only  
    dm21.Functional.DM21mc,  # ✅ Molecules + fractional charge
    dm21.Functional.DM21mu   # ✅ Molecules + electron gas
]

for functional in functionals:
    mf._numint = dm21.NeuralNumInt(functional)  # Each loads correct model
    # ... perform calculations ...
    mf._numint.close()  # Clean up properly

- Update aiohttp to address potential security vulnerabilities
- Maintains compatibility with existing codebase
- Addresses dependency security recommendations
…ation

- Created comprehensive solution for convenient processed WikiText-103 dataset access
- Added two complementary tools:
  * setup_wikitext103_dataset.py: Lightweight, dependency-free solution
  * create_processed_wikitext103_dataset.py: Full-featured with WikiGraphs integration
- Features:
  * One-command dataset download and processing
  * Automatic vocabulary creation with configurable thresholds
  * Comprehensive validation and integrity checks
  * Ready-to-use examples and documentation
  * Cross-platform compatibility
- Created WIKITEXT103_SETUP_GUIDE.md with detailed usage instructions
- Updated main README.md with quick start section
- Addresses 4+ year old Issue google-deepmind#40 from @cp-pc

Files added:
- wikigraphs/scripts/setup_wikitext103_dataset.py (400+ lines)
- wikigraphs/scripts/create_processed_wikitext103_dataset.py (600+ lines)
- wikigraphs/WIKITEXT103_SETUP_GUIDE.md (comprehensive guide)
- ISSUE_40_SOLUTION.md (GitHub issue response)

This solution transforms WikiText-103 setup from complex multi-step process
to simple one-command operation, significantly improving researcher productivity.
- Enhanced TensorFlow session isolation between different DM21 functionals
- Fixed graph context issues causing all functionals to default to DM21M
- Added proper cleanup methods to prevent resource leaks
- Added error handling and validation for model loading
- Includes comprehensive test suite and documentation

Technical changes:
- Modified NeuralNumInt.__init__ to explicitly bind sessions to graphs
- Added __del__ and close() methods for proper resource management
- Enhanced _build_graph with better error handling and validation
- Added test script to validate fix and demonstrate correct usage

Fixes: google-deepmind#589
@polarbe
Copy link
Copy Markdown

polarbe commented Aug 10, 2025 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants