Refactor Overly Complex config.py

[xiaopeng0822]

The sdgsystem/configs/config.py file has grown to 818 lines, making it difficult to maintain, understand, and extend. Multiple config classes have extremely long from_dict methods (150+ lines), violating the Single Responsibility Principle.

Metrics

• Lines: 818 (should be split into multiple files)
• Classes: 20+ config classes in one file
• Cyclomatic Complexity: High in from_dict methods
• Method Length: Some methods exceed 150 lines

Issues

1. Discoverability - Hard to find specific config classes
2. Merge Conflicts - High risk when multiple people edit
3. Testing - Difficult to test in isolation
4. Cognitive Load - Too much to understand in one file
5. Reusability - Config classes can't be easily imported separately

Proposed Structure

sdgsystem/configs/
├── __init__.py              # Re-export all configs
├── base.py                  # BaseConfig, common utilities
├── model.py                 # ModelConfig, ProviderConfig
├── generation.py            # GenerationConfig, RewriteConfig
├── tasks/
│   ├── __init__.py
│   ├── text.py             # TextTaskConfig, LocalConfig, WebConfig
│   └── image.py            # ImageTaskConfig, VQAConfig
├── training.py             # TrainerConfig, SFTConfig, GRPOConfig
├── evaluation.py           # EvaluationConfig, DeepEvalConfig
├── server.py               # ServerConfig, CORSConfig (new!)
└── loaders.py              # YAML/dict loading utilities

Benefits

✅ Maintainability - Each file < 200 lines
✅ Clarity - Related configs grouped together
✅ Testability - Can test each module independently
✅ Extensibility - Easy to add new config types
✅ Navigation - IDE can jump to files directly

Migration Strategy

1. Create new directory structure
2. Extract configs incrementally (maintain backward compatibility)
3. Update imports in dependent files
4. Keep __init__.py with re-exports for compatibility
5. Update documentation

[Harvey-zhou]

Yes the large config.py file and long from_dict methods should probably be refactored. We will look into simplifying those by leveraging Pydantic's built-in parsing before splitting the config file. Thanks for raising this issue!