Skip to content

fix(config): keep QlibConfig.registered safe in joblib worker subprocesses#2212

Open
genisis0x wants to merge 1 commit into
microsoft:mainfrom
genisis0x:fix/2038-config-registered-worker
Open

fix(config): keep QlibConfig.registered safe in joblib worker subprocesses#2212
genisis0x wants to merge 1 commit into
microsoft:mainfrom
genisis0x:fix/2038-config-registered-worker

Conversation

@genisis0x
Copy link
Copy Markdown

Summary

  • Fixes Backtest/joblib worker crash: AttributeError: No such registered in self._config #2038. QlibConfig.registered is implemented as a @property that returns self._registered, but because the overridden __setattr__ routes attribute writes through self.__dict__["_config"], the _registered flag actually lives inside the _config dict. If _config is rebuilt from _default_config without the _registered key (e.g. inside a freshly-spawned joblib worker that re-imports qlib, or transiently inside Config.reset() before QlibConfig.__init__ re-sets the flag), the property's inner self._registered access raises AttributeError, Python falls back to Config.__getattr__ for the original attribute name registered, and the worker dies with:

    AttributeError: No such ``registered`` in self._config
    

    This terminates backtests along PortfolioMetrics._cal_benchmark -> get_higher_eq_freq_feature -> D.features -> ParallelExt -> inst_calculator -> register_from_C.

  • Make the property defensive: read _registered out of self.__dict__["_config"] via a safe .get() chain so a missing key returns False instead of triggering the property -> __getattr__ cascade.

  • Harden register_from_C with getattr(C, "registered", False) so the second top-level caller (qlib/config.py:115) is double-protected against the same failure mode.

Validation

  • uv run pytest tests/misc/test_config_registered.py -- 4 passed.
  • Pre-fix sanity check: stashing the patch and running the new test suite reproduces the reporter's exact error message (AttributeError: No such ``registered`` in self._config) on test_registered_property_returns_false_when_key_missing and test_registered_property_returns_false_when_config_missing; reapplying the patch turns both green.
  • uv run black qlib/config.py tests/misc/test_config_registered.py -l 120 --check --diff -- clean.
  • uv run flake8 --ignore=E501,F541,E266,E402,W503,E731,E203 qlib/config.py -- clean.

Tests added

  • test_registered_property_returns_false_when_key_missing -- worker scenario where _config lacks the _registered key.
  • test_registered_property_returns_false_when_config_missing -- partially-constructed instance (e.g. mid-unpickle) where even _config is absent.
  • test_register_from_C_does_not_raise_when_registered_missing -- exact guarded boolean from the original bug report does not raise on the global C when the key is removed.
  • test_registered_property_after_normal_construction -- happy path still reflects the assigned value.

Fixes #2038

…esses

QlibConfig.registered is implemented as a property that reads
self._registered.  Because the overridden __setattr__ routes attribute
writes through self.__dict__["_config"], "_registered" lives inside
self._config rather than on the instance __dict__.  If self._config has
been rebuilt from _default_config (which does not contain "_registered")
-- for example in a freshly spawned joblib worker that re-imports qlib,
or after Config.reset() inside QlibConfig.__init__ before QlibConfig sets
self._registered -- the property's inner self._registered access raises
AttributeError, Python falls back to Config.__getattr__ for the original
name ("registered"), and the worker crashes with:

    AttributeError: No such ``registered`` in self._config

This terminates backtests that spawn joblib workers along the
PortfolioMetrics._cal_benchmark -> get_higher_eq_freq_feature ->
D.features -> ParallelExt -> inst_calculator -> register_from_C path
(issue microsoft#2038).

Make the property defensive: read "_registered" out of
self.__dict__["_config"] with a safe ``get`` chain so a missing key
returns False instead of triggering the property -> __getattr__ cascade.
Also harden register_from_C with ``getattr(C, "registered", False)`` so
the second top-level caller is double-protected.

Add regression tests for:
- registered returns False when "_registered" key is absent from _config,
- registered returns False on a partially-initialised QlibConfig
  (mirrors unpickle / fresh-import state seen in workers),
- ``getattr(C, "registered", False)`` does not raise when the key is
  missing on the global C,
- registered returns the assigned value on the happy path so the fix
  does not regress normal behaviour.

Fixes microsoft#2038
@genisis0x
Copy link
Copy Markdown
Author

Read through the CLA — all good.

@microsoft-github-policy-service agree

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Backtest/joblib worker crash: AttributeError: No such registered in self._config

1 participant