Skip to content

Conversation

@vict0rsch
Copy link
Collaborator

Not tested yet, feel free to do it yourself in the mean time.

This would be my preferred direction if we don't hit more configs-related obstacles.

@vict0rsch
Copy link
Collaborator Author

Make loading the data optional ; MAKE SURE this does not break GflowNet validation procedure which could expect the dataloaders

@vict0rsch
Copy link
Collaborator Author

vict0rsch commented Sep 24, 2024

Implemented a first version refactoring BaseTrainer.load(...) to use a new kwarg called prevent_load which is used in prepare_for_gfn as

config["prevent_load"] = {
    "logger": True,
    "loss": True,
    "datasets": True,
    "optimizer": True,
    "extras": True,
}

@lpodina
Copy link
Collaborator

lpodina commented Oct 1, 2024

On origin/catalyst_parallel for gflownet-dev and origin/faster_surface for ocp, and this command can trigger a permission denied error:
HYDRA_FULL_ERROR=1 python -W ignore main.py user=elena.podina +experiments=catalysts/asir_magpie logger.debug=True env.atomgraphconverter.n_cpu_threads=64 gflownet.optimizer.batch_size.forward=64 gflownet.optimizer.n_train_steps=3 +env.atomgraphconverter.sample_timeout=2 +env.atomgraphconverter.pyx_timeout=1

@vict0rsch vict0rsch marked this pull request as ready for review October 8, 2024 13:43
@vict0rsch vict0rsch requested a review from lpodina October 8, 2024 13:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants