Choice: Meta-Learning - Using prior data to warmstart optimization
This is our team's approach at finding a combination of search algorithm and multi-fidelity scheduler that performs well on the dataset deepweeds.
The search algorithm is a Bayesian Optimizer that employs a random forest regressor as a surrogate model. This surrogate model is warmstarted on objective function evaluations that were supplied.
The scheduler employs a prediction and grace period strategy to create unique adaptive fidelity behavior.
Download the Repository
git clone https://github.com/MarcSpeckmann/Meta-Learning-Using-Prior-Data-to-Warmstart-Optimization.gitFirst you need to install miniconda.
Make sure there is no environment that is already named "automl"
Subsequently, run these commands:
make installActivate the environment
conda activate automlYou may want to adjust the settings for how many concurrent trials to run and how many CPUs and GPUs to use per trial. These settings are found starting at line 281 in main.py
Our test system was able to handle up to about 12 concurrent trials, with fractional resource allocations.
Run the main experiment file
python main.pyFor reproducing the experiments detailed on the poster run the specific experiment_<searcher>_<scheduler>_<1,2,3>.py
The experiments were carried out on a computer with the following resources:
- AMD EPYC 7543 32-Core Processor (But only using 4 cores, 4vCPU)
- NVIDIA A100-SXM4-40GB
- 528 GB RAM
The test set accuracy will automatically be displayed after finishing the set runtime.