Draft
Conversation
adding weathergen JSON reader to develop
Contributor
Author
|
I have not yet looked at the map plots or mlflow. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Parameters for individual metrics are added in a new optional entry
metric_parametersin the eval config, for exampleWhen a metric is loaded or computed, corresponding parameters are taken from
metric_parameters.New JSON files for scores now have a top level key "scores", below which is a list of results for the same score with different parameter settings, with each list entry following the same format as before. Old files with no "scores" key are treated appropriately.
Upon loading a score, the reader goes through all elements of the JSON file in search of the right parameter settings. When saving scores,
metric_list_to_jsonnow checks whether the output file is already present. If it is, it goes through the file in search of the right parameters settings. If it finds them (possible when for example fsteps was changed), it replaces that entry with the new values. If the right parameters are not found, the new results are appended at the end of the JSON file.Issue Number
Closes #1475