Followup to the yaml helper refactor

[LecrisUT]

Splitting up a couple of comments from #4334

• ¹ ruame.yaml.YAML uses round-trip as the default parser, but if we do not save and re-use the yaml object, this does not really help us. to_yaml is example of that case where we throw away the yaml object afterwads.
• ² Related to the above, it would actually be useful to use the roundtrip to incrementally add to a file. I wonder if that is not what we actually see in Poor performance of tmt/testing-farm with thousands of tests #4406

Footnotes

1. https://github.com/teemtee/tmt/pull/4334#discussion_r2668330246 ↩

2. https://github.com/teemtee/tmt/pull/4334#discussion_r2668339049 ↩

[happz]

• 2 Related to the above, it would actually be useful to use the roundtrip to incrementally add to a file. I wonder if that is not what we actually see in Poor performance of tmt/testing-farm with thousands of tests #4406

We effectively overwrite parts of that file, replacing one pending result record with 1 or more actual results once a test finishes. I.e. not a simple append, that would have been possible before pending outcome was introduced.

[LecrisUT]

• 2 Related to the above, it would actually be useful to use the roundtrip to incrementally add to a file. I wonder if that is not what we actually see in Poor performance of tmt/testing-farm with thousands of tests #4406

We effectively overwrite parts of that file, replacing one pending result record with 1 or more actual results once a test finishes. I.e. not a simple append, that would have been possible before pending outcome was introduced.

Yes, but that should still be covered by the roundtrip yaml loader. But if as part of the dumper we re-read the file every time, then any efficiency that the roundtrip yaml was doing is probably gone. Probably we cannot tell for certain without trying it out or at least profiling that case.

[happz]

• 2 Related to the above, it would actually be useful to use the roundtrip to incrementally add to a file. I wonder if that is not what we actually see in Poor performance of tmt/testing-farm with thousands of tests #4406

We effectively overwrite parts of that file, replacing one pending result record with 1 or more actual results once a test finishes. I.e. not a simple append, that would have been possible before pending outcome was introduced.

Yes, but that should still be covered by the roundtrip yaml loader. But if as part of the dumper we re-read the file every time, then any efficiency that the roundtrip yaml was doing is probably gone. Probably we cannot tell for certain without trying it out or at least profiling that case.

We don't reread the file, we don't read it at all in this scenario. We take internal list of results, ExecutePlugin._results, swap the pending result for actual one(s), and serialize that list into a results.yaml file, overwriting it with new content.

[LecrisUT]

We don't reread the file, we don't read it at all in this scenario. We take internal list of results, ExecutePlugin._results, swap the pending result for actual one(s), and serialize that list into a results.yaml file, overwriting it with new content.

Ah indeed, when we do to_yaml we do not pass it a reference of the file. But then I can see two sources of slow down:

• Creating the yaml from scratch everytime to_yaml is called. We are definitely experiencing this slow down at least. How measurable this is, if there even is one 🤷
• Rewriting to the file with the same data. Technically we could speed this up by skipping to the line we actually changed, but not sure if there is a library that would support this

[happz]

We don't reread the file, we don't read it at all in this scenario. We take internal list of results, ExecutePlugin._results, swap the pending result for actual one(s), and serialize that list into a results.yaml file, overwriting it with new content.

Ah indeed, when we do to_yaml we do not pass it a reference of the file. But then I can see two sources of slow down:

No data measured for tmt, but:

• Creating the yaml from scratch everytime to_yaml is called. We are definitely experiencing this slow down at least. How measurable this is, if there even is one 🤷

Absolutely, this will be costly. See it happen in Artemis, same issue, large list of structures serialized into YAML, not cheap. This is where another format would benefit us, even a simple JSON does not care about indentation, multiline strings, all the formatting YAML requires. A binary format might be even better.

• Rewriting to the file with the same data. Technically we could speed this up by skipping to the line we actually changed, but not sure if there is a library that would support this

No idea how hard is this one hitting, needs to be measured.

After finishing to_yaml/from_yaml, I have a PR ready to add to_state/from_state, helpers tmt would use to save these files, without worrying about the format. That one would be configurable via envvar, and then I would hit it with some profiler, adding to_*/from_* helpers for couple more formats to experiment with.