-
Notifications
You must be signed in to change notification settings - Fork 1
Test Spec serialisation
There are (currently) 3 forms that test metadata can take:
Until now I've been using the existing code for serialising the tests from the frameworks, which takes the fields it needs from the test classes and sends them as-is (aside from some slight formatting of the exceptions to normalise paths, add the root path and strip newlines) to the extension which then has to parse that JSON and then figure out how to manipulate it to fit the format needed by the Test Explorer API.
This results in doing a lot of unnecessary work and means that the code trying to manipulate the data into the right forms doesn't even have all the data it needs in some cases (e.g. RSpec groups/contexts).
I think it would make much more sense to have the formatters/reporters manipulate the data they get from the framework classes where they can still query the framework for any additional data they might need, convert it into the form needed to make VSCode TestItems, and send that to the extension as JSON (with a schema to validate it on both ends). Not only does this make life easier, and make it clear as to what would be needed to add any other test runners as there's a clearer boundary, but it also means that the code to handle any particular test framework is where the TestItem IDs can be handled.
This would be a huge improvement - I've been using the relative filenames as IDs until now, with either line numbers or RSpec style location descriptors (e.g. foo.rb[123] or foo.rb[1:3:3:4]) for specs within a file which kind of works, but involves an awful lot of string manipulation and means that the extension needs to know far more than it should about how any given framework's tests are structured. If we allow the test framework formatter to pick its own IDs, we can use the schema to enforce that they're unique per parent, and stop using the TestItem ID as a combination of ID and location for building up the command to run tests. It should only be used as an ID that has no other assumed meaning, and buiding up the command for running a test should be able to rely solely on the TestItem's uri and range attributes.
This in turn would hopefully then make it easier to support remote workflows, where the URI might not just be a file:// one