Spike: property access

[ajtucker]

path_first, path_all, hash key [:some_key] access and direct property access have differing semantics, e.g. direct property access seems to raise an exception if a property isn't available in the JSON vs. path_first which will just return nil.

We want to allow the use of type signatures in a separate Gem to effectively annotate the dynamically created types with expected properties, so may be able to introspect at run-time, or else create some abstract classes in order to determine whether direct property access using the respond_to_missing? and method_missing functions.

Direct property access seems to not be used much in RP, but we should look at other project(s) that use this library to see whether they just use path_first and path_all.

Extra points for using dry-monads.

[ajtucker]

I've added a test that exhibits this behaviour difference. For RP Applications, fetching a PBO application and looking for a marketingName property, item.path_first('marketingName') returns nil (as there's no value for this property), whereas item.marketingName raises a NoMethodError.

When creating the sapi application/instance, we pass a reference to the sapi application YAML and this is used to load the "endpoint specification" YAML files under the endpointSpecs directory. Currently, this will ignore the model.yaml file, as the type: model is not in the list of endpoint types.

We want to load this model.yaml file and parse out the global properties and the properties applicable to each class, so that when we get a JSON response back, we can determine whether a property is allowed on the class/object (as given by the @type). We should probably include the transitive subclasses too.

We could also parse out the optional, multi etc. fields to figure out whether to e.g. raise an exception when a required value isn't present, or return a Maybe (for zero-to-one optional properties) or a list/array (for zero-to-many) or some clever monad for lists that must have at least one element.

[joescottdave]

For Food Alerts Editor and data dot food editor, I've gone for everything being a required field, with null for single values, empty arrays for lists, and at least one example of a list that must contain at least one, that instantiates with a blank version of that thing.

Ian always used to talk a lot about some pattern where each class had its own "null" variant, but I'm not sure if it had a name or how to implement it.

[ajtucker]

For Food Alerts Editor and data dot food editor, I've gone for everything being a required field

How do you know what the fields are: you create a class listing the fields you know about for a given @type?

The idea in this ticket is to glean the list of fields from the model.yaml file at run-time so that instead of checking the returned JSON for a key (field/property) which may not be there if there's no value, we can check the pre-parsed list of properties for the given @type.

Ian always used to talk a lot about some pattern where each class had its own "null" variant, but I'm not sure if it had a name or how to implement it.

This could well be the monad pattern from functional programming. E.g.
https://www.npmjs.com/package/typescript-monads has Maybe where values are either none<T> or maybe<T> for some type T.

[ajtucker]

For Food Alerts Editor and data dot food editor, I've gone for everything being a required field

How do you know what the fields are: you create a class listing the fields you know about for a given @type?

The idea in this ticket is to glean the list of fields from the model.yaml file at run-time so that instead of checking the returned JSON for a key (field/property) which may not be there if there's no value, we can check the pre-parsed list of properties for the given @type.

Ian always used to talk a lot about some pattern where each class had its own "null" variant, but I'm not sure if it had a name or how to implement it.

This could well be the monad pattern from functional programming. E.g.
https://www.npmjs.com/package/typescript-monads has Maybe where values are either none<T> or maybe<T> for some type T.

[joescottdave]

I hand wrote the types from what I could infer from the data and corrected them when something new arose, so they're far from safe. I also use Plain JavaScript Objects and not classes, because Vue's reactivity system cannot track the changes to values returned by class methods.

JavaScript classes also break when you're using server-side rendering, because they need to be "rehydrated" client-side

[bogdanadrianmarc]

I've hit a couple of blockers while doing some investigation into this. First is that sapi-client-ruby treats every JSON response as a SapiResource, meaning the gem doesn't know what is the real type of a resource. We can figure that out using the @type field from the JSON response, and then look into the model and figure out what fields this can have. However, if we were to do this it doesn't change the functionality much as we need to return a result from the method_missing method as this is expected by Reg Prod for example. This means we would only have a value to return if that field is actually in the data (which is what we are already checking), so I'm unsure how to proceed after this investigation? If the issue is that path_first and path_all return nil if the field is missing, we can fix that by returning nil in the method_missing method as well

[ajtucker]

The resource wrapper code in https://github.com/epimorphics/sapi-client-ruby/blob/main/lib/sapi_client/resource_wrapper.rb figures out whether there is a class corresponding to the @type field in the returned JSON, so I don't think we need to worry about this part.

This ticket is about parsing the model.yaml file in a similar way to the Rake task in the API gems.

When the sapi-ruby-client is created, we send it the location of the application.yaml (see e.g. fsa-regulated-products-api) file and from there it reads through the YAML files in the endpoints folder, checking the type: in each YAML file against the list of understood file types, see https://github.com/epimorphics/sapi-client-ruby/blob/main/lib/sapi_client/sapi_endpoint.rb#L16-L20.

It currently ignores the model.yaml file as this has type: model. Instead of ignoring it, I'd like us to parse this file during initialisation and build a lookup table (hash) from class URI (you'll need to combine the namespace and rest of the path, e.g. class: "geo-def:Country" needs to be http://data.food.gov.uk/codes/geographies/def/terms/Country).

The lookup should then give you another lookup from a given property name (e.g. label) to some definition of that property (e.g. whether it is required; whether it is multi-valued).

This series of lookups, from class URI (given by one of the @type values) then property name, can then be used in the responds_to?() and method_missing() methods in SapiClient::SapiResource so that even if there is no key in the returned JSON, we can still return nil rather than an exception.

Note that if we returned true for responds_to?() for any given property so that we could return nil for property that isn't in the JSON, this would fail to catch errors/typos — we want to only return true to responds_to?() when the property is declared on that class in the model.yaml file.

One thing to beware of is that a property may be declared on the superclass (see the subClassOf: declarations in the model.yaml file).

It would be great to develop this test-first, or at the very least with some tests. There are already fixture files in the test/fixtures directory that we could use.

[bogdanadrianmarc]

Thanks @ajtucker for clarifying this, there's a PR up now with all the changes here: #73 . Added unit tests for the new functionality as well

[ajtucker]

Thanks @bogdanadrianmarc, I'll take a look. Have a great weekend!