RFC: Caching field values

[AdamHillier]

Feature description and motivation

At the moment, field values on component instances behave much like instance attributes of generic Python class instances. One value exists per instance, and if it is mutable then an access after modification will return the same, modified value, e.g.:

@component
class A:
    foo: List[int] = Field(lambda: [1, 2, 3])

class B:
    def __init__(self):
        self.foo = [1, 2, 3]

a = A()
b = B()

assert a.foo == b.foo == [1, 2, 3]

a.foo.append(4)
b.foo.append(4)

assert a.foo == b.foo == [1, 2, 3, 4]

We could change this behaviour so that field values instead behave much more like @property values, i.e. the value is not 'cached' on the instance and instead re-generated on every access. See discussion here for a motivation of this different behaviour: larq/zoo#148 (comment).

Current implementation

For a full explanation of how components access field values, see the docstring of the _wrap_getattribute method in component.py:

zookeeper/zookeeper/core/component.py

Lines 116 to 144 in 2b8812a

	"""
	The logic for this overriden `__getattribute__` is as follows:
	During component instantiation, any values passed to `__init__` are stored
	in a dict on the instance `__component_instantiated_field_values__`. This
	means that a priori the `__dict__` of a component instance is empty (of
	non-dunder attributes).
	Field values can come from three sources, in descending order of priority:
	1) A value that was passed into `configure` (e.g. via the CLI), which is
	stored in the `__component_configured_field_values__` dict on the
	component instance or some parent component instance.
	2) A value that was passed in at instantiation, which is stored in the
	`__component_instantiated_field_values__` dict on the current component
	instance (but not any parent instance).
	3) A default value obtained from the `get_default` factory method of a
	field defined on the component class of the current instance if it has
	one, or otherwise from the factory of the field on the component class
	of the nearest parent component instance with a field of the same name,
	et cetera.
	Once we find a field value from one of these three sources, we set the value
	on the instance `__dict__` (i.e. we 'cache' it).
	This means that if we find a value in the instance `__dict__` we can
	immediately return it without worrying about checking the three cases above
	in order. It also means that each look-up other than the first will incur no
	substantial time penalty.
	"""

New implementation

It would be straightforward to implement @property-esqe behaviour for default values which are passed into fields, as mutable default values are already generated from lambdas, and there's no issue with immutable default values being cached .

However, it would be much more difficult to implement for values passed in through the CLI. Consider the configuration CLI argument foo=[1, 2, 3]. We receive this as a string, and parse it into a Python value (in this case a list) to be used as the value for the field foo. If we wanted to return a new instance of this list on each access of foo, we would either need to be able to deep-clone generic mutable objects, or we would have to hold on to the configuration value as a string, and re-parse it into a Python value each time.

It's an open question whether we are happy for the behaviour of default values vs cli-overriden values to be different.

[leonoverweel]

Thanks for the detailed RFC!

It's an open question whether we are happy for the behaviour of default values vs cli-overriden values to be different.

I'd say no, because this could lead to some very hard to diagnose bugs for a user who is unaware of these implementation details.

If we wanted to return a new instance of this list on each access of foo, we would either need to be able to deep-clone generic mutable objects, or we would have to hold on to the configuration value as a string, and re-parse it into a Python value each time.

In general I'm usually in favor of having a bit of internal implementation ugliness as a way to support more intuitive/robust/harder-to-break behavior for the end user, so these options don't sound too bad to me. I'd probably go for the latter, since it sounds easiest.

(Same reasoning as favoring giving up some cleanliness/minimalism in model code to enable larq/zoo#61, for example.)

[AdamHillier]

In general I'm usually in favor of having a bit of internal implementation ugliness as a way to support more intuitive/robust/harder-to-break behavior for the end user, so these options don't sound too bad to me. I'd probably go for the latter, since it sounds easiest.

This is a very valid point. I guess then the question becomes which behaviour we actually want. We can also support both (we already have a keyword arg to Field - allow_missing - and we could add cache_value as well, in which case we would need to decide what the default is).

[lgeiger]

Thanks for the detailed explanation!

I didn't think of the CLI use case, but I also think it would be very confusing if we'd have different behaviours for default vs CLI values.

If we wanted to return a new instance of this list on each access of foo, we would either need to be able to deep-clone generic mutable objects, or we would have to hold on to the configuration value as a string, and re-parse it into a Python value each time.

Can we just wrap the value into a lambda function if it is mutable and store that function on the instance? That way the implementation logic is also quite consistent: We'd always store the argument of Field on the instance. If users pass in mutable values via the CLI, we'd implicitely wrap them into a lambda functions to be consistent with how the Python API works.

One other approach would be to keep the current behaviour (Field ≈ cached_property) and encourage users to not use Field in this case like this and do larq/zoo#149 instead.

we could add cache_value as well, in which case we would need to decide what the default is.

I thought about introducing something like a PartialField, but I think having a keyword argument is cleaner.

In general having

@Field
def input_quantizer(self):
    return lq.quantizers.SteSign(clip_value=1.25)

behave like @property is very intuitive for me, though the same argument could be made the other way around.

[AdamHillier]

If users pass in mutable values via the CLI, we'd implicitely wrap them into a lambda functions to be consistent with how the Python API works.

If we have a single mutable object then wrapping it inside a lambda won't solve the issue because each call to the lambda will return a reference to the same object. I think we would have to do the CLI string parsing inside the lambda.

[AdamHillier]

In general having

@Field
def input_quantizer(self):
    return lq.quantizers.SteSign(clip_value=1.25)

behave like @property is very intuitive for me, though the same argument could be made the other way around.

I completely agree. I think having cache_value default to false is a reasonable thing to do.

[lgeiger]

If we have a single mutable object then wrapping it inside a lambda won't solve the issue because each call to the lambda will return a reference to the same object.

Ah yeah makes sense, since we want to replicate this behaviour:

class A:
    @property
    def foo(self):
        return [1, 2, 3]

class B:
    foo = [1, 2, 3]

a = A()
b = B()

assert a.foo == b.foo == [1, 2, 3]

a.foo.append(4)
b.foo.append(4)

assert a.foo == [1, 2, 3]
assert b.foo == [1, 2, 3, 4]

I think we would have to do the CLI string parsing inside the lambda.

What about using copy.deepcopy in the lambda?

[AdamHillier]

What about using copy.deepcopy in the lambda?

Cool, I hadn't come across that before but it looks like it would work. Especially as the values that can be passed in through the CLI (i.e. parsed from a string) are a very small subset of possible Python objects, and no doubt exclude any 'difficult' ones which would be hard to clone.

[AdamHillier]

Okay, so the concrete proposal is that Field gains a cache_value keyword argument, defaulting to false. When set to true, Field accesses have their current behaviour. When set to false, Field accesses behave like @property in the way specified above.

Are we all happy to move forward with this?

[lgeiger]

Will the bahaviour of immutable values like numbers change depending on cache_value?

Are we all happy to move forward with this?

👍

[AdamHillier]

Will the bahaviour of immutable values like numbers change depending on cache_value?

Unless we require the immutable defaults to be passed in with lambdas we can't really not cache immutable values, but I don't think that matters?

[lgeiger]

Unless we require the immutable defaults to be passed in with lambdas we can't really not cache immutable values, but I don't think that matters?

I agree, just wanted to make sure that this applies only to values we pass via lambda functions.

[AdamHillier]

I agree, just wanted to make sure that this applies only to values we pass via lambda functions.

Makes sense, and yes to be clear this will apply to mutable defaults passed with lambda functions and we'll use the copy.deepcopy module to do the same for things like lists that are passed in through the CLI.

[AdamHillier]

Apologies to come back to this, but I have some further questions after partially implementing this.

It seems to me that behaviour could be quite confusing when you mix cached and not cached fields with parent/child components.

@component
class Child:
    foo: List[int] = Field(cache_value=False)

@component
class Parent:
    child: Child = ComponentField(Child)
    foo: List[int] = Field(lambda: [1, 2, 3], cache_value=True)

p = Parent()
configure(p, {})

assert p.foo == [1, 2, 3]
assert p.child.foo == [1, 2, 3]

p.foo.append(4)

assert p.foo == [1, 2, 3, 4]
assert p.child.foo == ???   # What should this be?

It seems to me that either behaviour here would be confusing. We either don't respect value inheritence, or we don't respect the fact that the child field iscache_value=False.

The solution would probably be to disallow cache_value=False if any 'parent field' has cache_value=True. But what happens if the parent is a super-class defined in another package, that can't be modified (especially as the default for a field is cache_value=True).