__doc__ property

[dg-pb]

Hello

I have a code:

class A:
    string = 'Hello'
    @property
    def __doc__(self):
        return self.string


class MyCustomProxy(wrapt.ObjectProxy):
    pass


if __name__ == '__main__':
    a = A()
    dmw = wrapt.ObjectProxy(a)
    dmwc = MyCustomProxy(a)
    print(dmw.__doc__)
    print(dmwc.__doc__)
    a.string = 'Yes'
    print(dmw.__doc__)
    print(dmwc.__doc__)

Result:

Hello
Hello
Yes
Hello

Python implementation of the proxy object calls doc from its ProxyObject.@Propert(doc).

However, C implementation hard-copies doc value to the proxy if inheriting from it.

Is this expected?

[GrahamDumpleton]

Not sure. There has to be special code for certain dunder attributes in the pure Python implementation because of how the Python object model works.

class _ObjectProxyMethods(object):

    # We use properties to override the values of __module__ and
    # __doc__. If we add these in ObjectProxy, the derived class
    # __dict__ will still be setup to have string variants of these
    # attributes and the rules of descriptors means that they appear to
    # take precedence over the properties in the base class. To avoid
    # that, we copy the properties into the derived class type itself
    # via a meta class. In that way the properties will always take
    # precedence.

    @property
    def __module__(self):
        return self.__wrapped__.__module__

    @__module__.setter
    def __module__(self, value):
        self.__wrapped__.__module__ = value

    @property
    def __doc__(self):
        return self.__wrapped__.__doc__

    @__doc__.setter
    def __doc__(self, value):
        self.__wrapped__.__doc__ = value

    # We similar use a property for __dict__. We need __dict__ to be
    # explicit to ensure that vars() works as expected.

    @property
    def __dict__(self):
        return self.__wrapped__.__dict__

    # Need to also propagate the special __weakref__ attribute for case
    # where decorating classes which will define this. If do not define
    # it and use a function like inspect.getmembers() on a decorator
    # class it will fail. This can't be in the derived classes.

    @property
    def __weakref__(self):
        return self.__wrapped__.__weakref__

class _ObjectProxyMetaType(type):
    def __new__(cls, name, bases, dictionary):
        # Copy our special properties into the class so that they
        # always take precedence over attributes of the same name added
        # during construction of a derived class. This is to save
        # duplicating the implementation for them in all derived classes.

        dictionary.update(vars(_ObjectProxyMethods))

        return type.__new__(cls, name, bases, dictionary)

class ObjectProxy(with_metaclass(_ObjectProxyMetaType)):

   ...

The line:

dictionary.update(vars(_ObjectProxyMethods))

should only be copying the function definition from the class type, not an instance, so the property shouldn't I don't think be dereferenced at that point.

So would need to work if this is working as intended and where __doc__ property is invoked.

[GrahamDumpleton]

This may have more to do with strange Python rules about lookup order for __doc__ when mixing Python derived class, with C base class vs Python base class. Behaviour between Python and C objects isn't always exactly the same and strange things happen with some of the dunder methods.

[dg-pb]

Python implementation works fine and metaclass takes care of things for all inherited classes, C implementation is where things "malfunction". And I like C for obvious reasons.

I found that what solves it is:

class MyCustomProxy(wrapt.ObjectProxy):
    @property
    def __doc__(self):
        return self.__wrapped__.__doc__

However, if I inherit from it, then I need to do the same for sub-class again...

Another solution is to mix & match:

class MyCustomProxy(wrapt.ObjectProxy, metaclass=_ObjectProxyMetaType):
    pass

class MyCustomProxy2(MyCustomProxy):
    pass

This way, python's meta implementation will take care of things on top of C implementation, even if inherited from it again.

I would look at it, but I am quite basic at C, so all I can do now is to point it out and give couple of hacks for it.

[GrahamDumpleton]

The disparity with the C object proxy base class will be because it uses a C level getter/setter function slot for handling access to __doc__. A property in pure Python doesn't quite work the same when consider lookup resolution order from memory. To try and make the same one might have to avoid using getter/setter slots and add a descriptor object as attribute, similar to how property objects work. The overhead of that will be a bit more, although still small, and code will be bit more complicated in C implementation. Not sure how hard that would be to do.

[dg-pb]

Just for curiosity, which functionality of ProxyObject benefits most from C's implementation over python's?

Construction seems to be about 5x, however attribute access, such as doc look similar.

Also, __doc__ access of C object with python's meta is actually slower than just using pure python: 0.038 vs 0.023, which is the same as C: 0.022

[GrahamDumpleton]

Wasn't a case of choosing one over the other for a reason. When doing C based Python objects you just use the most obvious mechanism it provides for it. These strange situations like overriding __doc__ and that they behaved differently only became apparent many years after the original code was written.

[GrahamDumpleton]

If the question was more around why bother with C implementation at all, was because back in Python 2.X days when machines were slower, any pure Python overhead could be noticeable, especially with the original use for wrapt, which was in New Relic Python web application performance monitoring instrumentation. So much was being instrumented and being continually exercised, pure Python variant had a noticeable overhead, which is last thing you want when instrumenting production systems.

[dg-pb]

Thank you, that answered it.

[dg-pb]

Do I leave this so that someone can take a look, or do I close this?