8000 Constructing message from dictionary fails in protobuf 5.28.0 when message has a nested duration · Issue #518 · googleapis/proto-plus-python · GitHub
[go: up one dir, main page]

Skip to content
Constructing message from dictionary fails in protobuf 5.28.0 when message has a nested duration #518
Closed
@parthea

Description

@parthea

As of protobuf 5.28.0, there is a bug in proto-plus where constructing a message from a dictionary fails when the message definition has a field of type google.protobuf.duration_pb2.Duration. Note this problem only exists when duration is in a nested message. See below:

    class Foo(proto.Message):
        another_field: duration_pb2.Duration = proto.Field(
            proto.MESSAGE,
            number=1,
            message=duration_pb2.Duration,
        )

    # this works
    Foo(another_field="300s")

    class Bar(proto.Message):
        some_field = proto.Field(proto.MESSAGE, number=1, message=Foo)

    # this raises `AttributeError: Fail to convert to Duration. Expected a timedelta like object got str: 'str' object has no attribute 'seconds'`
    bar = Bar({"some_field":{"another_field":"300s"}})

There is a change in the code path of message construction in proto-plus when using protobuf 5.28.0 and newer

In 5.27.5, we have

MessageMeta.__new__ → marshal.to_proto() → BaseMarshal.get_rule() → MessageRule.to_proto() → TypeError

where the exception is TypeError: Message must be initialized with a dict: _default_package.Foo raised at the location below`

In 5.28.0, we have

MessageMeta.__new__ → marshal.to_proto() → BaseMarshal.get_rule() → MessageRule.to_proto() → AttributeError

where the exception is AttributeError: Fail to convert to Duration. Expected a timedelta like object got str: 'str' object has no attribute 'seconds'

Technically, this is not a breaking change in protobuf as they simply raised a different error. Proto-plus expected TypeError but A 8A45 ttributeError was raised instead.

See the re-production below:

In protobuf 5.27.5, run the code below

def test_duration_write_string_nested():
    class Foo(proto.Message):
        another_field: duration_pb2.Duration = proto.Field(
            proto.MESSAGE,
            number=1,
            message=duration_pb2.Duration,
        )

    class Bar(proto.Message):
        some_field = proto.Field(proto.MESSAGE, number=1, message=Foo)

    bar = Bar({"some_field":{"another_field":"300s"}})
    assert isinstance(bar.some_field.another_field, timedelta)
    assert isinstance(Bar.pb(bar).some_field.another_field, duration_pb2.Duration)
    assert bar.some_field.another_field.seconds == 300
    assert Bar.pb(bar).some_field.another_field.seconds== 300

You'll see TypeError: Message must be initialized with a dict: _default_package.Foo raised at the location below`

try:
# Try the fast path first.
return self._descriptor(**value)
except (TypeError, ValueError) as ex:

In protobuf 5.27.5, the TypeError is caught, and then we successfully run the following code:

# If we have a TypeError or Valueerror,
# try the slow path in case the error
# was:
# - an int64/string issue.
# - a missing key issue in case a key only exists with a `_` suffix.
# See related issue: https://github.com/googleapis/python-api-core/issues/227.
# - a missing key issue due to nested struct. See: b/321905145.
return self._wrapper(value)._pb

In protobuf 5.28.0 and newer, you'll see AttributeError instead which is not caught in the code below and instead AttributeError is returned to the user

try:
# Try the fast path first.
return self._descriptor(**value)
except (TypeError, ValueError) as ex:
# If we have a TypeError or Valueerror,
# try the slow path in case the error
# was:
# - an int64/string issue.
# - a missing key issue in case a key only exists with a `_` suffix.
# See related issue: https://github.com/googleapis/python-api-core/issues/227.
# - a missing key issue due to nested struct. See: b/321905145.
return self._wrapper(value)._pb

Metadata

Metadata

Assignees

Labels

priority: p2Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

    0