8000 Imports from adjacent .proto files does not work · Issue #441 · danielgtaylor/python-betterproto · GitHub
[go: up one dir, main page]
Skip to content

Imports from adjacent .proto files does not work #441

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
BananaLoaf opened this issue Nov 24, 2022 · 2 comments
Open

Imports from adjacent .proto files does not work #441

BananaLoaf opened this issue Nov 24, 2022 · 2 comments

Comments

@BananaLoaf
Copy link

First of all, I looked at the issue #408 and I am absolutely sure that I have betterproto[compiler] of version 2.0.0b5 installed

I have this file structure:

protos
├── common.proto
└── devices
    └── rpi.proto

With protos/common.proto:

syntax = 'proto3';
package org.company.name.common;

message DeviceId {
  string device_id = 1;
}

and protos/devices/rpi.proto:

syntax = 'proto3';
package org.company.name.rpi;

import "common.proto";
import "google/protobuf/empty.proto";

service RPI {
  rpc Start(common.DeviceId) returns (google.protobuf.Empty) {}
  rpc Stop(common.DeviceId) returns (google.protobuf.Empty) {}

I then generate betterproto files:

poetry run python -m grpc_tools.protoc \
        -I protos \
        --python_betterproto_out=package_name/grpc \
        protos/common.proto \
        protos/devices/rpi.proto \

The BUG

In package_name.grpc.org.company.name.rpi any mentions of DeviceId are presented as _common__.DeviceId, with _common__ not being imported. Launching that code does not work either, python has no idea what _common__ is.
@BananaLoaf BananaLoaf mentioned this issue Mar 31, 2023
Open
@JulianNeuberger
Copy link
JulianNeuberger commented Aug 15, 2023 •
edited
Loading

I stumbled over this problem today and was able to reproduce it in version 2.0.0b6

Given the two proto files below:

dependency.proto

syntax = "proto3";

package report.dependency;

message DependencyMessage {}

main.proto

syntax = "proto3";

import "dependency.proto";

package report.main;

message ReturnMessage {}

service MainService {
  rpc SomeCall(report.dependency.DependencyMessage) returns (ReturnMessage);
}

Running python -m grpc_tools.protoc --proto_path . --python_betterproto_out=. main.proto results in the following, broken python code:

from dataclasses import dataclass
from typing import (
    TYPE_CHECKING,
    Dict,
    Optional,
)

import betterproto
import grpclib
from betterproto.grpc.grpclib_server import ServiceBase


if TYPE_CHECKING:
    import grpclib.server
    from betterproto.grpc.grpclib_client import MetadataLike
    from grpclib.metadata import Deadline


@dataclass(eq=False, repr=False)
class ReturnMessage(betterproto.Message):
    pass


class MainServiceStub(betterproto.ServiceStub):
    async def some_call(
        self,
        dependency_dependency_message: "_dependency__.DependencyMessage",
        *,
        timeout: Optional[float] = None,
        deadline: Optional["Deadline"] = None,
        metadata: Optional["MetadataLike"] = None
    ) -> "ReturnMessage":
        return await self._unary_unary(
            "/report.main.MainService/SomeCall",
            dependency_dependency_message,
            ReturnMessage,
            timeout=timeout,
            deadline=deadline,
            metadata=metadata,
        )


class MainServiceBase(ServiceBase):
    async def some_call(
        self, dependenc
A3C5
y_dependency_message: "_dependency__.DependencyMessage"
    ) -> "ReturnMessage":
        raise grpclib.GRPCError(grpclib.const.Status.UNIMPLEMENTED)

    async def __rpc_some_call(
        self,
        stream: "grpclib.server.Stream[_dependency__.DependencyMessage, ReturnMessage]",
    ) -> None:
        request = await stream.recv_message()
        response = await self.some_call(request)
        await stream.send_message(response)

    def __mapping__(self) -> Dict[str, grpclib.const.Handler]:
        return {
            "/report.main.MainService/SomeCall": grpclib.const.Handler(
                self.__rpc_some_call,
                grpclib.const.Cardinality.UNARY_UNARY,
                _dependency__.DependencyMessage,
                ReturnMessage,
            ),
        }

Note the missing import from .. import dependency as _dependency__ which would be needed in lines 32, 50, 56, and 67.

Using the message DependencyMessage as a output type generates the import as expected, i.e. the following proto file does not show the problem described above

main.proto

syntax = "proto3";

import "dependency.proto";

package report.main;

message ReturnMessage {}

service MainService {
  rpc SomeCall(report.dependency.DependencyMessage) returns (report.dependency.DependencyMessage);
}

My unsophisticated fix would be to add this line to the top of plugin.models.ServiceMethodCompiler.__post_init:

self.py_input_message_type

Simply using the property seems to fix the problem, as the underlying method for building the import statements adds the needed imports to output_file.imports, which in turn makes them generate properly. This is why using the imported Message type as a return type generates the import properly: we use the property self.py_output_message_type in the check of line 726

I'm sure there is a better way though :)

@BananaLoaf
Copy link
Author

@JulianNeuberger I ended up forking it

https://github.com/BananaLoaf/python-bananaproto

I fixed this issue and changed some stuff I needed to change

@Dimitrioglo Dimitrioglo mentioned this issue Jan 31, 2024
Open
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@JulianNeuberger @BananaLoaf
0