8000 Proto Best Practices — references to googleapis/googleapis common objects · Issue #254 · protocolbuffers/protocolbuffers.github.io · GitHub
[go: up one dir, main page]
Skip to content

Proto Best Practices — references to googleapis/googleapis common objects #254

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
software-artificer opened this issue May 14, 2025 · 14 comments
Open

Proto Best Practices — references to googleapis/googleapis common objects #254

software-artificer opened this issue May 14, 2025 · 14 comments

Comments

@software-artificer
Copy link

👋🏻 hey folks!

Not sure if this is an actual issue or just missing documentation, so feel free to point me out to the right place.

I am wondering how do people deal with "common" protobuf messages that are referenced in the "Proto Best Practices" document but are not shipped with the protoc compiler. I am talking about google.types.* specifically here, which are located in the googleapis/googleapis. Feels a bit strange relying on someone's else API schemas.

  • Why aren't these messages part of the well-known types shipped with the protobuf?
  • How do people generally make them available in the CI and to the developers? Copy? Clone? Use googleapis/api-common-protos?
  • Does it make sense to include these into the protocolbuffers project itself?

Thanks in advance!
@Logofile
Copy link
Member

Thanks for submitting your question. Can you point me to an example of the common messages referenced in the Proto Best Practices topic? I'll work to get an answer to you--and potentially update the docs with it, as well--once I better understand the question. Thanks!

@software-artificer
Copy link
Author

@Logofile thanks for your prompt response. And of course, should have done so ahead of time, sorry. Here's the excerpt pointing to googleapis:

protocolbuffers.github.io/content/best-practices/dos-donts.md

Lines 147 to 169 in 354da49

* [`interval`](https://github.com/googleapis/googleapis/blob/master/google/type/interval.proto)
is a time interval independent of time zone or calendar (for example,
2017-01-15T01:30:15.01Z - 2017-01-16T02:30:15.01Z).
* [`date`](https://github.com/googleapis/googleapis/blob/master/google/type/date.proto)
is a whole calendar date (for example, 2005-09-19).
* [`month`](https://github.com/googleapis/googleapis/blob/master/google/type/month.proto)
is a month of year (for example, April).
* [`dayofweek`](https://github.com/googleapis/googleapis/blob/master/google/type/dayofweek.proto)
is a day of week (for example, Monday).
* [`timeofday`](https://github.com/googleapis/googleapis/blob/master/google/type/timeofday.proto)
is a time of day (for example, 10:42:23).
* [`field_mask`](https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/field_mask.proto)
is a set of symbolic field paths (for example, f.b.d).
* [`postal_address`](https://github.com/googleapis/googleapis/blob/master/google/type/postal_address.proto)
is a postal address (for example, 1600 Amphitheatre Parkway Mountain View,
CA 94043 USA).
* [`money`](https://github.com/googleapis/googleapis/blob/master/google/type/money.proto)
is an amount of money with its currency type (for example, 42 USD).
* [`latlng`](https://github.com/googleapis/googleapis/blob/master/google/type/latlng.proto)
is a latitude/longitude pair (for example, 37.386051 latitude and
-122.083855 longitude).
* [`color`](https://github.com/googleapis/googleapis/blob/master/google/type/color.proto)
is a color in the RGBA color space.

All of the links there point to googleapis/googleapis repository, with the excepiton of field_mask which is shipped with protobuf includes.

@Logofile
Copy link
Member

🤦 I don't know why I didn't see that section when I reviewed it. I blame the fact that it was later I the day on a Friday.

I'm not aware of the reason for the separation of the content, but since that's a code issue rather than a documentation issue, I recommend adding an issue in https://github.com/protocolbuffers/protobuf/issues. If you'd like to see a CI/CD best practice paragraph added to the documentation, let's leave this issue open to run alongside the code repository issue.

@software-artificer software-artificer mentioned this issue May 20, 2025
Closed
@software-artificer
Copy link
Author

@Logofile I know the feel and thank you very much for your guidance.

I went ahead and created the issue in the protocolbuffers/protobuf. Let's keep this one open until we get any more clarification there. If the common types will be included in the protoc package this could safely be closed. If not, then we can figure out how to properly document the best practices of fetching these proto files and making them available in CI/CD and development environments.

@Logofile
Copy link
Member

It sounds like we're in agreement. 😊

@mkruskal-google
Copy link
Member

@Logofile Closing protocolbuffers/protobuf#21849 and punting this back to you as a documentation bug 🙂. See my response there, it seems really unlikely we'd ever add new WKTs given the problems we've had with them

@Logofile
Copy link
Member

Thanks! I'l work with @software-artificer in the other issues to get the docs updated.

@Logofile
Copy link
Member
Logofile 8000 commented May 21, 2025

Oops. I got the issues mixed up. This one is the one to keep open. 🤦 Reopening....

@Logofile Logofile reopened this May 21, 2025
@software-artificer
Copy link
Author

Considering we got quite strong "bad vibes" for well-known and common types as the challenges that are associated with them, how do you think we should go about documenting this?

  • Should we keep WKTs as recommended, provided they're there to stay, but suggest that people get inspired by the Google's API design for other types? I.e. get the idea of how people design types for particular use cases and just to your needs kind-of style...
  • Or should we just recommend pulling Google's types into their projects and document how to do this? Which on its own presents the challenge of them not being updated often, as far as I can tell they are still not using editions. But also potentially introducing breaking changes without consumers being aware, because Google is free to iterate on their API however they feel comfortable.

I am leaning towards just saying "use WKTs and get inspired by some of the designs from Google", but I am of course will follow the your guidance here @Logofile as a mainatiner.

I have recently ran into an issue where I updated from proto3 to editions and this caused breaking changes in some generated code, despite not having any changes to the messages themselves, so even something like this could be a problem...

@Logofile
Copy link
Member

I'm so glad to have your collaboration on this. I primarily see how Googlers use protobufs, so having an OSS perspective is really helpful.

I prefer your second option for a couple of reasons:

  • We don't change implementation details about our WKTs often at all, and that's by design. They should be well-thought-out and fairly static... that's part of the appeal.
  • We notify protobuf users about upcoming breaking changes (see https://protobuf.dev/news for a collection of them). So if developers want to stay on top of changes, they have a mechanism to do so.

Related to your recent experience with moving from proto3 to editions, I'm curious what breakage you ran into. Did you create an issue for it? We're working on tooling that will automate the transition from proto2/proto3 to editions to hopefully eliminate (or at least greatly reduce) any implementation-specific sharp corners.

@software-artificer
Copy link
Author

Thank you 🙇‍♂ and I appreciate your support on this.

Let me clarify things a bit so we are definitely on the same page.

When I am saying WKTs I specifically mean the well-known types that are shipped with the protobuf package. I believe that we both are in agreement that they should stay there and be used, recommended and documented. These include things like google.protobuf.StringValue or google.protobuf.BoolValue, etc.

When I am talking about "Google's types" I specifically mean ones from googleapis/googleapis located in the google.type package — they aren't part of the protobuf.dev and aren't shipped with the compiler. If Google APIs were to change and evolve, I presume they won't be covered by the breaking changes in the protobuf.dev news/changelog, hence I was inclined to document them differently perhaps, as in: "feel free to get inspired by their design, but make your own". Does this make sense?

When it comes to the breaking changes that I have recently observed in one of our projects it was when we flipped from proto3 to editions and got some generated code in Go changed from using concrete values to using pointers.

@Logofile
Copy link
Member

Ah, thanks for that clarification! I didn't realize that we separate our Well-Known Types from other types that are provided by Google, but under a different package. So you're not asking about the WKT (which are available when you install protobuf), but rather asking that we document how to retrieve non-WKT Google types? If I'm still off, do please correct me. 😄

I will check with the engineering team to find out what our recommendations are and update this topic with that information.

@software-artificer
Copy link
Author

You are spot on! Keep me posted on what you will learn from the eng team and thank you.

@Logofile
Copy link
Member
Logofile commented Jun 3, 2025

I'm back from vacation! I'll try to work on this later this week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@software-artificer @mkruskal-google @Logofile
0