E540 [GarbageCollector] only store typeMeta and objectMeta in the gc store by caesarxuchao · Pull Request #28480 · kubernetes/kubernetes · GitHub
[go: up one dir, main page]

Skip to content

Conversation

caesarxuchao
Copy link
Contributor
@caesarxuchao caesarxuchao commented Jul 5, 2016

GC only needs to know the apiVersion, kind, and objectMeta of an object. This PR makes the stores of GC only save these fields.
cc @kubernetes/sig-api-machinery


This change is Reviewable

@caesarxuchao caesarxuchao added the sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. label Jul 5, 2016
@caesarxuchao caesarxuchao self-assigned this Jul 5, 2016
@k8s-github-robot k8s-github-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. release-note-label-needed labels Jul 5, 2016
@caesarxuchao caesarxuchao force-pushed the gc-save-only-meta branch 3 times, most recently from aae495e to a4f19d2 Compare July 6, 2016 21:23
@caesarxuchao caesarxuchao changed the title [WIP][GarbageCollector] only store typeMeta and objectMeta in the gc store [GarbageCollector] only store typeMeta and objectMeta in the gc store Jul 6, 2016
@caesarxuchao caesarxuchao assigned krousey and lavalamp and unassigned caesarxuchao Jul 6, 2016
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the behavior if no codec is provided?
It seems like there is a default Codec. Would you like to add a comment for it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. Will do.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought there was a codec in the restclient config-- if not, consider adding it there instead of adding a parameter here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dynamic.NewClient ignored it and overrode it with the dynamic codec.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually made dynamic.NewClient to respect the config. I'll see if I can squeeze the codec into the config.

@k8s-github-robot k8s-github-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 7, 2016
@k8s-github-robot k8s-github-robot added kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 25, 2016
@smarterclayton
Copy link
Contributor

My point was that unrecognized types is the current edge case, and for
everything else we fall back to unstructured. Having an optimizing codec
for unstructured makes great sense (since by definition we don't know
anything). I'm not 100% opposed to having that for everything, but since
we already have that machinery for proto I still think it's going to be
less of a stretch in the short term to use our existing machinery, add the
transform, and optimize unrecognized later.

The thing is the payload of object meta can change, and will change at some
point in the future. I don't want to build in bypasses that bite us later
(oh, i just assumed object meta would never change).

That's not a "I think this is completely wrong", but it is a "this is
bypassing a fundamental abstraction we have in place for a reason". I want
to be extra judicious about doing that.

On Mon, Jul 25, 2016 at 5:59 PM, k8s-merge-robot notifications@github.com
wrote:

Labelling this PR as size/XL


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#28480 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABG_p4o3ZS2d5_w1iN9BAEwJX20EpgV1ks5qZTHUgaJpZM4JEvyT
.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the danger is that MetaOnly won't work if we use incompatible ObjectMeta across different groups in the future. @smarterclayton @lavalamp is this going to happen in the near future?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have a mechanism to convert TPR or unrecognized types anyway.
Which means that swagger and/or api discovery has to communicate that info
if we do have to solve that. By putting this in we probably aren't making
that problem worse... except for our own types.

On Mon, Jul 25, 2016 at 11:20 PM, Chao Xu notifications@github.com wrote:

In pkg/controller/garbagecollector/metaonly/metaonly.go
#28480 (comment)
:

  • "io"
  • "reflect"
  • "strings"
  • "k8s.io/kubernetes/pkg/api/unversioned"
  • "k8s.io/kubernetes/pkg/api/v1"
  • "k8s.io/kubernetes/pkg/runtime"
  • "k8s.io/kubernetes/pkg/util/json"
    +)

+// MetaOnly allows decoding only the apiVersion, kind, and metadata fields of
+// JSON data.
+// TODO: enable meta-only decoding for protobuf.
+type MetaOnly struct {

  • unversioned.TypeMeta json:",inline"
  • v1.ObjectMeta json:"metadata,omitempty"

I think the danger is that MetaOnly won't work if we use incompatible
ObjectMeta across different groups in the future. @smarterclayton
https://github.com/smarterclayton @lavalamp
https://github.com/lavalamp is this going to happen in the near future?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes/kubernetes/pull/28480/files/cca957905d7ed2af91d0444ecb3dd717ac8c9825#r72182360,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABG_p3ws2Zajl7FyzwA3YwRNuGEp5X4iks5qZXzpgaJpZM4JEvyT
.

@k8s-github-robot k8s-github-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 27, 2016
if simpleStorage.deleted != ID {
t.Errorf("Unexpected delete: %s, expected %s", simpleStorage.deleted, ID)
}
simpleStorage.deleteOptions.GetObjectKind().SetGroupVersionKind(unversioned.GroupVersionKind{})
Copy link
Contributor Author
@caesarxuchao caesarxuchao Aug 1, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These changes are needed because I change the UseOrCreateObject. The decoder uses the passed in into object instead of creating a new one. In turn the versioning decoder skips the conversion so the gvk is kept. I think this behavior is an innocuous bug.

@lavalamp
Copy link
Contributor
lavalamp commented Aug 4, 2016

I think this is close, @caesarxuchao if you can address the comments here I think we can get this in soon?

@caesarxuchao
Copy link
Contributor Author

@lavalamp Comments addressed. PTAL.

@caesarxuchao
Copy link
Contributor Author

@k8s-bot e2e test this, issue #28656


// PrintAsMetadataOnlyObject is a helper function that converts an interface{} to
// *MetadataOnlyObject and then convert it to a human-readable string.
func PrintAsMetadataOnlyObject(obj interface{}) string {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this function needed at all? Looking at the call sites, I'm pretty sure you can just get rid of it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. I'll remove it.

@lavalamp
Copy link
Contributor
lavalamp commented Aug 9, 2016

just the one nit. Please fix & squash, the apply the label. Thanks!

@caesarxuchao caesarxuchao added release-note-none Denotes a PR that doesn't merit a release note. lgtm "Looks good to me", indicates that a PR is ready to be merged. and removed release-note-label-needed labels Aug 9, 2016
@caesarxuchao
Copy link
Contributor Author

Thanks. Squashed and self applied the label per comment.

@k8s-bot
Copy link
k8s-bot commented Aug 9, 2016

GCE e2e build/test passed for commit 4d23506.

@k8s-github-robot
Copy link

@k8s-bot test this [submit-queue is verifying that this PR is safe to merge]

@k8s-bot
Copy link
k8s-bot commented Aug 9, 2016

GCE e2e build/test passed for commit 4d23506.

@k8s-github-robot
Copy link

Automatic merge from submit-queue

@k8s-github-robot k8s-github-robot merged commit a9b6f69 into kubernetes:master Aug 9, 2016
k8s-github-robot pushed a commit that referenced this pull request May 21, 2018
Automatic merge from submit-queue (batch tested with PRs 64069, 64087). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

garbage collector `NewMetadataCodecFactory` cleanup

`NewMetadataCodecFactory` was introduce by #28480, but not used now.

/assign @deads2k 

**Release note**:

```release-note
NONE
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note-none Denotes a PR that doesn't merit a release note. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants
0