8000 Datastore: expose serializedSize for Entity. by suraj-qlogic · Pull Request #6842 · googleapis/google-cloud-java · GitHub 8000
[go: up one dir, main page]

Skip to content

Conversation

@suraj-qlogic
Copy link
Contributor

Fixes #6389

@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Nov 20, 2019
@elharo elharo added the kokoro:run Add this label to force Kokoro to re-run the tests. label Nov 20, 2019
@yoshi-kokoro yoshi-kokoro removed the kokoro:run Add this label to force Kokoro to re-run the tests. label Nov 20, 2019
return properties;
}

/** Returns the entity size. */
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Returns the number of bytes in the serialized protobuf."

Is this definitely a fixed and reproducible value for a given protobuf?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes,Once build it would be fixed number.

}

@Test
public void testEntitySize() throws Exception {
Copy link
Contributor

Choose a reason for hiding this comment

8000

The reason will be displayed to describe this comment to others. Learn more.

Does this really throw exception?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No,This method does not throw exception.

BaseEntity(Builder<K, ?> builder) {
this.key = builder.key;
this.properties = ImmutableSortedMap.copyOf(builder.properties);
this.serializedSize = this.toPb().getSerializedSize();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the cost of calculating this for a large entity? Most of the time we don't need this value. Perhaps this could be done in demand and not stored or only lazily stored in the field?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cost could be varied it depends on user entered properties. Does this look fine?

/** Returns the number of bytes in the serialized protobuf. */
  public int getSerializedSize() {
    if (serializedSize == 0) {
      serializedSize = this.toPb().getSerializedSize();
    }
    return serializedSize;
  }

Here we are calculating entity size when demand.

@codecov
Copy link
codecov bot commented Nov 20, 2019

Codecov Report

Merging #6842 into master will decrease coverage by 0.44%.
The diff coverage is 100%.

Impacted file tree graph

@@              Coverage Diff              @@
##             master    #6842       +/-   ##
=============================================
- Coverage     45.72%   45.28%    -0.45%     
+ Complexity    26528     6251    -20277     
=============================================
  Files          2615      453     -2162     
  Lines        282766    56282   -226484     
  Branches      32745     7229    -25516     
=============================================
- Hits         129299    25486   -103813     
+ Misses       143350    29140   -114210     
+ Partials      10117     1656     -8461
Impacted Files Coverage Δ Complexity Δ
...c/main/java/com/google/cloud/datastore/Entity.java 76.92% <100%> (+1.92%) 7 <1> (+1) ⬆️
.../cloud/datastore/testing/LocalDatastoreHelper.java 80.59% <0%> (-4.48%) 17% <0%> (ø)
...main/java/com/google/cloud/storage/BucketInfo.java 81.04% <0%> (-1.96%) 84% <0%> (ø)
...gle/cloud/bigtable/data/v2/BigtableDataClient.java 89.58% <0%> (-1.91%) 32% <0%> (ø)
...eas/src/main/java/io/grafeas/v1/GrafeasClient.java 61.63% <0%> (-1.47%) 59% <0%> (+14%)
...anner/jdbc/ClientSideStatementValueConverters.java 92.22% <0%> (-1.04%) 0% <0%> (ø)
...ogle/cloud/spanner/jdbc/ReadOnlyStalenessUtil.java 70.27% <0%> (-0.97%) 25% <0%> (ø)
...le/cloud/storage/contrib/nio/CloudStoragePath.java 75.67% <0%> (-0.69%) 51% <0%> (ø)
...le/cloud/spanner/jdbc/JdbcSqlExceptionFactory.java 45.71% <0%> (-0.67%) 7% <0%> (ø)
...ery/datatransfer/v1/DataTransferServiceClient.java 65.35% <0%> (-0.51%) 55% <0%> (+13%)
... and 2320 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5a2c0ae...2693e5f. Read the comment docs.

@pmakani pmakani requested a review from BenWhitehead November 21, 2019 05:14
Copy link
Contributor
@BenWhitehead BenWhitehead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've spent some time thinking about this and I feel like this is more of a utility method than a property of the entity. Entities are immutable, but in order to not incur a penalty of adding making size available serializedSize would need to be non-final which opens up other issues.

Instead of this being a field with a getter, can you add a new static method to com.google.cloud.datastore.Entity:

  public static int calculateSerializedSize(BaseEntity<? extends IncompleteKey> e) {
    return e.toPb().getSerializedSize();
  }

This way we don't impose a penalty to those users that don't need this operation.

@pmakani pmakani added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Nov 22, 2019
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Nov 22, 2019
return new Builder().fill(entityPb).build();
}

public static int calculateSerializedSize(BaseEntity<? extends IncompleteKey> e) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a javadoc comment to explain the usage of this method (and to help people find it when they google for this type of thing).

@suraj-qlogic suraj-qlogic requested a review from elharo November 25, 2019 06:50
/**
* Returns the serialized protobuf size of provided entity object.
*
* @param entity entity object to calculate serialize size.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"object whose size is measured"

checkNotNull(e);
return e.toPb().getSerializedSize();
/**
* Returns the serialized protobuf size of provided entity object.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Returns the size in bytes of the protobuf form of the provided entity."

@BenWhitehead BenWhitehead added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Nov 25, 2019
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Nov 25, 2019
@BenWhitehead BenWhitehead merged commit 88c48ef into googleapis:master Nov 25, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla: yes This human has signed the Contributor License Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

datastore: determine entity size using api

6 participants

0