8000 Small fixes and more tests for jdk Converters by lrytz · Pull Request #7970 · scala/scala · GitHub
[go: up one dir, main page]

Skip to content

Small fixes and more tests for jdk Converters #7970

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Apr 17, 2019
Merged

Conversation

lrytz
Copy link
Member
@lrytz lrytz commented Apr 5, 2019

Include tests from java8-compat, and add some new ones.

Fixes scala/bug#11468

@scala-jenkins scala-jenkins added this to the 2.13.1 milestone Apr 5, 2019
@lrytz
Copy link
Member Author
lrytz commented Apr 5, 2019

Also has a few fixes

  • Adds baseStepperShape, needed because anyStreamShapePrototype actually can work on arbitrary Steppers, not just AnyStepper
  • Add asJavaSeqStream / asJavaParStream to Byte/Short/Char/Float arrays
  • WrappedString.stepper works correctly when requesting an AnySepper instead of an IntStepper...

Review by @Ichoran

@lrytz lrytz requested a review from Ichoran April 5, 2019 16:09
@Ichoran
Copy link
Contributor
Ichoran commented Apr 8, 2019

I'll try to get to this tomorrow--I probably won't have time today. Thanks for grabbing my fixes.

@SethTisue SethTisue added the library:collections PRs involving changes to the standard collection library label Apr 8, 2019
Copy link
Contributor
@Ichoran Ichoran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since it's mostly tests I didn't read through exceedingly carefully; everything looks good that I saw. The one thing I caught is that the newest collections don't seem to be part of the tests: CollisionProofHashMap, VectorMap, and TreeSeqMap. But collection-laws tests those three and covers many of the cases here, so I don't think it's critical.

(I'm a little wary of the mima exceptions because they might mask actual incompatibilities later if we need to change something, but I guess we're starting bincompat with RC1?)

lrytz and others added 5 commits April 12, 2019 10:43
There are a few open issues
  - ChampStepper doesn't return all elements (Rex is working on a fix)
  - TableStepper.trySplit has a bug, it was changed to always return
    `null` in a previous commit (Rex is also fixing this)
  - The `ORDERED` flag is often not correct, in particular when creating
    a Stepper of a Set, and there's no specific implementation. Then
    the implementation falls back to IterableOnce, and IteratorStepper
    has the `ORDERED` flag
All `trySplit` instances now pass in collection-laws.

The Champ split is not as even as it could be; better would be
to extract the sizes from the nodes and use that to wisely pick
the partition points.  However, it's not binary compatible.
@lrytz
Copy link
Member Author
lrytz commented Apr 12, 2019

CollisionProofHashMap, VectorMap, and TreeSeqMap

added those

wary of the mima exceptions

We'll reset MiMa to 0 on the 2.13.0 release, so we can still break bin compat for now.

@Ichoran
Copy link
Contributor
Ichoran commented Apr 12, 2019

Changes look good. I was uncertain at the time whether to use TableStepper for LinkedHashMap, but I think you're right that the correct decision is to not.

@lrytz
Copy link
Member Author
lrytz commented Apr 12, 2019

Thanks!

@lrytz
Copy link
Member Author
lrytz commented Apr 12, 2019

@adriaanm this is ready.

@lrytz
Copy link
Member Author
lrytz commented Apr 15, 2019

I added a few more commits. The last one is to be discussed. I changed the references to primitives to the boxed types in the explicit conversion methods. For example

-    def toJavaOptionalDouble(o: Option[Double]): OptionalDouble = o match {
+    def toJavaOptionalDouble(o: Option[jl.Double]): OptionalDouble = o match {

The reason is that Scala primitives in generic signatures erase to Object, so the toJavaOptionalDouble is really not useful for Java programmers - even though that's the primary audience. For Scala programmers, it's recommended to use the extension methods in OptionConverters.Ops._ instead.

The problem is that using the explicit conversion methods in Scala is now pretty annoying, which might be very confusing to poeple.

As an alternative, we could duplicate the XConverters, have a version for Java and one for Scala. I'm not sure what's the best way forward....

@smarter
Copy link
Member
smarter commented Apr 15, 2019

The reason is that Scala primitives in generic signatures erase to Object

Do you know why?

@lrytz
Copy link
Member Author
lrytz commented Apr 15, 2019

scala/bug#4214 / e42733e

@lrytz lrytz changed the title More tests for Java Converters Small fixes and more tests for jdk Convertes Apr 16, 2019
@lrytz
Copy link
Member Author
lrytz commented Apr 16, 2019

@adriaanm I removed the last commit (to use boxed types instead of scala primitives) here, will split it off to a separte PR. So this one is ready.

@lrytz
Copy link
Member Author
lrytz commented Apr 16, 2019

@viktorklang I copied the FutureConvertersTest over from java8-compat, and there are failures... Could you take a look? I'm not sure how to find out what's going on.

@viktorklang
Copy link
Contributor

@lrytz I think I fixed the test here: viktorklang@c141ee3

The problem is that the callback is both executed synchronously and completed synchronously, which means that the calling thread is waiting for the latch to trigger, so it cannot trigger the latch. I.e. the test is wrongly written.

I "fixed" the test by changing the order of the completion and the countdown, but an equal solution would be to switch to *Async versions of the methods (sadly there are no such options for all of the methods tested.

I also fixed a test which demands that obtrude throws UnsupportedOperationException.

Let me know if you want to discuss this further!

@viktorklang
Copy link
Contributor

One item of note that I discovered while doing this is that there does not seem to be a cohesive story about whether to call converters toX or asX

@SethTisue SethTisue changed the title Small fixes and more tests for jdk Convertes Small fixes and more tests for jdk Converters Apr 16, 2019
@Ichoran
Copy link
Contributor
Ichoran commented Apr 16, 2019

It is supposed to be the case that asX means that it's a view onto the underlying whatever, whereas toX dispenses with the original after the new thing is constructed.

Maybe this implementation detail is not sufficiently important to warrant a seemingly random selection of to vs as methods, or maybe the rule isn't consistently applied, but that's what it's supposed to be.

@lrytz
Copy link
Member Author
lrytz commented Apr 16, 2019

I followed that rule consistently in the scala.jdk package, hopefully I got it right everywhere.

@lrytz
Copy link
Member Author
lrytz commented Apr 16, 2019

@viktorklang will look tomorrow at your proposed changes, thanks. I wonder why the test passes on java8-compat (with 2.13.0-RC1, scala/scala-java8-compat#139).

@viktorklang
Copy link
Contributor

@lrytz Hmmm, I do not know why it succeeds on the other version. Let me know if you have any follow-up questions tomorrow.

@lrytz
Copy link
Member Author
lrytz commented Apr 17, 2019

Oh, I see that I skipped a lot of method overrides when porting FuturesConvertersImpl.CF over to the Scala repository. Adding them fixes the test. So this is totally my fault, sorry...

@viktorklang
Copy link
Contributor

@lrytz Np! I'm glad the test caught it though! :-)

A3E2

@lrytz lrytz merged commit b7f2511 into scala:2.13.x Apr 17, 2019

override def thenCompose[U](fn: JFunction[_ >: T, _ <: CompletionStage[U]]): CompletableFuture[U] = thenComposeAsync(fn)

override def whenComplete(fn: BiConsumer[_ >: T, _ >: Throwable]): CompletableFuture[T] = whenCompleteAsync(fn)
Copy link
Contributor
FF3F

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lrytz @SethTisue @viktorklang Why all these methods are delegated to the async ones?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a comment above:

    // Ensure that completions of this future cannot hold the Scala Future's completer hostage

It seems these overrides were always there scala/scala-java8-compat@b3fca14

Haven't thought about it in detail, but maybe there are separate thread pools for Scala and Java futures, and it's to avoid running on the wrong one?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazingly, I could scroll back three weeks in Discord and find the link I posted.

https://github.com/scala/scala-java8-compat/blob/main/src/main/scala/scala/compat/java8/FutureConverters.scala#L57

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
library:collections PRs involving changes to the standard collection library
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Infinite loop in TableStepperBase on trySplit
9 participants
0