8000 Revert #5664 because the binary incompatible change leaks via erasure by adriaanm · Pull Request #5821 · scala/scala · GitHub
[go: up one dir, main page]

Skip to content

Revert #5664 because the binary incompatible change leaks via erasure #5821

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from adriaanm:revert-5664-binco
Apr 4, 2017

Conversation

adriaanm
Copy link
Contributor
@adriaanm adriaanm commented Apr 3, 2017

This reverts #5664 because the binary incompatible change leaks via erasure. We'll need to cut 2.11.10 and declare 2.11.9 a dud.

Originally discovered and reported in detail by @xuwei-k and @sjrd at scala-js/scala-js#2847 (comment) (see also the 2.11.9 release issue)

@adriaanm adriaanm added this to the 2.11.10 milestone Apr 3, 2017
@adriaanm adriaanm requested a review from szeiger April 3, 2017 19:42
@adriaanm
Copy link
Contributor Author
adriaanm commented Apr 3, 2017

@szeiger, @lrytz could you summarize your findings here for the release notes.

@lrytz
Copy link
Member
lrytz commented Apr 3, 2017

I'll do that, will write something.

@adriaanm adriaanm added the release-notes worth highlighting in next release notes label Apr 3, 2017
@lrytz
Copy link
Member
lrytz commented Apr 3, 2017

PR #5664 added a number of optimzied implementations for certain collection operations on a specific type. For example, List.filter received a new implementation overriding the one inherited from TraversableLike. In a simplified view, 2.11.8 had the following structure:

package test.collection.immutable
trait TraversableLike[+A, +Self] {
  def filter(p: A => Boolean): Self = ???
}
class List[+A] extends TraversableLike[A, List[A]]

Due to type erasure, method filter in classfile TraversableLike has return type Object.

We cannot add an overriding implementation of filter in class List in a binary compatible way, as noted in a comment on PR #5653:

class List[+A] extends TraversableLike[A, List[A]] {
  override def filter(p: A => Boolean): List[A] = ???
}

The new method would have return type List in bytecode (to perform the override, the Scala compiler would generat a "bridge" method). An invocation in client code of this new, more specific method would not work with the 2.11.8 library on the classpath.

To prevent this issue, an intermediate class was added. This class was supposed to be completely internal and never appear in user code.

private[immutable] trait FilteredTraversableInternal[+A, +Repr] extends TraversableLike[A, Repr] {
  override def filter(p: A => Boolean): Repr = ???
}
class List[+A] extends TraversableLike[A, List[A]] with FilteredTraversableInternal[A, List[A]]

The new trait is private to package immutable, so user code cannot introduce references to it. Unfortunately, the erasure phase of the Scala compiler does introduce references to this new class into user code. This means that user code compiled with 2.11.9 cannot run with a 2.11.8 library, as the class is missing.

Assume we have the following user code:

class C {
  def m(o: Option[List[String]]): Unit = o.get.filter(_ => true)
}

Compiling this with the original collections code and de-compiling C.m with cfr gives

public void m(Option<List<String>> o) {
  ((TraversableLike)o.get()).filter(...);
}

Due to erasure, the method Option.get has return type Object in bytecode. The Scala compiler introduces a cast to TraversableLike in order to make the invocation of method filter valid - the Object class does not have such a method.

If we re-compile the same example with the FilteredTraversableInternal intermediate class, the cast inserted by the Scala compiler now uses the FilteredTraversableInternal, because that is the class in which filter is defined. This way, a reference to the new trait finds its way into user code.

@lrytz
Copy link
Member
lrytz commented Apr 3, 2017

we also need a short version of that.. :)

@adriaanm
Copy link
Contributor Author
adriaanm commented Apr 4, 2017

Thanks @lrytz! @szeiger, let me know when you feel this is ready to go.

@szeiger
Copy link
Contributor
szeiger commented Apr 4, 2017

FTR, I don't think there is any part of this that can be salvaged. Everything relies on injecting a private superclass and is therefore not binary compatible.

@szeiger szeiger merged commit 94dd1dc into scala:2.11.x Apr 4, 2017
xuwei-k added a commit to xuwei-k/json4s that referenced this pull request Apr 5, 2017
xuwei-k added a commit to squeryl/squeryl that referenced this pull request Apr 5, 2017
xuwei-k added a commit to scalaz/scalaz that referenced this pull request Apr 5, 2017
xuwei-k referenced this pull request in woost/wust2 Apr 5, 2017
xuwei-k referenced this pull request in mcanlas/spawning-pool Apr 5, 2017
xuwei-k referenced this pull request in SystemFw/base.g8 Apr 5, 2017
xuwei-k added a commit to xuwei-k/monix that referenced this pull request Apr 5, 2017
alexandru pushed a commit to monix/monix that referenced this pull request Apr 5, 2017
xuwei-k referenced this pull request in 3tty0n/play-json-xml Apr 5, 2017
xuwei-k added a commit to unfiltered/unfiltered-scalate.g8 that referenced this pull request Apr 5, 2017
xuwei-k added a commit to unfiltered/unfiltered-war.g8 that referenced this pull request Apr 5, 2017
xuwei-k added a commit to xuwei-k/iarray that referenced this pull request Apr 5, 2017
xuwei-k added a commit to xuwei-k/nobox that referenced this pull request Apr 5, 2017
xuwei-k added a commit to nscala-time/nscala-time that referenced this pull request Apr 5, 2017
@SethTisue SethTisue modified the milestones: 2.11.11, 2.11.10 May 1, 2017
@SethTisue SethTisue removed the release-notes worth highlighting in next release notes label May 1, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
4B31
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants
0