8000 Merge 2.12.x to 2.13.x [ci: last-only] by SethTisue · Pull Request #5862 · scala/scala · GitHub
[go: up one dir, main page]

Skip to content

Merge 2.12.x to 2.13.x [ci: last-only] #5862

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 37 commits into from
Apr 24, 2017

Conversation

SethTisue
Copy link
Member
@SethTisue SethTisue commented Apr 24, 2017

four merge conflicts, all trivially resolvable

motive: get the sbt version bump and the MathJax spec fix
onto 2.13.x. performance improvements, too. (note that
the partest version bump was already here.)

% git checkout -b merge-2.12-to-2.13-apr-24
% git hist -n 1 | cat
*   483be9a038 - (HEAD -> merge-2.12-to-2.13-apr-24, origin/2.13.x, 2.13.x) Merge pull request #5860 from lrytz/scalacheckUpdate (4 days ago) <Lukas Rytz>
% export mb=$(git merge-base origin/2.12.x 2.13.x)
% git log --graph --oneline --decorate $mb..origin/2.12.x | cat
*   709a75223f (origin/HEAD, origin/2.12.x) Merge pull request #5861 from lrytz/t5717-msg
|\
| * 3733536023 Update error message for t5717
|/
*   7cc1f86473 Merge pull request #5857 from retronym/ticket/10232
|\
| * 9f522c6b3c Fix lambda deserialization in classes with 252+ lambdas
| * d9343a7f10 Remove dead code catch in LambdaDeserializer
* |   81459e475c Merge pull request #5812 from retronym/faster/specialization
|\ \
| * | e416a259a4 Optimize specializedTypeVars
| * | fe4788b97b Only do specialation definalization once per run
| * | e65b714b48 Optimize SpecializeTypes#satisfiable
| * | 8ae0fdab1e Avoid needless work in the specialization info transform in the backend
| * | 5cd3442419 Better diagnostic for failing jvm/future-spec
|  /
* |   3009880b61 Merge pull request #5820 from retronym/faster/backend
|\ \
| * | 070ab67f39 Review feedback: resurrect assertion, use LabelDefFinder.apply
| * | bebb1886de Optimize method descriptor creation
| * | 1ae858cecf Remove expensive assertion in the backend
| * | 53dd4e430e Cache ClassSymbol.javaBinaryNameString
| * | 455729e6f5 Use AnyRefMap in labelReferences
| * | 7a6dc1abbf Avoid excessive file stats during classfile writing
| * | b3975a5a16 Optimize label defs finder in the backend
|  /
* |   6e7fd660d6 Merge pull request #5824 from retronym/faster/sourcefileattr
|\ \
| |/
|/|
| * 27be3a2ef2 Avoid excessive IO in classfile parser
*   db80860e11 Merge pull request #5856 from SethTisue/partest-1.1.1
|\
| * 7236da05f6 upgrade to partest 1.1.1
* |   a3ec4a08f4 Merge pull request #5855 from SethTisue/post-2.12.2-version-bump
|\ \
| * | 0c8bdf632f bump version to 2.12.3
| |/
* |   4efab51a1b Merge pull request #5775 from SethTisue/sbt-0.14
|\ \
| |/
|/|
| * 8b4a033f2c upgrade to sbt 0.13.15
* d61301de77 Merge pull request #5851 from adriaanm/2.12-catchup-2.11
*   771fe005b4 No-op merge 2.11.x into 2.12.x
|\
| *   8a413ba7cc (tag: v2.11.11) Merge pull request #5846 from adriaanm/t10261-2.11
| |\
| | * 77917e94c7 Actually retract clashing synthetic apply/unapply [backport]
| |/
* |   17e160d845 Merge 2.11.x into 2.12.x
|\ \
| |/
| * 5167b691bb MathJax CDN change and version bump
* | f668845a40 No-op merge 2.11.x into 2.12.x
|/
* 7a5df5702c Update README.md
* a3987d86ef Bump on 2.11.10 release
% git merge origin/2.12.x
Auto-merging versions.properties
CONFLICT (content): Merge conflict in versions.properties
Auto-merging src/reflect/scala/reflect/runtime/JavaUniverseForce.scala
CONFLICT (content): Merge conflict in src/reflect/scala/reflect/runtime/JavaUniverseForce.scala
Auto-merging src/reflect/scala/reflect/internal/Definitions.scala
CONFLICT (content): Merge conflict in src/reflect/scala/reflect/internal/Definitions.scala
Auto-merging src/compiler/scala/tools/nsc/transform/SpecializeTypes.scala
Auto-merging src/compiler/scala/tools/nsc/backend/jvm/opt/BytecodeUtils.scala
Auto-merging src/compiler/scala/tools/nsc/backend/jvm/BTypes.scala
Auto-merging build.sbt
CONFLICT (content): Merge conflict in build.sbt
Recorded preimage for 'build.sbt'
Recorded preimage for 'src/reflect/scala/reflect/internal/Definitions.scala'
Recorded preimage for 'src/reflect/scala/reflect/runtime/JavaUniverseForce.scala'
Recorded preimage for 'versions.properties'
Automatic merge failed; fix conflicts and then commit the result.
% emacs build.sbt
% emacs src/reflect/scala/reflect/internal/Definitions.scala
% emacs src/reflect/scala/reflect/runtime/JavaUniverseForce.scala
% emacs versions.properties
% git add -u
% git commit

8000
retronym and others added 30 commits March 30, 2017 10:23
Any types that needed to be specialized to support callsites in the current
run would have already been info transformed during the specalization
tree transform of those call sites.

The backend requires further type information, e.g, to know about
inner/enclosing class relationships. This involves calls to `sym.info`
for classes on the classpath that haven't yet been info transformed.

During that process, all base classes of such types are also info
transformed. The specialization info transformer for classes then
looks at the members of the classes to add specialialized variants.

This is undesirable on grounds of performance and the risk of
encountering stub symbols (references to types absent from the
current compilation classpath) which can manifest as compiler crashes.
We know that `subst(tp1) <:< subst(tp2)` a priori (and cheaply!)
if `tp2` is `Any`, which is commonly the case.
Reworks e4b5c00 to perform the flag mutation once per run,
at the conclusion of the specialization tree transform, rather
than once per compilation unit. The old approach was O(NxM), where
N is the number of compilation units and M is the number of
specialized overloads.
Most commonly, this method will return an empty set. This commit focuses on
making that happen with a minimum of garbage, indirection, and info forcing.

  - Use mutable buffers to collect results, rather than appending sets
  - Avoid forcing the specialization info transform on all referenced
    types just to see if they have specialzied type parmeteters, we can
    phase travel back to typer to lookup this.
  - Record the entry for the RHS of the DefDef is a dedicated
    field to avoid immediately looking it up in a hash map after
    traversal
  - Use an AnyRefMap to avoid BoxesRuntime hashCode/equals
  - Use getOrElse rather than withDefaultValue to profit from a fast
    path in AnyRefMap.
The existing implementation pessimistically checks that all
parent directories of the about-to-be-written class file are
indeed directories.

This commit bypasses this logic for the common case of writing
to a regular directory on disk, and optimistically assumes that
the parent directory exists. If an exception is thrown during
writing, it attempts to create the parent directory.

This still avoids a compiler crash if a parent directory is
actually a file, which is tested by the existing test,
`run/t5717.scala`.
This is a hot method in the backend, and we save some cycles by
avoiding BoxesRuntime.
The backend uses this string as a key to the map of BTypes,
and as such needs to call `javaBinaryNameString` for each method
or field reference in the code. Even though we have previously
optimized the creation of this string by bypassing the Name
abstraction and by correctly sizing string builders, we can
still speed things up by caching the resulting String on its
ClassSymbol.
These assertions don't seem to pay their way anymore, the surrounding
implementation of the backend has matured, and they involve collections
operations that are too costly to be called in such a hot path in the backend.
Thread a single StringBuilder through the component's stringification
methods, rather than using string concat (ie, separates StringBuilders)
at each level.
The class file parser, used to read the java-defined classes from
the classpath, includes logic to search the output path for a .java
source file that corresponds to the value of the SourceFile attribute.

I haven't been able to figure out the rationale for that fix, but
it involves a non-neglible overhead, so this commits disables it
in the batch compiler.
Fix table
we do not speak of sbt 0.13.14. move along, move along.
Also make this whole retraction of apply/unapply in case of a
clashing user-defined member conditional on `-Xsource:2.12`.

It turns out, as explained by lrytz, that the retraction mechanism
was fragile because it relied on the order in which completers are run.
We now cover both the case that:

  - the completer was run, the `IS_ERROR` flag was set, and the
  symbol was unlinked from its scope before `addSynthetics`
  in `typedStat` iterates over the scope (since the symbol is
  already unlinked, the tree is not added, irrespective of its flags).
  For this case, we also remove the symbol from the synthetics in
  its unit (for cleanliness).

  - the completer is triggered during the iteration in `addSynthetics`,
  which needs the check for the `IS_ERROR` flag during the iteration.

Before, the completer just unlinked the symbol and set the IS_ERROR flag,
and I assumed the typer dropped a synthetic tree with a symbol with
that flag, because the tree was not shown in -Xprint output.
In reality, the completer just always happened to run before the
addSynthetics loop and unlinked the symbol from its scope in the
test cases I came up with (including the 2.11 community build).

Thankfully, the 2.12 community build caught my mistake, and lrytz provided
a good analysis and review.

Fix scala/bug#10261
Actually retract clashing synthetic apply/unapply
MathJax CDN update
Merge 2.11.x into 2.12.x [ci: last-only]
and use 2.12.2 as starr

also update a copyright date we missed before
I should have removed the try/catch when I removed the code path that
could throw that exception in 131402f.
retronym and others added 7 commits April 21, 2017 22:44
Improve performance of the backend
Create a lambda deserializer per group of target methods,
and call these sequentially trapping the particular pattern
of exception that is thrown when a target method is absent
from the map.

Fixes scala/bug#10232

```
  // access flags 0x100A
  private static synthetic $deserializeLambda$(Ljava/lang/invoke/SerializedLambda;)Ljava/lang/Object;
    TRYCATCHBLOCK L0 L1 L1 java/lang/IllegalArgumentException
   L0
    ALOAD 0
    INVOKEDYNAMIC lambdaDeserialize(Ljava/lang/invoke/SerializedLambda;)Ljava/lang/Object; [
      // handle kind 0x6 : INVOKESTATIC
      scala/runtime/LambdaDeserialize.bootstrap(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;[Ljava/lang/invoke/MethodHandle;)Ljava/lang/invoke/CallSite;
      // arguments:
      // handle kind 0x6 : INVOKESTATIC
      Test$.$anonfun$main$1$adapted(Lscala/Function1;)Ljava/lang/Object;,
      // handle kind 0x6 : INVOKESTATIC
      Test$.$anonfun$lambdas$1(Ljava/lang/Object;)Ljava/lang/String;,

      ...

      Test$.$anonfun$lambdas$249(Ljava/lang/Object;)Ljava/lang/String;,
      // handle kind 0x6 : INVOKESTATIC
      Test$.$anonfun$lambdas$250(Ljava/lang/Object;)Ljava/lang/String;
    ]
    ARETURN
   L1
    ALOAD 0
    INVOKEDYNAMIC lambdaDeserialize(Ljava/lang/invoke/SerializedLambda;)Ljava/lang/Object; [
      // handle kind 0x6 : INVOKESTATIC
      scala/runtime/LambdaDeserialize.bootstrap(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;[Ljava/lang/invoke/MethodHandle;)Ljava/lang/invoke/CallSite;
      // arguments:
      // handle kind 0x6 : INVOKESTATIC
      Test$.$anonfun$lambdas$251(Ljava/lang/Object;)Ljava/lang/String;,
      // handle kind 0x6 : INVOKESTATIC
      Test$.$anonfun$lambdas$252(Ljava/lang/Object;)Ljava/lang/String;,

      ...

      // handle kind 0x6 : INVOKESTATIC
      Test$.$anonfun$lambdas$256(Ljava/lang/Object;)Ljava/lang/String;
    ]
    ARETURN
    MAXSTACK = 2
    MAXLOCALS = 1

```
Fix lambda deserialization in classes with 252+ lambdas
We started checking the error message of this test only recently,
in scala#5835. In 7a6dc1a, the backend was
8000
 changed to use java.nio, and
the error message changed. Its PR was not rebased on the tip of
2.12.x, so the change of error message went unnoticed.

Fixes scala/scala-dev#377
Update error message for t5717
@scala-jenkins scala-jenkins added this to the 2.12.3 milestone Apr 24, 2017
@SethTisue SethTisue changed the title Merge 2.12.x to 2.13.x Merge 2.12.x to 2.13.x [ci: last-only] Apr 24, 2017
@SethTisue SethTisue requested a review from lrytz April 24, 2017 17:00
@SethTisue SethTisue changed the base branch from 2.12.x to 2.13.x April 24, 2017 17:00
@SethTisue SethTisue modified the milestones: 2.13.0-M2, 2.12.3 Apr 24, 2017
@SethTisue
Copy link
Member Author
SethTisue commented Apr 24, 2017

/nothingtoseehere

Scabot got confused (because I initially targeted the wrong branch), but there is a green validate-main run for 775bb11 at https://scala-ci.typesafe.com/job/scala-2.13.x-validate-main/196/console

@lrytz ready for review/merge

lrytz
lrytz approved t A372 hese changes Apr 24, 2017
@lrytz lrytz merged commit 66bd43b into scala:2.13.x Apr 24, 2017
@SethTisue SethTisue deleted the merge-2.12-to-2.13-apr-24 branch April 24, 2017 19:39
@lrytz
Copy link
Member
lrytz commented Apr 24, 2017

I double checked to use "Create Merge Commit" :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants
0