8000 Prepare for 2.13 library modularization [ci: last-only] by adriaanm · Pull Request #5677 · scala/scala · GitHub
[go: up one dir, main page]

Skip to content

Prepare for 2.13 library modularization [ci: last-only] #5677

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 11 commits into from adriaanm:modularize-v2-prep

Conversation

adriaanm
Copy link
Contributor
@adriaanm adriaanm commented Feb 7, 2017

In 2.13, the following packages will move out of scala-library
(to their own module):

  • scala.concurrent
  • scala.util (Try and Either move to the scala package)
  • scala.io (see also better-files)
  • scala.ref
  • scala.sys (not recommended for use)
  • scala.compat (entirely deprecated)
  • scala.text (entirely deprecated)

This commit cuts some ties to prepare for this move.

To compensate for the associated deprecations in your own code:

  • compat.Platform.EOL --> add import java.lang.System.{lineSeparator => EOL}
  • compat.Platform.currentTime --> add import java.lang.System.{currentTimeMillis => currentTime}
  • compat.Platform.arraycopy --> add import java.lang.System.arraycopy
  • sys.error --> throw new RuntimeException (or subclass)
  • sys.exit --> System.exit
  • sys.props --> System.getProperties().asScala (with import scala.collection.JavaConverters._)
  • sys.env --> System.getenv().asScala
  • sys.runtime --> Runtime.getRuntime
  • sys.addShutdownHook --> Runtime.getRuntime.addShutdownHook(new Thread(() => ...))
  • Console.BOLD --> AnsiColor.BOLD (with import scala.io.AnsiColor)
  • Console.readLine --> Console.in.readLine()
  • concurrent.Lock --> java.util.concurrent.locks.ReentrantLock (lock/unlock)

TODO

  • verify last commit preserves sys.process semantics.
  • deprecations in test
  • osgi
  • bincompat method this(java.lang.String)Unit in class scala.sys.ShutdownHookThread
  • run/lazy-concurrent.scala [output differs]
  • run/t2318.scala java.security.AccessControlException: access denied ("java.util.PropertyPermission" "scala.control.noTraceSuppression" "read")

@scala-jenkins scala-jenkins added this to the 2.12.2 milestone Feb 7, 2017
@adriaanm adriaanm changed the title WIP: Modularize v2 prep [ci: last-only] WIP: Prepare for 2.13 library modularization [ci: last-only] Feb 7, 2017
@adriaanm adriaanm added the WIP label Feb 7, 2017
@adriaanm adriaanm changed the title WIP: Prepare for 2.13 library modularization [ci: last-only] Prepare for 2.13 library modularization [ci: last-only] Feb 7, 2017
@adriaanm adriaanm force-pushed the modularize-v2-prep branch 2 times, most recently from 9f0a931 to b58b8fd Compare February 7, 2017 01:28
@adriaanm
Copy link
Contributor Author
adriaanm commented Feb 7, 2017

Deprecations

  • run/constrained-types.scala
  • run/noInlineUnknownIndy
  • run/sd275-java
  • run/sd275.scala
  • run/shutdownhooks.scala
  • run/t2464
  • run/t4671.scala
  • run/t5463.scala
  • run/t5717.scala
  • run/t5940.scala
  • run/t6240-universe-code-gen.scala
  • run/t6440.scala
  • run/t6440b.scala
  • run/t6502.scala
  • run/t7439
  • run/t7455
  • run/t7582-private-within
  • run/t7741a
  • run/t7741b
  • run/t8502.scala
  • run/t8502b.scala
  • run/t8549.scala
  • run/t8907.scala
  • run/t9268
  • run/typetags_without_scala_reflect_manifest_lookup.scala
  • run/typetags_without_scala_reflect_typetag_lookup.scala
  • run/typetags_without_scala_reflect_typetag_manifest_interop.scala
  • run/various-flat-classpath-types.scala
  • jvm/innerClassEnclMethodJavaReflection.scala
  • jvm/interpreter.scala

@dwijnand
Copy link
Member
dwijnand commented Feb 7, 2017

For clarity, "to their own modules" means: moved out of the repo, switch groupId to org.scala-lang.modules and new versions, right? Like XML and parser-combinators? Or is it more conservative than that?

@adriaanm
Copy link
Contributor Author
adriaanm commented Feb 7, 2017 via email

@AlecZorab
Copy link

Do you expect the same treatment to be applied to the collections library, either for 2.13 or for some hypothetical later release?

@adriaanm
Copy link
Contributor Author
adriaanm commented Feb 8, 2017 via email

@AlecZorab
Copy link
AlecZorab commented Feb 8, 2017 via email

@adriaanm
Copy link
Contributor Author
adriaanm commented Feb 8, 2017

I agree that the coupling should be reduced, as I tried to acknowledge in my message.

However, the more important point is that, in my opinion, it is crucial for Scala adoption that there be only one Scala collections API. Library interop crucially relies on agreeing on many of the core collection types, even if the compiler and spec could be reworked not to rely on our specific implementations (using a name-based approach as with for comprehensions).

I'm happy to see extensions built on top of the collections, but wholesale replacement is not feasible for a widely used library (again, in my opinion).

@SethTisue
Copy link
Member

can we discuss this elsewhere, e.g. on http://contributors.scala-lang.org? it isn't directly relevant to this pull request, which is more limited in scope.

@AlecZorab
Copy link

I'd suggest that exactly that did happen for actors, but I'll certainly concede that multiple collection libraries might result in some level of inconvenience for people needing to go between libraries depending on different libs.

As Seth suggests, I'll reserve further comment for another venue

8000

@adriaanm
Copy link
Contributor Author
adriaanm commented Feb 9, 2017

(Testing out scala/scabot#68, but something seems off -- it did work at #5662!)

@SethTisue
Copy link
Member

review by @SethTisue

@@ -27,7 +27,7 @@ trait ScalaSettings extends AbsScalaSettings
* defaults to the value of CLASSPATH env var if it is set, as in Java,
* or else to `"."` for the current user directory.
*/
protected def defaultClasspath = sys.env.getOrElse("CLASSPATH", ".")
protected def defaultClasspath = Option(System.getenv("CLASSPATH")).getOrElse(".")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is scala.util.Properties.{envOrElse, envOrNone} for that :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As Sébastien said, this is one of the ties I'd like to cut.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the goal of this PR is to cut dependencies, so this comment is out of scope: I'm in favor of shipping this kind of convenience (envOrElse) with Scala (in a form that makes it easily available, I understand we want to provide an opt-out).

@@ -1,7 +1,7 @@
package scala.reflect.macros
package compiler

import scala.compat.Platform.EOL
import java.lang.System.{lineSeparator => EOL}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There already was Properties.lineSeparator too, though it doesn't really add anything at all.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As Sébastien said, this is one of the ties I'd like to cut.

val budgetProp = scala.sys.Prop[String]("scalac.patmat.analysisBudget")
if (budgetProp.isSet) {
reportWarning(s"Please remove -D${budgetProp.key}, it is ignored.")
if (System.getProperty("scalac.patmat.analysisBudget") != null) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's where all the Properties.{propOrXXX, propIsSet, ...} methods are useful. I hadn't even realized it was all duplicated in sys.props until I saw it being used in the REPL.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The scala.sys.Prop is a bit over-designed for my tastes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would just re-read the source whenever I needed to use it.

@sjrd
Copy link
Member
sjrd commented

@gourlaysama From the PR description, IIUC, the idea is also not to rely on scala.util.Properties.

@gourlaysama
Copy link
Contributor

Ah, never mind, I hadn't seen that scala.util was going away too.

I would argue for moving Properties into the scala package then. Some of the things in there are useful to have all in one place in the core, like the stuff read from library.properties.

@sjrd
Copy link
Member
sjrd commented Feb 10, 2017

The stuff read from library.properties is not portable. It doesn't work on Scala.js. Moving something non-portable to the top scala._ package would be a serious mistake, IMO.

Besides, even though system properties overall do work in Scala.js, the only reason it is supported is because of JUnit. They shouldn't otherwise be used, IMO.

@adriaanm
Copy link
Contributor Author

Moving something non-portable to the top scala._ package would be a serious mistake, IMO.

Agreed. This is very much the goal here: the core should be easily portable across platforms. (Of course, as in scala.js and scala-native, parts of the Java stdlib will have to be reimplemented on those platforms.)

@adriaanm adriaanm modified the milestones: 2.12.2, WIP Feb 16, 2017
@SethTisue
Copy link
Member
SethTisue commented Mar 17, 2017

Why is scala.sys being deprecated

Eh? It isn't.

I've responded at some length over at https://contributors.scala-lang.org/t/modularization-of-scala-2-13-standard-library/635

@lrytz
Copy link
Member
lrytz commented Mar 30, 2017

scala.util (Try and Either move to the scala package)

What about these:

  • scala.util.matching.Regex
  • scala.util.hashing (MurmurHash3 has various uses, also in the compiler)
  • scala.util.control
  • scala.util.Random
  • scala.util.Sorting

@jvican
Copy link
Member
jvican commented Mar 30, 2017

scala.util.matching.Regex
scala.util.Random
scala.util.Sorting
scala.util.matching.Regex

I think it makes sense to remove these.

scala.util.hashing (MurmurHash3 has various uses, also in the compiler)

Honestly, I've used this so much, and I've seen so many people relying on it, that I'm a little bit reluctant on moving this away. I think it should stay.

@lrytz
Copy link
Member
lrytz commented Mar 30, 2017

One thing I never understood was the history of scala.util.Properties vs scala.sys.props, why do we have both / was the older not deprecated? Is scala.util.Properties going to stay? [edit: i see now that scala.util.Properties does more by reading the library.properties file]

About scala.util.matching.Regex, I don't think we can remove it, because it's used in collections (StringLike.r). The same is true for scala.util.control.Breaks. ControlThrowable, NonFatal are used in many places.

I think we really need to keep those core things in the core library.

Are we going to have an "util" module? Sounds a bit like a dumping ground :) But some util things are quite useful, and we want to provide the "batteries included" experience (with easy to get modules).

scala.concurrent is used in the repl. If we move it to a module we'll introduce a new bootstrap cycle. Maybe the repl moving out as well? Maybe we want to use futures in the compiler implementation at some point? I guess we'll have to use jdk apis. Also scala.io has a few uses in the compiler / repl, these should move to java.nio I guess.

@jvican
Copy link
Member
jvican commented Mar 30, 2017

Keeping control and hashing makes sense since they have value on their own.

However, I'm not sure if we should leave these in the core even if Scalac depends on them:

scala.util.matching.Regex
scala.util.Random
scala.util.Sorting

This functionality seems auxiliary. It's not clear if we should keep them because:

  • These utils are easily portable to the Scala side. The burden to maintain these simple wrappers around Java libs is close to zero (they haven't been touched in a while).
  • The contents of core should have a clear purpose. These seem to be just facilities but are not built around a common idea.

That being said, I think that a high-level discussion on the contents of the core would be beneficial. It's obvious to me that collections are necessary, but utilities that are not as visible as collections should be picked with care and a clear intention.

Should we raise this discussion in the next SIP meeting? We can probably identify common ground between the two compilers and find the most urgent needs that could benefit all Scala developers. I don't know if this makes sense, so I think it would be useful to bring it up to debate. /cc @odersky @adriaanm @SethTisue @sjrd @xeno-by @heathermiller @dragos @jsuereth.

Copy link
Member
@lrytz lrytz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes for cutting ties look good to me, let me know what you think about my other remarks.

@@ -45,6 +45,7 @@ package object sys {
*
* @return the result of `java.lang.Runtime.getRuntime()`
*/
@deprecated("use Runtime.getRuntime", "2.12.2")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as discussed (half?) offline, i think this pr should not deprecate things that will move out to separate modules. it would be better to define those modules first, carve them out, and then evolve the codebase there, hopefully with new contributors and energy :)

I won't copy this comment to other deprecations introduced in this PR, they are easy to find.

@@ -27,7 +27,7 @@ trait ScalaSettings extends AbsScalaSettings
* defaults to the value of CLASSPATH env var if it is set, as in Java,
* or else to `"."` for the current user directory.
*/
protected def defaultClasspath = sys.env.getOrElse("CLASSPATH", ".")
protected def defaultClasspath = Option(System.getenv("CLASSPATH")).getOrElse(".")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the goal of this PR is to cut dependencies, so this comment is out of scope: I'm in favor of shipping this kind of convenience (envOrElse) with Scala (in a form that makes it easily available, I understand we want to provide an opt-out).

@lrytz lrytz modified the milestones: 2.13.0-M1, WIP Apr 3, 2017
@SethTisue SethTisue modified the milestones: 2.13.0-M2, 2.13.0-M1 Apr 4, 2017
@lrytz
Copy link
Member
lrytz commented Apr 5, 2017

now in #5830

@SethTisue
Copy link
Member
SethTisue commented Dec 19, 2017

@nafg pointed out on the contributors forum that if Try moves up to scala.Try, Success and Failure would move with them, which seems questionable

@nafg
Copy link
Contributor
nafg commented Dec 19, 2017 via email

@floating-cat
Copy link

Current Option, Try, Either, List's subtypes package are same to its parent, and LazyList's subtypes are put inside the companion (There must be some other ADTs in standard lib I don't know).

Should we apply the some rules to these ADTs (put their subtypes inside the companion), or just treat some special?

@som-snytt
Copy link
Contributor

I'm not sure the fate of regex was fixed?

It would nice if support were in a form like xml, where the std lib doesn't depend on an impl, but usages of "p".r do. Then whatever impl on the classpath is used. I don't know off the cuff what that would look like.

I hope somebody actually takes a big pair of scissors and cuts Adriaan's tie when this is all complete.

@SethTisue
Copy link
Member

we've run out of time in the 2.13 cycle — let's pick this up again for 2.14

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

0