8000 Merge pull request #63 from axel22/master · scala/docs.scala-lang@31f558d · GitHub
[go: up one dir, main page]

Skip to content

Commit 31f558d

Browse files
committed
Merge pull request #63 from axel22/master
Typos fixed, some additions to the pc-overview docs
2 parents 2e97cbc + 79c7c9e commit 31f558d

File tree

8 files changed

+482
-259
lines changed

8 files changed

+482
-259
lines changed

overviews/parallel-collections/architecture.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,8 @@ subsets of elements of the whole parallel collection. And similar to normal
5252
In general, collections are partitioned using `Splitter`s into subsets of
5353
roughly the same size. In cases where more arbitrarily-sized partions are
5454
required, in particular on parallel sequences, a `PreciseSplitter` is used,
55-
which inherits `Splitter`.
55+
which inherits `Splitter` and additionally implements a precise split method,
56+
`psplit`.
5657

5758
### Combiners
5859

@@ -86,12 +87,12 @@ regular collections framework's corresponding traits, as shown below.
8687

8788
[<img src="{{ site.baseurl }}/resources/images/parallel-collections-hierarchy.png" width="550">]({{ site.baseurl }}/resources/images/parallel-collections-hierarchy.png)
8889

89-
<center><b>Hierarchy of Scala's Collections and Parallel Collections Libraries<b></center>
90-
90+
<center><b>Hierarchy of Scala's Collections and Parallel Collections Libraries</b></center>
91+
<br/>
9192

9293
The goal is of course to integrate parallel collections as tightly as possible
9394
with sequential collections, so as to allow for straightforward substitution
94-
of sequntial and parallel collections.
95+
of sequential and parallel collections.
9596

9697
In order to be able to have a reference to a collection which may be either
9798
sequential or parallel (such that it's possible to "toggle" between a parallel

overviews/parallel-collections/concrete-parallel-collections.md

Lines changed: 103 additions & 23 deletions
Large diffs are not rendered by default.

overviews/parallel-collections/configuration.md

Lines changed: 32 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -10,22 +10,25 @@ num: 7
1010

1111
## Task support
1212

13-
Parallel collections are modular in the way operations are scheduled. Each parallel
14-
collection is parametrized with a task support object which is responsible for
15-
scheduling and load-balancing tasks to processors.
16-
17-
The task support object internally keeps a reference to a thread pool implementation
18-
and decides how and when tasks are split into smaller tasks.
19-
To learn more about the internals of how exactly this is done, see the tech report \[[1][1]\].
20-
21-
There are currently a few task support implementations available for parallel collections.
22-
The `ForkJoinTaskSupport` uses a fork-join pool internally and is used by default on JVM 1.6 or greater.
23-
The less efficient `ThreadPoolTaskSupport` is a fallback for JVM 1.5 and JVMs that do not support
24-
the fork join pools. The `ExecutionContextTaskSupport` uses the default execution context implementation
25-
found in `scala.concurrent`, and it reuses the thread pool used in
26-
`scala.concurrent` (this is either a fork join pool or a thread pool executor, depending on the JVM version).
27-
The execution context task support is set to each parallel collection by default, so parallel collections
28-
reuse the same fork-join pool as the future API.
13+
Parallel collections are modular in the way operations are scheduled. Each
14+
parallel collection is parametrized with a task support object which is
15+
responsible for scheduling and load-balancing tasks to processors.
16+
17+
The task support object internally keeps a reference to a thread pool
18+
implementation and decides how and when tasks are split into smaller tasks. To
19+
learn more about the internals of how exactly this is done, see the tech
20+
report \[[1][1]\].
21+
22+
There are currently a few task support implementations available for parallel
23+
collections. The `ForkJoi 6D4E nTaskSupport` uses a fork-join pool internally and is
24+
used by default on JVM 1.6 or greater. The less efficient
25+
`ThreadPoolTaskSupport` is a fallback for JVM 1.5 and JVMs that do not support
26+
the fork join pools. The `ExecutionContextTaskSupport` uses the default
27+
execution context implementation found in `scala.concurrent`, and it reuses
28+
the thread pool used in `scala.concurrent` (this is either a fork join pool or
29+
a thread pool executor, depending on the JVM version). The execution context
30+
task support is set to each parallel collection by default, so parallel
31+
collections reuse the same fork-join pool as the future API.
2932

3033
Here is a way to change the task support of a parallel collection:
3134

@@ -41,33 +44,34 @@ Here is a way to change the task support of a parallel collection:
4144
scala> pc map { _ + 1 }
4245
res0: scala.collection.parallel.mutable.ParArray[Int] = ParArray(2, 3, 4)
4346

44-
The above sets the parallel collection to use a fork-join pool with parallelism level 2.
45-
To set the parallel collection to use a thread pool executor:
47+
The above sets the parallel collection to use a fork-join pool with
48+
parallelism level 2. To set the parallel collection to use a thread pool
49+
executor:
4650

4751
scala> pc.tasksupport = new ThreadPoolTaskSupport()
4852
pc.tasksupport: scala.collection.parallel.TaskSupport = scala.collection.parallel.ThreadPoolTaskSupport@1d914a39
4953

5054
scala> pc map { _ + 1 }
5155
res1: scala.collection.parallel.mutable.ParArray[Int] = ParArray(2, 3, 4)
5256

53-
When a parallel collection is serialized, the task support field is omitted from serialization.
54-
When deserializing a parallel collection, the task support field is set to the default value - the execution context task support.
57+
When a parallel collection is serialized, the task support field is omitted
58+
from serialization. When deserializing a parallel collection, the task support
59+
field is set to the default value-- the execution context task support.
5560

56-
To implement a custom task support, extend the `TaskSupport` trait and implement the following methods:
61+
To implement a custom task support, extend the `TaskSupport` trait and
62+
implement the following methods:
5763

5864
def execute[R, Tp](task: Task[R, Tp]): () => R
5965

6066
def executeAndWaitResult[R, Tp](task: Task[R, Tp]): R
6167

6268
def parallelismLevel: Int
6369

64-
The `execute` method schedules a task asynchronously and returns a future to wait on the result of the computation.
65-
The `executeAndWait` method does the same, but only returns when the task is completed.
66-
The `parallelismLevel` simply returns the targeted number of cores that the task support uses to schedule tasks.
67-
68-
69-
70-
70+
The `execute` method schedules a task asynchronously and returns a future to
71+
wait on the result of the computation. The `executeAndWait` method does the
72+
same, but only returns when the task is completed. The `parallelismLevel`
73+
simply returns the targeted number of cores that the task support uses to
74+
schedule tasks.
7175

7276

7377
## References

overviews/parallel-collections/conversions.md

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -11,23 +11,23 @@ num: 3
1111
## Converting between sequential and parallel collections
1212

1313
Every sequential collection can be converted to its parallel variant
14-
using the `par` method. Certain sequential collections have their
15-
direct parallel counterparts. For these collections the conversion is
16-
efficient - it occurs in constant time, since both the sequential and
14+
using the `par` method. Certain sequential collections have a
15+
direct parallel counterpart. For these collections the conversion is
16+
efficient-- it occurs in constant time, since both the sequential and
1717
the parallel collection have the same data-structural representation
18-
(one exception are mutable hash maps and hash sets which are slightly
18+
(one exception is mutable hash maps and hash sets which are slightly
1919
more expensive to convert the first time `par` is called, but
2020
subsequent invocations of `par` take constant time). It should be
2121
noted that for mutable collections, changes in the sequential collection are
2222
visible in its parallel counterpart if they share the underlying data-structure.
2323

24-
| sequential | parallel |
24+
| Sequential | Parallel |
2525
| ------------- | -------------- |
2626
| **mutable** | |
2727
| `Array` | `ParArray` |
2828
| `HashMap` | `ParHashMap` |
2929
| `HashSet` | `ParHashSet` |
30-
| `Ctrie` | `ParCtrie` |
30+
| `TrieMap` | `ParTrieMap` |
3131
| **immutable** | 10000 |
3232
| `Vector` | `ParVector` |
3333
| `Range` | `ParRange` |
@@ -43,9 +43,9 @@ vector.
4343

4444
Every parallel collection can be converted to its sequential variant
4545
using the `seq` method. Converting a parallel collection to a
46-
sequential collection is always efficient - it takes constant
46+
sequential collection is always efficient-- it takes constant
4747
time. Calling `seq` on a mutable parallel collection yields a
48-
sequential collection which is backed by the same store - updates to
48+
sequential collection which is backed by the same store-- updates to
4949
one collection will be visible in the other one.
5050

5151

@@ -61,7 +61,7 @@ into a `ParX` collection.
6161

6262
Here is a summary of all conversion methods:
6363

64-
| method | return type |
64+
| Method | Return Type |
6565
| -------------- | -------------- |
6666
| `toArray` | `Array` |
6767
| `toList` | `List` |

overviews/parallel-collections/ctries.md

Lines changed: 48 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -18,17 +18,24 @@ parallel counterparts. The only difference between the two is that the
1818
former traverses its elements sequentially, whereas the latter does so in
1919
parallel.
2020

21-
This is a nice property that allows you to write some algorithms more easily.
21+
This is a nice property that allows you to write some algorithms more
22+
easily. Typically, these are algorithms that process a dataset of elements
23+
iteratively, in which different elements need a different number of
24+
iterations to be processed.
25+
2226
The following example computes the square roots of a set of numbers. Each iteration
2327
iteratively updates the square root value. Numbers whose square roots converged
2428
are removed from the map.
2529

30+
case class Entry(num: Double) {
31+
var sqrt = num
32+
}
2633

2734
val length = 50000
2835

2936
// prepare the list
3037
val entries = (1 until length) map { num => Entry(num.toDouble) }
31-
val results = ParConcurrentTrieMap()
38+
val results = ParTrieMap()
3239
for (e <- entries) results += ((e.num, e))
3340

3441
// compute square roots
@@ -41,8 +48,25 @@ are removed from the map.
4148
}
4249
}
4350

51+
Note that in the above Babylonian method square root computation
52+
(\[[3][3]\]) some numbers may converge much faster than the others. For
53+
this reason, we want to remove them from `results` so that only those
54+
elements that need to be worked on are traversed.
55+
4456
Another example is the breadth-first search algorithm, which iteratively expands the nodefront
45-
until either it finds some path to the target or there are no more nodes to expand.
57+
until either it finds some path to the target or there are no more
58+
nodes to expand. We define a node on a 2D map as a tuple of
59+
`Int`s. We define the `map` as a 2D array of booleans which denote is
60+
the respective slot occupied or not. We then declare 2 concurrent trie
61+
maps-- `open` which contains all the nodes which have to be
62+
expanded (the nodefront), and `closed` which contains all the nodes which have already
63+
been expanded. We want to start the search from the corners of the map and
64+
find a path to the center of the map-- we initialize the `open` map
65+
with appropriate nodes. Then we iteratively expand all the nodes in
66+
the `open` map in parallel until there are no more nodes to expand.
67+
Each time a node is expanded, it is removed from the `open` map and
68+
placed in the `closed` map.
69+
Once done, we output the path from the target to the initial node.
4670

4771
val length = 1000
4872

@@ -63,8 +87,8 @@ until either it finds some path to the target or there are no more nodes to expa
6387

6488
// open list - the nodefront
6589
// closed list - nodes already processed
66-
val open = ParConcurrentTrieMap[Node, Parent]()
67-
val closed = ParConcurrentTrieMap[Node, Parent]()
90+
val open = ParTrieMap[Node, Parent]()
91+
val closed = ParTrieMap[Node, Parent]()
6892

6993
// add a couple of starting positions
7094
open((0, 0)) = null
@@ -97,13 +121,22 @@ until either it finds some path to the target or there are no more nodes to expa
97121
}
98122
println()
99123

100-
The concurrent trie data structure supports a linearizable, lock-free, constant
101-
time, lazily evaluated snapshot operation. The snapshot operation merely creates
124+
125+
The concurrent tries also support a linearizable, lock-free, constant
126+
time `snapshot` operation. This operation creates a new concurrent
127+
trie with all the elements at a specific point in time, thus in effect
128+
capturing the state of the trie at a specific point.
129+
The `snapshot` operation merely creates
102130
a new root for the concurrent trie. Subsequent updates lazily rebuild the part of
103131
the concurrent trie relevant to the update and leave the rest of the concurrent trie
104132
intact. First of all, this means that the snapshot operation by itself is not expensive
105133
since it does not copy the elements. Second, since the copy-on-write optimization copies
106134
only parts of the concurrent trie, subsequent modifications scale horizontally.
135+
The `readOnlySnapshot` method is slightly more efficient than the
136+
`snapshot` method, but returns a read-only map which cannot be
137+
modified. Concurrent tries also support a linearizable, constant-time
138+
`clear` operation based on the snapshot mechanism.
139+
To learn more about how concurrent tries and snapshots work, see \[[1][1]\] and \[[2][2]\].
107140

108141
The iterators for concurrent tries are based on snapshots. Before the iterator
109142
object gets created, a snapshot of the concurrent trie is taken, so the iterator
@@ -119,7 +152,14 @@ of the `size` method to amortized logarithmic time. In effect, this means that a
119152
the size only for those branches of the trie which have been modified since the last `size` call.
120153
Additionally, size computation for parallel concurrent tries is performed in parallel.
121154

122-
The concurrent tries also support a linearizable, lock-free, constant time `clear` operation.
123155

124156

157+
## References
158+
159+
1. [Cache-Aware Lock-Free Concurrent Hash Tries][1]
160+
2. [Concurrent Tries with Efficient Non-Blocking Snapshots][2]
161+
3. [Methods of computing square roots][3]
125162

163+
[1]: http://infoscience.epfl.ch/record/166908/files/ctries-techreport.pdf "Ctries-techreport"
164+
[2]: http://lampwww.epfl.ch/~prokopec/ctries-snapshot.pdf "Ctries-snapshot"
165+
[3]: http://en.wikipedia.org/wiki/Methods_of_computing_square_roots#Babylonian_method "babylonian-method"

0 commit comments

Comments
 (0)
0