You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: notes/techreport.tex
+29-29Lines changed: 29 additions & 29 deletions
Original file line number
Diff line number
Diff line change
@@ -29,9 +29,9 @@
29
29
30
30
\section{Introduction}
31
31
32
-
Scala-gopher is a library-level implementation of process algebra [Communication Sequential Processes, see \cite{Hoare85communicatingsequential} as ususally enriched by $\pi$-calculus \cite{Milner:1992:CMP:162037.162038} naming primitives] in scala. In addition to support of a 'limbo/go-like'\cite{Inferno:Limbo} \cite{golang} channels/goroutine programming style scala-gopher provide set of operations following typical scala idiomatic.
32
+
Scala-gopher is a library-level implementation of process algebra [Communication Sequential Processes, see \cite{Hoare85communicatingsequential} as ususally enriched by $\pi$-calculus \cite{Milner:1992:CMP:162037.162038} naming primitives] in scala. In addition to support of a 'limbo/go-like'\cite{Inferno:Limbo} \cite{golang} channels/goroutine programming style scala-gopher provides a set of operations following typical idiomatic scala.
33
33
34
-
At first, let's remind the fundamentals of a CSP model. The primary entities of this model are channels, coroutines and selectors. Coroutines are lightweight threads of execution, which can communicate with each other by passing messages between channels. Channels can be viewed as blocked multiproducer/multiconsumer queues. Sending message to unbuffered channel suspend producing coroutines until the moment when this message will have been read by some consumer. Buffered channels transfer control flow between sinks not on each message, but when internal channel buffer is full.
34
+
At first, let's remind the fundamentals of a CSP model. The primary entities of this model are channels, coroutines and selectors. Coroutines are lightweight threads of execution, which can communicate with each other by passing messages between channels. Channels can be viewed as blocked multiproducer/multiconsumer queues. Sending message to unbuffered channel suspends producing coroutines until the moment when this message will have been read by some consumer. Buffered channels transfer control flow between sinks not on each message, but when internal channel buffer is full.
35
35
In such way, communication via channel implicitly provides flow control functionality. At last, a selector statement is a way of coordination of several communication activities: like Unix select(2) system call, select statement suspends current coroutines until one of the actions (reading/writing to one of the channels in selector) will be possible.
36
36
37
37
Let's look at the one simple example:
@@ -56,7 +56,7 @@ \section{Introduction}
56
56
}
57
57
}
58
58
\end{Verbatim}
59
-
Here two channels and three goroutines are created. The first coroutine just generates consecutive numbers and send one to channel \verb|in|, second - accept this sequence as the initial state and for each number which has been read from state channel, write one to \verb|out| and produce next step by filtering previous. The result of the fold is the \verb|in| channel whith applied filters for each prime. The third coroutine just maps range to values to receive a list of first \verb|n| primes in Future.
59
+
Here two channels and three goroutines are created. The first coroutine just generates consecutive numbers and sends one to channel \verb|in|, second - accepts this sequence as the initial state and for each number which has been read from state channel, writes one to \verb|out| and produces next step by filtering previous. The result of the fold is the \verb|in| channel whith applied filters for each prime. The third coroutine just maps range to values to receive a list of first \verb|n| primes in Future.
60
60
61
61
If we look at the sequence of steps during code evaluation, we will see at first generation of
62
62
number, then checks in filters and then if a given number was prime - final output. Note, that goroutine is different from JVM thread of execution: sequential code chunks are executed in configurable executor service; switching between chunks does not use blocking operations.
@@ -68,14 +68,14 @@ \subsection{Go: Translations of hight-order functions to asynchronous form.}
68
68
69
69
The main entity of CSP is a 'process' which can be viewed as a block of code which handles specific events. In Go CSP processes are represented as goroutines (aka coroutines).
70
70
71
-
\verb|go[X](x:X):Future[X]| is a think wrapper arround SIP-22 async/await which do some preprocessing before async transformation:
71
+
\verb|go[X](x:X):Future[X]| is a thin wrapper arround SIP-22 async/await which does some preprocessing before async transformation:
72
72
\begin{itemize}
73
73
\item do transformation of hight-order function in async form.
74
74
75
75
76
76
Let $f(A \To B)\To C$ is a hight-order function, which accepts other function
77
77
$g:A \To B$ as parameter.
78
-
Let's say that $g$ in $f$ is {\i invocation-only } if $f$ not store $g$ in memory outside of $f$ scope and not return $g$ as part of return value. Only one action which $f$ can do with $g$ is invocation or passing as a parameter to other invocation-only function. If we look at Scala collection API, we will see, that near all hight-order functions there are invocation-only.
78
+
Let's say that $g$ in $f$ is {\i invocation-only } if $f$does not store $g$ in memory outside of $f$ scope and doesn't return $g$ as part of return value. Only one action which $f$ can do with $g$ is invocation or passing as a parameter to other invocation-only function. If we look at Scala collection API, we will see, that near all hight-order functions there are invocation-only.
79
79
80
80
Now, if we have $g$ which is invocation-only in $f$, and if we have function $g' : (A \To Future[B])$ let build function $f':(A\To Future[B])\to Future[C]$ that if $await(g')==await(g)$ then $await(f'(g'))==f(g))$ in next way
81
81
\begin{itemize}
@@ -106,7 +106,7 @@ \subsection{Go: Translations of hight-order functions to asynchronous form.}
106
106
(1 to n).mapAsync(i => out.aread)
107
107
\end{Verbatim}
108
108
109
-
Using this approach allows overcoming the inconvenience of async/await by allowing programmers use hight-order functions API inside asynchronous expression. Also, it is theoretically possible to generate asynchronous variants of API methods by transforming TASTY\cite{TASTY} representation of AST of synchronous versions. Thr similar technique is implemented in Nim \cite{Nim} programming language, where we can to generate both synchronious and asynchronious variants of a function from one definition.
109
+
Using this approach allows overcoming the inconvenience of async/await by allowing programmers to use hight-order functions API inside asynchronous expression. Also, it is theoretically possible to generate asynchronous variants of API methods by transforming TASTY\cite{TASTY} representation of AST of synchronous versions. A similar technique is implemented in Nim \cite{Nim} programming language, where we can generate both synchronious and asynchronious variants of a function from one definition.
110
110
111
111
\item do transformation of defer statement. This is just an implementation of error handling mechanism.
112
112
@@ -115,7 +115,7 @@ \subsection{Go: Translations of hight-order functions to asynchronous form.}
115
115
\subsection{Channels: callbacks organized as waits}
116
116
117
117
Channels in CSP are two-sided pipes between processes; Channels messages not only pass information between goroutines but also coordinate process execution.
118
-
In scala-gopher appropiative entities (\verb|Input[A]| for reading and \verb|Output[A]| for writing) implemented in fully asynchniously manner with help of callback-based interfaces:
118
+
In scala-gopher appropiative entities (\verb|Input[A]| for reading and \verb|Output[A]| for writing) implemented in fully asynchnious manner with help of callback-based interfaces:
119
119
120
120
\begin{Verbatim}[fontsize=\small]
121
121
trait Input[A]
@@ -158,12 +158,12 @@ \subsection{Channels: callbacks organized as waits}
158
158
the usual set of stream combinators: filter, map, zip, fold, etc.
159
159
160
160
Channel is a combination of input and output. In addition to well-known buffered and unbuffered
161
-
kinds of channels, scala-gopher provide some extra set of channels with different behavior and performance characteristics,
161
+
kinds of channels, scala-gopher provides some extra set of channels with different behavior and performance characteristics,
162
162
such as the channel with growing buffer (a-la actor mailbox) for connecting loosely coupled processes or one-time channel based on \verb|Promise|, which is automatically closed after sending one message.
163
163
164
164
\subsection{Selectors: process composition as event generation}
165
165
166
-
Mutually exclusive process composition (i.e. deterministic choice: $(a \to P)\square(b\to Q)$ in original Hoar notation ) usually represented in CSP=based languages as \verb|select| statement, which look's like ordinary switch. In a typical \verb|Go| program exists often repeated code pattern: select inside endless loop inside go statement.
166
+
Mutually exclusive process composition (i.e. deterministic choice: $(a \to P)\square(b\to Q)$ in original Hoar notation ) usually represented in CSP-based languages as \verb|select| statement, which looks like ordinary switch. In a typical \verb|Go| program exists often repeated code pattern: select inside endless loop inside go statement.
167
167
168
168
\begin{Verbatim}[fontsize=\small]
169
169
go {
@@ -180,7 +180,7 @@ \subsection{Selectors: process composition as event generation}
180
180
181
181
Appropriative expression in CSP syntax: $*[(c_{1} ? x \to P)\square(c_{2} ! y \to Q)]$
182
182
183
-
Scala-gopher provide\verb|select| pseudo-object which provide set of high-order pseudo-functions over
183
+
Scala-gopher provides\verb|select| pseudo-object which provides set of high-order pseudo-functions over
184
184
channels, which accept syntax of partial function over channel events:
185
185
186
186
\begin{Verbatim}[fontsize=\small]
@@ -223,14 +223,14 @@ \subsection{Selectors: process composition as event generation}
223
223
}
224
224
\end{Verbatim}
225
225
226
-
Here we see the special syntax for tuple state. Also note, that \verb|afold| macro assume that \verb|s match| must be the first statement in the argument pseudo-function. \verb|select.exit| is used for returning result from the flow.
226
+
Here we see the special syntax for tuple state. Also note, that \verb|afold| macro assumes that \verb|s match| must be the first statement in the argument pseudo-function. \verb|select.exit| is used for returning result from the flow.
227
227
228
-
Events which we can check in select match statement are reading and writing of channels and select timeouts. In future we will think about extending the set of notifications - i.e. adding channel closing and overflow notifications, which are rare needed in some scenarios.
228
+
Events which we can check in select match statement are reading and writing of channels and select timeouts. In future we will think about extending the set of notifications - i.e. adding channel closing and overflow notifications, which are needed in some rare scenarios.
229
229
230
230
\subsection{Transputer: an entity which encapsulates processing node. }
231
231
232
-
The idea is to have an actor-like object, which encapsulates processing node: i.e., read input data
233
-
from the set of input ports; write a result to the set of output ports and maintaine a local
<
2364
code>
232
+
The idea is to have an actor-like object, which encapsulates processing node: i.e., reads input data
233
+
from the set of input ports; writes a result to the set of output ports and maintaines a local
234
234
mutable state inside.
235
235
236
236
Example:
@@ -257,26 +257,26 @@ \subsection{Transputer: an entity which encapsulates processing node. }
257
257
}
258
258
\end{Verbatim}
259
259
260
-
Having set of such objects, we can build complex systems as combination of simple ones:
260
+
Having a set of such objects, we can build complex systems as combinations of simple ones:
261
261
\begin{itemize}
262
262
\item$a+b$ - parallel execution;
263
263
\item$replicate[A](n)$ - transputer replication, where we start in parallel $n$ instances
264
-
of $A$. Policy for prot replication can be configured - from sharing appropriative channel by each port to distributing or duplication of signals to ditinguish each instance.
264
+
of $A$. Policy for port replication can be configured - from sharing appropriative channel by each port to distributing or duplication of signals to ditinguish each instance.
265
265
\begin{Verbatim}[fontsize=\small]
266
266
val r = gopherApi.replicate[SMTTransputer](10)
267
267
( r.dataInput.distribute( (_.hashCode % 10 ) ).
268
268
.controlInput.duplicate().
269
269
out.share()
270
270
)
271
271
\end{Verbatim}
272
-
- here in \verb|r| we have ten instances of \verb|SMTTransputer|. If we send a message to \verb|dataInput| it will be directed to one of the instances (in dependency from the value of message \verb|hashcode|). If we send a message to the \verb|controlInput|, it will be delivered to each instance; output channel will be shared between all instances.
272
+
- here in \verb|r| we have ten instances of \verb|SMTTransputer|. If we send a message to \verb|dataInput| it will be directed to one of the instances (according to the value of message's\verb|hashcode|). If we send a message to the \verb|controlInput|, it will be delivered to each instance; output channel will be shared between all instances.
273
273
\end{itemize}
274
274
275
275
Transputers can participate in error handling scenarios in the same way as actors: for each transputer, we can define recovery policy and supervisor.
276
276
277
277
\subsection{ Programming Techniques based on dynamic channels }
278
278
279
-
Let's outline some programming techniques, well known in Go world but not easily expressible in current mainstream Scala streaming libraries.
279
+
Let's outline some programming techniques, well known in a Go world but not easily expressible in current mainstream Scala streaming libraries.
280
280
281
281
\begin{itemize}
282
282
\item Channel expression as an element of runtime state. Following this pattern allows a developer to maintain dynamics potentially recursive dataflows.
@@ -312,7 +312,7 @@ \subsection{ Programming Techniques based on dynamic channels }
312
312
313
313
Let we want provide API which must on request return some value to the caller. Instead of providing a method which will return a result on the stack we can provide endpoint channel, which will accept method arguments and channel where to return a result.
314
314
315
-
Next example illustrate this idea:
315
+
Next example illustrates this idea:
316
316
317
317
\begin{Verbatim}[fontsize=\small]
318
318
trait Broadcast[T]
@@ -351,7 +351,7 @@ \subsection{ Programming Techniques based on dynamic channels }
351
351
}
352
352
\end{Verbatim}
353
353
354
-
Broadcast provide API for creation of listeners and sending messages to all listeners.
354
+
Broadcast provides API for creation of listeners and sending messages to all listeners.
355
355
356
356
To register listener channel for receiving notification client sends this channel to newListener
357
357
@@ -363,35 +363,35 @@ \subsection{ Programming Techniques based on dynamic channels }
363
363
364
364
\section{ Connection with other models }
365
365
366
-
Exists many stream libraries for Scala with different sets of tradeoffs. At one side of
366
+
There are many stream libraries for Scala with different sets of tradeoffs. At one side of
367
367
spectrum, we have clear streaming models like akka-streams\cite{akka-streams} with comp set
368
368
of composable operations and clear high-level functionality but lack of flexibility,
369
-
from another side - very flexible but low-level models like actors.
369
+
at the other side - very flexible but low-level models like actors.
370
370
371
371
Scala-gopher provides uniform API which allows build systems from different parts of spectrum:
372
372
it is possible to build dataflow graph in a declarative manner and connect one with a dynamic part.
373
373
374
374
The reactive isolates model\cite{Prokopec:2015:ICE:2814228.2814245} is close to scala-gopher model with dynamically-grown channel buffer (except that reactive isolates support distributed case).
375
-
Isolate here corresponds to Transputer, Channel to Output and Events to gopher Input. Channels in reactive isolates are more limited: only one isolate which owns the channel can write to it when in the scala-gopher concept of channel ownity is absent.
375
+
Isolate here corresponds to Transputer, Channel to Output and Events to gopher Input. Channels in reactive isolates are more limited: only one isolate which owns the channel can write to it when in the scala-gopher concept of channel ownership is absent.
376
376
377
377
Communicating Scala Objects\cite{CSO} is a direct implementation of CSP model in Scala which allows
378
-
building expressions in internal Scala DSL, closed to origin Hoar notation with some extensions, like extended rendezvous for mapping input streams. Processes in CSO are not lightweight: each process requires Java thread which limits the scalability of this library until lightweight threading will be implemented on JVM level.
378
+
building expressions in internal Scala DSL, closed to original Hoar notation with some extensions, like extended rendezvous for mapping input streams. Processes in CSO are not lightweight: each process requires Java thread which limits the scalability of this library until lightweight threading will be implemented on JVM level.
379
379
380
380
Subscript\cite{vanDelft:2013:DCL:2489837.2489849} is a Scala extension which adds to language new constructions for building process algebra expressions. Although extending language can afford fine-grained interconnection of process-algebra and imperative language notation in far perspective, now it makes CPA constructs a second-class citizen because we have no direct representation of process and event types in Scala type system.
381
381
382
382
383
383
\section{ Conclusion and future directions }
384
384
385
-
Scala-gopher is a relatively new library which yet not reach 1.0 state, but we have the early experience
385
+
Scala-gopher is a relatively new library which has not yet reached 1.0 state, but we have the early experience
386
386
reports from using the library for building some helper components in an industrial software project.
387
387
388
-
In general, feedback is positive: developers enjoy relative simple mental model and ability freely use asynchronous operations inside hight-order functions. So, we can recommend to made conversion of invocation-only functions into async form to be available into the async library itself.
388
+
In general, feedback is positive: developers enjoy a relatively simple mental model and ability to freely use asynchronous operations inside higher-order functions. So, we can recommend to made conversion of invocation-only functions into async form to be available into the async library itself.
389
389
390
-
The area which needs more work: error handling in \verb|go| statements: now \verb|go| return
391
-
\verb|Future| which can hold a result of the evaluation or an exception. If we ignore statement result than we miss handling of exception; from another side, it unlikely handle errors there, because we must allow a developer to implement own error processing. The workaround is to use different method name for calling statement in the context with ignored return value, but it is easy to mix-up this two names.
390
+
The area which needs more work: error handling in \verb|go| statements: now \verb|go| returns
391
+
\verb|Future| which can hold a result of the evaluation or an exception. If we ignore statement result then we miss handling of exception; from another side, it unlikely handle errors there, because we must allow a developer to implement own error processing. The workaround is to use different method name for calling statement in the context with ignored return value, but it is easy to mix-up this two names.
392
392
We think, that right solution can be built on language level: we can bind handling of ignored value to type by providing appropriative implicit conversion or use special syntax for functions which needs
393
393
394
-
Also, we plan to extend model by adding notifications about channel-close and channel-overflow on write side which needed in some relatively rare usage scenarious.
394
+
Also, we plan to extend model by adding notifications about channel-close and channel-overflow on write side that are needed in some relatively rare usage scenarios.
395
395
396
396
Support of distributed communications can be the next logical step in future development. In our opinion, practical approach will differ from implementing location-transparency support for existing API. Rather we will think to enrich model with new channel types and mechanisms for work in explicitly distributed environment.
0 commit comments