It is probably not possible to use mutex with Grid. But I see no other option. 

I need to create a number of nodes in Grid to do parallel processing. But need to also make critical section around few places during execution  in the code (for example, writing to common file, or update SQLite db).

Grid package does not seem to have a command to do this. I found that the Threads package have mutex which is exactly what I wanted in order to make critical section.

But all my attempts to use mutex in the Grid fail. When I make mutex and pass it to Grid:-Launch call to be used by each node, I keep getting errors when it is used inside the node.

I do not know if I am just not passing it correctly, or simply Mutex is not supported in Grid. I tried many things, and none have worked.

If it is not supported, what other mechanism are there to allow one to synchronize access to some shared resource, so only one node is using at a time?

Grid has Wait and Barrier and WaitForFirst, but I do not see how these can be used for this.

Without the ability to be able to create critical section, then Grid package will not work for me.

Below is worksheet I have been using. I hope I am just making silly mistake in using the mutex with Grid.

ps. I prefer to use Grid and not Threads package to do parallel processing.

pps. I can see why mutex would not work in Grid, since mutex is process specific entity, and Grid uses completely separate processes for each node. So it will not work to use for synchronization between separate processes. But what else to use in Maple for this?

Download using_mutex_in_grid.mw

I never used the Maple Grid package. I've been learning it in the last few hrs and it looks nice and could be something I can use to speed my very slow script.

I am trying to see if I can use it to do parallel processing instead of using Batch script and calling cmaple.exe manually to solve one problem at a time sequentially.

I got basic flow working in the example below. Basically it goes like this.

Call Grid:-Lauach to run 10 separate Maple processes in parallel. From help, Grid will run 10 separate server.exe's.

Passing the function to run the list of problems to processes.

At the of the function, when it is done solving the 10 problems, there is Barrier. So when all 10 nodes are completed, this run is done.

Then I repeate this, passing the next 10 problems to solve and so on.

The only thing I am not clear on from help if I should worry about node 0 or just treat it as any other node in terms of using it to solve a problem or not. It seems node 0 is special. So do I need to check if node 0 is the one being run and not use it?

Could someome who knows Grid package better please check and review this code and see if they find any issues with it? As I will be basing all my tests on this frame work if I find it speed things. 

From this basic test, it seems to work ok for all nodes, including node 0. 

Download using_GRID.mw

Hello

I am experiencing some problems with procedures that are saved to a file and are read back to a new worksheet.  Grid:-Map and Grid:-Set, which work just fine when the procedures were defined in the same worksheet, complain about the input parameters.  I have tried to define the procedures again in the same worksheet, call Grid:-Set and etc but to no avail.   I would like to try to save all user variables except for the user-defined procedures and defined them again in the new worksheet, and see if that works. Is there an easy way to do that?

I am open to try whatever is suggested.

Many thanks.

Ed

Why I get "Error, invalid subscript selector" error in my code?

y1 := Grid:-Seq(UP1(s, U, V, W, Phi, Xi, N, a, b, II, JJ, A, B, Dd, M, Ns), s = 1 .. 7);
UKt := add(y1[i], i = 1 .. 7);

Error, invalid subscript selector

As you can see y1 is defined without any problem and have 7 seqments, but the next line warns invalid subscript selector.

I created a Github repository with files to reproduce the example that my question is based on.

I am using Grid:-Run to compute a 10000x3 matrix in each of 100 parallel runs. I then average the 100 matrices to obtain a final 10000x3 matrix, which I saved to an .m file.

It contains a variable called r with the matrix.

Each row of the matrix is a 3d point, and the entire matrix represents a trajectory on a sphere. I'd like to display an animation of this trajectory. Now, my question is not about how to do this, but rather about the memory requirements necessary to do so.

I can plot trajectories with, say, 1000 points, but apparently 10000 requires too much memory.

The Github repository I linked to contains an example of all of this.

The worksheet matrixTest.mw reads the matrix from the .m file and reads some procedures defined in the .mpl file that are used for creating the animation data.

Then, a call to the procedure animateMatrices basically calls plots:-display to display the matrix as an animation.

I've been able to display the first 5000 rows of the matrix.

What type of memory is running out when I try to display the animation of the full 10000 rows?

Here is a screenshot of my Activity Monitor right before I get the

Error, out of memory error while processing result

I am trying to perform a numerical integration of a matrix using parallel computing. I have found two packages to do that:

1. With Threads package

A:=Matrix([x,x^2],[2*x,sin(x)]);

Threads[Map](int,A,x=0..2,numeric=true);

This code does not parallelize at my Ryzen 5 processor and is using exactly 1 logical core during computation. The matrix is just an example, real matrices are much bigger.

kernelopts(multithreaded) returns true

kernelopts(numcpus) returns 6 when SMT is off and 12 when SMT is on (pretty weird though, it should always return 6)

2. With Grid package

A:=Matrix([x,x^2],[2*x,sin(x)]);

Threads[Map](int,A,x=0..2,numeric=true);

This leads to numerous messages:

"Error, attempting to assign to `int` which is protected.  Try declaring `local int`; see ?protect for details."

Meanwhile, the rezult seems OK and all cores are working to produce it.

What am I doing wrong?

I rewrite my code with Grid library using local vCPU of amazon 

discover no license of distributed HPC when setup command show need go to acresso.com

can Grid still be used and function with local 96 number of vCPU?

then when I test code, I can not pkill mserver in ps -aux in LInux 

can the performance really improved ? Because I suppose 3 to 5 minutes mserver will end and disappear from ps -aux but grid node of mserver still running

originally I can run 100 batches every day., but I have to monitor decrease of memory in order to determine whether can kill mserver for next batch 

I expect run 400 batches per day. But Can not kill mserver when using grid

In my code I had using time limit(30, ...) 

when using Grid seq of function , can lprint work normally to get results into text ?

i notice Grid up to 57, do I need to recalculate and revise code to fit 96 vCPU for grid number 96?

Hi MaplePrimes team,

I am aware that Maple is not designed for CAD and I am not here for you to help me build any advanced system like the example below.

Figure 1: Trimmed surface of fuselage to create airplane windows.

Here I my problem. Even using the adaptmesh option, creating serious trimmed surfaces requires large numbers of grids like for example [3000, 3000] and even beyond, which makes a mesh unnecessary on most of the surface and especially where we don’t need high resolution.

While if we limit the display volume with the Maple’s view command (view = [x₁..x₂, y₁..y₂, z₁..z₂]), the edges are always smooth regardless of the coordinates system or the type of mesh of the surface to be treated.

And so, the reason I’m writing this message is to ask you why creating trimmed surface with plot3d surface wouldn’t use Maple view’s algorithm to get smooth edges?

Future Alternative Solution:

I also take this opportunity to ask you when would it be possible to update the Maple’s grid option such that the point space will variable in term of surface variable. For example:

plot3d(<x(s,t), y(s,t), z(s,t)>, s = s₁..s₂, t = t₁..t₂, grid = [m(s,t), n(s,t)] )

I am sure that this solution, which for me does not seem to be difficult to implement, could be an alternative to enrich the mesh where we really need it and therefore a better control. What do you think?

Thank you in advance.

Best.

Guy.

Is possible to operate Grid Computing Toolbox together with Maple on one local multicore machine?

If yes, what is the proper instalation/configuration procedure on Windows or Linux?

Is there a way to use the Button Component to stop a maple computation? I have two long functions running on nodes via grid in my app and I want to have them stopped via special button, is it possible to do in maple?

I have several questions about Grid Toolbox. I am getting into distributed computing for a very large problem I have been working on for a long time; I am working on the simple examples that come with the Grid package, and I am not having much success. I am fairly experienced in Maple but I need help with this. My e-mail is harrisjtster@gmail.com.

Hi, I would like to parallelize the double for loop. I am computing pairwise SPolynomials like so

for i from 1 to n-1 do

for j from i+1 to ndo

spol:=Spol(poly[i],poly[j], tdeg(op(vars))):

...

od:

od: 

I would like to parallelize this code, here is what I have done:

at_node:=proc(p1,p2,vars, polynomials)

spol:=SPolynomial(p1,p2,tdeg(op(vars))):

if NormalForm(spol, polynomials, tdeg(op(vars)))=0

return [1,0]:

fi:

end proc:

Grid[Setup]("local", num_nodes=4):

Grid[Set](at_node): # below is the main loop that fails

for idx from 1 to n-1 do

out:=Grid[Seq](at_node(polynomials[idx], polynomials[j], vars, polynomials) j=idx+1, n):

od:

The loop above fails due to the error when calling normal form. It seems that the at_node function accepts an incorrect input for p1, a list of polynomials instead of a single polynomial. Is there a way to parallelize double for-loop with Grid like that? I am not sure where my error is.

 I recently tried using Maple 2020 to run example code for the Grid package provided in Section 15.8 of the Maple Programming Guide. Unfortunately, none of the sample code appears to work as advertised.

Firstly, when I try to run the first sequential Mandelbrot example provided, I get the following error:

Error, (in Mandelbrot:-MandelLoop) `break` outside of loop

This was easily fixed by replacing the break statement in the while with a boolean value flag, while k < iter and flag do (see attached image). With this minor change to the example provided in the manual, I was able to get the code sequential code to run in 45.743 seconds on a quad core laptop. The modifications to the sample code are highlighted in red below.

 MandelLoop := proc( X, Y, imageArray, i_low, i_high, j_low, j_high, iter, bailout )
        local flag, i, j, Xc, Yc, Xtemp, Ytemp, Xold, Yold, k, t;
        option hfloat;

        for i from i_low to i_high do
           for j from j_low to j_high do
               Xtemp := X[i];
               Ytemp := Y[j];
               Xc := Xtemp;
               Yc := Ytemp;
               k := 0;
               flag := true:
               while k < iter and flag do
                   Xold := Xtemp;
                   Yold := Ytemp;
                   Xtemp := Xold^2-Yold^2+Xc;
                   Ytemp := 2*Xold*Yold+Yc;
                   t := Xtemp^2+Ytemp^2;
                   if Xtemp^2+Ytemp^2 >= bailout then
                        imageArray[i, j, 1] := k - ln( ln( t ) )/ln(2.);
                        imageArray[i, j, 2] := imageArray[i, j, 1];
                        imageArray[i, j, 3] := imageArray[i, j, 1];
                        #break;
                        flag := false:
                  end if;
                  k := k+1;
               end do
           end do;
        end do;
    end proc:

The second, and seemingly more serious problem occurred when I tried to run the code in the last example that uses the Client/Server model to parallelise the Mandelbrot calculation. The code provided in the Programming Guide once again gives the break outside of loop error. However, after replacing the break statement in the Computeline procedure with a boolean flag as above, I found the code runs but does not enter either the Server procedure or the Computeline procedure that is actually supposed to do the work. This can be verified by including print statements in the Computeline or Server procedures. The parallelised code ran in less than one second, compared to 45 seconds for the sequential code. Unfortunately this was not due to a 45-fold speed up on my quad core laptop, but the fact that the code using the code parallelised using Grid Package did nothing whatsoever.

I have sent a number of e-mail requests to Maple Support asking them to explain these apparent errors and so far have received no response.

Any solutions to these problems from the Maple user community would be much appreciated.

Thanks.

John

Dear All,

I want to apply the ‘simplify’ command, in parallel, for the simplification of some parameters. Both Grid:-Map and Grid:-Run commands are tested. There is no error in both, whereas no simplification is implemented. It seems that the ‘simplify’ command correctly works on only ‘Master’ node, namely anywhere we are typing.

Can anyone help me to simplify in parallel. I examined two following codes.

1)

with (Grid);

for i from 1 to nops(dummy_UU1) do

freenode:=WaitForFirst():

Run(freenode,simplify,[dummy_UU1[i]],assignto='dummy_UU2'[i]):

end do:

Wait();

2)

dummy_UU2:=Map[tasksize=1](simplify,[seq(dummy_UU1[i],i=1..nops(dummy_UU1))]):

The following code is correctly executed and resulted in the simplification of dummy_UU1 components in serial.

for i from 1 to nops(dummy_UU1) do

dummy_UU2[i]:=simplify(dummy_UU1[i]):

end do:

Dear everyone,

I want to execute a simple example by using Grid:-Send and Grid:-Receive commands. I have two questions:
1) Why does the Grid:-GetLastResult(0) command not return anything?

2) I tried to send the value of a, namely 2, from node 0 to node 1. It seems the execution is not stopped by Grid:-Send(1,a) and a_1:=Grid:-Receive(0) while CPU is not working.
Would anyone guide me?

The answer to the second question is more important to me.

Best wishes

restart;

Grid:-Run(0,"a:=2;");

Grid:-GetLastResult(0);

Grid:-Get(0,a);

Grid:-Send(1,a);

a_1:=Grid:-Receive(0);