[go: up one dir, main page]

0% found this document useful (0 votes)
151 views72 pages

System Coupling Participant Library

System_Coupling_Participant_Library

Uploaded by

joesph killer
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
151 views72 pages

System Coupling Participant Library

System_Coupling_Participant_Library

Uploaded by

joesph killer
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 72

System Coupling Participant Library

ANSYS, Inc. Release 2020 R1


Southpointe January 2020
2600 ANSYS Drive
Canonsburg, PA 15317 ANSYS, Inc. and
ansysinfo@ansys.com ANSYS Europe,
Ltd. are UL
http://www.ansys.com registered ISO
(T) 724-746-3304 9001: 2015
(F) 724-514-9494 companies.
Copyright and Trademark Information

© 2020 ANSYS, Inc. Unauthorized use, distribution or duplication is prohibited.

ANSYS, ANSYS Workbench, AUTODYN, CFX, FLUENT and any and all ANSYS, Inc. brand, product, service and feature
names, logos and slogans are registered trademarks or trademarks of ANSYS, Inc. or its subsidiaries located in the
United States or other countries. ICEM CFD is a trademark used by ANSYS, Inc. under license. CFX is a trademark
of Sony Corporation in Japan. All other brand, product, service and feature names or trademarks are the property
of their respective owners. FLEXlm and FLEXnet are trademarks of Flexera Software LLC.

Disclaimer Notice

THIS ANSYS SOFTWARE PRODUCT AND PROGRAM DOCUMENTATION INCLUDE TRADE SECRETS AND ARE CONFID-
ENTIAL AND PROPRIETARY PRODUCTS OF ANSYS, INC., ITS SUBSIDIARIES, OR LICENSORS. The software products
and documentation are furnished by ANSYS, Inc., its subsidiaries, or affiliates under a software license agreement
that contains provisions concerning non-disclosure, copying, length and nature of use, compliance with exporting
laws, warranties, disclaimers, limitations of liability, and remedies, and other provisions. The software products
and documentation may be used, disclosed, transferred, or copied only in accordance with the terms and conditions
of that software license agreement.

ANSYS, Inc. and ANSYS Europe, Ltd. are UL registered ISO 9001: 2015 companies.

U.S. Government Rights

For U.S. Government users, except as specifically granted by the ANSYS, Inc. software license agreement, the use,
duplication, or disclosure by the United States Government is subject to restrictions stated in the ANSYS, Inc.
software license agreement and FAR 12.212 (for non-DOD licenses).

Third-Party Software

See the legal information in the product help files for the complete Legal Notice for ANSYS proprietary software
and third-party software. If you are unable to access the Legal Notice, contact ANSYS, Inc.

Published in the U.S.A.


Table of Contents
Introduction to the System Coupling Participant Library ............................................................................ 1
System Coupling Participant Library Developer's Guide ........................................................................... 3
Participant Steps in a Coupled Analysis ................................................................................................... 4
Command Line Arguments for Participant Solvers ................................................................................... 6
Coupling-Specific Command Line Arguments ................................................................................... 6
Parallel Execution-Specific Command-Line Arguments ...................................................................... 7
Other Command-Line Arguments ..................................................................................................... 8
Execution in a Parallel Environment ......................................................................................................... 8
Using a Supported MPI Distribution .................................................................................................. 8
Implementing the Interfaces in Parallel ....................................................................................... 9
Running the Participant in Parallel ............................................................................................. 10
Using an Unsupported MPI Distribution .......................................................................................... 10
Creating System Coupling Participant Setup Files Using the Participant Library ...................................... 10
Access to Heavyweight Data ................................................................................................................. 12
Relevant Concepts .......................................................................................................................... 13
Supported Heavyweight Data Formats ............................................................................................ 13
Data Access Structures .................................................................................................................... 15
Scalar Data Structure Examples ................................................................................................. 16
Vector Data Structure Examples ................................................................................................ 17
Other Data Structure Examples ................................................................................................. 23
Data Access Functions .................................................................................................................... 25
Creating Restart Points and Restarting a Coupled Analysis ..................................................................... 25
Setting Up for Restarts .................................................................................................................... 26
Creating Restart Points ................................................................................................................... 26
Restarting the Analysis from a Restart Point ..................................................................................... 27
Multi-Region Coupling Interfaces .......................................................................................................... 28
Mesh Definition for Multi-Region Coupling Interfaces ...................................................................... 28
Variable Field Data for Multi-Region Coupling Interfaces .................................................................. 32
Debugging Tools .................................................................................................................................. 34
Standalone Mode ........................................................................................................................... 34
Mesh Validity Check ........................................................................................................................ 35
Migration Guide ................................................................................................................................... 36
Migrating from Pre-2020 R1 Releases .............................................................................................. 36
Compiling, Linking, and Executing Applications That Use the Participant Library .................................... 37
Supported Languages .................................................................................................................... 38
System Coupling Participant Library Resources ............................................................................... 38
Examples ........................................................................................................................................ 39
Linux Examples ......................................................................................................................... 40
Windows Examples ................................................................................................................... 43
System Coupling Participant Library Tutorials ......................................................................................... 47
Heat Transfer in Square Channel Air Flow Tutorial .................................................................................. 47
Verify Prerequisites ......................................................................................................................... 47
Problem Description and Participant Setup ..................................................................................... 48
Get the Necessary Files ................................................................................................................... 50
Part 1: Serial Version of the Mock Solver ........................................................................................... 52
Build the Mock Solver ............................................................................................................... 52
Run Mock Solver in Standalone Mode ....................................................................................... 52
Perform the Coupled Analysis Using the Mock Solver ................................................................ 53
Visualize the Results in CFD-Post ............................................................................................... 53
Part 2: The Parallel Version of the Mock Solver .................................................................................. 54

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. iii
System Coupling Participant Library

Parallel Mock Solver Source Code .............................................................................................. 55


Build the Parallel Mock Solver ................................................................................................... 56
Run the Parallel Mock Solver in Standalone Mode ...................................................................... 56
Perform the Coupled Analysis Using the Parallel Mock Solver ..................................................... 56
Transient Oscillating Plate Damping Tutorial .......................................................................................... 57
Verify Prerequisites ......................................................................................................................... 57
Problem Description and Participant Setup ..................................................................................... 57
Get the Necessary Files ................................................................................................................... 59
Build the Damping Solver ............................................................................................................... 60
Run the Coupled Analysis ............................................................................................................... 62
Visualize the Results in CFD-Post ..................................................................................................... 62
Restart the Coupled Analysis from an Intermediate Point ................................................................. 64

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
iv of ANSYS, Inc. and its subsidiaries and affiliates.
List of Figures
1. Sequence of participant steps for a steady analysis ..................................................................................... 5
2. Sequence of participant steps for a transient analysis ................................................................................. 6
3. Fluent geometry ...................................................................................................................................... 49
4. Mock solver geometry ............................................................................................................................. 50
5. Fluent temperature results after co-simulation with the mock solver ......................................................... 54
6. Structural solver geometry ...................................................................................................................... 58
7. Surface mesh in the damping solver ........................................................................................................ 59
8. Plot of plate tip x-displacements over time for different damping coefficients ........................................... 64
9. Comparison of results between the continuous and restarted runs ........................................................... 65

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. v
Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
vi of ANSYS, Inc. and its subsidiaries and affiliates.
Introduction to the System Coupling Participant Library
The System Coupling Participant Library includes application programming interfaces (APIs) that allow
for the connection of solvers to System Coupling.

Note:

"System Coupling" refers to the current version of the System Coupling application, which
in previous releases was designated as "System Coupling 2.0."

Prerequisites
This documentation assumes that you've met the following prerequisites:

You're familiar with basic functionality available in System Coupling. Before you start working
with the System Coupling Participant Library, it is recommended that you first:

• Complete the following command-line tutorials from the Tutorials with Command-Line Interface (CLI)
Workflows page of the System Coupling Tutorials package:

– Oscillating Plate Two-Way Fluid-Structure Interaction

– Coil-and-Core Induction Heating Co-Simulation

– Electromagnetic-Thermal Bus Bar Co-Simulation

– Electric Motor Co-Simulation

• Review the System Coupling User's Guide.

• Ensure that you become familiar with System Coupling's data model and commands, as described in
the System Coupling Settings and Commands Reference.

You're a participant solver developer, with some programming experience in one of the suppor-
ted languages (p. 38) and the ability to modify the participant solver build process. These APIs
are intended to be connected with the participant solver inside the solver iteration loop, so you must
include the System Coupling Participant Library as a dependency.

Contents
For details on the System Coupling Participant Library's functionality and usage, see the following
documentation:

• System Coupling Participant Library Developer's Guide (p. 3)

• System Coupling Participant Library Tutorials (p. 47)

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 1
Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
2 of ANSYS, Inc. and its subsidiaries and affiliates.
System Coupling Participant Library Developer's Guide
System Coupling provides a co-simulation solver to perform multiphysics analyses. The System Coupling
Participant Library provides interfaces that allow coupling participant solvers to connect to the co-
simulation solver and to take part in the coupled analysis.

• Each coupling participant (also referred to as "participant," "participant solver," or "solver") is an instance
of a standalone application that provides a solution to one or more physical models.

• A coupled analysis is the process in which the coupled participants are executed in an iterative sequence
for the purpose of converging the overall coupled solution. A coupled analysis is controlled by the
System Coupling application, rather than the participants.

Participants declare which regions can be used in the coupled analysis. Each region can declare input
variables and output variables.

• Input variables represent physical quantities for which the participant expects System Coupling to
provide values.

• Output variables represent physical quantities for which System Coupling expects the participant to
provide values.

Note:

• Currently, only surface regions are supported in the participant library.

• All values are assumed to be in SI units.

Once the participant solver is instrumented with System Coupling APIs, it can participate in a coupled
analysis with other coupled participants, taking advantage of all the functionality provided by System
Coupling.

For more information, see:


Participant Steps in a Coupled Analysis
Command Line Arguments for Participant Solvers
Execution in a Parallel Environment
Creating System Coupling Participant Setup Files Using the Participant Library
Access to Heavyweight Data
Creating Restart Points and Restarting a Coupled Analysis
Multi-Region Coupling Interfaces
Debugging Tools
Migration Guide
Compiling, Linking, and Executing Applications That Use the Participant Library

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 3
System Coupling Participant Library Developer's Guide

Participant Steps in a Coupled Analysis


During a coupled analysis, the following sequence of steps are traversed:

1. Connection to System Coupling

The connection is established by providing the host name, port number, and participant name.

2. Heavyweight data access and initialization

Register access to the participant's mesh and variable values. Initialize the heavyweight data.

3. Coupled analysis initialization

Notify System Coupling that the analysis can be initialized. After this step, additional data access
functions can no longer be registered.

4. Coupled analysis loop

Enter a coupled analysis loop until the analysis is complete. This step is different, depending on
whether the analysis is steady or transient.

• Steady analysis

For a steady analysis, there is the coupling iteration loop. The participant should enter the
coupling iteration loop and exit it only when there are no more coupling iterations to be
done.

• Transient analysis

For a transient analysis, there are two nested loops:

– The outer loop is the coupling time step loop. The participant should enter the coupling time
step loop and exit it only when there are no more coupling time steps to be done.

– The inner loop is the coupling iteration loop. The participant should enter the coupling iteration
loop and exit it only when there are no more coupling iterations to be done within the current
coupling time step.

Inside the coupling iteration loop, the participant should iteratively update inputs, perform its
solver iterations, and then update outputs.

a. Inputs update

Notify System Coupling that inputs can be updated. After this step, the inputs are up-to-date
for the current solver iteration. The participant solver can perform its solver iterations after
the inputs update.

b. Outputs update

Notify System Coupling that outputs are updated and provide solver convergence status. Before
updating the outputs, the participant must update all output data so that it can be consumed
by System Coupling.

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
4 of ANSYS, Inc. and its subsidiaries and affiliates.
Participant Steps in a Coupled Analysis

5. Coupled analysis shutdown

This step is reached after exiting from the coupled analysis loop. System Coupling will be discon-
nected. The solver should proceed to an orderly shutdown after this step.

This process is depicted in Figure 1: Sequence of participant steps for a steady analysis (p. 5) and Fig-
ure 2: Sequence of participant steps for a transient analysis (p. 6).

Figure 1: Sequence of participant steps for a steady analysis

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 5
System Coupling Participant Library Developer's Guide

Figure 2: Sequence of participant steps for a transient analysis

Command Line Arguments for Participant Solvers


System Coupling starts participant solvers as child processes at the beginning of a coupled analysis.
Therefore, during the coupled analysis setup, participant solver executable will need to be set inside
System Coupling. See the System Coupling User's Guide for more details.

It is recommended that you create a wrapper script (either batch script, shell script, or any other kind
of script) around the actual solver executable. This script can parse the arguments provided by System
Coupling and set any environment variables, as needed.

For more information, see:


Coupling-Specific Command Line Arguments
Parallel Execution-Specific Command-Line Arguments
Other Command-Line Arguments

Coupling-Specific Command Line Arguments


The following command line arguments will always be provided by System Coupling:

--schost HOSTNAME
Name of the host where System Coupling is running. This value should be passed in as input into the
participant library function that establishes the connection with System Coupling.

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
6 of ANSYS, Inc. and its subsidiaries and affiliates.
Command Line Arguments for Participant Solvers

--scport PORTNUMBER
Port number on which System Coupling is listening for incoming connections from the participant
solvers. This value should be passed in as input into the participant library function that establishes the
connection with System Coupling.

--scname PARTICIPANTNAME
Unique name for this participant, given by System Coupling. This value should be passed in as input into
the participant library function that establishes the connection with System Coupling.

--screstart RESTART_STRING
This command-line argument is used only during a restart run. The provided string is used to tell the
participant solver which restart point to use. During a previous run, the participant solver provided this
string to System Coupling. For details, see Creating Restart Points and Restarting a Coupled Analys-
is (p. 25).

Below is an example of an execution command that System Coupling may generate:

solver.exe --schost host1 --scport 12345 --scname DEFAULT-1

Parallel Execution-Specific Command-Line Arguments


For a coupled analysis, System Coupling typically allocates computing resources for each participant.
For a participant that uses the participant library, System Coupling will pass available computing re-
sources via a command-line argument when starting the participant.

For more information, see Execution in a Parallel Environment (p. 8).

Local parallel execution:


To enable local parallel execution, the following command line argument will be passed to the parti-
cipant:

-np NPROCS

Here, NPROCS is replaced with the number of processes. For example, if the participant solver is to
be executed on local machine using four processes, the command line argument will be as follows:

-np 4

Distributed parallel execution:


To enable distributed parallel execution, the following command line argument will be passed to the
participant:

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 7
System Coupling Participant Library Developer's Guide

-hostlist HOSTLIST

Here, HOSTLIST is replaced with the comma-separated list of machines and number of cores available
on that machine. For example, if there are two machines available named hostA and hostB, and
each machine has four cores available, then the command line argument will be as follows:

-hostlist hostA:4,hostB:4

Other Command-Line Arguments


If other command line arguments need to be passed to the participant solver, this can be accomplished
by adding those arguments in the ExecutionControl.AdditionalArguments parameter in
the System Coupling data model. For example, the following snippet sets the custom command line
argument for participant DEFAULT-1 in the System Coupling script:

participant = DatamodelRoot().CouplingParticipant['DEFAULT-1']
participant.ExecutionControl.AdditionalArguments = '--input input.file'

For more information on the data model structure, see the System Coupling User's Guide.

Execution in a Parallel Environment


The System Coupling Participant Library supports execution of the participant solver in a parallel Message
Passing Interface (MPI) environment.

The participant library uses Fluent's MPI wrapper library for run-time switching between the compatible
MPI distributions. For a full list of available MPIs for the Windows and Linux platforms, see the Fluent
product documentation. The participant library is expected to be compatible with those MPIs.

The second part of the Heat Transfer in Square Channel Air Flow (p. 47) tutorial demonstrates the usage
of the parallel APIs.

For more information, see:


Using a Supported MPI Distribution
Using an Unsupported MPI Distribution

Using a Supported MPI Distribution


For information on executing in a parallel environment using a supported MPI distribution, see:
Implementing the Interfaces in Parallel
Running the Participant in Parallel

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
8 of ANSYS, Inc. and its subsidiaries and affiliates.
Execution in a Parallel Environment

Implementing the Interfaces in Parallel


If the participant solver is using one of the supported MPIs, then the relevant MPI communicator
should be provided when establishing the connection to System Coupling. The participant library
interfaces should be implemented in each rank of that communicator. The participant library will
manage all parallel processing of mesh and variable data internally. If some elements and/or nodes
are duplicated in multiple partitions (e.g. due to the mesh overlap layer), then the participant library
can be expected to handle such scenarios robustly.

The following sections provide examples of using the participant library to implement interfaces
in parallel:

C++

...
// initialize the MPI
MPI_Init(&argc, &argv);
...
std::string scHost, scname, buildInfo;
unsigned short int scPort;
...
// pass the MPI communicator when connecting to System Coupling
sysc::SystemCoupling sc(scHost, scPort, scName, MPI_COMM_WORLD, buildInfo));
...

...
/* initialize the MPI */
MPI_Init(&argc, &argv);
...
const char* scHost, scName, buildInfo;
unsigned short int scPort;
...
/* pass the MPI communicator when connecting to System Coupling */
syscConnectParallel(scHost, scPort, scName, MPI_COMM_WORLD, buildInfo);
...

Fortran

...
integer :: err_code
character(len=256) :: scHost
integer :: scPort
character(len=256) :: scName
character(len=256) :: buildInfo
...
! initialize the MPI
call MPI_Init(err_code)
...
! pass the MPI communicator when connecting to System Coupling
syscConnectF(scHost, scPort, scName, MPI_COMM_WORLD, buildInfo)
...

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 9
System Coupling Participant Library Developer's Guide

Running the Participant in Parallel


In order to link to the correct MPI wrapper at run-time, the path to the correct wrapper directory
should be set in the PATH environment variable on Windows or LD_LIBRARY_PATH environment
variable on Linux. For example, if the participant solver is using Intel MPI library, then those envir-
onment variables should contain the following path:

<Ansys Installation Path>/SystemCoupling/runTime/<platform>/cnlauncher/fluent/fluent<version>/


multiport/mpi_wrapper/<platform>/intel

This will link to the MPI wrapper library in that directory (mpi_wrapper.dll on Windows and
libmpi_wrapper.so on Linux).

If, for example, the IBM MPI library is used, then the ibmmpi directory should be used instead of
intel in the path above.

For examples of compiling, linking, and executing a program that uses the participant library and
a supported MPI, see Compiling, Linking, and Executing Applications That Use the Participant Lib-
rary (p. 37).

Using an Unsupported MPI Distribution


If the participant solver is using an MPI distribution that is not supported by the Fluent's MPI wrapper
library, then all participant library interfaces should be implemented/used only on one process (e.g.,
process with rank 0). The participant solver should manage all parallel collection of mesh and variable
data for access by the participant library. In this case, the stub MPI wrapper should be used at run-
time, which is located in the following directory:

<Ansys Installation Path>/SystemCoupling/runTime/<platform>


/cnlauncher/fluent/fluent<version>/multiport/mpi_wrapper/<platform>/stub

For more detailed examples, see Compiling, Linking, and Executing Applications That Use the Participant
Library (p. 37).

Creating System Coupling Participant Setup Files Using the Participant


Library
System Coupling Participant (SCP) setup files are XML-formatted files that contain participant setup
information that is relevant to System Coupling. More information about them can be found in the
System Coupling User's Guide.

The setup file should be generated during the participant solver setup and prior to starting System
Coupling.

The participant library provides a utility to create the setup files via a function called writeSetupFile
so that the library can be used during the setup stage, as well as during the coupled analysis stage.

The following sections provide examples of using the participant library to create the setup files:

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
10 of ANSYS, Inc. and its subsidiaries and affiliates.
Creating System Coupling Participant Setup Files Using the Participant Library

C++
In C++, create a sysc::SystemCoupling instance and provide an empty host name.

After that, add regions and variables.

After the completeCouplingSetup method is called, the setup file can be created using the
writeSetupFile method, which that takes in the sysc::SetupFileInfo structure.

// create struct with setup file information


sysc::SetupFileInfo setupFileInfo("example.scp");

// start write mode


sysc::SystemCoupling sc("", 0, "test");

// add region(s) and variable(s)


sysc::Variable variable("variable", sysc::Temperature);
sysc::Region region("region", sysc::Surface);
region.addOutputVariable(variable);
sc.addRegion(region);

// complete coupling setup


sc.completeCouplingSetup(sysc::Transient);

// write the setup file


bool isWritten = sc.writeSetupFile(setupFileInfo);

C
In C, start standalone mode by passing an empty host name to the syscConnect function.

After that, add regions and variables.

After the syscCompleteCouplingSetup function is called, the setup file can be created using the
syscWriteSetupFile function, which takes in the SyscSetupFileInfo structure.

/* create struct with SCP write information */


SyscSetupFileInfo setupFileInfo;
setupFileInfo.setupFileName = "example.scp";

/* start standalone mode */


SyscError ret;
ret = syscConnect("", 0, "test");

/* add region(s) and variable(s) */


SyscRegion region;

region.name = "region";
region.topology = SyscSurface;
ret = syscAddRegion(region);

SyscVariable variable;
variable.name = "variable";
variable.quantityType = SyscTemperature;
variable.location = SyscNode;
ret = syscAddInputVariable(region, variable);

/* complete coupling setup */


ret = syscCompleteCouplingSetup();

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 11
System Coupling Participant Library Developer's Guide

/* write the setup file */


int isWritten = syscWriteSetupFile(setupFileInfo);

Fortran
In Fortran, start standalone mode by passing an empty host name to the syscConnectF function.

After that, add regions and variables.

After the syscCompleteCouplingSetupF function is called, the setup file can be created using
the syscWriteSetupFileF function, which takes in the SyscSetupFileInfoF type.

type(SyscErrorF) :: ret
type(SyscSetupFileInfoF) :: setupFileInfo
type(SyscRegionF) :: region
type(SyscVariableF) :: variable
logical :: isWritten

setupFileInfo%setupFileName = "example.scp"

! start standalone mode


ret = syscConnectF("", 0, "test")

! add region(s) and variable(s)

region%regionName = "FSIN_1"
region%topology = SyscSurface

variable%variableName = "TEMP"
variable%quantityType = SyscTemperature
variable%location = SyscNode

ret = syscAddRegionF(region)
ret = syscAddOutputVariableF(region, variable)

! complete coupling setup


ret = syscCompleteCouplingSetupF(analysisType)

! write the setup file


isWritten = syscWriteSetupFileF(setupFileInfo)

Access to Heavyweight Data


Heavyweight data is data representing mesh and variable field values. Often, solvers store heavyweight
data in contiguous arrays in memory, and the participant library design takes advantage of this. The
participant solver provides heavyweight data access to the participant library, but retains ownership of
all heavyweight data (i.e., it is responsible for allocation and deallocation of those arrays).

For more information, see:


Relevant Concepts
Supported Heavyweight Data Formats
Data Access Structures
Data Access Functions

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
12 of ANSYS, Inc. and its subsidiaries and affiliates.
Access to Heavyweight Data

Relevant Concepts
Input data
Input data is for input variables provided to the participant by System Coupling. The participant library
needs to have read and write access to this data.

Output data
Output data is for output variables provided to System Coupling by the participant, including mesh
data. The participant library needs to have read-only access to this data.

Scalar
Scalar data has one component. Temperature values are an example of scalar data.

Vector
Vector data has three components. Nodal coordinates values are an example of vector data.

Data access function


A data access function is a callback function implemented by the participant solver and registered with
the participant library. The participant library will call this function when it needs to access the heavy-
weight data. All required data access functions must be registered prior to coupled analysis initializa-
tion (p. 4).

Supported Heavyweight Data Formats


The compatible formats for heavyweight data are scalar arrays and vector arrays.

Scalar Arrays

64-bit floating point arrays (for scalar variable field values)

• C/C++ double

• FORTRAN real(kind=8)

32-bit floating point arrays (for scalar variable field values)

• C/C++ float

• FORTRAN real(kind=4)

32-bit signed integer arrays (for mesh connectivity information)

• C/C++ int

• FORTRAN integer(kind=4)

64-bit signed integer arrays (for mesh connectivity information)

• C/C++ long long int

• FORTRAN integer(kind=8)

16-bit unsigned integer arrays (for mesh connectivity information)

• C/C++ unsigned short int

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 13
System Coupling Participant Library Developer's Guide

64-bit unsigned integer arrays (for mesh connectivity information)

• C/C++ unsigned long long int

Vector Arrays

32-bit floating point arrays (for vector variable field values and for mesh node coordinates)

• C/C++ float

• FORTRAN real(kind=4)

64-bit floating point arrays (for vector variable field values and for mesh node coordinates)

• C/C++ double

• FORTRAN real(kind=8)

Different layouts of vector data are supported:

Compact layout

(e.g., a single array storing each vector, one after another)

• [x0, y0, z0, x1, y1, z1, …, xN, yN, zN]

Split layout

(e.g., three separate arrays, one for each vector component)

• [x0, x1, .., xN]

• [y0, y1, .., yN]

• [z0, z1, .., zN]

Note:

• If vector data is stored in a single array where the vector components are separated (e.g.,
[x0, x1, ..., y0, y1, ..., z0, z1, zN]), then it can be handled as a special
case of split layout.

• If vector data is stored in a two-dimensional array, then it can be handled as a special case
of one of the above layouts.

If the participant solver stores heavyweight data in a format compatible with the participant library,
then this reduces the need to make unnecessary copies of the data and thus improves memory effi-
ciency during the coupled analysis. The participant solver should implement and register data access
functions that provide direct access to that heavyweight data.

If the data is stored in a format that is not directly compatible with the participant library, then the
participant solver should create compatible heavyweight data structures and register data access
functions that provide access to that heavyweight data. In such a case, the participant solver should
make sure that this heavyweight data and the original data are synchronized at certain steps during
the coupled analysis (p. 4):

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
14 of ANSYS, Inc. and its subsidiaries and affiliates.
Access to Heavyweight Data

• Before the coupled analysis initialization is entered, the heavyweight input and output data are
expected to be available. Output data, including the mesh data, should contain valid initial values.

• After the inputs update completes, all input variable values will be up-to-date in the heavyweight
data. The participant should make sure that the latest input values are available in the original data
structures before doing solver iterations

• Before the outputs update is entered, the participant should make sure that the heavyweight data
contains the updated values for all the output variables, as well as for the mesh.

Data Access Structures


Access to heavyweight data is provided by creating data access data structures and implementing
data access functions (callback functions) that provide these structures to the participant library.

This section discusses the following data access structures:

• OutputScalarData

• OutputVectorData

• InputScalarData

• InputVectorData

• SurfaceMesh

The SurfaceMesh data access structure groups the following set of output data objects that
describe surface mesh:

– OutputScalarData for node ids

– OutputVectorData for node coordinates

– OutputScalarData for element node counts

– OutputScalarData for element node ids

These structures require three pieces of information during their initialization (depending on the
target language, some of this information is deduced automatically and does not need to be explicitly
specified):

1. Primitive type of data (e.g., float, double, unsigned 16-bit integer, etc.)

2. Access to the heavyweight data (e.g., array pointers).

3. Extents of the data (e.g., array sizes)

The following examples show how to create data access structures to provide access to heavyweight
data to the participant library:
Scalar Data Structure Examples
Vector Data Structure Examples
Other Data Structure Examples

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 15
System Coupling Participant Library Developer's Guide

Scalar Data Structure Examples


The following sections provide examples of scalar data structures:
Example: Input Scalar Data
Example: Output Scalar Data

Example: Input Scalar Data


In this example, the participant solver has values of an input scalar variable stored in memory as a
double-precision array. To provide access to this variable, create the InputScalarData structure
as shown below.

C++

// heavyweight array(s)
double temperature[] = { 1.0, 2.0, 3.0, 4.0 };
std::size_t numScalars = 4;

// creating corresponding data access structure(s)


sysc::InputScalarData temperatureData(temperature, numScalars);

/* heavyweight array(s) */
double temperature[] = { 1.0, 2.0, 3.0, 4.0 };
size_t numScalars = 4;

/* creating corresponding data access structure(s) */


SyscInputScalarData temperatureData;
temperatureData.primitiveType = SyscDouble;
temperatureData.data = temperature;
temperatureData.size = numScalars;

Fortran

For Fortran, a helper function syscGetInputScalarDataF is provided for this case.

! heavyweight array(s)
real(kind=8), target :: temperature(4) = (/ 1.0, 2.0, 3.0, 4.0 /)

! creating corresponding data access structure(s)


type(SyscInputScalarDataF) :: temperatureData
temperatureData = syscGetInputScalarDataF(temperature)

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
16 of ANSYS, Inc. and its subsidiaries and affiliates.
Access to Heavyweight Data

Example: Output Scalar Data


In this example, the participant solver has values of an output scalar variable stored in memory as
a single-precision array. To provide access to this variable, create the OutputScalarData structure
as shown below.

C++

// heavyweight array(s)
float heatflow[] = { 1.0, 2.0, 3.0, 4.0 };
std::size_t numScalars = 4;

// creating corresponding data access structure(s)


sysc::OutputScalarData heatflowData(heatflow, numScalars);

/* heavyweight array(s) */
float heatflow[] = { 1.0, 2.0, 3.0, 4.0 };
size_t numScalars = 4;

/* creating corresponding data access structure(s) */


SyscOutputScalarData heatflowData;
heatflowData.primitiveType = SyscFloat;
heatflowData.data = heatflow;
heatflowData.size = numScalars;

Fortran

For Fortran, a helper function syscGetOutputScalarDataF is provided for this case.

! heavyweight array(s)
real(kind=4), target :: heatflow(4) = (/ 1.0, 2.0, 3.0, 4.0 /)

! creating corresponding data access structure(s)


type(SyscOutputScalarDataF) :: heatflowData
heatflowData = syscGetOutputScalarDataF(heatflow)

Vector Data Structure Examples


Vector data storage can be categorized as either compact or split storage. The following sections
provide examples for both categories:
Example: Compact Input Vector Data In a One-Dimensional Array
Example: Compact Output Vector Data In a Two-Dimensional Array
Example: Split Input Vector Data In Separate Arrays
Example: Split Output Vector Data In a Two-Dimensional Array
Example: Split Output Vector Data In a One-Dimensional Array

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 17
System Coupling Participant Library Developer's Guide

Example: Compact Input Vector Data In a One-Dimensional Array


In this example, the participant solver has values of an input vector variable stored in memory as
a compact one-dimensional array. To provide access to this variable, create the InputVectorData
structure as shown below.

Note that the size provided is 10 — there are 10 vectors, each with 3 components. So, the total
size of the array in memory is 30, but the size passed in here must be 10, not 30.

C++

// heavyweight array(s)
double displacement[] = { 0, 0, 0, 0.25, 0, 0, 0.5, 0, 0, 0.75, 0, 0, 1, 0, 0, 0, 0.1, 0, 0.25, 0.1,
0, 0.5, 0.1, 0, 0.75, 0.1, 0, 1, 0.1, 0 };
std::size_t numVectors = 10;

// creating corresponding data access structure(s)


sysc::InputVectorData displacementData(displacement, numVectors);

/* heavyweight array(s) */
double displacement[] = { 0, 0, 0, 0.25, 0, 0, 0.5, 0, 0, 0.75, 0, 0, 1, 0, 0, 0, 0.1, 0, 0.25, 0.1,
0, 0.5, 0.1, 0, 0.75, 0.1, 0, 1, 0.1, 0 };
size_t numVectors = 10;

/* creating corresponding data access structure(s) */


SyscInputVectorData displacementData;
displacementData.primitiveType = SyscDouble;
displacementData.data0 = displacement;
displacementData.data1 = NULL;
displacementData.data2 = NULL;
displacementData.size = numVectors;

Fortran

For Fortran, a helper function syscGetInputVectorDataCompactF is provided for this case.

! heavyweight array(s)
real(kind=8), target :: displacement(30) = (/ 0D0, 0D0, 0D0, 0.25D0, 0D0, 0D0, 0.5D0, 0D0, 0D0, 0.75D0, 0D0, 0
integer(kind=8) :: numVectors = 10

! creating corresponding data access structure(s)


type(SyscInputVectorDataF) :: displacementData
displacementData = syscGetInputVectorDataCompactF(displacement)

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
18 of ANSYS, Inc. and its subsidiaries and affiliates.
Access to Heavyweight Data

Example: Compact Output Vector Data In a Two-Dimensional Array


In this example, the participant solver has values of an output vector variable stored in memory as
a compact two-dimensional array. To provide access to this variable, create the OutputVectorData
structure as shown below.

Note that the size provided is 2 — there are 2 vectors, each with 3 components.

C++

// heavyweight array(s)
const std::size_t numVectors = 2;
const std::size_t numComponents = 3;
float force[numVectors][numComponents] = {{0.0, 0.0, 0.0}, {1.0, 1.0, 1.0}};

// creating corresponding data access structure(s)


sysc::OutputVectorData forceData(&force[0][0], numVectors);

/* heavyweight array(s) */
#define NUMVECTORS 2
#define NUMCOMPONENTS 3
float force[NUMVECTORS][NUMCOMPONENTS] = {{0.0, 0.0, 0.0}, {1.0, 1.0, 1.0}};

/* creating corresponding data access structure(s) */


SyscOutputVectorData forceData;
forceData.primitiveType = SyscFloat;
forceData.data0 = &force[0][0];
forceData.data1 = NULL;
forceData.data2 = NULL;
forceData.size = NUMVECTORS;

Fortran

For Fortran, a helper function syscGetOutputVectorDataCompactF is provided for this case.

Note that Fortran stores data in column-major order, so a compactly stored two-dimensional array
has component as the first index and vector as the second index.

! heavyweight array(s)
integer(kind=8), parameter :: numVectors = 2
integer(kind=4), parameter :: numComponents = 3
real(kind=4), target :: force(numComponents, numVectors)
force(1, 1) = 0D0
force(2, 1) = 0D0
force(3, 1) = 0D0
force(1, 2) = 1D0
force(2, 2) = 1D0
force(3, 2) = 1D0

! creating corresponding data access structure(s)


type(SyscOutputVectorDataF) :: forceData
forceData = syscGetOutputVectorDataCompactF(force)

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 19
System Coupling Participant Library Developer's Guide

Example: Split Input Vector Data In Separate Arrays


In this example, the participant solver has values of an input vector variable stored in memory as
three separate arrays, one array per component. To provide access to this variable, create the In-
putVectorData structure as shown below.

Note that the size provided is 4 — there are 4 vectors, each with 3 components.

C++

// heavyweight array(s)
double dx[] = { 1.0, 2.0, 3.0, 4.0 };
double dy[] = { 1.0, 2.0, 3.0, 4.0 };
double dz[] = { 1.0, 2.0, 3.0, 4.0 };
std::size_t numVectors = 4;

// creating corresponding data access structure(s) */


sysc::InputVectorData displacementData(dx, dy, dz, numVectors);

/* heavyweight array(s) */
double dx[] = { 1.0, 2.0, 3.0, 4.0 };
double dy[] = { 1.0, 2.0, 3.0, 4.0 };
double dz[] = { 1.0, 2.0, 3.0, 4.0 };
size_t numVectors = 4;

/* creating corresponding data access structure(s) */


SyscInputVectorData displacementData;
displacementData.primitiveType = SyscFloat;
displacementData.data0 = dx;
displacementData.data1 = dy;
displacementData.data2 = dz;
displacementData.size = numVectors;

Fortran

For Fortran, a helper function syscGetInputVectorDataSplitF is provided for this case.

! heavyweight array(s)
integer(kind=8), parameter :: numVectors = 4
real(kind=8), target :: dx(numVectors) = (/ 1.0, 2.0, 3.0, 4.0 /)
real(kind=8), target :: dy(numVectors) = (/ 1.0, 2.0, 3.0, 4.0 /)
real(kind=8), target :: dz(numVectors) = (/ 1.0, 2.0, 3.0, 4.0 /)

! creating corresponding data access structure(s)


type(SyscInputVectorDataF) :: displacementData
displacementData = syscGetInputVectorDataSplitF(dx, dy, dz)

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
20 of ANSYS, Inc. and its subsidiaries and affiliates.
Access to Heavyweight Data

Example: Split Output Vector Data In a Two-Dimensional Array


In this example, the participant solver has values of an output vector variable stored in memory as
a split two-dimensional array. To provide access to this variable, create the OutputVectorData
structure as shown below.

Note that the size provided is 2 — there are 2 vectors, each with 3 components.

C++

// heavyweight array(s)
const std::size_t numVectors = 2;
const std::size_t numComponents = 3;
double coords[numComponents][numVectors] = {{0.0, 1.0}, {0.0, 1.0}, {0.0, 1.0}};

// creating corresponding data access structure(s)


sysc::OutputVectorData coordsData(&coords[0][0], &coords[1][0], &coords[2][0], numVectors);

/* heavyweight array(s) */
#define NUMVECTORS 2
#define NUMCOMPONENTS 3
double coords[NUMCOMPONENTS][NUMVECTORS] = {{0.0, 1.0}, {0.0, 1.0}, {0.0, 1.0}};

/* creating corresponding data access structure(s) */


SyscOutputVectorData coordsData;
coordsData.primitiveType = SyscDouble;
coordsData.data0 = &coords[0][0];
coordsData.data1 = &coords[1][0];
coordsData.data2 = &coords[2][0];
coordsData.size = NUMVECTORS;

Fortran

For Fortran, a helper function syscGetOutputVectorDataSplitF is provided for this case.

Note that Fortran stores data in column-major order, so a non-compactly stored two-dimensional
array has the vector as the first index and the vector component as the second index.

! heavyweight array(s)
integer(kind=8), parameter :: numVectors = 2
integer(kind=4), parameter :: numComponents = 3
real(kind=8), target :: coords(numVectors, numComponents)
coords(1, 1) = 0D0
coords(1, 2) = 0D0
coords(1, 3) = 0D0
coords(2, 1) = 1D0
coords(2, 2) = 1D0
coords(2, 3) = 1D0

! creating corresponding data access structure(s)


type(SyscOutputVectorDataF) :: coordsData
coordsData = syscGetOutputVectorDataSplitF(coords)

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 21
System Coupling Participant Library Developer's Guide

Example: Split Output Vector Data In a One-Dimensional Array


In this example, the participant solver has values of an output vector variable stored in memory as
a split one-dimensional array. To provide access to this variable, create the OutputVectorData
structure as shown below.

Note that the size provided is 2 — there are 2 vectors, each with 3 components.

C++

// heavyweight array(s)
double force[] = {0.0, 1.0, 0.0, 1.0, 0.0, 1.0};
std::size_t numVectors = 2;

// creating corresponding data access structure(s)


sysc::OutputVectorData forceData(&force[0], &force[2], &force[4], numVectors);

/* heavyweight array(s) */
double force[] = {0.0, 1.0, 0.0, 1.0, 0.0, 1.0};
size_t numVectors = 2;

/* creating corresponding data access structure(s) */


SyscOutputVectorData forceData;
forceData.primitiveType = SyscDouble;
forceData.data0 = &force[0];
forceData.data1 = &force[2];
forceData.data2 = &force[4];
forceData.size = numVectors;

Fortran

! requires iso_c_binding
use iso_c_binding

! heavyweight array(s)
integer(kind=8) :: numVectors = 2
real(kind=8), target :: force(6) = (/ 0D0, 1D0, 0D0, 1D0, 0D0, 1D0 /)

! creating corresponding data access structure(s)


type(SyscOutputVectorDataF) :: forceData
forceData%primitiveType = SyscDouble
forceData%dataPtr1 = c_loc(force(1))
forceData%dataPtr2 = c_loc(force(3))
forceData%dataPtr3 = c_loc(force(5))
forceData%dataSize = numVectors

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
22 of ANSYS, Inc. and its subsidiaries and affiliates.
Access to Heavyweight Data

Other Data Structure Examples


The following sections provide examples of other types of data structures:
Example: Surface Mesh
Example Input Scalar in a Single STL Vector Container (C++ only)

Example: Surface Mesh


In this example, the participant solver has arrays representing surface mesh (unit square, single
quadrilateral mesh element) stored in different formats. To provide access to these arrays, create
the corresponding set of output data structures as shown below.

C++

// heavyweight array(s)
std::size_t numNodes = 4;
std::size_t numElems = 1;
std::size_t numElemNodeIds = 4;
long long int nodeIds[] = { 1, 2, 3, 4 };
double x[] = { 0, 1, 1, 0 };
double y[] = { 0, 0, 1, 1 };
double z[] = { 0, 0, 0, 0 };
std::vector<unsigned short int> elemNodeCounts(numElems, 4);
long long int elemNodeIds[] = { 1, 2, 3, 4 };

// creating corresponding data access structure(s)


sysc::SurfaceMesh mesh(sysc::OutputScalarData(nodeIds, numNodes),
sysc::OutputVectorData(x, y, z, numNodes),
sysc::OutputScalarData(elemNodeCounts),
sysc::OutputScalarData(elemNodeIds, numElemNodeIds));

/* heavyweight array(s) */
size_t numNodes = 4;
size_t numElems = 1;
size_t numElemNodeIds = 4;
long long int nodeIds[] = { 1, 2, 3, 4 };
double x[] = { 0, 1, 1, 0 };
double y[] = { 0, 0, 1, 1 };
double z[] = { 0, 0, 0, 0 };
unsigned short int elemNodeCounts[] = { 4 };
long long int elemNodeIds[] = { 1, 2, 3, 4 };

/* creating corresponding data access structure(s) */


SyscOutputScalarData nodeIdsData;
nodeIdsData.primitiveType = SyscInt64;
nodeIdsData.data = nodeIds;
nodeIdsData.size = numNodes;

SyscOutputVectorData nodeCoordsData;
nodeCoordsData.primitiveType = SyscDouble;
nodeCoordsData.data0 = x;
nodeCoordsData.data1 = y;
nodeCoordsData.data2 = z;
nodeCoordsData.size = numNodes;

SyscOutputScalarData elemNodeCountsData;
elemNodeCountsData.primitiveType = SyscUnsignedInt16;

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 23
System Coupling Participant Library Developer's Guide

elemNodeCountsData.data = elemNodeCounts;
elemNodeCountsData.size = numElems;

SyscOutputScalarData elemNodeIdsData;
elemNodeIdsData.primitiveType = SyscInt64;
elemNodeIdsData.data = elemNodeIds;
elemNodeIdsData.size = numElemNodeIds;

Fortran

Note that for Fortran, one can make use of the helper functions syscGetOutputScalarDataF
and syscGetOutputVectorDataSplitF as shown below.

! heavyweight array(s)
integer(kind=8), target :: nodeIds(4) = (/ 1, 2, 3, 4 /)
real(kind=8), target :: x(4) = (/ 0, 1, 1, 0 /)
real(kind=8), target :: y(4) = (/ 0, 0, 1, 1 /)
real(kind=8), target :: z(4) = (/ 0, 0, 0, 0 /)
integer(kind=4), target :: elemNodeCounts(1) = (/ 4 /)
integer(kind=8), target :: elemNodeIds(4) = (/ 1, 2, 3, 4 /)

! creating corresponding data access structure(s)


type(SyscOutputScalarDataF) :: nodeIdsData
type(SyscOutputVectorDataF) :: nodeCoordsData
type(SyscOutputScalarDataF) :: elemNodeCountsData
type(SyscOutputScalarDataF) :: elemNodeIdsData

nodeIdsData = syscGetOutputScalarDataF(nodeIds)
nodeCoordsData = syscGetOutputVectorDataSplitF(x, y, z)
elemNodeCountsData = syscGetOutputScalarDataF(elemNodeCounts)
elemNodeIdsData = syscGetOutputScalarDataF(elemNodeIds)

Example Input Scalar in a Single STL Vector Container (C++ only)


In this example, the participant solver has an input scalar variable stored in memory as a single STL
vector, using compact storage.

To provide access to this array, create the InputScalarData structure as shown below.

Note that the size does not need to be provided because it can be automatically deduced from
the STL vector size.

C++

// heavyweight array(s)
std::size_t numScalars = 15;
std::vector<double> coords(numScalars, 0.0);

// creating corresponding data access structure(s)


sysc::InputVectorData coordsData(coords);

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
24 of ANSYS, Inc. and its subsidiaries and affiliates.
Creating Restart Points and Restarting a Coupled Analysis

Data Access Functions


To provide access to the heavyweight data, the participant must also implement data access functions
which return the data access structures (p. 15) and register them with the participant library. These
functions are called from within the participant library (i.e. they are callback functions).

The following functions may need to be implemented:

• Function to access surface mesh (surface mesh access)

• Function to access input scalar variable data (input scalar variable access)

• Function to access input vector variable data (input vector variable access)

• Function to access output scalar variable data (output scalar variable access)

• Function to access output vector variable data (output vector variable access)

There is no need to implement and register functions that will not be used. For example, if there are
no scalar output variables, there is no need to implement and register a function for output scalar
variable access.

For information on how to implement data access functions, see Heat Transfer in Square Channel Air
Flow Tutorial (p. 47).

C++
Data access function prototypes are defined in the following header file:

• SystemCouplingParticipant/FunctionTypes.hpp

C
Data access function prototypes are defined in the following header file:

• syscDataAccess.h

Fortran
Data access function interfaces are defined in the following header file (located either in the Fortran-
FreeForm or the FortranFixedForm subdirectory, depending on the desired Fortran layout):

• syscDataAccessF.fi

Creating Restart Points and Restarting a Coupled Analysis


System Coupling supports the creation of restart points during the coupled analysis and, subsequently,
restarts of the coupled analysis from any previously created restart point. For more information, see
the System Coupling User's Guide.

The System Coupling Participant Library provides the APIs needed to create restart points. For more
information, see:

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 25
System Coupling Participant Library Developer's Guide

Setting Up for Restarts


Creating Restart Points
Restarting the Analysis from a Restart Point

Setting Up for Restarts


During the setup stage, the participant should indicate whether the restarts are supported. The System
Coupling Participant setup file (*.scp) can have an XML element <RestartsSupported> for that
purpose. For details, see the System Coupling User's Guide

When using the participant library to create the setup file, the restartsSupported flag can be
set in the Setup File Info structure. Once that structure is created, the setup file can be written. For
details, see Creating System Coupling Participant Setup Files Using the Participant Library (p. 10).

C++

// create a struct of type sysc::SetupFileInfo


// and set the restarts supported flag to true
std::string scpFileName = "example.scp";
bool restartsSupported = true;
sysc::SetupFileInfo setupFileInfo(scpFileName, restartsSupported);

/* create a struct of type SyscSetupFileInfo


and set the restarts supported flag to true */
SyscSetupFileInfo setupFileInfo;
setupFileInfo.setupFileName = "example.scp";
setupFileInfo.restartsSupported = 1;

Fortran

! create a variable of type SyscSetupFileInfoF


! and set the restarts supported flag to true
type(SyscSetupFileInfoF) :: setupFileInfo
setupFileInfo%setupFileName = "example.scp"
setupFileInfo%restartsSupported = .TRUE.

Creating Restart Points


During the coupled analysis, if the participant solver supports restarts, a callback function should be
registered that can be called to create the restart points. This function will be called during outputs
update if a restart point needs to be created. The callback function should be implemented and re-
gistered prior to the coupled analysis initialization. The function is expected to return a string that
identifies that restart point. This string, in turn, will be provided to the participant solver during a
restart run.

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
26 of ANSYS, Inc. and its subsidiaries and affiliates.
Creating Restart Points and Restarting a Coupled Analysis

C++

...
// sample implementation of the restart callback function
std::string createRestartPoint()
{
std::string restartPoint;
... // solver-specific implementation, sets string value
return restartPoint;
}
...
// System Coupling object
sysc::SystemCoupling sc(host, port, name);
...
// register restart point creation callback
sc.registerSurfaceMeshAccess(&createRestartPoint);
...

...
/* sample implementation of the restart callback function */
const char* createRestartPoint()
{
char* restartPoint;
/* solver-specific implementation, sets string value */
...
return restartPoint;
}
...
/* register restart point creation callback */
syscRegisterRestartPointCreation(&createRestartPoint);
...

Fortran

...
! sample implementation of the restart callback function
function createRestartPoint() result(restartPoint)
character(len=SyscFilePathLen) :: restartPoint
! solver-specific implementation, sets string value
...
end function createRestartPoint
...
! register restart point creation callback
syscRegisterRestartPointCreationF(createRestartPoint)

Restarting the Analysis from a Restart Point


When restarting the coupled analysis, System Coupling will provide the string that was returned when
the restart point was created. This string will be passed into the participant solver via the following
command line argument:

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 27
System Coupling Participant Library Developer's Guide

--screstart RESTART_STRING

The participant solver is expected to resume the analysis from the state it was in when the restart
point was created. The participant steps in a coupled analysis are the same as for the initial run. For
details, see Participant Steps in a Coupled Analysis (p. 4).

Multi-Region Coupling Interfaces


System Coupling supports multi-region coupling interfaces, where more than one region is specified
per side. Generally, the preferred approach is to use one region per coupling interface — that is, to
agglomerate multiple topological regions into a single coupling region inside the participant solver.
However, if agglomeration is not feasible, then the implementation of the participant library APIs should
be consistent with the rules described in this section.

For more information on implementing multi-region coupling interfaces, see:


Mesh Definition for Multi-Region Coupling Interfaces
Variable Field Data for Multi-Region Coupling Interfaces

Mesh Definition for Multi-Region Coupling Interfaces


When providing access to region mesh for a multi-region case, care should be taken when handling
shared nodes. Shared nodes are nodes that are used in the mesh definition in more than one region.
Example 1: Two regions with shared mesh nodes (p. 29) illustrates this scenario. In this example, there
are two surface regions defined, and each region has a simple mesh that consists of one quadrilateral
element. Region L has 4 mesh nodes, as does Region R. However, when taken together, there are
only six unique nodes between these two regions, because the two nodes in the middle (marked A
and B) are shared nodes.

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
28 of ANSYS, Inc. and its subsidiaries and affiliates.
Multi-Region Coupling Interfaces

Example 1: Two regions with shared mesh nodes

System Coupling will detect a shared node by its node id. For example, if node A is given an id of 4
when defining the mesh for Region L and the same id of 4 when defining mesh for the Region R,
then System Coupling will interpret the node A as shared between the two regions. To further illustrate
this example, surface mesh arrays for both regions are shown in Example 2: Mesh arrays for the case
with shared nodes (p. 30).

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 29
System Coupling Participant Library Developer's Guide

Example 2: Mesh arrays for the case with shared nodes

As shown in Example 2: Mesh arrays for the case with shared nodes (p. 30), nodes with ids 3 and 4
are repeated in the definitions of both regions. From the above example, it is obvious that some re-
dundant information is provided for shared nodes — node coordinates are provided multiple times,
once for each shared node for each region. For a given shared node, System Coupling will use only
one set of coordinates (from one of the regions) and will disregard all others. Therefore, it is critical
that nodal coordinates for shared nodes are identical.

To contrast the case with shared nodes and the case without shared nodes, compare Example 2: Mesh
arrays for the case with shared nodes (p. 30) to Example 3: Mesh arrays for the case with non-shared
nodes (p. 31). The latter example shows the case without shared nodes. Note that this is a different
setup and may result in a different solution.

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
30 of ANSYS, Inc. and its subsidiaries and affiliates.
Multi-Region Coupling Interfaces

Example 3: Mesh arrays for the case with non-shared nodes

Finally, these two regions can be agglomerated into a single region, as shown in Example 4: Mesh
arrays for the case where regions are agglomerated into one (p. 32). If the two regions above are to
be used in a single coupling interface, then agglomeration is preferred but not necessary.

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 31
System Coupling Participant Library Developer's Guide

Example 4: Mesh arrays for the case where regions are agglomerated into one

Variable Field Data for Multi-Region Coupling Interfaces


When working with multi-region cases, one should also be careful with how the variable field data
is handled on shared nodes. The least error-prone approach for a multi-region case is to provide
variable data defined on elements, because while nodes may be shared between regions, elements
are not. This is recommended whether the variable is conservative or profile-preserving. However,
this is not always possible — for instance, nodal displacements are inherently nodal. If variable data
is defined on mesh nodes, different treatment is required, depending on whether the variable is
conservative or not.

Values for conservative variables are accumulated on shared nodes by System Coupling. Example 5: Nod-
al conservative variable (heat rate) values for the multi-region case with shared nodes (p. 33) shows
an example of a conservative nodal variable (heat rate) for two regions, where each region consists
of one quadrilateral mesh face (see previous section for how to define the mesh for this example).
Nodes A and B are shared between these two regions.

Note that in Region L, nodes A and B each have heat rate value of 2 Watts. In Region R, nodes A
and B each have heat rate values of 3 Watts. Since heat rate is a conservative variable, the values on
shared nodes will be accumulated. Therefore, when both regions are taken together, the shared nodes

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
32 of ANSYS, Inc. and its subsidiaries and affiliates.
Multi-Region Coupling Interfaces

A and B will each have heat rate value of 5 Watts. The sum of heat rate values for both regions in
this example is 30 Watts.

Example 5: Nodal conservative variable (heat rate) values for the multi-region case with shared
nodes

Values for profile-preserving variables should be identical on shared nodes. Example 6: Nodal profile-
preserving variable (temperature) values for the multi-region case with shared nodes (p. 34) shows
an example of profile-preserving nodal variable (temperature) for the same two regions as in the
previous example. Note that shared nodes A and B each have identical temperature values in both
Region L and Region R. To get the average temperature on these two regions, the sum of temperature
nodal values is divided by the number of unique nodes. In this example, the average temperature is
(500 + 300 + 800 + 600 + 500 + 300) / 6 = 500 [K].

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 33
System Coupling Participant Library Developer's Guide

Example 6: Nodal profile-preserving variable (temperature) values for the multi-region case
with shared nodes

Note:

Sometimes, variable values on shared nodes can differ. For example, a shared node can
exist between a region that represents fluid flow inlet and a region that represents a non-
slip wall. In this situation, fluid flow velocity on the inlet node could be non-zero, but on
the wall node the velocity will be zero. However, in these situations, one would not want
to include both these regions in one coupling interface. If multiple regions with shared
nodes are included in one coupling interface, then the profile-preserving variable values
on those shared nodes should be identical.

Debugging Tools
The participant library includes the following tools to help with debugging:
Standalone Mode
Mesh Validity Check

Standalone Mode
Once the APIs are implemented, you can use the standalone mode to run the participant solver
without connecting to System Coupling. When running in standalone mode, the participant solver
will run for a few coupling time steps (if transient) and coupling iterations. At each iteration, it will
call all registered data access functions and print the contents of heavyweight data into a text file.

The standalone mode allows you to check for run-time issues more quickly and can help to provide
a measure of confidence that the APIs are implemented correctly. Before running a solver as part of
an overall coupled analysis, you should first confirm that the participant solver can run successfully
in standalone mode.

To run in standalone mode, provide an empty host name when the connection to System Coupling
would be typically established. No other changes are required to run in standalone mode. For a

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
34 of ANSYS, Inc. and its subsidiaries and affiliates.
Debugging Tools

demonstration of how to run a participant solver in standalone mode, see Heat Transfer in Square
Channel Air Flow Tutorial (p. 47).

C++

std::string scHost = "";


unsigned short int scPort = 0;
std::string partName = "test"
sysc::SystemCoupling sc(scHost, scPort, partName);

const char* scHost = "";


const unsigned short scPort = 0;
const char* partName = "test";
SyscError ret = syscConnect(scHost, scPort, partName, buildInfo);

Fortran

character(len=256) :: scHost
integer :: scPort = 0
character(len=256) :: partName
...
scHost = ""
ret = syscConnectF(scHost, scPort, partName)

Mesh Validity Check


A mesh validity check is available as a debugging tool. This is a function that can be called to detect
problems in the mesh that is provided to System Coupling and should be called after coupled analysis
initialization is done. It returns the mesh validity status and a message containing information about
a problem that was detected.

For example, it will check to verify that array sizes are consistent — that the element node ids array
contains only ids that are found in the node ids array, etc.

Note:

• This is not an exhaustive validity check — the mesh can still have problems even if all checks
pass. However, it can provide a measure of confidence that the mesh being provided to System
Coupling is correct.

• To ensure optimal performance, this function should not be called in production workflows.

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 35
System Coupling Participant Library Developer's Guide

C++

sysc::SystemCoupling sc(...);
sysc::Region region(...);
...
sysc::MeshValidityStatus meshStatus = sc.checkMeshValidity(region);
if (meshStatus.isInvalid) {
std::cout << meshStatus.message;
}

SyscRegion region;
...
SyscError ret = syscCheckMeshValidity(region);
if (ret.retcode != 0) {
printf(ret.message);
}

Fortran

type(SyscRegionF) :: region
type(SyscErrorF) :: ret
...
ret = syscCheckMeshValidityF(region)
if (ret%retcode .NE. SyscStatusOk) then
print *, trim(ret%message)
endif

Migration Guide
The information in this section is provided to help migrate the System Coupling APIs implementation
between the releases.

Migrating from Pre-2020 R1 Releases


This section is provided to help migrate the APIs from releases prior to 2020 R1.

Setup APIs
The APIs that are related to setup (e.g. add region, add input/output variable, complete coupling
setup) should no longer be used during the coupled analysis. These APIs will remain available and
can be used to generate the setup (SCP) files during the setup stage.

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
36 of ANSYS, Inc. and its subsidiaries and affiliates.
Compiling, Linking, and Executing Applications That Use the Participant Library

For information on creating the setup files, see Creating System Coupling Participant Setup Files Using
the Participant Library (p. 10).

For the most up-to-date information on the steps in a coupled analysis, see Participant Steps in a
Coupled Analysis (p. 4).

Heavyweight Data Access Functions (C++ Only)


This applies to only C++ APIs. Heavyweight data access functions now take region and variable names
rather than objects. For example, the surface mesh access function prototype has changed as follows:

// LEGACY PROTOTYPE:
sysc::SurfaceMesh getSurfaceMesh(const sysc::Region& region);

// NEW PROTOTYPE:
sysc::SurfaceMesh getSurfaceMesh(const std::string& regionName);

Data Access Helper Functions (Fortran only)


This applies only to Fortran APIs. The following Fortran helper functions have been modified to take
an additional argument nElems:

• syscGetInputScalarDataF

• syscGetOutputScalarDataF

• syscGetInputVectorDataSplitF

• syscGetInputVectorDataCompactF

• syscGetOutputVectorDataSplitF

• syscGetOutputVectorDataCompactF

For the most up-to-date information on the usage of these helper functions, see Access to Heavyweight
Data (p. 12).

Compiling, Linking, and Executing Applications That Use the Participant


Library
The instructions for linking and compiling the System Coupling Participant Library depend on the target
platform, the target language, the MPI distribution, and the compiler.

For more information, see:


Supported Languages
System Coupling Participant Library Resources
Examples

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 37
System Coupling Participant Library Developer's Guide

Supported Languages
System Coupling interfaces are provided for the following target languages:

Language Standard
C++ C++11 or later
C C89 or later
Fortran 2003 or later (both fixed-form and free-form)

System Coupling Participant Library Resources


Resources are available at the following locations:

Public Header Files:

• <ANSYSInstallationPath>/SystemCoupling/runTime/<platform>/include/Sys-
temCouplingParticipant

Note:

All public header files for the C++, C, and Fortran interfaces are located in the above
directory. Both fixed and free form headers are available for Fortran codes.

Link-Time Dependencies

Windows

• ANSYSInstallationPath>/SystemCoupling/runTime/<platform>/lib/SysC.Sys-
temCouplingParticipantParticipant.lib

– SysC.SystemCouplingParticipantParticipant.lib

– SysC.SystemCouplingParticipantFortran.lib (only if using Fortran APIs)

• <ANSYSInstallationPath>/SystemCoupling/runTime/<platform>/cnlauncher/flu-
ent/fluent<MultiportVersion>/multiport/mpi_wrapper/<platform>/<mpi>

– mpi_wrapper.lib

Linux

• <ANSYSInstallationPath>/System Coupling/runTime/<platform>/bin

– SysC.SystemCouplingParticipant.so

– SysC.SystemCouplingParticipantFortran.so (only if using Fortran APIs)

• <ANSYSInstallationPath>/SystemCoupling/runTime/<platform>/cnlauncher/flu-
ent/fluent<MultiportVersion>/multiport/mpi_wrapper/<platform>/<mpi>

– libmpi_wrapper.so

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
38 of ANSYS, Inc. and its subsidiaries and affiliates.
Compiling, Linking, and Executing Applications That Use the Participant Library

Run-Time Dependencies

• <ANSYSInstallationPath>/System Coupling/runTime/<platform>/bin

– SysC.SystemCouplingParticipant.dll (Windows)

– SysC.SystemCouplingParticipantFortran.dll (Windows, only if using Fortran APIs)

– SysC.SystemCouplingParticipant.so (Linux)

– SysC.SystemCouplingParticipantFortran.so (Linux, only if using Fortran APIs)

• <ANSYSInstallationPath>/System Coupling/runTime/<platform>/bin/compiler

• <ANSYSInstallationPath>/SystemCoupling/runTime/<platform>/cnlauncher/flu-
ent/fluent<MultiportVersion>/multiport/mpi_wrapper/<platform>/<mpi>

If not using a supported MPI distribution (this includes not using MPI at all), stub MPI wrapper
should be used (replace <mpi> with "stub" above).

The above directories must be included in the PATH environment variable (on Windows) or
LD_LIBRARY_PATH environment variable (on Linux) at run-time.

Examples
In the following examples, mock solver applications from the Heat Transfer in Square Channel Air
Flow (p. 47) tutorial are built with different configurations. These examples can be used as a reference
when building more complex applications. The actual build steps depend on your system configuration,
compilers, and other details. The source code for these applications is provided with the System
Coupling Participant Library tutorial package.

In the following examples, replace:

• <ANSYSInstallationPath> with the correct ANSYS installation path

• <MultiportVersion> with the correct version of the Fluent Multiport library

• <IntelMPIPath> with the correct path to the Intel MPI library

The following compiler and MPI versions are used in the examples:

Linux

• g++, gcc, and gfortran 6.3.0

• Intel(R) Fortran Compiler, Version 19.0.5.281

• Intel(R) MPI Library for Linux OS, Version 2017 Update 4

Windows

• Microsoft (R) C/C++ Optimizing Compiler Version 19.10.25027 for x64

• Intel(R) Visual Fortran Intel(R) 64 Compiler Version 19.0.4.245

• Intel(R) MPI Library for Windows OS, Version 2018 Update 3

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 39
System Coupling Participant Library Developer's Guide

The following sections provide examples for Linux (p. 40) and Windows (p. 43).

Linux Examples
The following Linux examples are available:

• C++ (p. 40)

• C (p. 40)

• ???

• ???

• C++ Parallel Version Using Intel MPI (p. 41)

• C Parallel Version Using Intel MPI (p. 41)

• ???

• ???

• C++ Parallel Version Using Fluent MPI Wrapper (p. 42)

• Fortran Parallel Version Using Fluent MPI Wrapper (p. 42)

• Executing in Standalone Mode (p. 42)

• Executing Parallel Version in Standalone Mode Using Intel MPI (p. 43)

C++

g++ -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11 -I<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/


include -L<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/bin -L<ANSYSInstallationPath>/
SystemCoupling/runTime/linx64/cnlauncher/fluent/fluent<MultiportVersion>/multiport/mpi_wrapper/
lnamd64/stub -o ChannelFlowMockSolver ChannelFlowMockSolver.cpp -lSysC.SystemCouplingParticipant
-lmpi_wrapper

g++ -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11 -I<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/


include -L<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/bin -L<ANSYSInstallationPath>/
SystemCoupling/runTime/linx64/cnlauncher/fluent/fluent<MultiportVersion>/multiport/mpi_wrapper/
lnamd64/stub -o ChannelFlowMockSolver ChannelFlowMockSolver.cpp -lSysC.SystemCouplingParticipant
-lmpi_wrapper

Fortran
GNU Fortran Compiler

gfortran -I<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/include/

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
40 of ANSYS, Inc. and its subsidiaries and affiliates.
Compiling, Linking, and Executing Applications That Use the Participant Library

SystemCouplingParticipant/FortranFixedForm -L<ANSYSInstallationPath>/SystemCoupling/
runTime/linx64/bin -L<ANSYSInstallationPath> /SystemCoupling/runTime/linx64/
cnlauncher/fluent/fluent<MultiportVersion>/multiport/mpi_wrapper/lnamd64/stub
-o ChannelFlowMockSolver ChannelFlowMockSolver.f -lSysC.SystemCouplingParticipantFortran
-lmpi_wrapper

Intel Fortran Compiler

ifort -I<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/include/
SystemCouplingParticipant/FortranFixedForm -I<IntelMPIPath>/intel64/include
-L<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/bin -L<IntelMPIPath>/intel64/lib
-L<IntelMPIPath>/intel64/lib/release -L<ANSYSInstallationPath>/SystemCoupling/runTime/
linx64/cnlauncher/fluent/fluent<MultiportVersion>/multiport/mpi_wrapper/lnamd64/stub
-o ChannelFlowMockSolverMPI ChannelFlowMockSolverMPI.f -lSysC.SystemCouplingParticipantFortran
-lmpifort lmpi -lmpi_wrapper -lgfortran

C++ Parallel Version Using Intel MPI

g++ -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11 -I<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/


include -I<IntelMPIPath>/intel64/include -L<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/bin
-L<IntelMPIPath>/intel64/lib/release -L<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/bin
-L<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/cnlauncher/fluent/fluent<MultiportVersion>/
multiport/mpi_wrapper/lnamd64/stub -o ChannelFlowMockSolverMPI ChannelFlowMockSolverMPI.cpp
-lSysC.SystemCouplingParticipant -lmpi -lmpi_wrapper

C Parallel Version Using Intel MPI

gcc -I<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/include -I<IntelMPIPath>/intel64/include


-L<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/bin -L<IntelMPIPath>/intel64/lib/release
-L<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/bin -L<ANSYSInstallationPath>/SystemCoupling/
runTime/linx64/cnlauncher/fluent/fluent<MultiportVersion>/multiport/mpi_wrapper/lnamd64/stub
-o ChannelFlowMockSolverMPI ChannelFlowMockSolverMPI.c -lSysC.SystemCouplingParticipant -lmpi
-lmpi_wrapper

Fortran Parallel Version Using Intel MPI


GNU Fortran Compiler

gfortran -I<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/include/
SystemCouplingParticipant/FortranFixedForm -I<IntelMPIPath>/intel64/include -L<ANSYSInstallationPath>/
SystemCoupling/runTime/linx64/bin -L<IntelMPIPath>/intel64/lib -L<IntelMPIPath>/intel64/lib/release
-L<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/cnlauncher/fluent/fluent<MultiportVersion>/
multiport/mpi_wrapper/lnamd64/stub -o ChannelFlowMockSolverMPI ChannelFlowMockSolverMPI.f
-lSysC.SystemCouplingParticipantFortran -lmpifort -lmpi -lmpi_wrapper

Intel Fortran Compiler

ifort -I<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/include/SystemCouplingParticipant/

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 41
System Coupling Participant Library Developer's Guide

FortranFixedForm -I<IntelMPIPath>/intel64/include -L<ANSYSInstallationPath>/SystemCoupling/runTime/


linx64/bin -L<IntelMPIPath>/intel64/lib -L<IntelMPIPath>/intel64/lib/release -L<ANSYSInstallationPath>/
SystemCoupling/runTime/linx64/cnlauncher/fluent/fluent<MultiportVersion>/multiport/mpi_wrapper/lnamd64/
stub -o ChannelFlowMockSolverMPI ChannelFlowMockSolverMPI.f -lSysC.SystemCouplingParticipantFortran
-lmpifort -lmpi -lmpi_wrapper -lgfortran

C++ Parallel Version Using Fluent MPI Wrapper

g++ -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11 -I<ANSYSInstallationPath>/SystemCoupling/runTime/


linx64/include -I<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/cnlauncher/fluent/
fluent<MultiportVersion>/multiport/mpi_wrapper/include -L<ANSYSInstallationPath>/SystemCoupling/
runTime/linx64/bin -L<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/cnlauncher/fluent/
fluent<MultiportVersion>/multiport/mpi_wrapper/lnamd64/stub -o ChannelFlowMockSolverMPI
ChannelFlowMockSolverMPI.cpp -lSysC.SystemCouplingParticipant -lmpi_wrapper

Fortran Parallel Version Using Fluent MPI Wrapper


Note that to link the MPI Wrapper, include 'mpif.h' statement in the application must be
replaced with the following C preprocessor directive:

#include "mpif.h"

The C preprocessor must be used. The GNU Fortran compiler option for this is -cpp.

gfortran -cpp -I<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/include/


SystemCouplingParticipant/FortranFixedForm -I<ANSYSInstallationPath>/SystemCoupling/runTime/
linx64/cnlauncher/fluent/fluent<MultiportVersion>/multiport/mpi_wrapper/include
-L<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/bin -L<ANSYSInstallationPath>/SystemCoupling/
runTime/linx64/cnlauncher/fluent/fluent<MultiportVersion>/multiport/mpi_wrapper/lnamd64/stub
-o ChannelFlowMockSolverMPI ChannelFlowMockSolverMPI.f -lSysC.SystemCouplingParticipantFortran
-lmpi_wrapper

Executing in Standalone Mode


1. Add the following locations to the LD_LIBRARY_PATH environment variable:

• <ANSYSInstallationPath>/SystemCoupling/runTime/linx64/bin

• <ANSYSInstallationPath>/SystemCoupling/runTime/linx64/bin/compiler

• <ANSYSInstallationPath>/SystemCoupling/runTime/linx64/cnlauncher/flu-
ent/fluent<MultiportVersion>/multiport/mpi_wrapper/lnamd64/stub

2. Now, execute the program in standalone mode:

./ChannelFlowMockSolver --scname="test"

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
42 of ANSYS, Inc. and its subsidiaries and affiliates.
Compiling, Linking, and Executing Applications That Use the Participant Library

Executing Parallel Version in Standalone Mode Using Intel MPI


1. Add the following locations to the LD_LIBRARY_PATH environment variable:

• <ANSYSInstallationPath>/SystemCoupling/runTime/linx64/bin

• <ANSYSInstallationPath>/SystemCoupling/runTime/linx64/bin/compiler

• <ANSYSInstallationPath>/SystemCoupling/runTime/linx64/cnlauncher/flu-
ent/fluent<MultiportVersion>/multiport/mpi_wrapper/lnamd64/intel

2. Now, execute the program in standalone mode locally using two processes:

mpirun -n 2 ./ChannelFlowMockSolverMPI --scname="test"

Windows Examples
The following Linux examples are available:

• C++ (p. 43)

• C (p. 43)

• Fortran (p. 44)

• C++ Parallel Version Using Intel MPI (p. 44)

• C Parallel Version Using Intel MPI (p. 44)

• Fortran Parallel Version Using Intel MPI (p. 44)

• C++ Parallel Version Using Fluent MPI Wrapper (p. 44)

• Fortran Parallel Version Using Fluent MPI Wrapper (p. 45)

• Executing in Standalone Mode (p. 45)

• Executing Parallel Version in Standalone Mode Using Intel MPI (p. 45)

C++

g++ -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11 -I<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/


include -L<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/bin -L<ANSYSInstallationPath>/
SystemCoupling/runTime/linx64/cnlauncher/fluent/fluent<MultiportVersion>/multiport/mpi_wrapper/
lnamd64/stub -o ChannelFlowMockSolver ChannelFlowMockSolver.cpp -lSysC.SystemCouplingParticipant
-lmpi_wrapper

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 43
System Coupling Participant Library Developer's Guide

cl /EHsc /I"<ANSYSInstallationPath>\SystemCoupling\runTime\winx64\include" ChannelFlowMockSolver.c


/FeChannelFlowMockSolver.exe /link /subsystem:console /LIBPATH:"<ANSYSInstallationPath>\
SystemCoupling\runTime\winx64\lib" SysC.SystemCouplingParticipant.lib

Fortran

ifort -I"<ANSYSInstallationPath>\SystemCoupling\runTime\winx64\include\SystemCouplingParticipant\
FortranFixedForm" /FeChannelFlowMockSolver.exe ChannelFlowMockSolver.f /link /subsystem:console
/LIBPATH:"<ANSYSInstallationPath>\SystemCoupling\runTime\winx64\lib"
SysC.SystemCouplingParticipantFortran.lib

C++ Parallel Version Using Intel MPI

cl /EHsc /I"<ANSYSInstallationPath>\SystemCoupling\runTime\winx64\include" /I"<IntelMPIPath>\


intel64\include" ChannelFlowMockSolverMPI.cpp /FeChannelFlowMockSolverMPI.exe /link
/subsystem:console /LIBPATH:"<ANSYSInstallationPath>\SystemCoupling\runTime\winx64\lib"
SysC.SystemCouplingParticipant.lib /LIBPATH:"<IntelMPIPath>\intel64\lib" impi.lib

C Parallel Version Using Intel MPI

cl /EHsc /I"<ANSYSInstallationPath>\SystemCoupling\runTime\winx64\include" /I"<IntelMPIPath>\


intel64\include" ChannelFlowMockSolverMPI.c /FeChannelFlowMockSolverMPI.exe /link /
subsystem:console /LIBPATH:"<ANSYSInstallationPath>\SystemCoupling\runTime\winx64\lib"
SysC.SystemCouplingParticipant.lib /LIBPATH:"<IntelMPIPath>\intel64\lib" impi.lib

Fortran Parallel Version Using Intel MPI

ifort -I"<ANSYSInstallationPath>\SystemCoupling\runTime\winx64\include\SystemCouplingParticipant\
FortranFixedForm" -I"<IntelMPIPath>\intel64\include" /FeChannelFlowMockSolverMPI.exe
ChannelFlowMockSolverMPI.f /link /subsystem:console /LIBPATH:"<ANSYSInstallationPath>\SystemCoupling\
runTime\winx64\lib" SysC.SystemCouplingParticipantFortran.lib /LIBPATH:"<IntelMPIPath>\intel64\lib"
impi.lib

C++ Parallel Version Using Fluent MPI Wrapper

cl /EHsc /I"<ANSYSInstallationPath>\SystemCoupling\runTime\winx64\include"
/I"<ANSYSInstallationPath>\SystemCoupling\runTime\winx64\cnlauncher\fluent\
fluent<MultiportVersion>\multiport\mpi_wrapper\include" ChannelFlowMockSolverMPI.cpp
/FeChannelFlowMockSolverMPI.exe /link /subsystem:console /LIBPATH:"<ANSYSInstallationPath>\
SystemCoupling\runTime\winx64\lib" SysC.SystemCouplingParticipant.lib
/LIBPATH:"<ANSYSInstallationPath>\SystemCoupling\runTime\winx64\cnlauncher\fluent\
fluent<MultiportVersion>\multiport\mpi_wrapper\win64\stub" mpi_wrapper.lib

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
44 of ANSYS, Inc. and its subsidiaries and affiliates.
Compiling, Linking, and Executing Applications That Use the Participant Library

Fortran Parallel Version Using Fluent MPI Wrapper


Note that to link the MPI Wrapper, include mpif.h statement in the application must be replaced
with the following C preprocessor directive:

#include "mpif.h"

The C preprocessor must be used. The Intel Fortran compiler option for this is -fpp.

ifort -fpp -I"<ANSYSInstallationPath>\SystemCoupling\runTime\winx64\include\SystemCouplingParticipant\


FortranFixedForm" -I"<ANSYSInstallationPath>\SystemCoupling\runTime\winx64\cnlauncher\fluent\
fluent<MultiportVersion>\multiport\mpi_wrapper\include" /FeChannelFlowMockSolverMPI.exe
ChannelFlowMockSolverMPI.f /link /subsystem:console /LIBPATH:"<ANSYSInstallationPath>\SystemCoupling\
runTime\winx64\lib" SysC.SystemCouplingParticipantFortran.lib /LIBPATH:"<ANSYSInstallationPath>\
SystemCoupling\runTime\winx64\cnlauncher\fluent\fluent<MultiportVersion>\multiport\mpi_wrapper\
win64\stub" mpi_wrapper.lib

Executing in Standalone Mode


1. Add the following locations to the PATH environment variable:

• <ANSYSInstallationPath>\SystemCoupling\runTime\winx64\bin

• <ANSYSInstallationPath>\SystemCoupling\runTime\winx64\bin\compiler

• <ANSYSInstallationPath>\SystemCoupling\runTime\winx64\cnlauncher\flu-
ent\fluent<MultiportVersion>\multiport\mpi_wrapper\win64\intel

2. Now, execute the program in standalone mode:

ChannelFlowMockSolver.exe --scname="test"

Executing Parallel Version in Standalone Mode Using Intel MPI


1. Add the following locations to the PATH environment variable:

• <ANSYSInstallationPath>\SystemCoupling\runTime\winx64\bin

• <ANSYSInstallationPath>\SystemCoupling\runTime\winx64\bin\compiler

• <ANSYSInstallationPath>\SystemCoupling\runTime\winx64\cnlauncher\flu-
ent\fluent<MultiportVersion>\multiport\mpi_wrapper\win64\intel

2. Now, execute the program in standalone mode locally using two processes:

mpiexec -localonly -noprompt -n 2 ChannelFlowMockSolverMPI.exe --scname="test"

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 45
Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
46 of ANSYS, Inc. and its subsidiaries and affiliates.
System Coupling Participant Library Tutorials
The following tutorials demonstrate how to implement the APIs provided by the System Coupling Par-
ticipant Library:
Heat Transfer in Square Channel Air Flow Tutorial
Transient Oscillating Plate Damping Tutorial

Heat Transfer in Square Channel Air Flow Tutorial


This tutorial gives instructions on using System Coupling Participant Library APIs to create a mock
solver and use it within the System Coupling co-simulation infrastructure. It shows how to run a two-
solver coupled analysis with the mock solver and the ANSYS Fluent solver as participants.

The latter part of the tutorial demonstrates the use of parallel interfaces. The mock solver code is
modified to be executed within the MPI environment.

Topics covered:

• Steady analysis

• Temperature and heat flow transfer

• Coupling to ANSYS Fluent

• Parallel execution

To perform the tutorial, follow the instructions in the following sections:


Verify Prerequisites
Problem Description and Participant Setup
Get the Necessary Files
Part 1: Serial Version of the Mock Solver
Part 2: The Parallel Version of the Mock Solver

Verify Prerequisites
Ensure that the following prerequisites are met. You've installed:

• A compiler for one of the supported languages (p. 38)

• ANSYS installation, including Fluent

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 47
System Coupling Participant Library Tutorials

Problem Description and Participant Setup


In this example, air is heated as it flows through a rectangular channel. The Fluent solver is used to
model the air flow, and a simple mock solver is used to supply the Heat Flow to Fluent via System
Coupling.

Set up both coupling participants:


Fluent Setup
Mock Solver Setup

Fluent Setup
The Fluent setup involves the flow of air through a rectangular 1-meter-long channel with a square
0.1 by 0.1 [m] cross-section. The geometry is discretized using tetrahedral mesh, as shown in Fig-
ure 3: Fluent geometry (p. 49).

• The inlet is the surface coincident with the YZ plane, with a velocity of 0.1 m/s. Temperature of the
air is 273.15 [K] (0 [C])

• The outlet is the side opposite from the inlet, with 0 Pa gauge pressure.

• The region participating in the coupled analysis is the surface coincident with the XY plane. This
makes the System Coupling interface region a rectangle extending from origin to point (1, 0.1, 0).

• The remaining three sides form a wall.

• Energy model is activated.

• Turbulence model is k-epsilon (with default settings).

• The case is set up to perform five Fluent solver iterations per coupling iteration.

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
48 of ANSYS, Inc. and its subsidiaries and affiliates.
Heat Transfer in Square Channel Air Flow Tutorial

Figure 3: Fluent geometry

Mock Solver Setup


The mock solver creates a rectangular surface region named "FSI" that overlaps Fluent's region that
participates in the coupled analysis, so it is also a rectangle extending from origin to point (1, 0.1, 0).

The mesh consists of four quad elements, evenly dividing the region, as shown in Figure 4: Mock
solver geometry (p. 50).

• It receives temperatures on mesh nodes as inputs on the FSI region.

• It sends heat rate on elements as outputs on the FSI region. Each of the four elements provides 5
[W], so a total of 20 [W] is supplied to Fluent by the mock solver.

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 49
System Coupling Participant Library Tutorials

Figure 4: Mock solver geometry

Get the Necessary Files


Download the ParticipantLibraryTutorials.zip tutorial package and extract it to a local
directory.

Inside the resulting directory, you will find a subdirectory called HeatTransferSquareChannel. This
will serve as the simulation working directory and contains all the files needed for this tutorial. The
contents of this subdirectory are:

• fluent

Directory containing the Fluent case file, fluent.cas.h5

• mocksolver

Directory containing the following files:

– ChannelFlowMockSolver: Linux binary executable

– ChannelFlowMockSolver.bat: Windows batch script wrapper around the Windows


executable (serial version)

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
50 of ANSYS, Inc. and its subsidiaries and affiliates.
Heat Transfer in Square Channel Air Flow Tutorial

– ChannelFlowMockSolver.c: C mock solver code and explanatory comments (serial


version)

– ChannelFlowMockSolver.cpp: C++ mock solver code and explanatory comments


(serial version)

– ChannelFlowMockSolver.exe: Windows binary executable (serial version)

– ChannelFlowMockSolver.f: Fortran (fixed-form) mock solver code and explanatory


comments (serial version)

– ChannelFlowMockSolver.sh: Linux shell script wrapper around the Linux executable


(serial version)

– ChannelFlowMockSolverMPI: Linux binary executable (parallel version)

– ChannelFlowMockSolverMPI.bat: Windows batch script for executing parallel mock


solver locally using Intel MPI

– ChannelFlowMockSolverMPI.c: C mock solver code and explanatory comments (par-


allel version).

– ChannelFlowMockSolverMPI.cpp: C++ mock solver code and explanatory comments


(parallel version)

– ChannelFlowMockSolverMPI.exe: Windows binary executable (parallel version)

– ChannelFlowMockSolverMPI.f: Fortran (fixed-form) mock solver code and explanatory


comments (parallel version)

– ChannelFlowMockSolverMPI.sh: Linux shell script for executing parallel mock solver


locally using Intel MPI

• fluent.scp

System Coupling Participant setup file containing the Fluent solver setup information to be
consumed by System Coupling

• mocksolver.scp

System Coupling Participant setup file containing the mock solver setup information to be
consumed by System Coupling

Note:

Note that this file has been pre-generated for this tutorial.

Refer to Creating System Coupling Participant Setup Files Using the Participant Library (p. 10)
in the System Coupling Participant Library Developer's Guide (p. 3) for information about
how to generate setup files using the participant library.

• runSerial.py

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 51
System Coupling Participant Library Tutorials

System Coupling script file that sets up and executes the coupled analysis using the serial
version of the mock solver

• runParallel.py

System Coupling script file that sets up and executes the coupled analysis using the parallel
version of the mock solver

Part 1: Serial Version of the Mock Solver


For simplicity, the first part of this tutorial uses the serial version of the mock solver and executes
both the mock solver and Fluent in serial. A more advanced parallel version of the same problem will
be demonstrated in the second part of this tutorial.

Build the Mock Solver


Build the mock solver program using a compiler for one of the supported languages (p. 38). Refer
to the Compiling, Linking, and Executing Applications That Use the Participant Library (p. 37) section
of the System Coupling Participant Library Developer's Guide (p. 3) for examples of how to build
this mock solver with different compilers on different platforms.

Note:

Pre-built mock solver executables are provided along with the source code, so this step
is optional.

Run Mock Solver in Standalone Mode


To confirm that the APIs were implemented successfully in the mock solver program, run the ex-
ecutable in standalone mode. This step is recommended before running the participant solver in
the overall coupled analysis to detect any implementation problems.

To run in standalone mode, an empty string should be provided for the System Coupling host
name. Since this mock solver program takes in the SC host name as the first command line argument,
iif the argument is omitted an empty host name will be used, as shown below:

• On Windows, from the command prompt:

ChannelFlowMockSolver.bat

• On Linux, from the shell:

./ChannelFlowMockSolver.sh

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
52 of ANSYS, Inc. and its subsidiaries and affiliates.
Heat Transfer in Square Channel Air Flow Tutorial

This will print a few lines to the console, containing host, port, and name information. It will execute
the mock coupled analysis for five coupling iterations.

Note:

The batch and the shell scripts use the environment variable AWP_ROOT201 to set up
the participant solver environment. Make sure that this environment variable is set to
the ANSYS installation path (e.g., C:\Program Files\ANSYS Inc\v201). Alternat-
ively, modify the scripts to use a different environment variable.

Perform the Coupled Analysis Using the Mock Solver


Execute the coupled analysis (in serial) by using the runSerial.py script:

<ANSYSInstallationPath>/SystemCoupling/bin/systemcoupling -R runSerial.py

Note:

Alternatively, you can run System Coupling interactively. Start System Coupling by running
<ANSYSInstallationPath>/SystemCoupling/bin/systemcoupling and
then type the contents of the script(s) into the command console line-by-line.

You can monitor the solution progress by watching the Transcript output.

If the case runs without issue, you've successfully created a mock solver and run a co-simulation
between Fluent and the mock solver.

Visualize the Results in CFD-Post


Use CFD-Post to view the Fluent results after the co-simulation run is complete. You'll be able to
visualize the air temperature as it gets heated while flowing through the rectangular channel.

1. Open CFD-Post.

2. Load the Fluent results file.

a. In the CFD-Post toolbar, select File > Load Results.

b. In the Load Results File dialog, select the Fluent results file that was generated during co-simu-
lation, fluent-1-00061.dat.h5.

The Fluent results file is loaded, and the geometry is displayed on the 3D Viewer tab of
CFD-Post's viewer.

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 53
System Coupling Participant Library Tutorials

3. Add a cut-plane along the air flow and visualize the air temperature by coloring the plane with
temperature.

a. In the CFD-Post toolbar, select Insert > Location > Plane, enter the name of the plane in the
dialog box, and click OK.

b. In the Details of Plane 1 view:

i. On the Geometry tab, set Method to ZX Plane and set the Y field to 0.05 [m].

ii. On the Color tab, set Mode to Variable and set Variable to Temperature.

This allows you to visualize the air temperature as the air flows through the channel and the
hot air is mixed with the colder air.

Figure 5: Fluent temperature results after co-simulation with the mock solver

Part 2: The Parallel Version of the Mock Solver


The second part of this tutorial demonstrates how to modify this case to run the mock solver in a
parallel MPI environment.

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
54 of ANSYS, Inc. and its subsidiaries and affiliates.
Heat Transfer in Square Channel Air Flow Tutorial

• The parallel mock solver source code is provided with the tutorial package.

• The batch script ChannelFlowMockSolverMPI.bat (Windows) or shell script ChannelFlowMock-


SolverMPI.sh (Linux) that are provided allow the mock solver to be executed locally in parallel using
the Intel MPI distribution that comes with System Coupling.

• The Fluent solver will also run in local parallel, using its default MPI.

Parallel Mock Solver Source Code


The original mock solver has been modified to work in a parallel MPI environment. The main differ-
ence is the initializeArrays() function/subroutine that partitions the mesh elements among
the processes based on the number of processes available.

In a more realistic solver, the partitioning can be done according to an algorithm suitable for that
solver. In this example, the partitioning details are hard-coded for simplicity. Suppose we label each
of the four mesh elements in the mock solver A, B, C, and D, as shown in the schematic below.

(0,0.1,0) (1,0.1,0)
6-----7-----8-----9-----10
| | | | |
| A | B | C | D |
| | | | |
1-----2-----3-----4-----5
(0,0,0) (1,0,0)

The following table shows how the mesh elements are distributed among the processes, depending
on the total number of available processes.

Number of
Rank Elements Nodes
processes
A, B, 1, 2, 3, 4, 5, 6, 7,
1 0
C, D 8, 9, 10
2 0 A, B 1, 2, 3, 6, 7, 8
2 1 C, D 3, 4, 5, 8, 9, 10
3 0 A 1, 2, 6, 7
3 1 B 2, 3, 7, 8
3 2 C, D 3, 4, 5, 8, 9, 10
4 or more 0 A 1, 2, 6, 7
4 or more 1 B 2, 3, 7, 8
4 or more 2 C 3, 4, 8, 9
4 or more 3 D 4, 5, 9, 10
4 or No
4 or more No nodes
greater elements

Note that the above partitioning requires that some mesh nodes are duplicated among different
processes. The participant library is robust to having nodes and elements duplicated in multiple
processes.

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 55
System Coupling Participant Library Tutorials

The source code for the parallel version is provided with the same tutorial package, in each supported
language. Refer to that source code and the comments for more implementation details.

More details about implementing the participant library APIs for execution in a parallel environment
are provided in the Using an Unsupported MPI Distribution (p. 10) section of the System Coupling
Participant Library Developer's Guide (p. 3).

Build the Parallel Mock Solver


Build the mock solver program using a compiler for one of the supported languages (p. 38). Refer
to the Compiling, Linking, and Executing Applications That Use the Participant Library (p. 37) section
of the documentation for examples of how to build this mock solver with different compilers on
different platforms.

Run the Parallel Mock Solver in Standalone Mode


Build the mock solver program using a compiler for one of the supported languages (p. 38). Refer
to the Compiling, Linking, and Executing Applications That Use the Participant Library (p. 37) section
of the documentation for examples of how to execute this mock solver with different compilers on
different platforms.

Perform the Coupled Analysis Using the Parallel Mock Solver


The coupled analysis will be done using local parallel execution. Intel MPI distribution that comes
with System Coupling will be used to execute the parallel mock solver. Both Fluent and the parallel
mock solver will be executed using four processes.

To do that, use the -t4 command line argument when starting System Coupling to solve the case.
Execute the runParallel.py script to run System Coupling in batch mode:

<ANSYSInstallationPath>/SystemCoupling/bin/systemcoupling -R runParallel.py -t4

You can monitor the solution progress by watching the Transcript output.

During the execution, note the "Execution Information" section in the System Coupling
Transcript will show that both Fluent and the mock solver are executed in parallel using four pro-
cesses. The -np 4 command-line argument is passed to the mock solver, and the -t4 command-
line argument is passed to Fluent.

+=============================================================================+
| Execution Information |
+=============================================================================+
...
| DEFAULT-2 |
| Execution Command: |
| ...ChannelFlowMockSolverMPI.bat ... -np 4 |
...
| Fluid Flow (Fluent) |
| Execution Command: |
| ...fluent.exe ... -t4 |
...

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
56 of ANSYS, Inc. and its subsidiaries and affiliates.
Transient Oscillating Plate Damping Tutorial

+=============================================================================+

Once the coupled analysis completes, you've successfully created a parallel mock solver and run a
co-simulation between Fluent and the mock solver. The post-processing step is identical to that of
the serial run.

Transient Oscillating Plate Damping Tutorial


This tutorial demonstrates the implementation of System Coupling Participant Library APIs in a transient
coupled analysis. It shows how to set up and perform a transient coupled analysis using a simple test
solver and the ANSYS Mechanical solver as participants.

This example also shows how to implement the restart interfaces in the participant solver, how to create
restart points at specified intervals, and how to restart the coupled analysis from a previously generated
restart point.

Topics covered:

• Transient analysis

• Force and nodal displacements transfer

• Coupling to ANSYS Mechanical

• Restarts

To perform the tutorial, follow the instructions in the following sections:


Verify Prerequisites
Problem Description and Participant Setup
Get the Necessary Files
Build the Damping Solver
Run the Coupled Analysis
Visualize the Results in CFD-Post
Restart the Coupled Analysis from an Intermediate Point

Verify Prerequisites
Ensure that the following prerequisites are met. You've installed:

• A compiler for one of the supported languages (p. 38)

• ANSYS installation, including Mechanical

Problem Description and Participant Setup


This example is a modification of the "Oscillating Plate Two-Way Fluid-Structure Interaction" tutorials
(e.g., Oscillating Plate FSI with Fluent and Mechanical ) in the System Coupling Tutorials package.

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 57
System Coupling Participant Library Tutorials

The setup of the Mechanical participant is identical to that tutorial. A thin, flexible plate, made of a
linear elastic material, (Young's modulus = 2.5E6 [Pa], Poisson's ratio = 0.35, density = 2550 [kg m^-
3]) has a fixed support along its bottom face. A pressure load of 100 [Pa] is applied on one side of
the plate for the first 0.5 [s] of the simulation, which starts the oscillation. The setup is illustrated in
Figure 6: Structural solver geometry (p. 58).

Figure 6: Structural solver geometry

Since the purpose of this example is to demonstrate the use of Participant Library APIs for a transient
analysis, a simple damping force solver will be used that will couple with Mechanical via System
Coupling. This solver receives nodal displacements from Mechanical and calculate the damping
force using the following formula:
(1)

where is the damping coefficient and is the rate of change of nodal positions with respect to
time (nodal velocities).

The damping solver will then provide the calculated forces to Mechanical via System Coupling, and
these damping forces will cause the oscillations to diminish in amplitude. The higher the damping
coefficient value is, the faster the oscillations will diminish. The damping coefficient value of 1.0 will
be provided to the damping solver via a command line argument --dampcoeff 1.0.

The surface mesh for the damping solver consists of 11 quadrilateral elements that overlap the plate
surface in Mechanical. The surface mesh is shown in Figure 7: Surface mesh in the damping solv-
er (p. 59).

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
58 of ANSYS, Inc. and its subsidiaries and affiliates.
Transient Oscillating Plate Damping Tutorial

Figure 7: Surface mesh in the damping solver

Get the Necessary Files


Download the ParticipantLibraryTutorials.zip tutorial package and extract it to a local
directory.

Inside the resulting directory, you will find a subdirectory called OscillatingPlateDamping. This will
serve as the simulation working directory and contains all the files needed for this tutorial. The contents
of this subdirectory are:

• mapdl

Directory containing the Mechanical case file, ds.dat

• dampingsolver

Directory containing the following files:

– OscillatingPlateDamping: Linux binary executable

– OscillatingPlateDamping.bat: Windows batch script wrapper around the Windows ex-


ecutable

– OscillatingPlateDamping.c: Sample C damping solver code and explanatory comments

– OscillatingPlateDamping.cpp: Sample C++ damping solver code and explanatory com-


ments

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 59
System Coupling Participant Library Tutorials

– OscillatingPlateDamping.exe: Windows binary executable

– OscillatingPlateDamping.f: Sample Fortran (fixed-form) damping solver code and explan-


atory comments

– OscillatingPlateDamping.sh: Linux shell script wrapper around the Linux executable

• mapdl.scp

System Coupling Participant setup file containing the Mechanical solver setup information to be
consumed by System Coupling

• damping.scp

System Coupling Participant setup file containing the damping solver setup information to be
consumed by System Coupling.

Note:

Note that this file has been pre-generated for this tutorial.

Refer to Creating System Coupling Participant Setup Files Using the Participant Library (p. 10) in
the System Coupling Participant Library Developer's Guide (p. 3) for information about how to
generate setup files using the participant library

• run.py

System Coupling script file that sets up and executes the coupled analysis

• runRestart.py

System Coupling script file that restarts the coupled analysis from a previously created intermediate
restart point

Build the Damping Solver


Build the damping solver program using a compiler for one of the supported languages (p. 38).

Note:

Pre-built damping solver executables are provided along with the source code, so this step
is optional.

The source code is provided in the dampingsolver subdirectory. In the following examples, damping
solver applications are built with different configurations. The actual build steps depend on your
system configuration, compilers, and other details. Refer to the Compiling, Linking, and Executing
Applications That Use the Participant Library (p. 37) section of the System Coupling Participant Library
Developer's Guide (p. 3) for additional information.

The following compilers were used in these examples:

• Linux

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
60 of ANSYS, Inc. and its subsidiaries and affiliates.
Transient Oscillating Plate Damping Tutorial

– GNU (g++, gcc, and gfortran) 6.3.0

• Windows

– Microsoft (R) C/C++ Optimizing Compiler Version 19.10.25027 for x64

– Intel(R) Visual Fortran Intel(R) 64 Compiler Version 19.0.4.245

In the following examples, replace:

• <ANSYSInstallationPath> with the correct ANSYS installation path.

• <MultiportVersion> with the correct version of the Fluent Multiport library.

Linux
C++

g++ -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11 -I<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/


include -L<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/bin -L<ANSYSInstallationPath>/
SystemCoupling/runTime/linx64/cnlauncher/fluent/fluent<MultiportVersion>/multiport/mpi_wrapper/
lnamd64/stub -o OscillatingPlateDamping OscillatingPlateDamping.cpp -lSysC.SystemCouplingParticipant
-lmpi_wrapper

gcc -I<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/include -L<ANSYSInstallationPath>/


SystemCoupling/runTime/linx64/bin -L<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/
cnlauncher/fluent/fluent<MultiportVersion>/multiport/mpi_wrapper/lnamd64/stub -o
OscillatingPlateDamping OscillatingPlateDamping.c -lSysC.SystemCouplingParticipant -lmpi_wrapper

Fortran

gfortran -I<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/include/SystemCouplingParticipant/
FortranFixedForm -L<ANSYSInstallationPath>/SystemCoupling/runTime/linx64/bin -L<ANSYSInstallationPath>/
SystemCoupling/runTime/linx64/cnlauncher/fluent/fluent<MultiportVersion>/multiport/mpi_wrapper/lnamd64/
stub -o OscillatingPlateDamping OscillatingPlateDamping.f -lSysC.SystemCouplingParticipantFortran
-lmpi_wrapper

Windows
C++

cl /EHsc /I"<ANSYSInstallationPath>\SystemCoupling\runTime\winx64\include" OscillatingPlateDamping.cpp


/FeOscillatingPlateDamping.exe /link /subsystem:console /LIBPATH:"<ANSYSInstallationPath>\SystemCoupling\
runTime\winx64\lib" SysC.SystemCouplingParticipant.lib

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 61
System Coupling Participant Library Tutorials

cl /EHsc /I"<ANSYSInstallationPath>\SystemCoupling\runTime\winx64\include" OscillatingPlateDamping.c


/FeOscillatingPlateDamping.exe /link /subsystem:console /LIBPATH:"<ANSYSInstallationPath>\SystemCoupling\
runTime\winx64\lib" SysC.SystemCouplingParticipant.lib

Fortran

ifort -I"<ANSYSInstallationPath>\SystemCoupling\runTime\winx64\include\SystemCouplingParticipant\FortranFixedFor
/FeOscillatingPlateDamping.exe OscillatingPlateDamping.f /link /subsystem:console /LIBPATH:"<ANSYSInstallationPa
SystemCoupling\runTime\winx64\lib" SysC.SystemCouplingParticipantFortran.lib

Run the Coupled Analysis


Once you have created the damping solver program, run the coupled analysis using System Coupling.
To do so, execute the run.py script to run System Coupling in batch mode:

<Ansys Installation Directory>/SystemCoupling/bin/systemcoupling -R run.py

If the case runs without issue, you've successfully created a simple damping solver and run a co-
simulation between Mechanical and this damping solver.

Visualize the Results in CFD-Post


Use CFD-Post to view the Mechanical results after the co-simulation run is complete. You'll be able
to plot the plate deformation over time. One point at the top of the plate will be used to track the
displacement in the chart, which is a useful way to verify that the damping occurs due to the force
calculated in the damping solver.

1. Open CFD-Post.

2. Load the Mechanical solver results file.

a. In CFD-Post's toolbar, select File > Load Results.

b. In the Load Results File dialog, select the Mechanical results file that was generated during co-
simulation, file.rst.

The Mechanical results file is loaded and the geometry is displayed on the 3D Viewer tab
of CFD-Post's viewer.

3. Create a point in the solid domain, using node number 77.

This point is at the top corner of the solid body and will be used to track the deformation of the
plate.

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
62 of ANSYS, Inc. and its subsidiaries and affiliates.
Transient Oscillating Plate Damping Tutorial

a. In CFD-Post's toolbar, select Insert > Location > Pointand then click OK to accept the default
name.

b. In the Geometry tab's Details view:

i. Set Domains to All Structural Domains.

ii. Set Method to Node Number.

iii. Set the Node Number field to 77.

c. Click Apply.

On the model, cross-hairs appear on node number 77, so you can see where this point is
on the solid body.

4. Insert a chart for the data at node 77.

To view the deformation using the point you just created, insert an XY Transient Chart for the
data at this node (Point 1). In the chart you create, the x-axis is time and the y-axis is the total
mesh displacement.

a. In CFD-Post's toolbar, select Insert > Chart and then click OK to accept the default name.

b. In Details on the General tab, set Type to XY – Transient or Sequence.

c. On the Data Series tab:

i. Set Name to System Coupling.

ii. Set Location to Point 1.

d. On the X Axis tab, ensure that the Expression is Time.

e. On the Y Axis tab, set Variable to Total Mesh Displacement X.

5. Click Apply to generate the chart of mesh displacement over time.

After the chart is generated, note the damping that is visible in the plate's motion.

Figure 8: Plot of plate tip x-displacements over time for different damping coefficients (p. 64) shows
the chart of plate tip position over time for different damping coefficients. As expected, the higher
the damping coefficient is, the faster the plate oscillations diminish in amplitude.

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 63
System Coupling Participant Library Tutorials

Figure 8: Plot of plate tip x-displacements over time for different damping coefficients

Restart the Coupled Analysis from an Intermediate Point


In the initial coupled analysis, the restart point was generated for every 10th coupling time step. The
damping solver source code shows how to implement the restart APIs for this problem.

• The createRestartPoint function was implemented and registered.

• When the above function is called, it will create a text file and write current nodal coordinates and force
values. This is the only information that is required to cleanly restart the damping solver at a given time
step. The name of the restart file is returned from that function.

• When System Coupling restarts from a given coupling step, it will pass the restart file name as a command
line argument to the damping solver.

• The damping solver will read in the nodal coordinates and forces from the restart file prior to entering
the coupled analysis loop.

The coupled analysis will now be restarted from the 50th step (simulation time 5.0 [s]) and solved
through the 100th step (simulation time 10.0 [s]).

The restart script runRestart.py is provided with this tutorial. To run the coupled analysis, execute
the restart script in batch mode:

<Ansys Installation Directory>/SystemCoupling/bin/systemcoupling -R runRestart.py

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
64 of ANSYS, Inc. and its subsidiaries and affiliates.
Transient Oscillating Plate Damping Tutorial

The coupled analysis should restart and solve until the end. The ANSYS Mechanical results can be
post-processed the same way as for the initial run. Figure 9: Comparison of results between the con-
tinuous and restarted runs (p. 65) shows the chart of the plate tip position over time for the continuous
and restarted analysis. The results show that nearly identical displacements are achieved from the
restarted run. The very slight differences can be attributed to the limitations within System Coupling
and ANSYS Mechanical. See Restarting a Coupled Analysis in the System Coupling User's Guide for a
more detailed description of these limitations.

Figure 9: Comparison of results between the continuous and restarted runs

Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
of ANSYS, Inc. and its subsidiaries and affiliates. 65
Release 2020 R1 - © ANSYS, Inc. All rights reserved. - Contains proprietary and confidential information
66 of ANSYS, Inc. and its subsidiaries and affiliates.

You might also like