See
discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/317935582
Mesh generation for HPC problems: the potential of
SnappyHexMesh
Presentation June 2017
DOI: 10.13140/RG.2.2.25007.53923
CITATIONS READS
0 21
1 author:
Andrea Montorfano
Politecnico di Milano
45 PUBLICATIONS 173 CITATIONS
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
dynamic VOF simulation of GDI injector opening View project
Developments in transient modeling, moving mesh, turbulence and multiphase methodologies in OpenFOAM View project
All content following this page was uploaded by Andrea Montorfano on 27 June 2017.
The user has requested enhancement of the downloaded file.
Mesh generation for HPC problems:
the potential of snappyHexMesh
Dr. Andrea Montorfano
dip. di Energia, Politecnico di Milano
andrea.montorfano@polimi.it
Motivation
CFD work-flow
Pre-processing
Physical Geometrical Geometrical
MESH
problem model (CAD) model (CFD)
Solver setup
solution
Data
processing
Is solution
END
acceptable?
A. Montorfano, mesh generation for HPC problem with snappyHexMesh
Motivation
CFD Vision 2030:
Physical Geometrical Geometrical
MESH
problem model (CAD) model (CFD)
Solver setup
solution
Data
processing
Is solution
END acceptable
?
A single engineer/scientist must be able to conceive, create, analyze, and
interpret a large ensemble of related simulations in a time-critical period
(e.g. 24 hours), without individually managing each simulation.
Source: NASA Contractor Report CR2014-218178 (2014)
A. Montorfano, mesh generation for HPC problem with snappyHexMesh
Motivation
Mesh generation and adaptivity continue to be significant bottlenecks in
the CFD workflow
NASA should develop, fund, and sustain a base research and technology
(R&T) development program for simulation-based analysis and design
technologies:
1. High Performance Computing (HPC)
2. Physical Modeling
3. Numerical Algorithms
4. Geometry and Grid Generation
5. Knowledge Extraction
6. MDAO
Source: NASA Contractor Report CR2014-218178 (2014)
A. Montorfano, mesh generation for HPC problem with snappyHexMesh
Motivation
Panel Discussion at AIAA Aviation 2014
(participants poll)
A. Montorfano, mesh generation for HPC problem with snappyHexMesh
snappyHexMesh
First version released with OpenFOAM-1.5 (2008)
quite a low-profile announcement for the main non-
commercial meshing tool used along with OpenFOAM!
A. Montorfano, mesh generation for HPC problem with snappyHexMesh
Working principle
The Three Stages of SHM
0. A rigorously hexahedral base mesh is generated with an
external tool (e.g. blockMesh)
1. Mesh is refined according to surface intersection and
non-fluid regions are removed
2. Points are snapped onto the surface
3. Hex layers are extruded on specified solid
surfaces
A. Montorfano, mesh generation for HPC problem with snappyHexMesh
Working principle
SHM is based on the octree-refinement algorithm
Cells are refined based on:
Intersection with model surface
Intersection with model edges (aka feature edges)
Distance from surface
Refinement boxes/spheres/cylinders/etc
A. Montorfano, mesh generation for HPC problem with snappyHexMesh
Working principle
Each stage is iterative (until quality constraints are met)
Each stage can be run separately
A. Montorfano, mesh generation for HPC problem with snappyHexMesh
Working principle
The whole process is fully automatic
once the user has
Defined the input geometry
Properly compiled the input dictionary
These actually represent the hard points in the workflow
A. Montorfano, mesh generation for HPC problem with snappyHexMesh
Input geometry
SHM reads:
Pre-defined OpenFOAM primitives (box, sphere, cylinder, etc.)
Triangulated surface formats (STL, OBJ)
The geometry file has to be:
Closed (no holes, crevices)
Clean (no illegal faces)
Properly subdivided (each surface in the geometry file will turn into an
OpenFOAM boundary patch)
Usually the file supplied by the CAD guy cannot be given as-it-is to
SHM
A. Montorfano, mesh generation for HPC problem with snappyHexMesh
Input geometry
Solution #1: Talk with the CAD guy
Solution #2: Use OpenFOAM utilities for
surface manipulation
Solution #3: Use a non-commercial tool
(e.g. SALOME)
Solution #4: use a commercial grid-
generator to clean your STL
A. Montorfano, mesh generation for HPC problem with snappyHexMesh
Input dictionary
All parameters needed for the execution are specified in the
snappyHexMeshDict file
example file has 149 lines (not counting comments, of course)
and with case complexity the dictionary length increases as well
but most parameters can be found once for all, for a given case family!
The lack of a GUI allows an easy coupling with automation
scripting and the deployment of SHM in an HPC context
A. Montorfano, mesh generation for HPC problem with snappyHexMesh
snappyHexMesh in parallel
The three stages can be executed in parallel on several subdomains
Communication is needed to sync points on processor interfaces
Cell number can grow differently among processors
need for dynamic rebalancing
The decomposed mesh is ready to be run, without the need to reconstruct it
A. Montorfano, mesh generation for HPC problem with snappyHexMesh
snappyHexMesh in parallel
Dynamic load balancing is done automatically by ptscotch
Load balancing is time-consuming. Need for a trade-off between
slowdown due to imbalance and rebalancing overhead
A. Montorfano, mesh generation for HPC problem with snappyHexMesh
What you need
An OpenFOAM installation
MPI environment
ptscotch support in OpenFOAM
A good amount of memory: SHM uses a lot of RAM during its operation. A rough guide
is 4 GB per million cells.
Good-level I/O
A remote visualization facility (to debug your meshing process) operating in
client/server mode
A. Montorfano, mesh generation for HPC problem with snappyHexMesh
SHM workflow
Time to prepare the case is
negligible
Castellated mesh phase:
needs few iterations but causes
huge increase in cell count
increasing setup difficulty
increasing wall-clock time
(rebalancing overhead)
always converges
Snap phase:
needs more iterations, does not
increase number of cells
usually converges, some spots
may need attention
Add layer phase
needs still more iterations
adds significant number of cells
does not always converge
A. Montorfano, mesh generation for HPC problem with snappyHexMesh
SHM workflow
Looks like SHM is an user-unfriendly, difficult-to-setup, resource-demanding
tool, however it is used by a large part of the OF community:
considerable cells savings with respect to purely-tet mesh: up to 40% less
elements, due only to the element shape (cut hex vs tet)
Cut-hexes are better handled by OF solvers than tets (better accuracy)
completely automatic run
works in parallel on any number of processors, by default
no license cost
The real bottleneck is preparing the dictionary (requires some iterations),
however:
The same dictionary can be used for different cases of the same family
A large part of the dictionary can be used even for different case families
The really discriminating factor is the users expertise (open-source is not free!)
A. Montorfano, mesh generation for HPC problem with snappyHexMesh
SHM in industrial contexts #1
Scenario: parameter study
find Cx and Cz of vehicle as a function of wing pitch
input: car snappyHexMesh
geometry@ (in parallel)
simpleFoam
(in parallel)
Cx, postProcessing
Cz (in parallel)
repeat
On an HPC cluster the user can run N cases, with Np processor each, at the same
time
A. Montorfano, mesh generation for HPC problem with snappyHexMesh
SHM in industrial contexts #2
Scenario: shape optimization
Optimize aerodynamic surface to have max(Cz), with constraints C1CN
input: parametric
start surface profile:
STL generator
(or morpher)
snappyHexMesh
(in parallel)
(x, y, z) = f(p1pn)
p1.pn simpleFoam
(in parallel)
C1CN
optimization
end driver
Cx
Cz
postProcessing
(in parallel)
(e.g. DAKOTA)
The driver can solve multi-parameter optimization running several cases at the
same time
A. Montorfano, mesh generation for HPC problem with snappyHexMesh
Summary
The combination of:
high parallelism
no license cost
automatic mesh generation
easily interfacing with external scripts/drivers
makes snappyHexMesh a handy instrument for large-scale simulations in HPC
environments
The setup phase requires highly skilled operators
The production phase allows you to fire and forget
A single engineer/scientist must be
able to conceive, create, analyze,
and interpret a large ensemble of
related simulations in a time-
critical period (e.g. 24 hours),
without individually managing
each simulation.
A. Montorfano, mesh generation for HPC problem with snappyHexMesh
View publication stats
Thank
you!
Contact:
Andrea Montorfano, PhD
Dip. di Energia, Politecnico di Milano
+39 02 2399 3804
andrea.montorfano@polimi.it
