V11.

cover

Front cover
Course Exercises Guide
IBM Spectrum Scale Basic
Administration for Linux and AIX
Course code H005G   ERC 4.0
May 2018 edition
Notices
This information was developed for products and services offered in the US.
IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative
for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not
intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or
service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate
and verify the operation of any non-IBM product, program, or service.
IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this
document does not grant you any license to these patents. You can send license inquiries, in writing, to:
IBM Director of Licensing
IBM Corporation
North Castle Drive, MD-NC119
Armonk, NY 10504-1785
United States of America
INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY
KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some jurisdictions do not allow disclaimer
of express or implied warranties in certain transactions, therefore, this statement may not apply to you.
This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein;
these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s)
and/or the program(s) described in this publication at any time without notice.
Any references in this information to non-IBM websites are provided for convenience only and do not in any manner serve as an
endorsement of those websites. The materials at those websites are not part of the materials for this IBM product and use of those
websites is at your own risk.
IBM may use or distribute any of the information you provide in any way it believes appropriate without incurring any obligation to you.
Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other
publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other
claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those
products.
This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible,
the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to
actual people or business enterprises is entirely coincidental.
Trademarks
IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corp., registered in many
jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM
trademarks is available on the web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml.
© Copyright International Business Machines Corporation 2014, 2018.
This document may not be reproduced in whole or in part without the prior written permission of IBM.
US Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
V11.3
Contents

TOC

Contents
Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv

Exercises description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v

Exercise 1. Cluster node preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-1

Part 1: Remote lab access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2
Part 2: Cluster node preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-4

Exercise 2. Installation and cluster configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-1

Part 1: Remote lab access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2
Part 2: Install the Spectrum Scale software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-3
Part 3: Create the Spectrum Scale Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-8
Part 4: Start Spectrum Scale and verify the status of all nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-9
Part 5: Create the Network Shared Disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-10
Part 6: Create a Spectrum Scale cluster file system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-13

Exercise 3. Cluster management and configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-1

Part 1: Explore Spectrum Scale configuration files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2
Part 2: Configuration changes: Adding a node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-4
Part 3: Add NSDs to an active cluster file system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-10
Part 4: Rebalance file system data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-11
Part 5: Delete a disk from an active cluster file system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-12

Exercise 4. Storage pools, filesets, and policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-1

Part 1: Prepare the environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2
Part 2: Create a file system with two storage pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2
Part 3: Create filesets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-5
Part 4: Create a file placement policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-6
Part 5: Verify that the policy was properly installed using the mmlspolicy command.Test the placement
policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-6
Part 6: File management policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-8

Exercise 5. Replication and snapshots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-1

Part 1: Enable replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2
Part 2: Change the failure group on the existing NSDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2
Part 3: Replicate a file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-3
Part 4: Replicate all data in the file system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-4
Part 5: Use a snapshot to backup a file system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-5
Part 6: Restore a file system to a previous snapshot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-6
Part 7: File clones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-6

© Copyright IBM Corp. 2014, 2018 iii

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Trademarks

TMK

Trademarks
The reader should recognize that the following terms, which appear in the content of this training
document, are official trademarks of IBM or other companies:
IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business
Machines Corp., registered in many jurisdictions worldwide.
The following are trademarks of International Business Machines Corporation, registered in many
jurisdictions worldwide:
AIX® BigInsights® Express®
GPFS™ IBM Elastic Storage™ IBM FlashSystem®
IBM Spectrum Accelerate™ IBM Spectrum Archive™ IBM Spectrum Control™
IBM Spectrum Protect™ IBM Spectrum Scale™ IBM Spectrum Storage™
IBM Spectrum Virtualize™ IBM Spectrum™ InfoSphere®
Linear Tape File System™ Passport Advantage® Power Systems™
Power® POWER8® Redbooks®
Storwize® System Storage® Tivoli®
XIV®
Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.
Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other
countries, or both.
UNIX is a registered trademark of The Open Group in the United States and other countries.
Other product and service names might be trademarks of IBM or other companies.

© Copyright IBM Corp. 2014, 2018 iv

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.3
Exercises description

pref

Exercises description
This course includes the following exercises:
• Cluster node preparation
• Installation and cluster configuration
• Cluster management and configuration
• Storage pools, filesets, and policies
• Replication and snapshots
In the exercise instructions, you can check off the line before each step as you complete it to track
your progress.
Most exercises include required sections, which should always be completed. In this course it is
necessary to complete these sections before you can start later exercises. If you have sufficient
time and want an extra challenge, some exercises might also include optional sections that you can
complete.

© Copyright IBM Corp. 2014, 2018 v

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.3
Exercises description

pref
Text highlighting
The following text highlighting conventions are used throughout this book:
Bold Identifies interface controls, and graphical objects such as buttons,
labels and icons that the user selects.
Italics Identifies publication titles, is used where the word or phrase is meant
to stand out from the surrounding text, and identifies parameters
whose actual names or values are to be supplied by the user.
Monospace Identifies file names, directory names, file paths, attributes, variables,
file listings, SMIT menus, code examples, and input and output that
you would see displayed on a terminal.
Monospace bold Identifies commands, subroutines, daemons, and text the user would
type.

© Copyright IBM Corp. 2014, 2018 vi

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 1. Cluster node preparation

EXempty

Exercise 1. Cluster node preparation

Estimated time
00:30

Overview
This exercise introduces the lab environment. The steps also make sure that the nodes in your
assigned lab environment are ready for Spectrum Scale installation and cluster configuration.

Objectives
At the end of the exercise, you should be able to:
• Validate your assigned nodes for Spectrum Scale installation
• Ensure proper network name/address resolution
• Examine the disk configuration
• Configure node to node communication

Requirements
• This workbook
• Access to a set of nodes configured for this course

© Copyright IBM Corp. 2014, 2018 1-1

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 1. Cluster node preparation

EXempty
Exercise instructions

Preface
• Two versions of these instructions are available: one with hints and one without. You can use
either version to complete this exercise. Also, do not hesitate to ask the instructor if you have
questions.
• All exercises in this chapter depend on the availability of specific equipment in your classroom.
• The typical lab environment for this course consists of set of virtual machines (VMs) that are
hosted in a cloud environment. Each student is provided with a URL that allows access to a set
of assigned VMs.
• The VM environment that is used for this course is a common configuration that is shared with a
number of other courses. This course does not use all of the assigned VMs in the lab exercises.
• The diagram below depicts the typical configuration.

Part 1: Remote lab access

Your instructor will provide you with connection information for a set of virtual machines (VMs) that
you will use for the lab exercises in this course. You will use a subset of these VMs to configure a
Spectrum Scale (GPFS) cluster in the lab exercises.
__ 1. Open a web browser, and connect to the URL provided by your instructor. You should see a
set of six VMs. The set of VMs consists of:
- Four Linux VMs that can be used as Spectrum Scale cluster nodes
- One Linux VM that functions as an iSCSI server to provide shared disks to the other
Linux VMs
- One Windows 7 VM that provides a graphical interface from which you can interact with
the Linux VMs

© Copyright IBM Corp. 2014, 2018 1-2

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 1. Cluster node preparation

EXempty
The set of VMs should be similar to the following example:

__ 2. Initially the VMs are in the Powered off state. You must start the VMs in order to use them.
__ a. First, start the VM called Host-5 by clicking the Run button for the VM. This VM functions
as an iSCSI server that provides shared disks to the other Linux VMs, and must be up
and running before the other Linux VMs are started.

__ b. When Host-5 has a state of Running, start the remaining VMs by clicking the Run button
for the entire set.

__ 3. When all of the VMs have a state of Running, click the icon for the Windows VM. Next,
select the Student ID, and log in. The password is: passw0rd
You will use the GUI of the Windows VM to access the command line environment of Linux
VMs as directed in later exercise steps.
In this course, you will use the first three Linux VMs (vm01, vm02, and vm03) to create
various Spectrum Scale cluster configurations. The table below shows the hostname, IP
address, and login information for these VMs.
VM name Host name IP address Login ID Password
Host-1 vm01 192.168.1.1 root passw0rd
Host-2 vm02 192.168.1.2 root passw0rd

© Copyright IBM Corp. 2014, 2018 1-3

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 1. Cluster node preparation

EXempty
VM name Host name IP address Login ID Password
Host-3 vm03 192.168.1.3 root passw0rd
__ 4. Open a terminal session to vm01.
__ a. Double-click the PuTTY icon on the desktop of your Windows VM GUI.

__ b. Open a terminal session to vm01 by double-clicking the vm01 entry in the Saved
Sessions list. Alternatively, you can enter the IP address for vm01 and then click Open.
__ c. Log in as the root user.
__ 5. Using the procedure that is described in the previous step, open one terminal session to
vm02, and one session to vm03. Log in to each session as the root user.

Part 2: Cluster node preparation

Perform all of the remaining steps of this exercise on each of your assigned VMs vm01, vm02, and
vm03. This verifies that each node is configured correctly and is ready to be used as part of a
Spectrum Scale cluster. You can run all of the steps in sequence on vm01, and then repeat the
sequence of steps on vm02, and then vm03. Alternatively, you can run each individual step on all
three nodes at the same time before proceeding to the next step.

Note

You can review the Spectrum Scale (formerly GPFS) FAQ compatibility list to ensure the OS levels
on your VMs are supported.
The Spectrum Scale FAQ:
https://www.ibm.com/support/knowledgecenter/en/STXKQY/gpfsclustersfaq.html

__ 6. The exercises in this course use Spectrum Scale V5.0. Check that the operating system on
the node is supported by the version of Spectrum Scale that will be installed. To determine
the operating system release information for Red Hat Linux, perform the following
command:
cat /etc/redhat-release

© Copyright IBM Corp. 2014, 2018 1-4

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 1. Cluster node preparation

EXempty
__ 7. To ensure proper name resolution, the /etc/hosts file on each node must contain the
hostname and IP address of all nodes in the cluster. If the /etc/hosts file does not include
the hostname and IP address information for vm01, vm02, and vm03, then you need to add
it before you proceed.
__ 8. Ping all of the nodes that will be part of the cluster, including the node from which you are
running the test. This ensures that name resolution is working as expected, and that each
node has networking configured correctly.
__ 9. The Spectrum Scale daemon that runs on each cluster node must be able to communicate
and run commands remotely over a network on all other cluster nodes without being
prompted for a user ID or password. This includes a node being able to run commands on
itself. For this course SSH is being used to provide this functionality.
__ a. Generate a key pair for the node by running the following command:
ssh-keygen -t rsa -N ""
Note: there is no space between the quotation marks, and no passphrase is needed.
Press the Enter key when prompted to save the key in the default location.
__ 10. After generating the key pair for a node, the public component must be copied onto each
node that you want to be able to log in to without being prompted for a password. Use the
command sequence shown below to copy the public key file from the current node onto all
the nodes that will be part of the cluster:
ssh-copy-id -i root@vm01
ssh-copy-id -i root@vm02
ssh-copy-id -i root@vm03
Enter yes if prompted to continue connecting. When prompted, enter the root password for
the target node.
__ 11. Once you have copied the public key to all three nodes that will be part of the cluster, run
the command sequence shown below. This verifies that the root user on the current node
can log in to all of the nodes that will be part of the cluster without being prompted for a
password.
ssh vm01 date
ssh vm02 date
ssh vm03 date

Note

Once ssh has been properly configured on all three nodes that will be part of the cluster, you can do
almost everything you need to do from vm01.

__ 12. Run the following command to ensure that no log in banners are displayed when ssh is
used to log in to the current node using an appropriate key:
touch ~/.hushlogin

© Copyright IBM Corp. 2014, 2018 1-5

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 1. Cluster node preparation

EXempty
__ 13. Examine the PATH environment variable to make sure that it includes the
/usr/lpp/mmfs/bin directory. If the PATH environment variable does not include the
directory, then you need to add it.
There are a number of different methods that can be used to automatically update the PATH
environment variable with a custom setting. One method is to add the customization of the
PATH environment variable to the end of the /root/.bashrc file. Remember to source the
file after adding the PATH customization to update the current environment.
__ 14. Verify that the shared disks that will be configured in the Spectrum Scale cluster are
available on the current node. The commands to verify the disks depend on the type of
shared disk technology being used, and the particular operating system being used on the
current node. The typical lab environment for this course uses iSCSI technology to
implement the shared disk configuration.
__ a. Use the command shown below to display the iSCSI configuration for the current node.
The output should indicate that the node has access to eight iSCSI disks, labeled as
sde, sdf, sdg, sdh, sdi, sdj, sdk, and sdl.
iscsiadm -m session -P 3

Note

Spectrum Scale does not require that each disk have the exact same device name on each node in
the cluster. However, it easier for us as humans to keep track of the disks if they do have the same
device name on each node. If sdb on one node is the same physical disk as sde on a different
node, the Spectrum Scale daemon is able to map the NSD ID properly. When you configure a disk
as an NSD, Spectrum Scale assigns a new NSD ID to the device and that information is placed in a
reserved area at the beginning of the disk. That header is read every time Spectrum Scale starts
and is used by the Spectrum Scale daemon when accessing and managing NSDs.

Important

All of the steps that are performed in Part 2, "Cluster node preparation" of this exercise must be
performed on all three of the nodes (vm01, vm02, and vm03) that will be configured in the
Spectrum Scale cluster in later exercises.

End of exercise

© Copyright IBM Corp. 2014, 2018 1-6

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 2. Installation and cluster configuration

EXempty

Exercise 2. Installation and cluster

configuration
Estimated time
01:30

Overview
This exercise covers the steps that are required to install IBM Spectrum Scale. The exercise also
covers the steps that are required for basic cluster configuration for a two node cluster.

Objectives
At the end of the exercise, you should be able to:
• Install the Spectrum Scale software
• Create a Spectrum Scale cluster
• Configure network shared disks (NSDs)
• Create a Spectrum Scale file system

Requirements
• This workbook
• Access to a set of nodes configured for this course

© Copyright IBM Corp. 2014, 2018 2-1

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 2. Installation and cluster configuration

EXempty
Exercise instructions

Preface
• Two versions of these instructions are available: one with hints and one without. You can use
either version to complete this exercise. Also, do not hesitate to ask the instructor if you have
questions.
• All exercises in this chapter depend on the availability of specific equipment in your classroom.
• The typical lab environment for this course consists of set of virtual machines (VMs) that are
hosted in a cloud environment. Each student is provided with a URL that allows access to a set
of assigned VMs.
• The VM environment that is used for this course is a common configuration that is shared with a
number of other courses. This course does not use all of the assigned VMs in the lab exercises.

Part 1: Remote lab access

Your instructor provided you with connection information for a set of virtual machines (VMs) that
you can use for the lab exercises in this course. VMs in the cloud environment that is used for this
course are placed in a suspended state after a period of inactivity. You might need to resume the
VMs before you can continue with the exercise.
__ 1. Open a web browser, and connect to the URL provided by your instructor.
__ a. If you see the listing of your assigned set of six VMs, proceed to Step 2.
__ b. If you still have the browser session that is connected to your Windows VM that you
started for the previous exercise, then move the cursor to check that the VM is still
running. If the Windows VM is running, then proceed to Step 2.
__ c. If the Windows VM session displays the following symbol, then the VM is in the
suspended state.

__ d. If the Windows VM is suspended, click the Environment VMs button on the left side of
the Skytap menu bar at the top of the screen, then select View All VMs from the list.

© Copyright IBM Corp. 2014, 2018 2-2

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 2. Installation and cluster configuration

EXempty
__ 2. VMs in the cloud environment that is used for this course are placed in a suspended state
after a period of inactivity. Before you continue with the exercise, ensure that all of your VMs
are in the Running state.
__ a. If your assigned VMs are in the Suspended state, first resume Host-5 by clicking the
Run button for the VM, as described in the previous exercise. Next, when Host-5 is in
the Running state, resume the remaining VMs.
__ 3. When all VMs are in the Running state, select the Windows VM to connect to the graphical
interface.
__ 4. If the VMs were suspended, close any existing PuTTY sessions, and establish new ones to
each of your assigned VMs vm01, vm02, and vm03, and log in as the root user.

Part 2: Install the Spectrum Scale software

You must perform all of the steps in this part of the exercise on each of your assigned VMs vm01,
vm02, and vm03. These steps install the Spectrum Scale software and build the GPFS Portability
Layer (GPL). You can run all of the steps in sequence on vm01, and then repeat the sequence of
steps on vm02, and then vm03. Alternatively, you can run each individual step on all three nodes at
the same time before you proceed to the next step.

Note

Copy and paste functions as expected between applications that are running in the Windows VM
GUI. For example, to copy text from one PuTTY session to another:
• First, click the left mouse button and drag the cursor to highlight the text to be copied from the
source PuTTY session. Releasing the left mouse button copies the highlighted text to the
clipboard in the Windows VM.
• Next, place the cursor in the destination PuTTY session and click the right mouse button to
paste the text from the clipboard in the Windows VM.

__ 5. Change directory to /spectrum-scale-course/software/base/5.0. This directory

contains the file Spectrum_Scale_STD_500_x86_64_LNX.tar.gz, which is the installer
image for IBM Spectrum Scale Standard Edition V5.0 for Linux on 64-bit x86 hardware.
__ 6. Use the ls -l command to list the contents of the directory.
__ 7. Use the tar command to extract the contents of the tar archive for IBM Spectrum Scale
Standard Edition V5.0 for Linux on 64-bit x86 hardware. Then, use the ls command to list
the files that were extracted.

© Copyright IBM Corp. 2014, 2018 2-3

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 2. Installation and cluster configuration

EXempty

Note

You should observe that two new files are extracted. One of the files is the README file, and the
other file is the self-extracting installer package. If you run the self-extracting installer package script with -?
as an argument, it shows the syntax and possible options available.
By default, the self-extracting installer attempts to start a graphical version of the license acceptance process
tool. If your node does not have an X11 GUI, then you can use the --text-only argument to accept the
license agreement from a terminal session. Using the --silent argument starts the license acceptance
process tool in silent mode so that you don’t need to manually accept the licenses interactively as the installer
is running.
The name of the Spectrum Scale self-extracting installer package file is made up of multiple elements. The
different elements indicate the edition (Standard, Advanced, or Data Management), version, and operating
system and hardware platform combination that is supported by the images that are contained in the file.

© Copyright IBM Corp. 2014, 2018 2-4

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 2. Installation and cluster configuration

EXempty
__ 8. Run the self-extracting installer package with the -? argument to view the different options
available for running this command.

© Copyright IBM Corp. 2014, 2018 2-5

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 2. Installation and cluster configuration

EXempty
__ 9. Run the self-extracting shell script to extract the RPM package files. Use either the
--text-only or the --silent argument. If desired, you can run the installer on one node
with the --text-only argument to experience the process of manually accepting the
license. You can then use the --silent argument when running the script on the remaining
nodes.

© Copyright IBM Corp. 2014, 2018 2-6

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 2. Installation and cluster configuration

EXempty
__ 10. By default the self-extracting package puts the Spectrum Scale product images files into a
directory hierarchy that is rooted at /usr/lpp/mmfs/5.0.0.0. Use the cd command to
change to the /usr/lpp/mmfs/5.0.0.0/gpfs_rpms directory, which contains the RPM
images suitable for Red Hat Linux.
__ 11. Use the rpm command to install the required RPM packages for Spectrum Scale Standard
Edition on Red Hat Linux:
- gpfs.base-5.0.*.rpm
- gpfs.gpl-5.0.*.noarch.rpm
- gpfs.msg.en_US-5.0.*.noarch.rpm
- gpfs.gskit-8.0.50*.rpm
- gpfs.license*.rpm
- gpfs.ext-5.0.*.rpm
- gpfs.compression-5.0.*.rpm
__ 12. The man pages for Spectrum Scale commands are supplied in a separate RPM, which is
not a required package. However, it is a good idea to install this RPM on at least one of your
nodes. Use the rpm command to install the documentation RPM package for Spectrum
Scale Standard Edition on Red Hat Linux:
- gpfs.docs-5.0.*.noarch.rpm

Note

The Linux operating system configuration that is used for the lab environment uses a LANG
environment setting of en_US.UTF-8. This setting appears to cause problems with the formatting of
Spectrum Scale man pages. If you set the LANG environment variable to a value of either en_US, or
C, the Spectrum Scale man pages are formatted correctly.
Setting the LANG environment variable can be performed as-needed when invoking the man
command to view a Spectrum Scale man page. For example: LANG=C man mmcrcluster

__ 13. Use the rpm command to list the Spectrum Scale packages that are installed on the node.
__ 14. Run the following command to build the GPFS Portability Layer (GPL) for Linux:
mmbuildgpl

Note

If you receive the message that is shown below when you run the mmbuildgpl command, then it
means that the /usr/lpp/mmfs/bin directory has not been added to the PATH environment
variable.
[root@vm02 gpfs_rpms]# mmbuildgpl
-bash: mmbuildgpl: command not found

© Copyright IBM Corp. 2014, 2018 2-7

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 2. Installation and cluster configuration

EXempty
Revisit Step 13 on page 1-6 of the previous exercise to ensure that the PATH environment variable
is configured correctly.

Important

All of the steps that are performed in Part 2, "Install the Spectrum Scale software" of this exercise
must be performed on all three of the nodes (vm01, vm02, and vm03). The nodes vm01 and vm02
will be configured as a cluster in the next part of the exercise. Although vm03 is not added to the
cluster in this exercise, it is best to complete the configuration at this time since it will be added to
the cluster in a later exercise.

Part 3: Create the Spectrum Scale Cluster

In this exercise, the cluster is initially created on two nodes (vm01 and vm02). In a later exercise,
you will add the node vm03.
For this part of the exercise, you are working mainly on the node vm01. Pay particular attention to
the instructions in each step to make sure that you are running the command on the correct node.
__ 15. First, on vm01 issue the mmcrcluster command without arguments to see the options that
are needed to create a cluster.
__ 16. On vm01, use the cd command to change to the /spectrum-scale-course/data directory.
__ 17. On vm01, use vi or a similar editor to create a node descriptor file called nodes.txt in the
/spectrum-scale-course/data directory. The node descriptor file contains information
about the nodes that are to be configured in the Spectrum Scale cluster. For this exercise,
you should include the information that is required for vm01 and vm02. Both nodes are to be
designated as quorum and manager nodes.
__ 18. Before creating the cluster, make sure that you have installed the Spectrum Scale RPMs on
the node vm02.
__ 19. Use the mmcrcluster command to create a new cluster. Specify /usr/bin/ssh as the
remote shell command, and /usr/bin/scp as the remote copy command. Specify clust1
as the name of the cluster.
__ 20. Use the mmlslicense command to check the license designation for the nodes in the
cluster.

__ 21. Use the mmchlicense command to assign the proper license designation to the nodes in the
newly created cluster.
__ 22. Run the mmlslicense command again to confirm that there are no nodes that still require a
license designation.

© Copyright IBM Corp. 2014, 2018 2-8

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 2. Installation and cluster configuration

EXempty

Note

Spectrum Scale requires that each node in the cluster is designated with the appropriate node
license. Spectrum Scale nodes that have a direct connection to the storage devices, nodes that
serve as quorum nodes, file system manager nodes, NSD server nodes, or nodes used to export
the Spectrum Scale file system data by using a network file sharing protocol such as HTTP, FTP, or
NFS require a server node license. Nodes that do not have direct connection to the storage are
considered client or consumer nodes, and as such can use the client node license.

__ 23. Use the mmlscluster command to display information about the cluster.

Note

Notice the name of the cluster.

• If no cluster name is specified on the mmcrcluster command line, the hostname of the first
quorum node is used as the cluster name.
• If the specified cluster name contains dots, it is assumed to be a fully qualified (unique) name.
• If the specified cluster name does not contain dots, then it is assumed to not be unique. In this
case, the domain name of the first quorum node is added to the specified cluster name. If the
first quorum node does not have a fully qualified hostname, then the short hostname of the
node is added to the specified cluster name.

Part 4: Start Spectrum Scale and verify the status of all nodes
__ 24. On vm01, use the mmgetstate command to check the current state of all the cluster nodes.
The current state is expected to be listed as down since Spectrum Scale has not been
started on either node yet.
__ 25. On vm01, start the Spectrum Scale daemon on all of the nodes in the cluster by using the
mmstartup command.
__ 26. When the cluster starts, use the mmgetstate command again to check the current state of
the cluster nodes and make sure that the nodes are active. It can take a few minutes for the
state of each node to show as active.

Note

If both nodes are in the active state, then the cluster is ready to use. If you have any status other
than active, then wait a few minutes and try the mmgetstate command again.

© Copyright IBM Corp. 2014, 2018 2-9

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 2. Installation and cluster configuration

EXempty

Important

The Linux nodes configured for the class are running Red Hat Enterprise Linux 7.1. The default
configuration of RHEL 7.1 is to use the xfs file system type for the default file systems used to store
the operating system.
When GPFS starts on a node that uses xfs for the root file system, a problem is detected by the abrt
subsystem, and sent to the root user as an email message. The subject line of the email is [abrt]
full crash report. The first line of the backtrace section included in the email message is:
:WARNING: at fs/xfs/xfs_aops.c:989 xfs_vm_writepage+0x58e/0x5d0 [xfs]()
In addition to the email message, the user will be informed of the problem with a message that is
displayed at login. For example:
ABRT has detected 1 problem(s). For more info run: abrt-cli list --since 1491239313
If the abrt-cli list command is run, the output is similar to the following example:
[root@vm03 ~]# abrt-cli list
id 3e929fc2891999700b62ea24f911acc35bc261cb
reason: WARNING: at fs/xfs/xfs_aops.c:989 xfs_vm_writepage+0x58e/0x5d0 [xfs]()
time: Tue 04 Apr 2017 02:21:13 PM EST
cmdline: BOOT_IMAGE=/vmlinuz-3.10.0-229.el7.x86_64 root=/dev/mapper/rhel00-root ro
crashkernel=auto rd.lvm.lv=rhel00/root rd.lvm.lv=rhel00/swap rhgb quiet LANG=en_US.UTF-8
package: kernel
count: 3
Directory: /var/tmp/abrt/oops-2017-03-04-14:21:13-8533-0
Reported: cannot be reported
The warning is generated because of the interaction of GPFS with the Linux kernel memory
management layer. The message is not a kernel Oops, but a Warning. This is a known issue, and
can safely be ignored. It does not affect the operation or performance of GPFS. One way in which
this problem can be avoided is to use the ext4 file system type instead of the default xfs file system
type when installing the RHEL operating system.

Part 5: Create the Network Shared Disks

In this part of the exercise, you use four of the eight shared disks provided by your instructor and
identified in the previous exercise.
Before a disk can be used in the Spectrum Scale cluster, it must be defined as a cluster NSD
device.
By default, a unique name is given to each NSD that is defined, such as “gpfs1nsd”. This unique
NSD name is stored in a special header area on the disk and becomes the global device name
used by Spectrum Scale.
When Spectrum Scale is started on a cluster node, by default it examines all of the disk devices on
that cluster node and determines whether a disk is an NSD. If it is, it maps the relationship between
the NSD name and the local device name on that node.

© Copyright IBM Corp. 2014, 2018 2-10

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 2. Installation and cluster configuration

EXempty
__ 27. Use vi or a similar editor to create an NSD stanza file called
/spectrum-scale-course/data/diskdesc.txt using the required stanza format, creating
one stanza entry for each disk device. Name the NSDs nsd1, nsd2, and so on. Example
stanza entries:
%nsd:
nsd=nsd1
device=/dev/sde
usage=dataAndMetadata
servers=vm01,vm02
%nsd:
nsd=nsd2
device=/dev/sdf
usage=dataAndMetadata
servers=vm01,vm02
%nsd:
nsd=nsd3
device=/dev/sdg
usage=dataAndMetadata
servers=vm02,vm01
%nsd:
nsd=nsd4
device=/dev/sdh
usage=dataAndMetadata
servers=vm02,vm01
- For sde and sdf, set the primary server to vm01, and the backup server to vm02.
- For sdg and sdh, set the primary server to vm02, and the backup server to vm01.
- Disk usage for all the devices is both data and metadata.

Note

If you do not specify a name for an NSD device in the stanza, the mmcrnsd command automatically
defines an NSD name for the device and updates the stanza in the file. Automatically assigned
names are of the form gpfs1nsd, gpfs2nsd, and so on.
It is recommended that you specify a server list for all NSDs. If you do not define a server list,
Spectrum Scale assumes that the disk is SAN-attached to all of the nodes in the cluster. In this
case, if a node loses the SAN connection it will not be able to use the NSD protocol to connect to
the storage by using a network connection.
You can have up to eight nodes in the servers list for each device. You should define a primary and
secondary NSD server as a minimum.
By default, when Spectrum Scale starts on a node, it discovers the NSDs that can be accessed by
using a direct SAN connection. If an NSD disk is direct SAN-attached to a node, Spectrum Scale
will use the local SAN path. If SAN access fails, Spectrum Scale can access the NSD device by
using the NSD protocol over the network. This can only be done if the NSD disk has a valid network
path defined.

© Copyright IBM Corp. 2014, 2018 2-11

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 2. Installation and cluster configuration

EXempty
The usage clause of the stanza is ignored by the mmcrnsd command, and passed unchanged to the
output file produced by the mmcrnsd command.

__ 28. Create a backup copy of the descriptor file; you will need it later during another exercise.
cd /spectrum-scale-course/data/
cp diskdesc.txt diskdesc_bak.txt
__ 29. Now use the mmcrnsd command to create the NSDs using the stanza file that you created in
Step 27. This is the process where unique NSD identifiers are written to each disk. This
ensures that the NSD devices are available to be used for a Spectrum Scale file system.

Note

If you get an error similar to mmcrnsd: Disk device sde refers to an existing NSD, then that
means that the disk already contains NSD information and has been used as an NSD disk in
another cluster. You can use the -v no argument with the mmcrnsd command to specify that the
disks are to be formatted as NSDs regardless of their previous state or content. The default for this
argument is -v yes which means that if the disk has been used as an NSD in the past and still has
NSD information on it the operation fails. Using the -v no argument is something that should be
used with extreme caution on production systems to prevent damage to NSD disks and Spectrum
Scale file systems.

__ 30. Now use the cat command to look at the diskdesc.txt file. The mmcrnsd command
comments out the stanza of any device that encountered an error and could not be
configured. In addition, the stanza for any device that did not specify an NSD name in the
original file is updated to show the new assigned NSD name. Do you notice any changes in
the file?
__ 31. List the NSD configuration by using the mmlsnsd command. At this point, all of the NSDs are
marked as free; this allows them to be used to create Spectrum Scale cluster file systems.

© Copyright IBM Corp. 2014, 2018 2-12

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 2. Installation and cluster configuration

EXempty
Part 6: Create a Spectrum Scale cluster file system
Now that a Spectrum Scale cluster and some NSD devices have been created, you can create a
cluster file system. In this part of the exercise, you use the four NSDs to create a new cluster file
system.
__ 32. Create a shared cluster file system with the attributes listed below by using the mmcrfs
command.
- Set the file system device name to be fs1
- Set the block size to 64 KB
- Set the file system mount point at /gpfs
- Use the NSDs listed in the stanza file /spectrum-scale-course/data/diskdesc.txt
- Set the file system to be automatically mounted when the mmfsd daemon is started

Note

If this operation fails, it is most likely because the disks have been used with a different Spectrum
Scale cluster file system in the past and still have old information that prevents them from being
used. Use the -v no argument at the end of the mmcrfs command to force the disks to be reused to
create the new file system.

__ 33. Use the mmmount command to mount the file system on all the nodes.
__ 34. Use the mmlsfs command to list the attributes of the newly created file system.
__ 35. Use the mmdf command to list the file system usage information.
__ 36. To query all disks in the fs1 file system with the number of blocks automatically scaled to an
easy-to-read value, run the mmdf command with the --block-size auto argument.

Note

One of the things you might notice in the mmdf output is that one of the four disks has a slightly
larger amount of free space than the other disks in fs1. This is because Spectrum Scale places a
file system descriptor file on some of the disks, but not all of the disks. This information will be
discussed in the next unit in more detail.

__ 37. How many inodes are currently used in the file system? ______________
__ 38. Use the df command to verify that the file system is mounted properly.

End of exercise

© Copyright IBM Corp. 2014, 2018 2-13

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 3. Cluster management and configuration

EXempty

Exercise 3. Cluster management and

configuration
Estimated time
01:15

Overview
In this exercise, you examine the IBM Spectrum Scale structure, and see where the cluster
information and tools are placed.

Objectives
At the end of this exercise, you should be able to:
• Document where Spectrum Scale configuration information is stored
• Make configuration changes to a Spectrum Scale cluster
• Add a node to an existing cluster
• Add NSDs to an existing file system

Requirements
• This workbook
• Access to a set of nodes configured for this course
• Exercise 1 and exercise 2 completed successfully

© Copyright IBM Corp. 2014, 2018 3-1

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 3. Cluster management and configuration

EXempty
Exercise instructions

Preface
• Two versions of these instructions are available: one with hints and one without. You can use
either version to complete this exercise. Also, do not hesitate to ask the instructor if you have
questions.
• All exercises in this chapter depend on the availability of specific equipment in your classroom.
• The typical lab environment for this course consists of set of virtual machines (VMs) that are
hosted in a cloud environment. Each student is provided with a URL that allows access to a set
of assigned VMs.
• The VM environment that is used for this course is a common configuration that is shared with a
number of other courses. This course does not use all of the assigned VMs in the lab exercises.

Part 1: Explore Spectrum Scale configuration files

You can find the Spectrum Scale executable files, data files, and log files under three directories on
the system:
•/usr/lpp/mmfs
•/var/mmfs/
•/var/adm/ras
__ 1. Open a PuTTY session to vm01, and log in as the root user. Refer to Part 1, "Remote lab
access," on page 2-2 of the previous exercise to determine whether you need to resume
any suspended VMs.
__ 2. Explore the content of the directories that are under /usr/lpp/mmfs.
__ 3. What is the name of the sample utility for backing up a Spectrum Scale file system to Tivoli
Storage Manager? (Hint: It is found in the /usr/lpp/mmfs/bin directory).
___________________________________
__ 4. What subdirectories are in /var/mmfs?
▪ ____________________
▪ ____________________
▪ ____________________
▪ ____________________
▪ ____________________
▪ ____________________
▪ ____________________
▪ ____________________
▪ ____________________
__ 5. Find the mmsdrfs file. The mmsdrfs file is a text file that can be viewed, but should never be
changed with a text editor. Open or view contents of the file. What kind of information does
this file appear to contain?

© Copyright IBM Corp. 2014, 2018 3-2

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 3. Cluster management and configuration

EXempty
________________________________________________________________________

Important

Do not ever edit the mmsdrfs file manually. The file should be backed up for disaster recovery
purposes.

__ 6. What type of Spectrum Scale information appears in the /var/adm/ras directory?

________________________________________________________________________

© Copyright IBM Corp. 2014, 2018 3-3

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 3. Cluster management and configuration

EXempty
Part 2: Configuration changes: Adding a node
In this part of the exercise, you have an opportunity to change some configuration settings and see
what happens. You also add a third node (vm03) to your existing cluster.
__ 7. Open a PuTTY session to vm03 and log in as the root user.
__ 8. Make sure that the following commands can be run from vm03 without being prompted for a
password.
ssh vm01 date
ssh vm02 date
ssh vm03 date
If any of these commands prompt you for a password, return to Part 2, "Cluster node
preparation," on page 1-4 of Exercise 1 and complete the required configuration.
__ 9. On vm03, use the rpm -qa |grep gpfs command to verify that Spectrum Scale was
properly installed. If Spectrum Scale was not properly installed, return to Part 2, "Install the
Spectrum Scale software," on page 2-3 of Exercise 2 and complete steps required for
Spectrum Scale installation.
__ 10. On vm03, use the echo $PATH command to verify that the /usr/lpp/mmfs/bin directory is
properly defined in the PATH. If this directory is not in the path, then you will not be able to
run any Spectrum Scale commands.
__ 11. The lab environment that is used for this course is shared by a number of other courses.
Currently, vm03 has direct access to the same set of shared disks already being used by
vm01 and vm02. In this course, vm03 is to be configured as an NSD client, where the node
ignores any direct attachment to NSD devices, and instead sends all Spectrum Scale block
I/O over the TCP/IP network to the NSD servers vm01 and vm02. The easiest way to do this
is to create a script that does not produce a device list. This causes Spectrum Scale to stop
looking for locally attached devices.
On vm03, create the file /var/mmfs/etc/nsddevices that contains the following two lines:
#!/bin/ksh
exit 0
When Spectrum Scale is started on vm03, it runs this script to look for direct attached NSD
devices. The exit 0 indicates that the script completed successfully, and that Spectrum
Scale should not continue to detect locally attached devices.
__ 12. On vm03, make sure that the /var/mmfs/etc/nsddevices script is executable.
chmod +x /var/mmfs/etc/nsddevices
__ 13. On vm01, use the mmaddnode command to add vm03 to your Spectrum Scale cluster. Use a
node designation of nonquorum-client. A client designation provides a lower license cost
and from an application point of view, the Spectrum Scale file system behaves the same as
on the other nodes. Since you are adding a single node, it is easier to use the command line
interface to provide the node designation rather than creating a node file.
__ 14. Use the mmlslicense command to view the cluster license designations. Do any of your
nodes require license designations?

© Copyright IBM Corp. 2014, 2018 3-4

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 3. Cluster management and configuration

EXempty
__ 15. Use the mmchlicense command to assign the client license designation to the newly added
node.

© Copyright IBM Corp. 2014, 2018 3-5

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 3. Cluster management and configuration

EXempty
__ 16. Use the mmlslicense command again to verify that vm03 now has the cluster license
designation of client.
__ 17. Use the mmlsnode and the mmlscluster commands to list all the nodes in your cluster and
their designations.

© Copyright IBM Corp. 2014, 2018 3-6

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 3. Cluster management and configuration

EXempty
__ 18. Use the mmgetstate command to check to see whether the Spectrum Scale mmfsd daemon
is active on all three nodes. If the mmfsd daemon is not currently active on the newly added
node, use the mmstartup command to activate it.

Note

If all three nodes are in the active state, then the cluster and all cluster nodes are ready to use. If
any nodes have a status other than active, then wait for 3 - 5 minutes and try the mmgetstate
command again.

__ 19. On vm03, use the df command to verify that the node has access to the /gpfs file system.

© Copyright IBM Corp. 2014, 2018 3-7

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 3. Cluster management and configuration

EXempty
__ 20. Run the mmlsdisk fs1 -m command on vm01, vm02, and vm03. Compare the results.
What do you notice?
_______________________________________________________________________
__ 21. Now that vm03 has been added to the cluster, you can use it to run cluster management
commands if you so desire. For these next lab steps, the author of these materials chose to
use vm03 to run the commands, but you can instead use vm01 or vm02.
__ a. Use the mmlsmgr command to view the file system manager for the file system fs1.
__ 22. Use the mmchmgr command to change the file system manager to vm02, and then verify that
the change was successful.
__ 23. Can vm03 become the file system manager for fs1?
__ 24. Use the mmchmgr command to restore the file system manager role for fs1 back to vm01.

© Copyright IBM Corp. 2014, 2018 3-8

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 3. Cluster management and configuration

EXempty
__ 25. Use the mmlsmgr -c command to see which node is acting as the cluster manager.

Note

The mmchmgr -c command can be used to configure a different node to act as the cluster manager.

__ 26. Use the mmlsconfig command to view the cluster configuration.

__ 27. Is the Spectrum Scale daemon configured to autoload?
__ 28. Use the mmchconfig command to set autoload so that the IBM Spectrum Scale daemon
starts automatically on each node when the operating system starts.
__ 29. Use the mmlsconfig command to verify that the autoload attribute was changed.

Note

The mmlsconfig output shows only certain parameters that have been changed from the default
value, or set by Spectrum Scale. In the next step, you will see a method to view a full list of
parameters and their current values.

__ 30. Use the mmfsadm dump config command to view all of the Spectrum Scale parameters.
The output can be long so you might want to pipe the output to more or pg so that you can
look through it.
__ 31. What is the value of pagepool parameter? ____________________
__ 32. Use the mmchconfig command to change the pagepool value to 800 MB, and have the
change take effect immediately. Confirm the change by running the mmfsadm command
again.

Note

This step is just an example of how to dynamically change the pagepool size to an arbitrary value,
which in this case is 800 MB. Typically this value would be adjusted only to meet file system activity
needs.
The default pagepool value is either one-third of the physical memory on the node or 1 GB,
whichever value is smaller.
For more information about tuning the pagepool size, see the mmchconfig man pages and the
Spectrum Scale WIKI page:
https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/General%20Parallel
%20File%20System%20%28GPFS%29/page/Tuning%20Parameters

© Copyright IBM Corp. 2014, 2018 3-9

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 3. Cluster management and configuration

EXempty
Part 3: Add NSDs to an active cluster file system
Spectrum Scale does not use a Logical Volume Manager. However, the size of an existing file
system can be dynamically changed by adding or removing NSD disks. If you need more disk
space on active and mounted file systems, you must add SAN LUNs defined as NSDs.

Note

Spectrum Scale does not support the dynamic resizing of an existing Spectrum Scale NSD disk by
a storage subsystem. If the size of the underlying structure (LUN) of an NSD is changed, it
damages the structure of the NSD and you lose all of the data on that NSD.

__ 33. On vm01 (or vm02), create a disk descriptor file /spectrum-scale/data/adddisk.txt.

This file is used for adding the new disks. Use the disks that are labeled /dev/sdi and
/dev/sdj and create the descriptor file by using the format as shown in the example below.
In the previous exercise, you defined an NSD name for each device in the stanza file when
configuring NSDs. This time you will let the mmcrnsd command define the NSD name
automatically.
Example command and output:
[root@skylab1 gpfs]# cat adddisk.txt
%nsd:
device=/dev/sdi
usage=dataAndMetadata
servers=vm01,vm02
%nsd:
device=/dev/sdj
usage=dataAndMetadata
servers=vm02,vm01
__ 34. Create a backup copy of the descriptor file. This allows you to compare the original data
with the resulting file after it has been modified by the mmcrnsd command.
cd /spectrum-scale-course/data/
cp adddisk.txt adddisk_bak.txt
__ 35. Use the mmcrnsd command to create the two new NSD devices for the cluster. You may
have to use the parameters -v no to force the creation of the NSDs.

© Copyright IBM Corp. 2014, 2018 3-10

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 3. Cluster management and configuration

EXempty
__ 36. Display the contents of both the adddisk.txt file and the adddisk_bak.txt file. What
differences do you observe?
_____________________________________________________________________

Note

In the previous exercise, you manually assigned a name of the form nsdN for each NSD in the
configuration file before running the mmcrnsd command. In this exercise, you allowed the mmcrnsd
command to automatically generate a name for each of the new NSDs (resulting in names of the
form gpfsNnsd). In a production environment, the administrator will choose one method of naming
(manual or automatic name assignment) when configuring NSDs at initial cluster creation. The
administrator typically continues to use the selected method when adding any additional NSDs.

__ 37. Verify that the NSD disks have been created by using the mmlsnsd command.
__ 38. Before you add the new NSDs to a file system, use the mmlsdisk command to list the disks
that are currently configured for fs1.
__ 39. Use the command that is shown below to create a large file in the fs1 file system. This file is
being placed there so that you can observe the data striping behavior of Spectrum Scale,
and to observe the behavior when extra space is added to a file system. It should take less
than 2 minutes for the command to complete.
dd if=/dev/zero of=/gpfs/bigfile bs=1M count=2048
__ 40. Use the mmdf command to examine the current capacity of the fs1 file system.
__ 41. Use the mmadddisk command to add the new NSDs to the fs1 file system.
__ 42. Use the mmlsdisk command to display the current configuration and state of the disks in the
fs1 file system, and to verify that the two NSDs have been properly added.
__ 43. Use the mmdf command to display the current capacity of the fs1 file system. You should
note that none of the space in the newly assigned disks is being used to store data.

Part 4: Rebalance file system data

Note

Spectrum Scale automatically spreads the file system data across all of the NSD disks that are
configured in the file system. Suppose that your file system is almost full and you add an additional
NSD to increase the file system size. When you add an NSD disk, that NSD disk is empty and the
existing disks in the file system are full, or at least contain data. As new files are created in the file
system the new space is allocated mainly from the new empty disk. This can lead to unbalanced I/O
for this file system.
You can restripe the file system to rebalance the spread of data across all of the NSDs in the file
system. The mmrestripefs command can be used to rebalance the data. You should consider the
necessity and benefits of restriping the data, and also consider the current demands on the system
before running this command. Restriping a large file system requires many insert and delete

© Copyright IBM Corp. 2014, 2018 3-11

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 3. Cluster management and configuration

EXempty
operations, and may affect system performance. In a production environment, this operation should
be done when the system demands are low.

__ 44. Use the mmrestripefs command to rebalance the existing data in the fs1 file system. It
may take a few minutes for the command to complete.
__ 45. After the mmrestripefs command completes, use the mmdf command again to display the
capacity of the fs1 file system. Now you should note that the data has been rebalanced
across all of the NSDs in the file system.

Part 5: Delete a disk from an active cluster file system

Removing a disk is accomplished with the mmdeldisk command, passing the file system name and
disk (NSD) you want to delete as parameters. As with the mmadddisk command, you can also
specify that the file system should be restriped by using the -r and -a options to run the restripe in
the background.
As with the mmrestripefs command, the mmdeldisk command handles this task easily when the
system utilization is low. Under load, it can take a significant amount of time. If this command fails,
run the mmgetstate -a command to verify the state of your cluster nodes. In the case of this lab
step, if there is a heavy amount of I/O activity from the previous mmrestripefs command you might
need to wait a few minutes before running mmdeldisk again.
__ 46. Use the mmdeldisk command to remove the two NSD disks you added to fs1 in Step 41. Be
sure to use the appropriate options to restripe the file system. It may take a few minutes for
the command to complete.

Note

The -a option indicates asynchronous rebalancing should be used. Use the mmlsdisk and mmlsnsd
commands to verify the successful removal of the disks. If replication was implemented, then the -r
option can be used to preserve the data and metadata replication factors.
The disks to be deleted by the mmdeldisk command must be up and running for the command to
succeed. You can verify this by using the mmlsdisk command. If you need to delete a damaged
disk, you must use the -p option with the mmdeldisk command so it can delete a stopped disk.

__ 47. Use the mmdf command again to verify that the two recently added NSDs have been
successfully removed from the file system.

End of exercise

© Copyright IBM Corp. 2014, 2018 3-12

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 4. Storage pools, filesets, and policies

EXempty

Exercise 4. Storage pools, filesets, and

policies
Estimated time
01:00

Overview
This exercise covers the implementation of IBM Spectrum Scale storage pools (tiered disks within a
file system) and the usage of storage pools. This exercise also covers the implementation (creation,
linking, and population) of Spectrum Scale filesets (file system subtrees). Lastly, the exercise also
covers the usage of the Spectrum Scale policy engine in the placement (ingestion) and migration of
data among the Spectrum Scale storage pools.

Objectives
At the end of the exercise, you should be able to:
• Create storage pools
• Create filesets
• Implement a placement policy
• Define and apply a file management policy

Requirements
• This workbook
• Access to a set of nodes configured for this course
• Exercises 1, 2, and 3 completed successfully

© Copyright IBM Corp. 2014, 2018 4-1

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 4. Storage pools, filesets, and policies

EXempty
Exercise instructions
• Two versions of these instructions are available: one with hints and one without. You can use
either version to complete this exercise. Also, do not hesitate to ask the instructor if you have
questions.
• All exercises in this chapter depend on the availability of specific equipment in your classroom.
• The typical lab environment for this course consists of set of virtual machines (VMs) that are
hosted in a cloud environment. Each student is provided with a URL that allows access to a set
of assigned VMs.
• The VM environment that is used for this course is a common configuration that is shared with a
number of other courses. This course does not use all of the assigned VMs in the lab exercises.

Part 1: Prepare the environment

In this exercise you have an opportunity to complete the following tasks:
• Create storage pools
• Allocate NSD devices to storage pools
• Manage storage pool usage
Before you create the storage pools, you must first delete the existing file system and its associated
NSD device definitions.
__ 1. Open a PuTTY session to vm01, and log in as the root user. Refer to the instructions in a
previous exercise to determine whether you need to resume any suspended VMs.
__ 2. Use the mmunmount command to unmount all cluster file systems on all nodes.
__ 3. Now use the mmdelfs command to delete the file system fs1.
__ 4. Use the mmlsnsd command to list any NSD disks that are defined in the cluster.
Now you are ready to create a file system that uses storage pools.

Part 2: Create a file system with two storage pools

Storage pools are defined when a file system is created. For this portion of the exercise, you create
a file system by using four of the existing NSDs that are part of the cluster. The file system has two
storage pools.
__ 5. Change directory to /spectrum-scale-course/data.
__ 6. Copy the backup of the original NSD descriptor file (diskdesc_bak.txt) to the file
pooldesc.txt.
cp diskdesc_bak.txt pooldesc.txt

© Copyright IBM Corp. 2014, 2018 4-2

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 4. Storage pools, filesets, and policies

EXempty
__ 7. Use vi or a similar editor to modify the pooldesc.txt file. The file should have the following
modifications:
- Add information to the stanzas for the nsd1 and nsd2 NSDs to place them in the system
storage pool.
- Modify the stanzas for the nsd3 and nsd4 devices to place them in a storage pool that is
labeled poolA
- Modify the stanzas for the nsd3 and nsd4 devices so that they are used only for data
storage.
__ 8. Use the mmcrfs command to create a file system by using the pooldesc.txt descriptor file
with the following attributes.
- Set the file system device name to be fs1
- Set the block size to 64 KB
- Set the file system mount point at /gpfs
- Use the NSDs listed in the stanza file /spectrum-scale-course/data/diskdesc.txt
- Set the file system to be automatically mounted when the mmfsd daemon is started

© Copyright IBM Corp. 2014, 2018 4-3

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 4. Storage pools, filesets, and policies

EXempty
__ 9. Verify that the file system was created correctly by using the mmlsfs command.

__ 10. Mount the file system on all nodes using the mmmount command.

© Copyright IBM Corp. 2014, 2018 4-4

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 4. Storage pools, filesets, and policies

EXempty
__ 11. Verify that the file system is mounted using the df command.
__ 12. Verify the storage pool configuration using the mmdf command.

Part 3: Create filesets

Spectrum Scale filesets are named sub-trees (directories) of a file system that can be used for
storage pool allocation, quotas, and fileset-level snapshots. You will create five filesets to organize
the data. Use vm01 to perform these tasks.
__ 13. List all of the filesets that are associated with the file system fs1. Only the root fileset
should be listed.
__ 14. Use the mmcrfileset command to create five filesets called fileset1, fileset2, fileset3,
fileset4, and fileset5.
__ 15. Verify that the filesets were created using the mmlsfileset command.
__ 16. What is the status of the newly added filesets? ___________________

© Copyright IBM Corp. 2014, 2018 4-5

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 4. Storage pools, filesets, and policies

EXempty
__ 17. Use the mmlinkfileset command to link each of the new filesets into the file system at a
junction path location under /gpfs with the same name as the fileset. For example, link the
fileset fileset1 to the junction path location /gpfs/fileset1.
__ 18. Verify that the filesets were linked using the mmlsfileset command.

Part 4: Create a file placement policy

Now that you have a file system with two storage pools (system and poolA) and some filesets, you
are ready to define a placement policy to instruct Spectrum Scale where you would like the file data
placed. By default, if the system pool can accept data, all file data is placed in the system storage
pool.
Spectrum Scale also evaluates the rules in the placement policy sequentially and stops when it
meets a matching rule. There should always be a default rule at the end of the policy file so that
Spectrum Scale knows where to place files that do not meet the criteria in the other defined rules.
Comments can also be included for ease of reading.
In this part of the exercise, you change the default behavior by creating a new policy file.
__ 19. Use vi or a similar editor to create a new policy file (on vm01). Place this new policy file in
/spectrum-scale-course/data and name it placementpolicy.txt. The new policy file
should include the following three placement rules:
- Any new data blocks for files that have a name that ends with “.dat” or “.DAT” are
placed in the poolA storage pool.
- Any new data blocks that are created for files in fileset5 are placed into the poolA
storage pool.
- Default - any new data blocks that are created for files in fileset1-fileset4 are placed in
the system storage pool.
__ 20. Use the mmchpolicy command to install the new policy file for the fs1 file system.

Part 5: Verify that the policy was properly installed using the mmlspolicy command.Test the
placement policies
In this part of the exercise you run some experiments to see how policies work. Use the table below
to track the results. You can determine the amount of free space by using the mmdf fs1 command.
Use the “Free KB in full blocks” column.
Table 1. Storage pool free space
Experiment system pool (free KB) poolA (free KB)
Pool usage before the start of the activity
After bigfile1 is created
After bigfile1.dat file is created
After bigfile2 file is created
After Migrate/delete policy is applied
__ 21. Before you create any files in the file system, use the mmdf command to determine the
amount of free space in each storage pool, and record the results in the table above.

© Copyright IBM Corp. 2014, 2018 4-6

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 4. Storage pools, filesets, and policies

EXempty
__ 22. Use the command that is shown below to create a 50 MB file in /gpfs/fileset1 called
bigfile1.
dd if=/dev/zero of=/gpfs/fileset1/bigfile1 bs=1M count=50
__ 23. After you create the file (bigfile1), use the mmdf command to determine the amount of free
space in each storage pool, and record the results in Table 1.
__ 24. Use the command that is shown below to create a second 50 MB file located in
/gpfs/fileset2 called bigfile1.dat.
dd if=/dev/zero of=/gpfs/fileset2/bigfile1.dat bs=1M count=50
__ 25. After you create the file (bigfile1.dat), use the mmdf command to determine the amount of
free space in each storage pool, and record the results in Table 1.
__ 26. Use the dd command to create a 50 MB file in /gpfs/fileset5 called bigfile2.
dd if=/dev/zero of=/gpfs/fileset5/bigfile2 bs=1M count=50
__ 27. After you create the file (bigfile2), use the mmdf command to determine the amount of free
space in each storage pool, and record the results in Table 1.
__ 28. Based on the information you recorded, use the blanks to list the storage pool used to store
each of the files you created.
bigfile1 ______________
bigfile1.dat ______________
bigfile2 ______________

Note

You can use the mmlsattr -L [full pathname of filename] command to validate your answers.
For example:
# mmlsattr -L /gpfs/fileset1/bigfile1

© Copyright IBM Corp. 2014, 2018 4-7

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 4. Storage pools, filesets, and policies

EXempty
__ 29. Run the following command sequence to create two more 50 MB files that will be used in
subsequent steps:
dd if=/dev/zero of=/gpfs/fileset3/bigfile3 bs=1M count=50
dd if=/dev/zero of=/gpfs/fileset4/bigfile4 bs=1M count=50
__ 30. Use the mmlsattr -L command to determine which storage pool bigfile3 and bigfile4 were
placed into.

Part 6: File management policies

Now that data files are stored in the fs1 cluster file system you are going to manage the movement
of them by using file management policies. After you create the policy file, you use the Spectrum
Scale policy engine (the internal component that evaluates rules and walks the metadata and
applies the rules) to test these changes.
__ 31. To begin, on vm01, create the file management policy file
/spectrum-scale-course/data/managepolicy.txt that implements the business rules.
The new management policy file should include the following file management business
rules:
- All files with names that start with the phrase “big” or “BIG” need to be moved to poolA
- All files that have names end with a “.dat” or “.DAT” extension should be deleted

Hint

If you run the mmapplypolicy command with the -I test flag, it will show details of what would be
performed based on the specified policy, but will not perform the operations.

__ 32. Test the rule set by using the mmapplypolicy command with the -I test flag.

© Copyright IBM Corp. 2014, 2018 4-8

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 4. Storage pools, filesets, and policies

EXempty
__ 33. Apply the policy using the mmapplypolicy command.
__ 34. Review the output of the mmapplypolicy command to answer these questions:
__ a. How many files were deleted?
__ b. How many files were migrated?
__ c. How many KB of data was migrated?
»
__ 35. Use the mmdf fs1 command to check the new free space values for both pools and record
the information in Table 1 on page 4-6.
__ 36. The example below shows a table that is completed using values from the cluster that was
used to create these materials. The numbers you obtain may be slightly different if you
created smaller or larger files than the exercise steps indicate.
Experiment system pool (free KB) poolA (free KB)
Pool usage before the start of the activity 3661248 4062848
After bigfile1 file is created 3610048 4062848
After bigfile1.dat file is created 3610048 4011648
After bigfile2 file is created 3610048 3960448
After Migrate/delete policy is applied 3661184 3858048

End of exercise

Optional exercise: External pools

This optional portion of the exercise simulates exporting data to an external storage pool, for
example a tape library.
__ 37. Use vi or a similar editor to create the file /usr/local/bin/expool1.ksh, with the following
contents:
#!/bin/ksh
dt=`date +%h%d%y-%H_%M_%S`
results=/tmp/FileReport_${dt}
echo one $1
if [[ $1 == 'MIGRATE' ]];then
echo Filelist
echo There are `cat $2 | wc -l` files that match >> ${results}
cat $2 >> ${results}
echo ----
echo - The file list report has been placed in ${results}
echo ----
fi
__ 38. Make the external pool script executable.
chmod +x /usr/local/bin/expool1.ksh

© Copyright IBM Corp. 2014, 2018 4-9

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 4. Storage pools, filesets, and policies

EXempty
__ 39. Now use vi or a similar editor to create the file
/spectrum-scale-course/data/listrule1.txt with the following contents:
RULE EXTERNAL POOL 'externalpoolA' EXEC '/usr/local/bin/expool1.ksh'
RULE 'MigToExt' MIGRATE TO POOL 'externalpoolA' WHERE FILE_SIZE > 2
__ 40. Use the mmapplypolicy command to run the external pool migration and then examine the
report placed into the /tmp directory.

End of optional exercise

© Copyright IBM Corp. 2014, 2018 4-10

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 5. Replication and snapshots

EXempty

Exercise 5. Replication and snapshots

Estimated time
01:00

Overview
In this exercise, you gain experience with Spectrum Scale replication and snapshots.

Objectives
At the end of this exercise, you should be able to:
• Enable data and metadata replication
• Change and verify a file's replication status
• Create a file system snapshot
• Restore a file from a snapshot
• Restore an entire file system from a snapshot

Requirements
• This workbook
• Access to a set of nodes configured for this course
• Exercises 1, 2, 3, and 4 completed successfully

© Copyright IBM Corp. 2014, 2018 5-1

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 5. Replication and snapshots

EXempty
Exercise instructions
• Two versions of these instructions are available: one with hints and one without. You can use
either version to complete this exercise. Also, do not hesitate to ask the instructor if you have
questions.
• All exercises in this chapter depend on the availability of specific equipment in your classroom.
• The typical lab environment for this course consists of set of virtual machines (VMs) that are
hosted in a cloud environment. Each student is provided with a URL that allows access to a set
of assigned VMs.
• The VM environment that is used for this course is a common configuration that is shared with a
number of other courses. This course does not use all of the assigned VMs in the lab exercises.

Preface
• Overview: Spectrum Scale can be configured in such a way as to maintain availability even
when storage subsystem failures occur. This level of protection requires at least double the
storage space and uses a mirroring-like concept Spectrum Scale calls replication.
• Multiple copies of metadata and data can be kept on separate storage subsystems for
availability purposes. Data and metadata can be replicated separately. Replication can be done
in a local cluster for local availability, or in a multi-site cluster for disaster recovery. The
Spectrum Scale documentation has more details on both configurations.
• The Shared Nothing cluster model with File Placement Optimizer supports up to three copies of
the data and metadata. The other cluster models support a maximum of two copies.

Part 1: Enable replication

__ 1. Open a PuTTY session to vm01, and log in as the root user. Refer to the instructions in a
previous exercise to determine whether you need to resume any suspended VMs.
__ 2. The maximum replication factor was established when the file system was created. If it is
not set explicitly, the default value for maximum replicas is set to two for both data and
metadata. A file system is enabled for replication when the maximum number of data and
metadata replicas is set to 2. To determine the values for these parameters, use the mmlsfs
command.

Note

If the maximum number of data and metadata replicas are not set to 2, you need to re-create the file
system or go back and complete exercise 4.

Part 2: Change the failure group on the existing NSDs

__ 3. View the current failure group settings by using the mmlsdisk command.

© Copyright IBM Corp. 2014, 2018 5-2

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 5. Replication and snapshots

EXempty

Note

The failure group should currently be set to a value of -1 (default) for all the NSDs. This value
indicates that the disk is not part of a failure group.

__ 4. Change the failure group value of each NSD in the fs1 file system to be a unique number.
Recall that the scope of replication is the storage pool, which means that the NSDs that are
part of a storage pool must belong to one of two different failure groups in order to
implement replication.
__ a. Change directory to /spectrum-scale-course/data.
__ b. Use vi or a similar editor to modify the pooldesc.txt stanza file that was used to create
the file system in the previous exercise. Add a failureGroup attribute to each stanza.
Use a failure group value of 1 for nsd1, 2 for nsd2, and so on.
__ c. Use the mmchdisk command to change the failure group values using the stanza file.
__ 5. Verify the changes by using the mmlsdisk command.

Note

No data has been moved. The output of the mmlsdisk command indicates that the file system is no
longer properly replicated. This is because the default replication level is still set to 1, which
effectively disables replication.

Part 3: Replicate a file

Replication status can be set at the file level. In this step you replicate the data and metadata of a
single file in the file system.
__ 6. Use the command that is shown below to create a 50 MB file in /gpfs/fileset1 called
bigfile10.
dd if=/dev/zero of=/gpfs/fileset1/bigfile10 bs=1M count=50
__ 7. Use the mmlsattr command to check the replication status of the file bigfile10.
__ 8. Use the mmchattr command to change the file replication status of bigfile10 so that it is
replicated in two failure groups.

Note

If the size of the file is large, you might notice that this command takes a few moments to run. As
you change the replication status of a file, the data is copied before the command completes,
unless you use the –I defer option.

__ 9. Now use the mmlsattr command again to check the replication status of the file bigfile10.

© Copyright IBM Corp. 2014, 2018 5-3

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 5. Replication and snapshots

EXempty
__ 10. Did you see a change in the replication status of the file?

Part 4: Replicate all data in the file system

You can replicate all of the data in a file system. In this part of the exercise, you change the default
replication status for the entire file system.
__ 11. Use the command that is shown below to create a 50 MB file in /gpfs/fileset1 called
bigfile11.
dd if=/dev/zero of=/gpfs/fileset1/bigfile11 bs=1M count=50
__ 12. Use the mmlsattr command to check the replication status of the file bigfile11.

© Copyright IBM Corp. 2014, 2018 5-4

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 5. Replication and snapshots

EXempty
__ 13. Use the mmchfs command to change the default replication status for fs1 so that both data
and metadata are replicated.
__ 14. Use the mmlsattr command to check the replication status of the file bigfile11.
__ 15. Has the replication status of bigfile11 changed? _________________
__ 16. The replication status of existing files does not change until mmrestripefs is run. Any new
files are created by using the new replication values. To test this, create a new 50 MB file in
/gpfs/fileset1 called bigfile12.
__ 17. Use the mmlsattr command to check the replication status of the file bigfile12.
__ 18. Is the file bigfile12 file replicated?
__ 19. Replicate the existing files in the file system by using the mmrestripefs command.
__ 20. Use the mmlsattr command to check the replication status of the file bigfile11.
__ 21. Is the file replicated?

Part 5: Use a snapshot to backup a file system

Overview: A snapshot is a point in time view of a file system. Spectrum Scale snapshots can be
used to create point-in-time copies of the file system state for backup or recovery purposes. Initially,
Spectrum Scale snapshots are copies of the metadata (not data) for the file system - it starts with a
shadow inode file. These snapshots grow over time. As new data blocks are written to the original
file system and existing data blocks are modified, the original data blocks are copied into the
snapshot.
Therefore, snapshots start small, and grow based on the amount of write activity for the file system.
A snapshot cannot be modified; it is read only.
In this part of the exercise you create a file system snapshot and restore a user deleted file from a
snapshot image.
__ 22. Use the command that is shown below to create a file that can be used to test and validate
the snapshot process.
echo "Hello World:snap1" > /gpfs/fileset1/snapfile1
__ 23. Use the mmcrsnapshot command to create a snapshot image of the fs1 file system and
label it snap1.
__ 24. Use the command that is shown below to modify the /gpfs/fileset1/snapfile1 file that is
being used to test and validate the snapshot process.
echo "Hello World:snap2" > /gpfs/fileset1/snapfile1
__ 25. Create a second snapshot image of the fs1 file system by using the mmcrsnapshot
command and label it snap2.
__ 26. Use the mmlssnapshot command to view the snapshots of the fs1 file system.
__ 27. Delete the file /gpfs/fileset1/snapfile1.
__ 28. Even though the test file has been deleted, it is still available in the snapshots that were
created when the file existed. Change directory to /gpfs/.snapshot and use the ls -l
command to view the contents of that directory.

© Copyright IBM Corp. 2014, 2018 5-5

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 5. Replication and snapshots

EXempty
__ 29. What directories do you see? _____________________
__ 30. Compare the contents of the snapfile1 file that is stored in each snapshot image.
[root@vm01 .snapshots]# cat snap1/fileset1/snapfile1
Hello World:snap1
[root@vm01 .snapshots]# cat snap2/fileset1/snapfile1
__ 31. Hello World:snap2Are the file contents the same? _______________.
__ 32. Copy the deleted test file back to its original location from the snap2 snapshot.

Part 6: Restore a file system to a previous snapshot

You can restore an entire file system to a previous snapshot for recovery or roll-back purposes.
__ 33. Use the mmcrsnapshot command to create a new snapshot of the fs1 file system and label
it snap3.
__ 34. Use the ls -l command to list the current contents of the /gpfs/fileset1 directory in the
fs1 file system.
__ 35. Remove the file /gpfs/fileset1/bigfile1 from the fs1 file system.
__ 36. Verify that the file was removed by viewing the contents of /gpfs/fileset1 again.
__ 37. Use the mmrestorefs command to restore the fs1 file system to the state captured in the
snap3 snapshot.
__ 38. Verify that the file /gpfs/fileset1/bigfile1 was restored.
__ 39. When you are done with a snapshot, you can delete it if you are sure you will not need it in
the future. List, delete, and verify the deletion of all snapshots of the fs1 file system.

Part 7: File clones

A file clone is a writable snapshot of an individual file. Cloning a file is similar to creating a copy of
a file, but the creation process is faster and more space efficient because no additional disk space
is used until the clone or the original file is modified.
Multiple clones of the same file can be created with no additional space overhead.
File clones can be created from a regular file or a file in a snapshot by using the mmclone command.
Creating a file clone from a regular file is a two-step process that uses the mmclone command with
the snap and copy keywords.
__ 40. Use the command that is shown below to create a 50 MB file in /gpfs/fileset1 called
file1 that can be used to test the clone process.
dd if=/dev/zero of=/gpfs/fileset1/file1 bs=1M count=50

© Copyright IBM Corp. 2014, 2018 5-6

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.2
Exercise 5. Replication and snapshots

EXempty
__ 41. Use the cd command to change directory to /gpfs/fileset1, then use the mmclone snap
command to create a read-only snapshot called snap1 of the file called file1.
__ 42. Use the mmclone copy command to create a writable clone called file2 of the read-only
snapshot called snap1 that you created in the previous step.
__ 43. Use the mmclone show command to show information about the file1 and file2 files.
For more information about file clones, see Chapter 27. Creating and managing file clones of the
IBM Spectrum Scale Administration Guide, which can be found online in the Knowledge Center.

End of exercise

© Copyright IBM Corp. 2014, 2018 5-7

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.3

backpg

© Copyright International Business Machines Corporation 2014, 2018.