Migration to Huawei Cloud
Module 7: Using and Migrating Storage Services
Objectives
⚫ Upon completion of this course, you will:
Understand how to select storage services on Huawei Cloud.
Understand how to migrate existing storage data to Huawei Cloud.
Be able to perform migration exercises.
3
Contents
1. Introduction to Storage Services
2. Migrating Object Storage to Huawei Cloud OBS
3. Migrating File Storage to Huawei Cloud
4
Storage Services on Huawei Cloud
Object Storage Service Scalable File Service Elastic Volume Service
(OBS) (SFS) (EVS)
5
OBS Highlights
⚫ Fully managed object storage
⚫ Support for access over the Internet
⚫ Unlimited storage space (48 TB per
object)
OBS ⚫ 99.9999999999% reliability
⚫ Event triggering capabilities
⚫ Cost-effective solutions
6
SFS Highlights
⚫ Fully managed file storage
⚫ Similar to a NAS server
⚫ Unlimited storage space
⚫ 99.99999999% reliability
SFS
⚫ Includes SFS Capacity-Oriented and SFS
Turbo.
To meet the needs from massive small files to
performance demanding workloads.
7
EVS Highlights
⚫ A block storage service that provides disks to
cloud servers
⚫ Three-copy redundancy in a single AZ and
99.9999999% durability
⚫ Up to 32 TB for a single disk
EVS
⚫ A variety of specifications, balancing both
costs and performance
⚫ Storage of the backup data to OBS
8
Contents
1. Introduction to Storage Services
2. Migrating Object Storage to Huawei Cloud OBS
3. Migrating File Storage to Huawei Cloud
9
Introduction to OMS
⚫ Object Storage Migration Service (OMS) is an efficient, easy-to-use online data migration
service that helps users easily and smoothly migrate object storage data from other public
clouds to Huawei Cloud.
Object storage of other cloud vendors
OMS
2023-1 2022-1
3-5 TB 60 TB
1 Compares whether the objects
exist in the destination bucket. OBS
Dir-1
obj1 Dir-2
obj6 Dir-3
obj10 Dir-4
obj16 Dir-5
obj21
2 Migrates the objects to
obj2 obj7 obj11 obj17 obj22 the destination bucket.
obj3 obj8 obj12 obj18 obj23 obj21
Checks whether the migrated
3 objects are consistent with
obj4 obj9 obj13 obj19 obj24
those in the source bucket.
··· ··· ··· ··· ···
Average migration speed: 20 TB/day; maximum migration speed: 40 TB/day
10
Functions ⚫ You can select files or folders, or use an ⚫ For a large-capacity, large-scale object
object list to choose what data will be migration, OMS provides a pre-migration evaluation
1 migrated. 2 to analyze how many and how large objects are in
⚫ You can also migrate objects generated after a the source bucket.
Flexible object specified time point or with certain prefixes. Automatic ⚫ The task can then be split into multiple sub-tasks
selection task splitting based on this evaluation to maximize concurrency
performance.
⚫ Parallel migration: OMS supports concurrent ⚫ OMS allows you to manage migration tasks by group.
Functions migration of multiple objects.
⚫ Archive data restoration: OMS automatically
3 restores archive data and migrates it to 4 ⚫ Migration tasks may take a long period of time,
destination buckets. and you can use the Simple Message Notification
⚫ Custom domain name: OMS allows you to migrate data (SMN) service to obtain migration results by
Efficient from a source bucket with a customized domain name. email, SMS message, or customized URL.
transmission ⚫ Object recording: OMS records any objects that Result
failed to be migrated and retransmits these failed notification
objects later.
⚫ OMS secures access with an access key (AK) and
1 ⚫ All you need to do is log in to the OMS 2 secret access key (SK) combination. Your
console, create a migration task, and wait for identity is authenticated before data
Highlights the migration to complete. Solid transmission to ensure only you have access to
Ease of use your data.
security
3 ⚫ OMS can use HTTPS connections for secure data 4
transmission.
⚫ OMS encrypts data migrated to Huawei Cloud to
Encrypted Encrypted make your data more secure.
transmission storage
11
Migration Principles
Object storage Huawei Cloud region
service
OMS
Console (GUI) nodes
S3 Obtain the source object
1
list and analyze the
bucket size, object Console
OSS quantity, and size
2 Users can perform
distribution.
Assessment Management migration
nodes Databases
nodes configurations on the
Azure
Firewall
console or manage
Assess Management Database tasks by calling APIs
via API Gateway.
Create migration
Support for the 3 tasks based on 4 Deliver the
object storage migration Replicate data
the analysis.
services of multiple tasks. 5 from the source
cloud vendors, objects to OBS
including AWS, Azure, buckets.
Aliyun, Tencent
Cloud, and Huawei Migration Migration Migration
OBS
Cloud. node node Migration node node
cluster
The migration node cluster can quickly migrate object storage data to OBS buckets and
allows for multi-task and multi-thread concurrent migration.
Control flow
Data flow
12
Evaluating Source Buckets (1)
1. List source
buckets.
13
Evaluating Source Buckets (2)
2. Evaluates the buckets to be migrated.
14
Creating Migration Tasks
3. Create migration tasks based on the evaluation results.
15
Application Scenarios
Data migration On-cloud DR
When migrating typical web applications to Huawei With OMS, users can easily migrate object
Cloud, users can use OMS to migrate object storage storage data from other cloud vendors to Huawei
data to OBS quickly. Cloud for disaster recovery.
Application
Scenarios
Pre-migration
Data restoration evaluation
With OMS, users can use data backups from other Users can perform pre-migration evaluation to
cloud vendors to quickly restore data to Huawei analyze how many and how large objects are in
Cloud. source buckets, predict the migration duration,
and formulate a proper migration solution.
16
Cases
Another cloud vendor Huawei Cloud
OMS
Load Data
balancer ELB
conversion
Cloud
Server ECS
Encrypted
transmission over the
Internet
MySQL
RDS Object OBS RDS
storage
17
Migration Process
Phase Implementation Procedure Description
Collect source information. Clarify the source region, bucket name, and list of objects.
Estimate the migration duration based on the total number of objects, total data volume of
Migration Estimate the migration duration.
objects, and object sizes in the migration source.
preparations
Collect incremental data Clarify the daily growth and distribution of the incremental data and the service cutover time
POC test
information. window.
Obtain information including the object access frequency, object size and quantity
Identify risks.
characteristics, and how many and how often objects are added and deleted.
Plan migration tasks. Plan migration tasks by bucket, folder, and file.
Migration
Decide the migration batch. Properly plan the migration batch and execution sequence in each batch.
implementation plan
Determine the incremental
Clarify how to implement the incremental migration and the service cutover time.
migration policy.
Test and verify the overall
Test on the full and incremental migration plan to verify the migration feasibility.
Drill and emergency migration plan.
plan Formulate the emergency plan for
Formulate an emergency plan for the incremental service cutover to ensure service continuity.
the incremental service cutover.
Track the migration process. View the processes of migration tasks in real time.
Predict any problems that may occur during the migration, respond to the problems timely, and
Migration assurance Predict risks.
provide countermeasures.
After the migration is complete, use tools to compare the source and destination data to
Compare data consistency.
ensure data consistency and integrity.
18
Recommended Migration Process
(Evaluation, task creation, failed object recording, and task monitoring)
An object storage migration process normally includes: pre-migration consultation, migration plan formulation,
bucket evaluation, task planning, migration task creation, incremental migration, consistency check, and migration
acceptance.
Perform multiple
migrations to reduce
the incremental data
volume.
Pre-migration Migration task Incremental Migration
Plan formulation Consistency check
OMS
OMS
consultation creation migration acceptance
OMS
OMS
OMS
Tool Tool
Bucket • Create migration • Migrate incremental data
evaluation OMS tasks in batches (generated after specific time
based on the points).
Task
OMS evaluation • Migrate incremental objects by
planning
results. listing and comparing the
Migration objects in the source and
schedule destination.
• Write incremental data to
specific source directories to
• If you have high requirements on accelerate incremental
service continuity during identification. During the migration, OMS:
migration, enable dual-write or
• Verifies the timestamp and object
retrieval before starting the
size.
migration.
• Lists and compares the objects in
the source and destination.
• Downloads the destination objects,
calculates their MD5 or CRC64
values, and compares the values with
those of source objects.
19
Contents
1. Introduction to Storage Services
2. Migrating Object Storage to Huawei Cloud OBS
3. Migrating File Storage to Huawei Cloud
20
Migrating File Storage to Huawei Cloud OBS
NFS-based shared Private line- or
file access Internet-based OBS access
(via API)
CSG Reducing storage
Migrating data to the cloud costs using
lifecycle
management
Server Different
OBS
storage classes
DES
Traditional backup
of local disks
Two ways are available for migration.
Self-owned data center
21
CSG Highlights
Building a bridge between the local
storage and remote cloud storage
NFS support similar to a single NAS,
compatible with all existing
Cloud Storage Gateway applications
(CSG) Massive storage, high performance, and
high data reliability
Easy DR and migration
22
DES Highlights
Disk-based and Teleport-based
transmission
Transportation through physical devices
Massive data transmission at a lower
Data Express Service cost than Internet
(DES) Built-in encryption to prevent data
loss
23
Restoring Data from OBS to A Variety of Storage
Start a server to migrate
data.
Migrate data to EVS.
◼ Attach an EVS disk of a desired
OBS
size to the server.
Storing the migrated data to
EVS ◼ Download data from OBS.
Storing the migrated
◼ Detach the EVS disk or create a
Server data to SFS SFS disk backup.
Migrate data to SFS.
◼ Create an SFS file system and
A region
mount it to the server.
◼ Download data from OBS.
24
Migrating NAS to Huawei Cloud SFS
On premises Cloud
Establish the network and use rsync, rclone, or scp
to migrate and synchronize data between the two
Server servers.
Mount the NFS
share and write
data to the Mount the NFS share
server. and write data to
NFS SFS. SFS
Server
Establish the network using a private line, mount both the NFS share
and SFS file system to the server, and then migrate and synchronize
data.
Two ways are available for migration.
25
Thank You.
Copyright©2023 Huawei Technologies Co., Ltd. All Rights Reserved.
The information in this document may contain predictive statements including,
without limitation, statements regarding the future financial and operating
results, future product portfolio, new technology, etc. There are a number of
factors that could cause actual results and developments to differ materially
from those expressed or implied in the predictive statements. Therefore, such
information is provided for reference purpose only and constitutes neither an
offer nor an acceptance. Huawei may change the information at any time
without notice.
26
