M6 Migrating SQL Server Databases To Google Cloud
M6 Migrating SQL Server Databases To Google Cloud
Hello everyone, Julianne here. There are several options available for migrating a
SQL Server database to Google Cloud. So Let’s explore those in some detail.
Objectives
A lift and shift strategy involves picking up and moving virtual machines into Google
Cloud. You could employ this strategy with your SQL Server databases by using
Compute Engine.
Alternatively, you could use a managed version of SQL Server provided by Cloud
SQL.
In any case, you also need to architect your SQL Server databases for security, high
availability, and disaster recovery.
You can also run SQL Server in a Kubernetes environment using Google Kubernetes
Engine.
Choose the right Google Cloud service to run SQL
Server workloads based on your use cases
Choose the right Google Cloud service to run SQL Server workloads based on your
use cases. Each of the three services offer advantages over the others.
Compute Engine offers the most straightforward and familiar option with a minimum of
changes from your existing SQL Server installation. You basically build a custom SQL
Server with the exact features you want, just as you would on an on-premises
machine. But it also means DBAs have to do all the same work they would on a local
server.
Cloud SQL offers all the advantages of a managed service and removes some of the
administrative burden from the DBAs; however, not all SQL Server features are
supported, primarily because it’s running on Linux. So it’s critical to ensure that none
of the unsupported features are important to you before you choose this option.
GKE offers flexibility in deploying the database into the same cluster as your
applications, as well as the ability to create hybrid deployments and portability to other
cloud environments.
SQL Server on
Compute Engine
SQL Server on Compute Engine is basically just relocating the server from a local
machine to a Compute Engine VM. This is often the easiest way to do the migration
itself, but may not be the best long-term option.
Google provides pre-built
SQL Server images
● Versions included: 2017, 2019, and 2022
● Editions included:
○ Express
○ Web
○ Standard
○ Enterprise
When you create the Compute Engine VM, you could start with a bare Windows
image and install your own SQL Server database onto it, but Google offers pre-built
versions of SQL Server in many different versions, which simplifies the whole process
of creating the VM.
SQL Server images included
shielded VM features
Using the Google-provided images also offers benefits beyond convenience during
the build process. They provide a higher level of security against attempts to hack
your server by providing mechanisms to detect and prevent unauthorized changes to
the software and to protect keys and certificates.
Secure boot ensures that only verified software runs on your VM. It uses digital
signatures to verify software components and stops the boot process if something has
been tampered with.
Shielded VMs use a custom chip called a Virtual Trusted Platform Module to protect
digital signatures.
And Integrity monitoring compares boot information to a stored baseline hash that is
stored in the vTPM.
Enable Shielded VM features when
configuring the virtual machine
Shielded VM ?
Turn on all settings for the most secure configuration.
The Shielded VM features are available when you configure the VM, and you can
enable just the ones you want.
Google includes Windows and SQL Server
license fee when VMs are run
Another advantage of using the pre-built Google images is that the license cost of
using Windows and SQL Server is built into the running costs of the machine. The
pricing varies depending on which edition of SQL Server you choose and how many
cores you configure the VM for. Still, it is often a lot easier to manage than manually
paying for your own SQL Server license.
However, if you already pay for licenses, there is a Microsoft License Mobility for
Google Cloud program. See the documentation for more information.
Add additional disks to store SQL Server
data when creating the virtual machine
● Choose SSD or Standard disks:
○ SSD is faster.
○ Standard is cheaper.
● Disks can be 10 GB to 64 TB:
○ Larger disks are faster.
● Can set up a snapshot schedule
for automated backups.
When configuring the VM, you also need to allocate storage space. This is done by
adding data disks.
As usual, they can be SSDs for faster performance or standard disks for cost savings.
You can configure the size to meet your needs, but remember that you pay for what
you allocate, not what you use.
And you can also schedule automatic disk snapshots to make disk backups, but
remember that this is different from a SQL Server backup.
Use Snapshot Schedules to
automatically backup disks
● Choose the regions you want
the snapshots to be stored in.
● Schedule snapshots during
off-peak hours.
● For Windows machines, enable
Volume Shadow Copy Service:
○ Safely creates snapshots
without needing machines
to be off.
Also, if you’re running a Windows instance, you can choose the Volume Shadow
Copy Service to allow snapshots without shutting down the SQL Server service.
Configure the VM network interface to provide
secure access to the SQL Server
Ultimately, you will have a SQL Server VM that clients will need to connect to. So in
order to reach the server, you need to configure the networking. Typically, you don’t
want an external IP address, but instead want to make a private network for the
clients to access the server. This provides more security.
Setting a Windows username and password to log in
1 3
To administer a Windows server, you will use RDP. In order to RDP into the server,
you will need to set a Windows username and password. This can be done from the
console or the CLI. First you specify any username you like, and then a strong
password will be generated.
As with all usernames and passwords, you will need to save or remember these. You
can always change the password after logging in.
SQL Server on Compute Engine best practices
There are many best practice recommendations, but they are all pretty
straightforward.
You want to follow the typical best practices for creating a VM, which include using a
firewall and the default network settings, as well as installing anti-virus software.
Additionally, you would configure the SQL Server portion in a way that’s similar to how
you would on a local machine. Namely, you would want separate disks for the log and
data files, as well as for tempdb and the Windows paging files.
Using SSDs is more expensive but offers better performance. If you can’t use SSDs
for all the drives, it’s probably most important to use them for the tempdb and paging
files first.
Also, although you have the server in the cloud and have turned on snapshots, you
can’t skip normal SQL Server maintenance. You still need to set up regular database
and log backups and schedule them with SQL Server Agent.
You can also search Marketplace for SQL and find
pre-configured VMs on both Windows and Linux
Base your decisions on how much time you want to invest in building your image and
how much custom configuration you need.
Configure SQL Server AlwaysOn Availability
Groups for high availability
● Requires an Active Directory
domain controller.
● One SQL Server is the main:
○ Runs the failover cluster
manager service.
● One or more replicated servers:
○ Each server has its own data SQL Server 1
FC Manager SQL Server 2
Active directory
controller
replicated from the main.
● If the main fails, it automatically
switches to the failover.
If you’re worried about the main SQL Server instance going down and need a solution
that guarantees the server is always available, you can use the SQL Server
AlwaysOn Availability Groups.
This requires an Active Directory domain controller. One server would be designated
as the main server, then one or more replicated servers would be in a standby mode,
constantly keeping their own copy of the data in sync with the main.
If the main fails, one of the standbys would take over as the main server and continue
the workload uninterrupted. Eventually, you can fix the main, resync the data, and
switch back to it.
https://cloud.google.com/compute/docs/instances/sql-server/configure-availability
Requirements for AlwaysOn
Availability Groups
● SQL Server Enterprise edition
○ IP address for SQL server
○ Alias IP address or Failover Cluster
○ Alias IP address for Availability Group Listener
● Active Directory domain controller
● Configure the Failover Cluster Manager on the main server
● Add databases to availability groups to enable replication
AlwaysOn Availability requires the more expensive Enterprise edition of SQL Server,
so if you need this feature, make sure you consider price when choosing the image to
build your VM.
You set up the main and the Active Directory, configure the Failover Cluster Manager
on the main server, and add databases to the availability groups to enable replication.
Another slightly different option for high availability is SQL Server Failover Cluster
Instances.
Using this option, you set up a single shared storage device. This would be like a
virtual SAN setup using Microsoft Storage Spaces Direct.
Then you have a main instance of SQL Server that uses that virtual SAN, but if it fails,
a standby node would become active and attach to the shared SAN to continue the
work uninterrupted.
https://cloud.google.com/compute/docs/instances/sql-server/configure-failover-cluster-
instance
Requirements for SQL Server Failover
Cluster Instances
● Two SQL Server Enterprise edition virtual machines
● Windows Server 2016 or 2019 Datacenter editions
● Active Directory domain controller
● Internal Google Cloud load balancer
This option also requires the Enterprise Edition of SQL Server, as well as the
Datacenter versions of Windows Server. Also, you need an AD domain controller and
an internal Google Cloud load balancer.
In this lab, you create a SQL Server VM and admin and client VMs. You then do some
basic administration and make a connection from a client to the database.
Lab Review
Creating SQL Server Databases
on Compute Engine
Compute Engine allows you to deploy SQL Server databases with no limitations.
Using Compute Engine, you can configure your SQL Server databases just as you
would on-premises.
SQL Server on Cloud SQL offers an easier alternative to fully building out a VM. It
comes with all the benefits of using a fully managed service compared to your own
VM, but as always, you have to make some trade-offs for that easier option.
Cloud SQL provides a managed
SQL Server solution
Details: Potential problems:
● Runs on Linux ● Doesn’t provide Windows-based instances
● Runs in a network provided by Google ● Limited SQL Server versions
● Automated backups and maintenance ● Not all SQL Server features are supported
● Easy to set up failover servers
The first notable difference is that the server runs on Linux, not Windows. Google
handles all the networking and backup and maintenance for you, and also makes it
easy to set up failover servers.
The downside of not having a Windows-based instance is that not all versions of SQL
Server are supported. Also, some of the features that rely on Windows are not
supported.
SQL Server features that are not
supported on Cloud SQL
● SQL Server Analysis Services (SSAS)
● Filetable and Filestream
● Database Mail
● Backing up to Microsoft Azure Blob Storage
● Hybrid buffer pool
● Link to Azure SQL Managed instance
● Machine Learning and R Services
● Server-level triggers
Until Microsoft’s recent association with Linux, SQL Server ran exclusively on
Windows, so many features heavily relied on Windows features.
This slide lists some of the features of SQL Server that Microsoft currently doesn't
support on Linux and are not supported on Cloud SQL. The complete list can be
found in the Google Cloud documentation.
Create a Cloud SQL for SQL Server instance
in minutes using the Console
It is very quick and easy to create a Cloud SQL instance of SQL Server. There’s a lot
less to configure than when building your own VM. Basically, you just need an
instance ID and a password and to choose the region for it. There are additional
options, but these are the ones you always need to set.
Automate instance creation with the CLI
It is also very easy to create a Cloud SQL machine using the CLI. Compare the
gcloud command for creating this instance to creating a VM, and you can see how
much simpler it is.
Automate instance creation with Terraform
settings {
tier = "db-n1-standard-4"
}
}
Or if you are using Terraform, Cloud SQL databases can be created with a simple
template.
As you saw earlier in the course, Terraform makes it easier to configure your data
center resources by defining what you want to configure with a simple configuration
language.
Choose SQL Server Edition and machine type
when configuring the database
You can also choose among several SQL Server editions and machine configurations.
Choose the edition based on
database scaling
Express 4 3.75 GB 10 GB $0
The choice of version is largely influenced by your database size needs. Pricing is
significantly different, so it’s important to plan ahead; otherwise, you might have to
migrate the databases to another server later, or you might spend much more than
you need.
There are four editions of SQL Server: Express, Web, Standard, and Enterprise.
Express edition has no license charge but is limited to a 10 GB database.
The other three editions require licensing; Enterprise being the most expensive. See
the table on the slide for vCPU, RAM, and database size features and licensing costs.
An important point to note is licensing is per-vCPU Core, but you are always charged
for at least 4 vCPUs.
Configure the machine type according
to required capacity
● Specify the number of vCPUs and RAM
for your instance.
● Specify disk space from 10 GB to 30 TB:
○ Enable automatic storage increases
to increase disk space as needed.
Cloud SQL instances can have either a private or public IP address, or both.
Choose only a private IP for a more
secure architecture
● A VPC peering is created between the
Cloud SQL database network and your
network.
● Only machines in your peered network
can access the database.
A private IP would make your instance more secure, but you’d have to configure
authorized clients to connect to that instance through the private IP using a peered
network.
Enabling Private IP creates a VPC peering between the Cloud SQL database network
and your network.
Then, only machines in your peered network can access the database.
A firewall protects machines with
public IP addresses
● Authorize one or more external
sources to allow access to the database.
● Use CIDR notation to allow IP
addresses or ranges.
● By default, no machine outside your
project has access to the database.
A firewall is used for protection in the case of public IPs and, by default, only allows
machines within the project to have access to the server.
However, you could authorize one or more external connections. As with configuring
any firewall rule, use CIDR notation to allow IP addresses or ranges.
Cloud SQL automates administration
You can also easily create a failover cluster by just enabling the High Availability
option.
The service will also automate typical maintenance tasks like patch management. As
with backups, choose a maintenance window when there is low utilization. If
maintenance requires a reboot, the server will be unavailable for a short period of
time.
Backups and restores can be done from
the Console or by using the CLI
You can also manually perform a backup or restore with either the console or the CLI.
Remember, these are not the same type of backups as native SQL Server database
and log backups. A Cloud SQL backup is a snapshot of the persistent disk with the
data. It can only be restored to another Cloud SQL database.
In addition, you can always run standard SQL Server backups as you would with any
SQL Server database.
Lab Intro
Administering a Highly Available
Cloud SQL for SQL Server Database
While Cloud SQL has some limitations, it makes setting up and administering a
database much easier. You can create failover servers by simply choosing an option.
Backup, restore, and admin tasks are automated.
In this lab, you create a SQL Server Cloud SQL instance that is secure and highly
available. You will perform a backup and a restore operation and connect to the
instance from a client using its private IP address.
Lab Review
Administering a Highly Available
Cloud SQL for SQL Server Database
In this lab, you created a secure, highly available SQL Server database using Cloud
SQL.
Running SQL Server on GKE offers some significant benefits over the other options.
Most notable is the ability to run the server across various cloud platforms or a hybrid
on-premises/off-premises option.
Additionally, you have a higher degree of automation available for creating CI/CD
pipelines.
It also provides the ability to move the server to the same cluster as the application for
better performance.
Use YAML to configure your SQL
Server database
To configure a database you need:
● Persistent Volume Claims
○ Reserve disk space from the cluster
for your database.
● Deployment
○ Configures the application.
○ Specify the Docker image, resources,
volumes, environment variables, etc.
● Service
○ Provides access to the database from
client applications.
You configure SQL Server databases to run in a Kubernetes cluster by using YAML
files.
Persistent Volume claims are used to allocate disk space for your database running in
the cluster.
And a Service configuration is used to provide access to your SQL Server from its
clients.
In the deployment, specify the Docker image for
the SQL Server version you want to deploy
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: mssql-deployment
spec:
replicas: 1
<< CODE OMITTED>>
spec:
terminationGracePeriodSeconds: 10
containers:
- name: mssql
image: mcr.microsoft.com/mssql/server:2017-latest
ports:
- containerPort: 1433
Here is an example of using a Microsoft standard Docker image for your server. This
image is maintained by Microsoft and stored in Docker Hub. Alternatively, you could
build your own Docker image with SQL Server installed if you needed a higher degree
of customization.
Official Microsoft SQL Server Docker
images can be found at Docker Hub
● Has details on versions, editions, and
configurations
● Required environment variables:
○ ACCEPT_EULA=Y
○ SA_PASSWORD=<your-pw>
○ MSSQL_PID=<your_product_id |
edition_name> (default: Developer)
Many Microsoft pre-built Docker images are available. Choose the version that suits
your needs and make sure to set the required environment variables.
You use license agreements to specify an initial SA password, the edition, and so on.
More information can be found at the Docker Hub web page for each image.
PersistentVolumeClaim reserves disk
space on the cluster
kind: PersistentVolumeClaim kind: PersistentVolumeClaim kind: PersistentVolumeClaim
apiVersion: v1 apiVersion: v1 apiVersion: v1
metadata: metadata: metadata:
name: mssql-mdf-volume name: mssql-ldf-volume
name: mssql-base-volume
spec: spec:
spec: accessModes: accessModes:
accessModes: - ReadWriteOnce - ReadWriteOnce
- ReadWriteOnce resources: resources:
resources: requests: requests:
requests: storage: 10Gi storage: 10Gi
storage: 10Gi
Use separate volumes for the database, data, and log files when configuring SQL Server.
Just as in a normal SQL Server configuration, you want the software, data, log, and
tempdb on different physical volumes. So, here you would configure different
Persistent Volume Claims for each.
Note that each PersistentVolumeClaim has a unique name, and the amount of disk
space for each is specified.
apiVersion: apps/v1beta1
kind: Deployment
<<CODE OMITTED >>
volumeMounts:
- name: mssql-base-volume
mountPath: /var/opt/mssql
Also in a deployment, - name: mssql-ldf-volume
mountPath: /var/opt/mssql/ldf
mount the volume - name: mssql-mdf-volume
mountPath: /var/opt/mssql/mdf
volumes:
claims and specify - name: mssql-base-volume
persistentVolumeClaim:
the paths claimName: mssql-base-volume
- name: mssql-mdf-volume
persistentVolumeClaim:
claimName: mssql-mdf-volume
- name: mssql-ldf-volume
persistentVolumeClaim:
claimName: mssql-ldf-volume
The Persistent Volume Claims from the previous slide are added as Volumes in the
pod you are configuring.
apiVersion: apps/v1beta1
kind: Deployment
<<CODE OMITTED >>
volumeMounts:
- name: mssql-base-volume
mountPath: /var/opt/mssql
Also in a deployment, - name: mssql-ldf-volume
mountPath: /var/opt/mssql/ldf
mount the volume - name: mssql-mdf-volume
mountPath: /var/opt/mssql/mdf
volumes:
claims and specify - name: mssql-base-volume
persistentVolumeClaim:
the paths claimName: mssql-base-volume
- name: mssql-mdf-volume
persistentVolumeClaim:
claimName: mssql-mdf-volume
- name: mssql-ldf-volume
persistentVolumeClaim:
claimName: mssql-ldf-volume
Then, the volumes are mounted using the mountPath property when you configure
the SQL Server container that will also run in the pod. Note that path names are
Linux, not Windows, paths.
In a deployment, you can configure
environment variables
apiVersion: apps/v1beta1
kind: Deployment
<<CODE OMITTED >>
env:
- name: ACCEPT_EULA
value: "Y"
- name: SA_PASSWORD
valueFrom:
secretKeyRef:
name: mssql-secrets
key: SA_PASSWORD
- name: MSSQL_DATA_DIR
value: /var/opt/mssql/mdf
You can also configure the required environment variables here. Check out the
SA_PASSWORD variable. This is being set from a Kubernetes secret.
Secrets are used to keep sensitive
data out of the configuration files
Storing sensitive data in plain text in a configuration file is never a good idea, so you
can use secrets to store the actual password and use the secret in the config file.
Here the password is being set using the CLI. Run this code before the deployment.
Service provides access to the
database running in the cluster
apiVersion: v1
kind: Service
metadata:
name: mssql-deployment
spec:
selector:
app: mssql
ports:
- protocol: TCP
port: 1433
targetPort: 1433
type: LoadBalancer
The service configuration provides access to the database running inside the cluster.
Note the port and target port variables: 1433 is the default SQL Server port.
The service type LoadBalancer creates a public IP address, allowing the database to
be accessed from outside the cluster. If you set the type to ClusterIP (or didn't set the
type property at all), the service would only have a private IP address. Thus, the
database would only be available from inside the cluster.
When the configuration is complete, use kubectl to
deploy the resources specified in the YAML files
After you configure the various files, you can use the kubectl command line interface
to put them all together to deploy the server.
Deploy each of the YAML files with the command kubectl apply -f and the name
of the file.
This lab gives you the opportunity to create a Kubernetes cluster and configure and
deploy a SQL Server database in it. Then, you connect to the database from a client
and look at an alternative way to deploy SQL Server to the cluster using Helm.
Lab Review
Running SQL Server on Google
Kubernetes Engine
In this lab, you created a Kubernetes cluster, and configured and deployed SQL
Server into the cluster.
You then connected to the SQL Server database from a client machine.
Running your SQL Server databases in a Kubernetes cluster is ideal when you want a
cross-cloud or hybrid-cloud solution that is automated.
Module Review
● Used Compute Engine to lift and shift SQL Server databases to Google Cloud.
● Employed Cloud SQL for managed SQL Server databases.
● Architected SQL Server for security, high availability, and disaster recovery.
● And configured SQL Server to run with Kubernetes on GKE.