Load Balancing MinIO Server
Load Balancing MinIO Server
Version 1.2.0
Table of Contents
1. About this Guide . . . . . . . . . . . . . . . Về . . . . .bản
. . . . . hướng
. . . . . . . . . dẫn
. . . . . .này
............................................3
2. Loadbalancer.org Appliances Supported. . Các . . . . . . thiết
. . . . . . bị
. . . Loadbalancer.org
. . . . . . . . . . . . . . . . . . . . . . .được
. . . . . . . hỗ
. . . . trợ
............3
3. Software Versions Supported . . . Các . . . . . . phiên
. . . . . . . .bản
. . . . . phần
. . . . . . . mềm
. . . . . . . được
. . . . . . . .hỗ
. . . .trợ
........................3
3.1. Loadbalancer.org Appliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3.2. MinIO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
4. MinIO Server . .Máy . . . . . .chủ
. . . . . MiniIO
..........................................................................3
4.1. Operating Modes. Các . . . . . .chế
. . . . . độ
. . . . vận
. . . . . .hành
.........................................................4
5. Load Balancing MinIO Server . . . . . . . . . . . . . . . . . . . . . .cân
Máy chủ Mini10 . . . . .bằng
. . . . . . .tải
.......................................4
5.1. MinIO Configuration . Cấu . . . . . . hình
. . . . . . Mini10
...............................................................4
Operating Mode . .Chế . . . . . .độ
. . . vận
. . . . . .hành
...............................................................4
5.2. Load Balancer Configuration Cấu . . . . . . hình
. . . . . . cân
. . . . . .bằng
. . . . . . .tải
...........................................4
Operating Mode . .Chế . . . . . độ
. . . . vận
. . . . . .hành
...............................................................4
Timeouts . .Thời . . . . . . gian
. . . . . . chờ
........................................................................4
Port Requirements . . . . . . . .cầu
Yêu . . . . về
. . . .cổng
. . . . . .kết
. . . .nối
....................................................4
SSL/TLS Termination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Health Checks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Deployment Concept . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
6. Loadbalancer.org Appliance – the Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
6.1. Virtual Appliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
6.2. Initial Network Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
6.3. Accessing the Appliance WebUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Main Menu Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
6.4. Appliance Software Update. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Determining the Current Software Version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Checking for Updates using Online Update. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Using Offline Update . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
6.5. Ports Used by the Appliance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
6.6. HA Clustered Pair Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
7. Running MinIO in Distributed Erasure Code Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
8. Appliance Configuration for MinIO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
8.1. a) Layer 7 VIP Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
8.2. b) Defining the Real Servers (RIPs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
8.3. c) Upload Your SSL Certificate to The Load Balancer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
8.4. d) Configure SSL Termination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
8.5. e) Finalizing the Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
9. Testing & Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
9.1. Using System Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
9.2. Obtaining information about the MinIO Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
10. Technical Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
11. Further Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
12. Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
12.1. Configuring HA - Adding a Secondary Appliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Non-Replicated Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Configuring the HA Clustered Pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
13. Document Revision History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
*Cân bằng tải: Phân phối lưu lượng truy cập mạng hoặc ứng dụng 1 cách có hiệu
quả trên nhiều server trong 1 cụm server farm, phân phối đồng đều tài nguyên
trên nhiều máy chủ.
1. About this Guide
This guide details the steps required to configure a load balanced MinIO Server environment utilizing
Loadbalancer.org appliances. It covers the configuration of the load balancers and also any MinIO Server
configuration changes that are required to enable load balancing.
For more information about initial appliance deployment, network configuration and using the Web User Interface
(WebUI), please also refer to the Administration Manual.
Some features may not be available or fully supported in all cloud platforms due to platform specific limitations.
For more details, please refer to the "Main Differences to our Standard (Non-Cloud) Product" section in the
appropriate cloud platform Quick Start Guide or check with Loadbalancer.org support.
The screenshots used throughout this document aim to track the latest Loadbalancer.org
software version. If you’re using an older version, or the very latest, the screenshots presented
here may not match your WebUI exactly.
3.2. MinIO
All versions
4. MinIO Server
MinIO Server is a high-performance open source S3 compatible object storage system designed for hyper-scale
private data infrastructure.
MinIO can be installed on a wide range of industry standard hardware. It can run as a standalone server, but it’s
full power is unleashed when deployed as a cluster with multiple nodes. From 4 to 32 nodes and beyond using
MinIO federation.
Data is protected against hardware failure and data corruption using erasure code at the object level and bitrot
protection. MinIO is highly available – a distributed cluster can loose up to half the disks on a single node and up
to half the nodes and continue to serve objects.
The use of the Strict Consistency data model ensures that an exact copy of all data is available from all nodes.
With Eventual Consistency, read operations could return old or stale data.
.
MinIO integrates with various authentication systems such as WSO2, OKTA and Active Directory to authenticate
applications and users. Data integrity is ensured using encryption and tamper proofing technology.
Standalone – runs on a single node with a single disk or for improved resilience a RAID array
Standalone Erasure Code – runs on a single node: object data and parity is striped across all drives in that
node
Distributed Erasure Code – runs on multiple nodes: object data and parity is striped across all disks in all
nodes, all objects are accessible from any working node
RAID in not required for the second and third options. Data is protected using object level
erasure coding and bitrot protection.
Using a load balancer ensures that connections are only sent to ready/available nodes and also that these
connections are distributed equally.
Timeouts
For MinIO Server, the load balancer’s client and server timeouts are set to 1 0 minutes.
Port Requirements
The following table shows the port(s) that are load balanced:
Port 9000 is the default port for MinIO but this can be changed if required by modifying the node
SSL/TLS Termination
To enable secure communication, SSL/TLS is terminated on the load balancer.
Health Checks
As mentioned here, MinIO includes 2 un-authenticated probe points that can be used to determine the state of
each MinIO node. In this guide, the health checks are configured to read the readiness probe /minio/health/ready.
Deployment Concept
The load balancer can be deployed as a single unit, although Loadbalancer.org recommends a
clustered pair for resilience & high availability. Please refer to Configuring HA - Adding a
Secondary Appliance for more details on configuring a clustered pair.
The same download is used for the licensed product, the only difference is that a license key file
(supplied by our sales team when the product is purchased) must be applied using the
appliance’s WebUI.
The VA has 4 network adapters. For VMware only the first adapter (eth0) is connected by
default. For HyperV, KVM, XEN and Nutanix AHV all adapters are disconnected by default. Use
the network configuration screen within the Hypervisor to connect the required adapters.
Be sure to set a secure password for the load balancer, when prompted during the setup routine.
There are certain differences when accessing the WebUI for the cloud appliances. For details,
please refer to the relevant Quick Start / Configuration Guide.
https://<IP-address-configured-during-the-network-setup-wizard>:9443/lbadmin/
You’ll receive a warning about the WebUI’s SSL certificate. This is due to the default self
signed certificate that is used. If preferred, you can upload your own certificate - for more
information, please refer to Appliance Security Features.
If you need to change the port, IP address or protocol that the WebUI listens on, please
refer to Service Socket Addresses.
Username: loadbalancer
Password: <configured-during-network-setup-wizard>
To change the password, use the WebUI menu option: Maintenance > Passwords.
3. If the latest version is already installed, a message similar to the following will be displayed:
4. If an update is available, you’ll be presented with a list of new features, improvements, bug fixes and security
related updates.
Do not navigate away whilst the update is ongoing, this may cause the update to fail.
6. Once complete (the update can take several minutes depending on download speed and upgrade version)
the following message will be displayed:
7. If services need to be reloaded/restarted or the appliance needs a full restart, you’ll be prompted accordingly.
Please contact support@loadbalancer.org to check if an update is available and obtain the latest
6. If services need to be reloaded/restarted or the appliance needs a full restart, you’ll be prompted accordingly.
TCP 22 * SSH
The ports used for SSH, GSLB, SNMP, the WebUI, the fallback page, the gateway service and the
shuttle service can be changed if required. For more information, please refer to Service Socket
Addresses.
1. For nodes 1 – 4:
set the hostnames using an appropriate sequential naming convention, e.g. minio1, minio2, minio3,
minio4
mount the disks using an appropriate sequential naming convention, e.g.
disk 1 → /mnt/minio-data1
disk 2 → /mnt/minio-data2
ensure that /etc/hosts refers to the nodes own allocated IP address rather than the 127.0.0.1 loopback
address
set the domain name of each node to an appropriate value, e,g. lbtestdom.com
2. Run the following commands on all nodes to start MinIO in Distributed Erasure Code mode:
export MINIO_ACCESS_KEY=<minio>
_export MINIO_SECRET_KEY=<minio123>_
The sequential naming convention used for the hostnames and the disks enables this
command format to be used.
Change the hostnames, domain name, access key and secret key to suit your
requirements.
3. Enter an appropriate name for the VIP in the Label field, e.g. MinIO-Cluster.
4. Set the Virtual Service IP address field to the required IP address, e.g. 192.168.110.65.
7. Click Update.
9. Scroll down to the Persistence section and set Persistence Mode to None.
10. Scroll down to the Health Checks section and set the Health Check to Negotiate HTTP (HEAD).
If preferred, the liveness probe (minio/health/live) can be used instead of the readiness
probe (minio/health/ready). For more details of both please refer to the MinIO monitoring
documentation available here.
12. Leave Response Expected blank – this will cause the load balancer to look for an HTTP 200 OK response
from each Real Server.
14. Enable (check) the Timeout checkbox and set both Client Timeout & Real Server Timeout to 10m (i.e. 10
minutes).
2. Enter an appropriate name for the server in the Label field, e.g. minio1.
3. Change the Real Server IP Address field to the required IP address, e.g. 192.168.110.60.
5. Click Update.
6. Now repeat these steps to add the other MinIO server nodes.
1. Using the WebUI, navigate to: Cluster Configuration > SSL Certificates.
2. Click Add a new SSL Certificate & select Upload prepared PEM/PFX file.
4. Browse to and select the certificate file to upload (PEM or PFX format).
2. Using the Associated Virtual Service drop-down, select the Virtual Service created above, e.g. MinIO-Cluster.
Once the VIP is selected, the Label field will be auto-populated with SSL-MinIO-Cluster.
This can be changed if preferred.
6. Click Update.
Once the load balancer and MinIO nodes are configured you can use the MinIO client, a web browser or an
alternative 3rd party S3 browser to view the buckets and objects. Connect to the VIP address on the load balancer
rather than one of the MinIO nodes
● minio1.lbtestdom.com:9000
Uptime: 43 minutes
Version: 2019-10-11T00:38:09Z
Storage: Used 901 MiB, Free 24 GiB
Drives: 2/2 OK
MEM usage
current 68 MiB
historic 68 MiB
● minio2.lbtestdom.com:9000
Uptime: 43 minutes
Version: 2019-10-11T00:38:09Z
Storage: Used 901 MiB, Free 24 GiB
Drives: 2/2 OK
MEM usage
current 68 MiB
historic 68 MiB
● minio3.lbtestdom.com:9000
Uptime: 43 minutes
Version: 2019-10-11T00:38:09Z
Storage: Used 901 MiB, Free 24 GiB
Drives: 2/2 OK
MEM usage
current 68 MiB
historic 68 MiB
● minio4.lbtestdom.com:9000
Uptime: 43 minutes
Version: 2019-10-11T00:38:09Z
Storage: Used 901 MiB, Free 24 GiB
Drives: 2/2 OK
MEM usage
current 68 MiB
historic 68 MiB
For Enterprise Azure, the HA pair should be configured first. For more information, please refer
to the Azure Quick Start/Configuration Guide available in the documentation library
The clustered HA pair uses Heartbeat to determine the state of the other appliance. Should the active device
(normally the Primary) suffer a failure, the passive device (normally the Secondary) will take over.
Non-Replicated Settings
A number of settings are not replicated as part of the Primary/Secondary pairing process and therefore must be
manually configured on the Secondary appliance. These are listed by WebUI menu option in the table below:
Local Configuration Network Interface Interface IP addresses, bonding configuration and VLANs
Configuration
Local Configuration System Date & time Time and date related settings
1. Deploy a second appliance that will be the Secondary and configure initial network settings.
2. Using the WebUI on the Primary appliance, navigate to: Cluster Configuration > High-Availability
Configuration.
3. Specify the IP address and the loadbalancer user’s password for the Secondary (peer) appliance as shown in
the example above.
Clicking the Restart Heartbeat button on the Primary appliance will also automatically restart
heartbeat on the Secondary appliance.
For more details on configuring HA with 2 appliances, please refer to Appliance Clustering for
HA.
For details on testing and verifying HA, please refer to Clustered Pair Diagnostics.
1.1.0 1 December 2021 Converted the document to AsciiDoc Move to new AH, RJC, ZAC
documentation
system
1.1.1 26 April 2022 Updated SSL related content to reflect New software RJC
latest software version release
1.1.2 28 September 2022 Updated layer 7 VIP and RIP creation Reflect changes in AH
screenshots the web user
interface
About Loadbalancer.org