ZFS on FreeBSD
Overview and iSCSI Example
What is ZFS
128bit filesystem
Combined volume management and filesystem
Released by Sun Microsystems to Replace aging UFS in
2006
Can store 256 quadrillion(256x1015) zettabytes (1
zettabyte = 1 billion terabytes)
256,000,000,000,000,000,000,000,000,000,000,000,000 (256x1036)
bytes (give or take a few terabytes)
Uses Pooled Storage Model
Open Source GPL and CDDL
ZFS Features
Data Integrity
Copy on Write transaction model
Raid Support (Striped, Mirror, and RaidZ/Raid5) in Pools/Volumes
Quick and Easy Snapshots
Data Compression and Deduplication
Block level full and differential replication
Robust Cache Architecture
Easy administration with command line, only two commands zpool and
zfs
Almost eliminates /etc/fstab
One the fly setting modifications
ZFS Limitations
ZFS is only fully supported on Solaris, OpenSolaris,
Open Indiana, and FreeBSD
64bit OS recommended
2GB of Ram per 1TB of disk space (Used mostly for
cache).
Poor performance on slow CPUs
Data Integrity
Every data block that is written has a checksum in
metadata (raid volumes have a checksum for each
redundant block)
Data verified against a checksum on every read
raid1 and up volumes have automatic error correction,
all volumes have automatic error detection
Data Integrity can find hardware issues in controllers,
disks, and memory
Data integrity can be disabled, not recommended.
Copy on Write (COW) Transaction Model
Any time data is written a new block is created instead
of reusing existing blocks
COW allows for extremely fast and efficient snapshots
(zero performance penalty unlike LVM2)
Old blocks are eventually reused if they have no
references in the current file system or and snapshot
ZFS Architecture
ZFS is a uses pooled storage model
pools are not mounted directly instead datasets
are created in a pool and they are then mounted
to access the filesystem
mounting a zfs is handled by the zfs daemon not
the UNIX mount command. Mount points are
defined in the dataset not /etc/fstab
Raid on ZFS (Pools)
Supports spaned, mirrored, and raid5 pools
(zpool create)
Supports easily replacing disks in pool
(zpool replace)
Supports cache devices
Supports easily adding disks to a pool (zpool
attach or zpool add)
ZFS (Zpool) Volume Management
Commands
zpool create pool device
create a new pool
zpool add pool device
add a device to a existing pool
zpool remove pool device
remove a device from a pool
zpool replace pool device new_device
replace a faulty device in a pool
zpool destroy pool
delete a pool from the system
zpool status
get the status of all available pools
More information is available in the zpool man page.
Snapshots
command syntax:
zfs snapshot Volume@snapshot
Snapshots are differential
Snapshots are created very quickly around 2 sec or less
Snapshots supported on volumes and sub volumes
previous snapshot data is accessible via the .zfs
directory on each volume
Replication/Backup
command syntax:
zfs send volume@snapshot
zfs receive volume@snapshot
zfs can only send snapshots
zfs send and receive allows to send a snapshot of
filesystem to anything (disk, usb storage, tape, or
network service)
using ssh or netcat zfs can be replicated to a remote
machine
Differential replication is possible if the receive end
has a previous snapshot
Compression and Deduplication
command syntax:
zfs set dedup=on|off volume
zfs set compression=on|off volume
Deduplication uses a checksum or verify to find
redundant data to consolidate. Really useful for data
that doesn't change much (Disk Images and Backups).
Compression uses gzip or lzjb to compress data
Both Deduplication and Compression both happen at
the file system level so the operating system sees the
uncompress or undeduped file
FreeBSD ZFS Installer
The install ISOs at http://www.freebsd.org do not setup ZFS by default
ZFS installer ISOs are available at http://mfsbsd.vx.sk
ZFS installer will use entire disk
Installation steps:
1. Boot from CD
2. login as root/mfsroot
3. run destroygeom (optional)
4. mount CD on /cdrom
5. run zfsinstall script
6. chroot to /tank/root and set root password, hostname and
IP
7. Reboot
Exporting ZFS using Samba
Advantages:
easy to administer, exports are based on directories,
Windows clients can easily connect, user based
authentication, Design for multiple connections per export
Disadvantages: I/O throughput relatively low, really only useful for using
export as a file share
Samba is available through the samba35 port
Config file is in /usr/local/etc/smb.conf
Init script is /usr/local/etc/rc.d/samba
Exporting ZFS using NFS
Advantages:
Disadvantages:
easy to administer, fair I/O throughput,Managed export
config, exports are based on directories not files
I/O throughput not as high as iSCSI over ethernet,
access is difficult for not Linux/Unix clients
ZFS uses the the distribution (system) NFS server in FreeBSD
nfs_server_enable and rpcbind_enable must be set to YES in /etc/rc.conf
zfs datasets are exported via the zfs dataset property sharenfs
zfs exports are stored in /etc/zfs/exports (this is a managed file do not
edit)
to share the dataset tank/vol1you would issue the command zfs set
sharenfs=on tank/vol1
nfs options such as alldirs, maproot, network, mask can be set using a zfs
set sharenfs=<option> dataset command
Exporting ZFS using iSCSI over Ethernet
Advantages:
Great I/O throughput, Works great for Virtual Machines,
Easy to connect nonUnix clients
Disadvantages:
Difficult to administer, Deamon reload required after each
config change, Initial setup can be trick without examples
iSCSI is provided using the istgt port (third party package)
istgt has two config files auth.conf and istgt.conf. They are both located in
/usr/local/etc/istgt/
istgt can export volumized datasets or raw diskimages (diskimages are easier)
each diskimage or dataset are exported as a Logical Unit in /usr/local/etc/istgt.
conf
The Windows iSCSI client does not like " " or "_" in Logical Unit entries, otherwise
istgt will accept them.
init script located in /usr/local/etc/rc.d/istgt. Set istgt_enable to YES in /etc/rc.
conf to start at boot.
Istgt Auth.conf Example
[AuthGroup1]
Comment "Unit Controller Users"
Auth
"user2" "secret123456"
[AuthGroup2]
Comment "Initiator Users"
Auth
"user2" "secret123456"
Istgt istgt.conf example
[PortalGroup1]
[Global]
Comment
"Global section"
Comment
"Portal Group 1"
NodeBase
"zfs1.sebeka.k12.mn.us"
Portal
DA1 10.1.0.95:3260
PidFile
/var/run/istgt.pid
AuthFile
/usr/local/etc/istgt/auth.conf
MediaDirectory
/var/istgt
Comment
"Initiator Group 1"
LogFacility
"local7"
InitiatorName
"ALL"
Timeout
30
Netmask
10.1.0.0/23
NopInInterval
20
DiscoveryAuthMethod
Auto
MaxSessions
64
MaxConnections
16
FirstBurstLength
65536
MaxBurstLength
262144
MaxRecvDataSegmentLength 262144
[UnitControl]
Comment
"Unit Controller"
AuthMethod
CHAP
AuthGroup
AuthGroup1
Portal
UC1 127.0.0.1:3261
Netmask
127.0.0.1
[InitiatorGroup1]
[LogicalUnit1]
TargetName
Mapping
AuthMethod
AuthGroup
UnitType
UnitInquiry
LUN0
sambaserversys
PortalGroup1 InitiatorGroup1
Auto
AuthGroup2
Disk
"FreeBSD" "iSCSI Disk" "0123" "10000001"
Storage /vol1/images/sambaserver_sys.raw auto
Leveraged Uses of ZFS
Shared Hypervisor Storgage (Dedupe,
Snapshots, and Rreprication)
Backup Storage (Dedupe/Compression,
Snapshots)
Software Installation Rollbacks (Snapshots)
Compressed Logging volumes (Compression)
Samba/NFS Server (Snapshots,Compression)
ZFS Online Resources
FreeBSD Wiki Page for ZFS
http://wiki.freebsd.org/ZFS
FreeBSD Handbook Chapter on ZFS
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/filesystems-zfs.html
Official ZFS Admin Guide
http://docs.oracle.com/cd/E19253-01/819-5461/
