Wednesday, August 15, 2012

FreeBSD RAID

I just finished reconfiguring my network file server - a long process.  I'm running FreeBSD, and the geom utilities for RAID configuration.

I have four disks - 3x 320GB, 1x 1TB.  I placed the 3 matching disks into a stripe, or RAID0.  This RAID0 disk is then in a mirror/RAID1 with the 1TB drive.

Configuring the stripe & mirror is done with the gstripe and gmirror commands.  Since the array is also a boot/root disk, I have entries in /boot/loader.conf:
geom_stripe_load="YES"
geom_mirror_load="YES"

These instantiate the stripe & mirror prior to reading fstab and completing boot.  Also /etc/fstab is set to the array device:
/dev/mirror/gm0b        none            swap    sw           0       0
/dev/mirror/gm0a        /               ufs     rw           1       1
/dev/mirror/gm0e        /tmp            ufs     rw           2       2
/dev/mirror/gm0f        /usr            ufs     rw           2       2
/dev/mirror/gm0d        /var            ufs     rw           2       2
The 1TB drive, ad7, contains 1 slice ad7s1.  The gm0 mirror is between the stripe, st0, and that slice.  I suppose I could have added the entire ad7 to the mirror instead.  ad7 is larger than st0, so in that situation gm0 would have been created with st0, and then ad7 would be inserted.

gmirror bases the size of the mirror on the first device it is created with:
gmirror label -v -b prefer gm0 /dev/ad7s1
In this case, I built gm0 with ad7s1, then inserted the stripe:
gmirror insert gm0 stripe/st0
One caveat is, when building a mirror on a slice (like ad7s1) it must be mounted read-only or not at all.  If you build it on a device (like ad7) then you can simply use
sysctl kern.geom.debugflags=17
The system will then let you write to the disk metadata.
To build on the root slice, I rebooted into single user mode, which mounts root as read-only.

I like this setup.  I've got RAID1 redundancy, and the 3-disk stripe is quite peppy.  Lastly, to take advantage of the speed of the stripe, I set the priority of the stripe in gm0 higher than ad7s1.  Since gm0 uses a 'prefer' balance mode, it will read from the stripe first.  I believe writes complete on the stripe as well, but I'm not certain.  I'm getting decent write speeds with dd commands.  I'll have to do a more thorough performance test some time in the future.

No comments:

Post a Comment