I’ve got six Crucial m4 SSDs with 256GB in a RAID5 array, resulting in 1.2TB useable disk space.The SSDs are connected via SATA3 to a Gigabyte 990FX UD7 board. I’m running 64bit Ubuntu 11.10 server.
The following two articles offer some good ideas on performance tuning the SSD RAID:
http://h3x.no/2011/07/09/tuning-ubuntu-mdadm-raid56
https://wiki.archlinux.org/index.php/Solid_State_Drives
The chuck size on /dev/md127 is 512k, which is the default in Ubuntu and suits the 512k erase block size of the Crucial m4 disks. I’ve decided to partition the disks with “gdisk”, using a GPT. The partition starts at sector 2048 to align with the SSD chunk size. The filesystem is ext4, using 4k block size and 128 blocks stride, which again matches the 512k EBS. There is no need to pass any special options to mkfs.ext4, it was all autodetected.
I’m using the following optimizations in /etc/rc.local:
md=`ls -l /dev/disk/by-id/md-uuid-f70a5a71:577d6424:8ca4b9cf:265a61a3 |
awk '{ print $NF }' | sed -e 's/[/\.]//g'`
echo 32768 > /sys/block/$md/md/stripe_cache_size
blockdev --setra 4096 /dev/$md
SSD=/dev/disk/by-id/ata-M4*
for s in $SSD
do
NODE=`ls -l $s | awk '{ print $NF }' | sed -e 's/[/\.]//g'`
echo noop > /sys/block/$NODE/queue/scheduler
done
Make sure /etc/rc.local is run by bash, not sh. The block device scheduler for the SSDs is set to “noop”, since there is no seek time on the SSD.
Here are some benchmarks:
root@localhost:/mnt/ssd# dd of=file.bin if=/dev/zero bs=1G count=10
10+0 records in
10+0 records out
10737418240 bytes (11 GB) copied, 20.4972 s, 524 MB/s
root@localhost:/mnt/ssd# dd if=file.bin of=/dev/null
20971520+0 records in
20971520+0 records out
10737418240 bytes (11 GB) copied, 16.53 s, 650 MB/s
650MB/s read speed is not too bad!
Probably add GRUB_CMDLINE_LINUX_DEFAULT=”bootdegraded=true” to /etc/default/grub to allow the array to boot in degraded state.
Note that there is no TRIM support for software raid5 yet (only levels 0,1,10 as of Linux 3.3), but support for levels 4,5,6 is in the making. Until then, a regular wipe of all SSDs may allow for steady performance.