In this part we will discuss using Software RAID in Linux and setting up a special
mdadm mirror between the ephemeral SSD and the EBS volume. Setting the flag
--write-mostly with EBS device on the mirror will ensure that the
md driver avoids reading from ebs if at all possible and send all reads to the SSD. This option was originally added when mirroring over a slow network interface, but performs equally well to concentrate reads on an SSD.
In our test environment we have a c3.8xlarge instance running, this is the disk configuration:
1 2 3 4 5 6 7
/dev/xvdc are the 320GB SSD ephemeral disks available to a c3.8xlarge instance while
/dev/xvdd is a provisioned IOPS EBS volume with 4000 iops.
fio on Ubuntu 14.04
mdadm is strangely not installed by default on ubuntu 14.04, so the first step is to install it and while we are at it, lets also install our io performance tester fio.
For some strange reason
mdadm depends on the
postifx and I got a pop asking me to select a mail delivery option which I selected localhost for – I don’t need this for now, if in production you might want to setup an SES SMTP with your new postfix installation.
Setting up software raid-1 mirror
First we will setup a standard software mirror or raid-1 device between
/dev/xvdc (SSD) and
/dev/xvdd (EBS) but tell md to only use the latter for writes.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Ignore the notes about the previous ext2 file system detected on the SSD, cloud-init on ubuntu formats ephemerals disks by default. Also ignore the mdadm message about the difference in device size.
Next lets verify what we have just configured by looking at
1 2 3 4 5 6 7
(W) next to
/dev/xvdd (EBS) indicating this is the
write-mostly device. We should now have a raw device appearing in
/dev/md1 with no file system:
Next let us setup an ext4 file system on the new raid-1 device:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
And mount it to
1 2 3 4 5
Now we are ready to start testing with fio and examine the results.
We conducted the standard io tests for read, write sequential and random workloads and here are the results of the md raid-1 array in comparison to the native SSD and EBS volumes:
- Sequential and random write results are identical to those of EBS, which is expected
- Sequential reads are almost identical to the results of the SSD which is great
- Disappointing random read results – this is where SSD’s excel. We only see a 50% improvement in performance from EBS but nowhere near the expected performance of the SSD.