Louwrentius

Intel and Apple released Thunderbolt a high-speed (10 Gigabit/s) interface, that seems to replace both USB and Firewire. It is mainly targeted at end-user systems allowing to connect peripherals with just a single cable to a computer. Thunderbolt devices, like external hard drives or displays can be daisy chained, like Firewire. In short, Thunderbolt removes the cable clutter and ads a significant speed bonus.

For NAS owners and storage enthusiasts, this is also a very interesting technology. Just like Firewire, it seems to support computer-to-computer communication. So Thunderbolt could be used as a high-speed link between your homegrown NAS device and your PC workstation. Or between two storage / server system.

The only downside to Thunderbolt is the maximum cable length of 3 meters between devices. Thunderbolt doesn't seem to be the ideal replacement for your Gigabit network, but if most of your computer systems are close to each other, it might be very interesting.

Hardware RAID controllers are considered 'the best' solution for high performance and high availability. However, this is not entirely true. Using a hardware RAID controller might even endanger your precious data.

For enterprise environments, where performance is critical, it is more important that the arrays keeps on delivering data at a high speed. Professional RAID controllers use TLER with TLER-enabled disks to limit the time spend on recovering bad sectors. If a disk encounters a bad sector, there is no time to pause and try to fix it. The disk is just dropped out of the RAID array after just a couple of seconds. At that moment, the array still performes relatively well, but there is no redundancy. If another disk fails (another bad sector?) the array is lost, with all its data.

More people are building NAS boxes for centralized storage of data, for private home use. Since disks are cheap, it is possible to create lots of storage capacity for little money. Creating backups of terabytes of data is however not cheap. Or you have to create two NAS boxes. But that is very expensive and not worth the effort.

People seem to spend lots of money on expensive enterprise level hardware RAID cards, not understanding that the whole TLER-mechanism causes an increased risk for their data. In enterprise environments, budgets are relatively big, and data is always backed up. They can afford to take the risk of losing a RAID array due to these backups. But consumers often don't have the money to spend on creating backups of terabytes of data. They just go for RAID 5 or RAID 6 and hope for the best.

For consumers, if the RAID array goes, all data is lost.

So consumers should choose a RAID solution that will do its best to recover from hardware failure. Performance is not so much an issue. Reliability is. So consumers do want disks to spend 'ages' on recovering bad sectors if this means that the RAID array will survive.

Linux software RAID and ZFS do not use TLER and therefore are a safer choice for your data then regular hardware RAID controllers. You may still use such controllers (but please test them properly) but only to provide SATA ports with individual disks, the RAID part should be handled by Linux.

So in my opinion, hardware RAID controllers are more expensive, require more expensive (enterprise) disks and are less safe for your data.

Just a small post.

To benchmark your Linux software RAID array as setup with MDADM, please use my new benchmark script. I used this script to create these results.

You may need to configure some values within the header of this file to make it fit your enviroment.

 DEVICES="/dev/sd[a-f]"
 NO_OF_DEVICES=6
 ARRAY=/dev/md5
 CHUNKS="4 8 16 32 64 128 256 512 1024"
 MOUNT=/storage
 LOG=/var/log/raid-test.log
 LOGDEBUG=/var/log/raid-test-debug.log
 LEVEL="0 5 6 10"
 TESTFILE=$MOUNT/test.bin
 TESTFILESIZE=10000 (IN MB, thus this is 10 GB)
 TRIES=5 (how many times to run a benchmark.)

By default, the script wil format the array using XFS, feel free to format it with another filesystem such as EXT4 or EXT3 or whatever you want to test.