Louwrentius

Mobo: Asus P5Q-EM DO ( 6x sata)

CPU: Core 2 Duo E7400

RAM: 4 GB

Controller; HighPoint RocketRAID 2314 (16x poorten)

PSU: CooMas Silent Pro M 600W ATX2 (definitely not overkill)

HD: Samsung 1 TB SAT2 HD103UJ (10x)

OS: Debian Lenny with custom kernel (backported)

FS: EXT4

RAID config: 10 disk RAID 6 based on Linux software RAID.

Performance: Read: 850 MB/s Write: 300 MB/s

Capacity: 7,5 GiB or 8 GB.

I am very pleased with the result. Points that should be improved are the temperature and noise. The PSU is a major overkill since this puppy draws less than 200 watt.

Bunny:~# ./show-hdd-temp.sh

Device Temperature

/dev/sda 32

/dev/sdb 34

/dev/sdc 33

/dev/sdd 36

/dev/sde 34

/dev/sdf 33

/dev/sdg 38

/dev/sdh 39

/dev/sdi 37

/dev/sdj 36

/dev/sdk 35

Personalities : [raid6] [raid5] [raid4]

md0 : active raid6 sdj[10] sdb[0] sdl[9] sdk[8] sdg[7] sdf[6] sdi[5] sdh[4] sde[3] sdd2 sdc1

7813463552 blocks super 0.91 level 6, 64k chunk, algorithm 2 [11/11] [UUUUUUUUUUU]

[======>..............] reshape = 30.1% (294904524/976682944) finish=585.3min speed=19413K/sec

I made a small script that converts the output of cat /proc/mdstat to an actual date and time telling you when the RAID rebuild / reshape is finished.

This is the link to the correct version of the script.

Example:

debian:~# ./raid-rebuild-eta.sh

Estimated time of finishing rebuild / reshape:
Mon Jun 29 00:45:07 CEST 2009

So after toying around with RAID 0 just for fun, time to get serious. I created a RAID 6 of 10 x 1 TB disks. This gives me raw device read speeds of 850 MB/s and write speeds of 300 MB/s. I think this is exactly what should be expected, but boy it is damn fast. Especially the write speed surprises me.

Still, if I format the device using EXT4, write speed stays the same. However, read speed drop to about 350 MB/s. I don't understand why. Maybe some misalignment? I don't know.

The issue with the dropped read speed is due to an incorrect chunk size of the array. The array now consists of 20 drives, providing 18 TB of storage using XFS. Read speads exceed 1.1 GB/s (that is not a typo) and write speeds are about 350 MB/s.