Linux hw raid6 performance using ext4 and xfs (updated)

The story revisited (2012)

The Linux Ext filesystem was definitely improved during the last couple of years.
The sequential write performance was improved a lot. I measure numbers around 2-300MB/s (Linux 2.6.32).

One small note: don't expect to see these numbers on a live system. In a real work environment (under light load) you'll see 30-70MB/sec sequential read and write performance. Random read and write will be even worse.

The original article:

Hardware config

  • 3WARE 9650SE-8 RAID Firmware Version = FE9X 4.08.00.006
  • 8 x WD SATA2GP 1.5TB WD15EADS 7200/32 HDD
  • ASROCK A790GHH/128m
  • 8G RAM
  • CPU AMD AM3 PHENOMII 810 X4

Measurement results

Summary:

  raw ext3 xfs
write
312 MB/s
137 MB/s
269 MB/s
read
419 MB/s
320 - 340 MB/s
410 MB/s

Details:

Following the instructions found on this 3ware page...

Raw write

# sync; time `dd if=/dev/zero of=/dev/sda1 bs=1M count=20480 && sync`
20480+0 records in
20480+0 records out
21474836480 bytes (21 GB) copied, 68.929 seconds, 312 MB/s

real    1m18.373s
user    0m0.033s
sys     0m24.271s

Raw read

# sync; time `dd if=/dev/sda1 of=/dev/null bs=1M count=20480 && sync`
20480+0 records in
20480+0 records out
21474836480 bytes (21 GB) copied, 51.2106 seconds, 419 MB/s

real    0m51.335s
user    0m0.017s
sys     0m15.656s

EXT3 write

# mke2fs -j -T largefile4 -b 4096 /dev/sda1
...
# sync; time `dd if=/dev/zero of=/mnt/pp bs=1M count=20480 && sync`
20480+0 records in
20480+0 records out
21474836480 bytes (21 GB) copied, 156.81 seconds, 137 MB/s

real    3m12.603s
user    0m0.049s
sys     0m36.227s

EXT3 read

# sync; time `dd if=/mnt/pp of=/dev/null bs=1M count=20480 && sync`
20480+0 records in
20480+0 records out
21474836480 bytes (21 GB) copied, 67.165 seconds, 320 MB/s

real    1m7.334s
user    0m0.016s
sys     0m12.980s

Different block size:

# sync; time `dd if=/mnt/pp of=/dev/null bs=4M count=20480 && sync`
5120+0 records in
5120+0 records out
21474836480 bytes (21 GB) copied, 63.088 seconds, 340 MB/s

real    1m3.222s
user    0m0.012s
sys     0m12.504s

XFS write

# sync; time `dd if=/dev/zero of=/mnt/pp bs=1M count=20480 && sync`
20480+0 records in
20480+0 records out
21474836480 bytes (21 GB) copied, 79.7094 seconds, 269 MB/s

real    1m29.821s
user    0m0.054s
sys     0m25.851s

XFS read

# sync; time `dd if=/mnt/pp of=/dev/null bs=1M count=20480 && sync`
20480+0 records in
20480+0 records out
21474836480 bytes (21 GB) copied, 52.4275 seconds, 410 MB/s

real    0m52.564s
user    0m0.024s
sys     0m11.487s