I have been entertaining the idea of migrating from EXT3 to either XFS or JFS. Two things annoy me about EXT3: the darn periodic fsck procedure (aaaagh! Windows scandisk all over again!) and its less than stellar speed performance. After many hours reading reviews, benchmarks, opinions and discussions in general, I reached the obvious conclusion: I would have to test this damn thing on my own if I wanted any real, useful conclusion. Well, even that kind of conclusion can be disputed. Then, after I put myself through the trouble, I felt that I should share the results for whatever it's worth. I also felt that LQ would be the perfect venue for that. So, without further ado, here is my very simplistic benchmark...
Test routine:
-
format a test partition with XFS or JFS;
-
cross-filesystem copy: copy a directory full of MP3 files from an ext3 partition to the test partition;
-
intra-filesystem copy: replicate that same directory (with another name, of course) within the test partition;
-
recursive deletion: delete the entire directories, one at a time.
Notes:
- that MP3 directory contains
5 GB of data in 1,317 files and 103 sub directories;
- the test partition was formatted and mounted on an
LUKS+cryptsetup encrypted underlying slot, which certainly slows everything down, but that is my real-world scenario anyway.
Machine:
Acer Aspire One netbook (yes, why not a netbook?)
1 GHz Atom CPU, 2 GB of RAM, 22 GB partition
Kubuntu 9.04 (Jaunty)
Kernel 2.6.28-15-generic
Command line procedures
XFS:
Code:
luc[141-0]#test> cryptsetup luksOpen /dev/sda3 crya3
luc[142-0]#test> mkfs.xfs -d agcount=4 -l size=32m /dev/mapper/crya3
luc[143-0]#test> mount /dev/mapper/crya3 /mnt/a3 -o noatime,nodiratime,osyncisdsync,logbufs=8
luc[144-0]#test> df -h | grep crya3
/dev/mapper/crya3 23G 4.2M 23G 1% /mnt/a3
luc[145-0]#test> time cp -aux '/music/Test dir' /mnt/a3/
1.312u 62.827s 14:31.81 7.3% 0+0k 10501312+10475304io 1pf+0w
luc[146-0]#test> cd /mnt/a3
luc[147-0]#a3> mkdir Replicate
luc[148-0]#a3> cp -aux 'Test dir' Replicate/
1.616u 76.048s 19:39.39 6.5% 0+0k 10476240+10475304io 2pf+0w
luc[149-0]#a3> time chown -R luc Replicate/
0.004u 0.088s 0:00.32 25.0% 0+0k 136+0io 1pf+0w
luc[150-0]#a3> time rm -rf Replicate/
0.020u 1.160s 0:01.79 65.9% 0+0k 60+0io 1pf+0w
luc[151-0]#a3> time rm -rf 'Test dir'
0.016u 1.000s 0:01.76 57.3% 0+0k 408+0io 0pf+0w
luc[152-0]#a3> df -h | grep crya3
/dev/mapper/crya3 23G 4.2M 23G 1% /mnt/a3
luc[153-0]#a3> umount /mnt/a3/
JFS:
Code:
luc[161-0]#test> jfs_mkfs -L testjfs -s 128 /dev/mapper/crya3
jfs_mkfs version 1.1.12, 24-Aug-2007
Warning! All data on device /dev/mapper/crya3 will be lost!
Continue? (Y/N) Y
|
Format completed successfully.
23437807 kilobytes total disk space.
luc[162-0]#test> mount /dev/mapper/crya3 /mnt/a3 -o noatime,nodiratime
luc[163-0]#test> df -h | grep crya3
/dev/mapper/crya3 23G 3.0M 23G 1% /mnt/a3
luc[164-0]#test> time cp -aux '/music/Test dir' /mnt/a3/
1.600u 73.300s 17:51.11 6.9% 0+0k 10503468+10490696io 2pf+0w
luc[165-0]#test> cd /mnt/a3
luc[166-0]#a3> mkdir Replicate
luc[167-0]#a3> cp -aux 'Test dir' Replicate/
1.408u 75.712s 19:02.84 6.7% 0+0k 9955020+10486592io 3pf+0w
luc[168-0]#a3> time chown -R luc Replicate/
0.004u 0.048s 0:00.20 20.0% 0+0k 88+0io 0pf+0w
luc[169-0]#a3> time rm -rf Replicate/
0.000u 0.156s 0:01.01 14.8% 0+0k 164+24io 1pf+0w
luc[170-0]#a3> time rm -rf 'Test dir'
0.008u 0.124s 0:04.64 2.5% 0+0k 1936+48io 0pf+0w
luc[171-0]#a3> df -h | grep crya3
/dev/mapper/crya3 23G 3.0M 23G 1% /mnt/a3
luc[172-0]#a3> umount /mnt/a3/
EXT3:
Code:
luc[181-0]#test> mkfs.ext3 -m 1 -b 2048 -O has_journal -O dir_index -L testext3 /dev/mapper/crya3
mke2fs 1.41.4 (27-Jan-2009)
Filesystem label=testext3
OS type: Linux
Block size=2048 (log=1)
Fragment size=2048 (log=1)
1466368 inodes, 11718902 blocks
117189 blocks (1.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=549453824
716 block groups
16384 blocks per group, 16384 fragments per group
2048 inodes per group
Superblock backups stored on blocks:
16384, 49152, 81920, 114688, 147456, 409600, 442368, 802816, 1327104,
2048000, 3981312, 5619712, 10240000
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 26 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
luc[182-0]#test> tune2fs -o journal_data_ordered /dev/mapper/crya3
luc[183-0]#test> mount /dev/mapper/crya3 /mnt/a3 -o noatime,nodiratime
luc[184-0]#test> df -h | grep crya3
/dev/mapper/crya3 22G 79M 22G 1% /mnt/a3
luc[185-0]#test> time cp -aux '/music/Test dir' /mnt/a3/
0.820u 71.592s 10:20.53 11.6% 0+0k 10502484+10475304io 2pf+0w
luc[186-0]#test> cd /mnt/a3
luc[187-0]#a3> mkdir Replicate
luc[188-0]#a3> cp -aux 'Test dir' Replicate/
0.832u 65.864s 11:13.49 9.9% 0+0k 10332432+10475304io 3pf+0w
luc[189-0]#a3> time chown -R luc Replicate/
0.016u 0.060s 0:01.03 6.7% 0+0k 716+0io 0pf+0w
luc[190-0]#a3> time rm -rf Replicate/
0.016u 2.576s 0:30.31 8.5% 0+0k 16532+0io 1pf+0w
luc[191-0]#a3> time rm -rf 'Test dir'
0.012u 2.564s 0:32.67 7.8% 0+0k 18456+0io 0pf+0w
luc[192-0]#a3> umount /mnt/a3/
Summary:
-
Formatting the XFS partition took
1 second. Sorry, I forgot to include that time measurement in the scripted procedure.
- Copying the 5 GB directory from an EXT3 partition to the XFS partition took
14 minutes and 31 seconds.
- Replicating the 5 GB directory within the XFS partition took
19 minutes and 39 seconds.
-
Deleting the replicated test directory recursively took
1.79 second! Fast! Who said that XFS is slow to delete?
-
Deleting the test directory recursively took
1.76 second!
-
Formatting the JFS partition took from
15 to 20 seconds. Sorry, I forgot to include that time measurement in the scripted procedure.
- Copying the 5 GB directory from an EXT3 partition to the JFS partition took
17 minutes and 51 seconds.
- Replicating the 5 GB directory within the JFS partition took
19 minutes and 02 seconds.
-
Deleting the replicated test directory recursively took
1 second!
-
Deleting the test directory recursively took
4.64 seconds. I found that odd. The two directories were identical.
-
Formatting the EXT3 partition took from
50 to 60 seconds. Sorry, I forgot to include that time measurement in the scripted procedure.
- Copying the 5 GB directory from an EXT3 partition to another EXT3 partition took
10 minutes and 20 seconds.
- Replicating the 5 GB directory within the JFS partition took
11 minutes and 13 seconds.
-
Deleting the replicated test directory recursively took
30 seconds. Slow!
-
Deleting the test directory recursively took
32 seconds. The only big disappointment in the EXT3 test.
A more concise breakdown:
format test partition
XFS: 1 second
JFS: 15-20 seconds
EXT3: 50-60 seconds
cross-FS copy:
XFS: 14:31.81 7.3%
JFS: 17:51.11 6.9%
EXT3: 10:20.53 11.6%
intra-FS copy (replicate directory):
XFS: 19:39.39 6.5%
JFS: 19:02.84 6.7%
EXT3: 11:13.49 9.9%
recursive deletion (first directory):
XFS: 0:01.79 65.9%
JFS: 0:01.01 14.8%
EXT3: 0:30.31 8.5%
recursive deletion (second directory):
XFS: 0:01.76 57.3%
JFS: 0:04.64 2.5%
EXT3: 0:32.67 7.8%
I am still not sure what conclusion I should take from this.
My main goal (besides overall performance) is achieving faster backups, which I always do with rsync across hard disks. Since I always use 'rsync --delete' a lot of deletion may be involved, so XFS would certainly bring me a lot of benefit. But XFS copies new data a lot slower than EXT3.
One could argue that XFS would make up for that with its quite competent xfsdump/restore tool, which makes incremental backups, including ability to interrupt and resume later. But I am not convinced that xfsdump/restore is a good alternative to rsync. If I ever migrated to XFS, I would probably remain faithful to rsync, so xfsdump/restore is not a compelling argument in favor of XFS.
XFS is also alluring because it can defrag a mounted file system, but I have no idea of how much impact the issue of fragmentation really has on file system performance. Judging from my many years using a very fragmented EXT3 file system every day, I would wager that the impact is pretty negligible.
I am basically torn between EXT3 and XFS. I am ruling JFS out because I think it fared poorly in the cross-filesystem copy test, which is the most important one for me.
I have no interest in the Reiser FS. I have never put a lot of trust in it and I understand it is now headed to oblivion anyway.
Like I said before, I am just posting it in case it is useful to someone. I know it's not comprehensive, it's quick and dirty. Take it with a grain of salt and a sprinkle of pepper, like any other benchmark. Related thoughts and ideas are welcome, thank you.
They said they will add one at some point. I use JFS, but have used XFS for a while.
