LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - General (http://www.linuxquestions.org/questions/linux-general-1/)
-   -   My own XFS - JFS - EXT3 benchmark (http://www.linuxquestions.org/questions/linux-general-1/my-own-xfs-jfs-ext3-benchmark-809670/)

lucmove 05-23-2010 06:05 PM

My own XFS - JFS - EXT3 benchmark
 
I have been entertaining the idea of migrating from EXT3 to either XFS or JFS. Two things annoy me about EXT3: the darn periodic fsck procedure (aaaagh! Windows scandisk all over again!) and its less than stellar speed performance. After many hours reading reviews, benchmarks, opinions and discussions in general, I reached the obvious conclusion: I would have to test this damn thing on my own if I wanted any real, useful conclusion. Well, even that kind of conclusion can be disputed. Then, after I put myself through the trouble, I felt that I should share the results for whatever it's worth. I also felt that LQ would be the perfect venue for that. So, without further ado, here is my very simplistic benchmark...

Test routine:
- format a test partition with XFS or JFS;
- cross-filesystem copy: copy a directory full of MP3 files from an ext3 partition to the test partition;
- intra-filesystem copy: replicate that same directory (with another name, of course) within the test partition;
- recursive deletion: delete the entire directories, one at a time.

Notes:
- that MP3 directory contains 5 GB of data in 1,317 files and 103 sub directories;
- the test partition was formatted and mounted on an LUKS+cryptsetup encrypted underlying slot, which certainly slows everything down, but that is my real-world scenario anyway.

Machine:
Acer Aspire One netbook (yes, why not a netbook?)
1 GHz Atom CPU, 2 GB of RAM, 22 GB partition
Kubuntu 9.04 (Jaunty)
Kernel 2.6.28-15-generic

Command line procedures

XFS:

Code:

luc[141-0]#test> cryptsetup luksOpen /dev/sda3 crya3
luc[142-0]#test> mkfs.xfs -d agcount=4 -l size=32m /dev/mapper/crya3
luc[143-0]#test> mount /dev/mapper/crya3 /mnt/a3 -o noatime,nodiratime,osyncisdsync,logbufs=8
luc[144-0]#test> df -h | grep crya3
/dev/mapper/crya3      23G  4.2M  23G  1% /mnt/a3

luc[145-0]#test> time cp -aux '/music/Test dir' /mnt/a3/
1.312u 62.827s 14:31.81 7.3%    0+0k 10501312+10475304io 1pf+0w                   

luc[146-0]#test> cd /mnt/a3
luc[147-0]#a3> mkdir Replicate
luc[148-0]#a3> cp -aux 'Test dir' Replicate/
1.616u 76.048s 19:39.39 6.5%    0+0k 10476240+10475304io 2pf+0w

luc[149-0]#a3> time chown -R luc Replicate/
0.004u 0.088s 0:00.32 25.0%    0+0k 136+0io 1pf+0w

luc[150-0]#a3> time rm -rf Replicate/
0.020u 1.160s 0:01.79 65.9%    0+0k 60+0io 1pf+0w

luc[151-0]#a3> time rm -rf 'Test dir'
0.016u 1.000s 0:01.76 57.3%    0+0k 408+0io 0pf+0w

luc[152-0]#a3> df -h | grep crya3
/dev/mapper/crya3      23G  4.2M  23G  1% /mnt/a3

luc[153-0]#a3> umount /mnt/a3/

JFS:

Code:

luc[161-0]#test> jfs_mkfs -L testjfs -s 128 /dev/mapper/crya3
jfs_mkfs version 1.1.12, 24-Aug-2007                                                     
Warning!  All data on device /dev/mapper/crya3 will be lost!                               

Continue? (Y/N) Y
  |           

Format completed successfully.

23437807 kilobytes total disk space.

luc[162-0]#test> mount /dev/mapper/crya3 /mnt/a3 -o noatime,nodiratime
luc[163-0]#test> df -h | grep crya3
/dev/mapper/crya3      23G  3.0M  23G  1% /mnt/a3

luc[164-0]#test> time cp -aux '/music/Test dir' /mnt/a3/
1.600u 73.300s 17:51.11 6.9%    0+0k 10503468+10490696io 2pf+0w

luc[165-0]#test> cd /mnt/a3
luc[166-0]#a3> mkdir Replicate
luc[167-0]#a3> cp -aux 'Test dir' Replicate/
1.408u 75.712s 19:02.84 6.7%    0+0k 9955020+10486592io 3pf+0w

luc[168-0]#a3> time chown -R luc Replicate/
0.004u 0.048s 0:00.20 20.0%    0+0k 88+0io 0pf+0w

luc[169-0]#a3> time rm -rf Replicate/
0.000u 0.156s 0:01.01 14.8%    0+0k 164+24io 1pf+0w

luc[170-0]#a3> time rm -rf 'Test dir'
0.008u 0.124s 0:04.64 2.5%      0+0k 1936+48io 0pf+0w

luc[171-0]#a3> df -h | grep crya3
/dev/mapper/crya3      23G  3.0M  23G  1% /mnt/a3

luc[172-0]#a3> umount /mnt/a3/

EXT3:

Code:

luc[181-0]#test> mkfs.ext3 -m 1 -b 2048 -O has_journal -O dir_index -L testext3 /dev/mapper/crya3

mke2fs 1.41.4 (27-Jan-2009)                                                               
Filesystem label=testext3                                                                 
OS type: Linux                                                                           
Block size=2048 (log=1)                                                                   
Fragment size=2048 (log=1)                                                               
1466368 inodes, 11718902 blocks                                                           
117189 blocks (1.00%) reserved for the super user                                         
First data block=0                                                                       
Maximum filesystem blocks=549453824                                                       
716 block groups                                                                         
16384 blocks per group, 16384 fragments per group                                         
2048 inodes per group                                                                     
Superblock backups stored on blocks:                                                     
        16384, 49152, 81920, 114688, 147456, 409600, 442368, 802816, 1327104,             
        2048000, 3981312, 5619712, 10240000                                               

Writing inode tables: done                           
Creating journal (32768 blocks): done               
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 26 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

luc[182-0]#test> tune2fs -o journal_data_ordered /dev/mapper/crya3
luc[183-0]#test> mount /dev/mapper/crya3 /mnt/a3 -o noatime,nodiratime
luc[184-0]#test> df -h | grep crya3
/dev/mapper/crya3      22G  79M  22G  1% /mnt/a3

luc[185-0]#test> time cp -aux '/music/Test dir' /mnt/a3/
0.820u 71.592s 10:20.53 11.6%  0+0k 10502484+10475304io 2pf+0w

luc[186-0]#test> cd /mnt/a3
luc[187-0]#a3> mkdir Replicate
luc[188-0]#a3> cp -aux 'Test dir' Replicate/
0.832u 65.864s 11:13.49 9.9%    0+0k 10332432+10475304io 3pf+0w

luc[189-0]#a3> time chown -R luc Replicate/
0.016u 0.060s 0:01.03 6.7%      0+0k 716+0io 0pf+0w

luc[190-0]#a3> time rm -rf Replicate/
0.016u 2.576s 0:30.31 8.5%      0+0k 16532+0io 1pf+0w

luc[191-0]#a3> time rm -rf 'Test dir'
0.012u 2.564s 0:32.67 7.8%      0+0k 18456+0io 0pf+0w

luc[192-0]#a3> umount /mnt/a3/


Summary:

- Formatting the XFS partition took 1 second. Sorry, I forgot to include that time measurement in the scripted procedure.
- Copying the 5 GB directory from an EXT3 partition to the XFS partition took 14 minutes and 31 seconds.
- Replicating the 5 GB directory within the XFS partition took 19 minutes and 39 seconds.
- Deleting the replicated test directory recursively took 1.79 second! Fast! Who said that XFS is slow to delete?
- Deleting the test directory recursively took 1.76 second!

- Formatting the JFS partition took from 15 to 20 seconds. Sorry, I forgot to include that time measurement in the scripted procedure.
- Copying the 5 GB directory from an EXT3 partition to the JFS partition took 17 minutes and 51 seconds.
- Replicating the 5 GB directory within the JFS partition took 19 minutes and 02 seconds.
- Deleting the replicated test directory recursively took 1 second!
- Deleting the test directory recursively took 4.64 seconds. I found that odd. The two directories were identical.

- Formatting the EXT3 partition took from 50 to 60 seconds. Sorry, I forgot to include that time measurement in the scripted procedure.
- Copying the 5 GB directory from an EXT3 partition to another EXT3 partition took 10 minutes and 20 seconds.
- Replicating the 5 GB directory within the JFS partition took 11 minutes and 13 seconds.
- Deleting the replicated test directory recursively took 30 seconds. Slow!
- Deleting the test directory recursively took 32 seconds. The only big disappointment in the EXT3 test.

A more concise breakdown:

format test partition
XFS: 1 second
JFS: 15-20 seconds
EXT3: 50-60 seconds

cross-FS copy:
XFS: 14:31.81 7.3%
JFS: 17:51.11 6.9%
EXT3: 10:20.53 11.6%

intra-FS copy (replicate directory):
XFS: 19:39.39 6.5%
JFS: 19:02.84 6.7%
EXT3: 11:13.49 9.9%

recursive deletion (first directory):
XFS: 0:01.79 65.9%
JFS: 0:01.01 14.8%
EXT3: 0:30.31 8.5%

recursive deletion (second directory):
XFS: 0:01.76 57.3%
JFS: 0:04.64 2.5%
EXT3: 0:32.67 7.8%

I am still not sure what conclusion I should take from this.

My main goal (besides overall performance) is achieving faster backups, which I always do with rsync across hard disks. Since I always use 'rsync --delete' a lot of deletion may be involved, so XFS would certainly bring me a lot of benefit. But XFS copies new data a lot slower than EXT3.

One could argue that XFS would make up for that with its quite competent xfsdump/restore tool, which makes incremental backups, including ability to interrupt and resume later. But I am not convinced that xfsdump/restore is a good alternative to rsync. If I ever migrated to XFS, I would probably remain faithful to rsync, so xfsdump/restore is not a compelling argument in favor of XFS.

XFS is also alluring because it can defrag a mounted file system, but I have no idea of how much impact the issue of fragmentation really has on file system performance. Judging from my many years using a very fragmented EXT3 file system every day, I would wager that the impact is pretty negligible.

I am basically torn between EXT3 and XFS. I am ruling JFS out because I think it fared poorly in the cross-filesystem copy test, which is the most important one for me.

I have no interest in the Reiser FS. I have never put a lot of trust in it and I understand it is now headed to oblivion anyway.

Like I said before, I am just posting it in case it is useful to someone. I know it's not comprehensive, it's quick and dirty. Take it with a grain of salt and a sprinkle of pepper, like any other benchmark. Related thoughts and ideas are welcome, thank you.

exvor 05-23-2010 06:43 PM

If you do not like the time ext3 takes to check a volume then you will kill yourself over the time XFS will take. XFS does not use a journal and will take as long as if you had the drive formatted into ext2. I do not have any thoughts on JFS because I have never used it. Also the periodic scan can be disabled with ext3. You should try out ext4 as well as it has some improvements or coming improvements like online defragmentation.

lucmove 05-23-2010 07:30 PM

Quote:

Originally Posted by exvor (Post 3978852)
If you do not like the time ext3 takes to check a volume then you will kill yourself over the time XFS will take. XFS does not use a journal and will take as long as if you had the drive formatted into ext2. I do not have any thoughts on JFS because I have never used it. Also the periodic scan can be disabled with ext3. You should try out ext4 as well as it has some improvements or coming improvements like online defragmentation.

Where/how did you acquire that information? Someone at this page says that "xfs_check is much faster than fsck.ext3".

The EXT4 filesystem is too new and untried for my standards. I won't have any part of it until 2013. I would rather go with the older ones. At least their problems are known and somewhat documented.

exvor 05-23-2010 08:58 PM

Ohh never mind XFS is journal-ed. I got the file systems mixed up when I was using XFS on my nas. Just ignore that part of my post.

As for check times I have never had a check on a ext3 system take very long. Well at least compared to a ext2 system.

lucmove 05-23-2010 09:22 PM

Quote:

Originally Posted by exvor (Post 3978905)
As for check times I have never had a check on a ext3 system take very long. Well at least compared to a ext2 system.

I have two 150 GB partitions and the compulsory fsck takes from 50 to 60 minutes to check EACH.

catkin 05-24-2010 03:06 AM

Quote:

Originally Posted by lucmove (Post 3978836)
- Deleting the replicated test directory recursively took 1 second!
- Deleting the test directory recursively took 4.64 seconds. I found that odd. The two directories were identical.

A buffering effect?

Thanks for sharing your testing and results :)

H_TeXMeX_H 05-24-2010 04:40 AM

With JFS make sure to use the deadline I/O scheduler, it will drastically increase performance in many cases. I'm not sure if it will make a difference in the tests that you did, but it might. When I was using CFQ with JFS it caused many videos to stutter and a/v desync, this must mean that something was very wrong with the way they interact.

XFS is more advanced than JFS, and about as fast, you can do sparse defrag with it as well, whereas JFS has not defrag :( They said they will add one at some point. I use JFS, but have used XFS for a while.


All times are GMT -5. The time now is 01:33 PM.