Need help interpreting my filesystem benchmark data
I decided to do my own filesystem benchmarking, because I didn't trust Phoronix.
So let me explain what I did, I booted the Slackware64 13.37 install disk, mounted a USB stick, formatted one of my partitions as various filesystems, and I mount it and ran a benchmarking script. First the commands I ran: Code:
mkfs.ext4 /dev/sda3 Code:
seq 1 10000 | time xargs touch 2> /mnt/usb/tf."$1" Code:
total Code:
%P Percentage of the CPU that this job got, computed as (%U + %S) / %E. I also calculated the CPU time = (%U + %S) = user + system. What does this all mean ? I know that btrfs is the fastest in real time, and jfs is the fastest in CPU time, but I can't relate the two to make some type of index or meaningful number. I want to know which of these filesystems is the most efficient, i.e. which one does things in least amount of time and with the least amount of CPU time. How do I integrate these calculations ? Thanks. |
Okay, so, I'm not really answering but more wondering (willing to learn here, so...) if the cosntellation of forces around the drive is'nt the defining factor as well? Throughput of the controller, bus speed of the motherboard...
Subscribed to this thread, this seems interesting... Thor |
Yes, it would matter. So far this is best resource I have found on the subject:
http://stackoverflow.com/questions/5...utput-of-time1 Still, it is hard to derive meaningful numbers from this data. |
1 Attachment(s)
I have searched a lot on this, and it looks like the best benchmark in this case is just the real time that each command took.
In that case here are the results in a graph: Also note that the deadline IO scheduler seems to be better than CFQ overall. EDIT: Probably, I will switch to btrfs when they get fsck support and are a bit more stable. |
I'm not sure what you are trying to do with your benchmarking.
I dont know why you would use noatime, its not widely used AFAIK (and it could 'skew' the results), and benchmaking filesystems on a USB flash drive could have _very_ different results to a HDD/SSD. |
I was just saving the results to a USB stick, the benchmarks were done on the HDD.
I was using a netbook for the benchmark, the ASUS Eee PC 1015PN. The HDD is Seagate Momentus 5400.6 series. I used noatime, because I always use this option. I don't see why it would skew results. |
I don't know whether I can answer any of your questions, but I hope that I can add to the discussion, a little.
Quote:
Quote:
Here is my argument:
If you can say that you are in one of those corner cases, you can probably say 'this one of the numbers is very much more important than that one; otherwise you have to say that you have to take both into account. Alternatively, you could say, after appropriate research 'in my use case, 70% A and 30% B is my best metric', but you'd always know that there is danger that in some slightly different workload 50;50 (or something else) would be closer. whatever you do about this, you may not choose the absolute best - there may not be an absolute best, considering different workloads, etc, etc - but you won't be horribly disadvantaged by taking a slightly non-optimal system. Quote:
using a computer in interactive mode, I want it to get back to me and let me get on with the next thing as soon as possible; if I've paid for the damn cpu and it uses it, I'm not going to complain about that, doing work is what I bought it for! If I was really, really, pushed, I could maybe get concerned about Watt/hours to complete some task, but I can only really see that as relevant in some embedded task (eg, some media computer) where temperatures might be a concern and it might be operating on battery, otherwise their are better things to worry about I'd argue that this applies to all GUI and CLI interactive stuff, but if you have a computer doing something extended in batch mode (folding at home, or calculating pi to more digits that seems ever to be useful are the only apps that come to mind, but there are probably others) you might want to take a different attitude. Quote:
Quote:
If you are unsure, or some volumes will use noatime and some won't, I still don't expect it to alter the ranking of the results significantly (but I could be wrong, because I haven't done any testing). BTW, when I did some testing on this, pushing on for 10 years ago, noatime was the biggest single time saver*, so I don't know why people would avoid it (or relatime, maybe). My experience is that, these days, distros often set 'noatime' as the default, which wasn't the case, back then. this is a bit of a disappointment personally, as back then, I had some 'secret sauce' to make a computer go faster, and now I don't, or at least, not to that extent. Quote:
You haven't mentioned journalling modes, and those can cause significant performance differences. I think there is a general rule 'the more advanced the filesystem, the more knobs you've got to change exactly how they work'. This creates a dilemma as to whether it is fair to, eg, use BTRFS's more advanced options (compression, space cache) in comparison with older filesystems that don't have such options. Until recently, Phoronix has alway dealt with this by just taking 'the default' (and 'the default' can change between different distros and over time, so just saying 'we used the default' doesn't tell you as much as you'd like, and you don't really know if the default is different between, eg, Ubuntu desktop and Ubuntu server), but in their more recent tests, they, at least, have started telling you what options they have taken, which is a step forward. Next time out, I'll dig out some links, if they are not all broken by now. * Back then, the second biggest time saver was reiserfs. There are very limited circumstances in which I would even consider using reiser3 today (something that accesses squillions of small files, and for which the individual files are not of the highest value, like the cache files on a squid caching box, maybe), but I might consider reiser4, after appropriate research. But, probably, I'd just go for ext4 or BTRFS, eventually, if I didn't want to do that much work. |
Quote:
Reading scripts is something I dont find easy, that (ande speeling) is the only time where my minor dyslexia really shows up. Quote:
If its a setting you always use, sure, set noatime. But that makes your benchmarking less generalised, to the point where it is of no use to anyone who isnt using noatime (IMO anyway, and I would have to get of my butt and do benchamrking with noatime set and compare it to 'standard' settings to be able to say that for sure). Torvalds has had a rant about noatime...like most things. ;) http://kerneltrap.org/node/14148 Quote:
I'm far more wary of 'gaming' benchmarking myself. Sometimes its worth it, sometimes the analysis is out and out illogical. Far more worring is that there are a metric butt-load of sites with 'gaming benchmarking' that has been cherry picked to make system 'A' look better than system 'B'. *edit- I'm not even that convinced by the whole 'time' issue. Heres my reasoning (and I've never done any benchmarking on this exact situation) Filesystem 'A' takes 30 seconds to write file 'A' and finalise. CPU and I/O use is high for 5 seconds as it writes the file, then drops off for the next 24 seconds, then there is one spike in CPU and I/O use as it finalises the writing. Filesystem 'B' takes 20 seconds to write file 'A' and finalise. CPU and I/O use is high for the whole 25 seconds. Which one is better? If your only concerned about CPU use, 'A' is better. If your concerned about disc I/O, 'A' wins again. But in total time 'B' is better. |
Quote:
But the bottom line is that it is clear. Back in the day, I used to claim that I could tell by just the responsiveness of the system whether atime was on or off, without even knowing the processor speed, looking at the fstab setting, etc. It is probably harder, these days, as there probably a bigger spread of levels of hardware performance, but it is that clear, it does make a noticeable difference. There is one factor that hasn't been mentioned so far; the number of cores. If you have several cores, who cares that you saturate one with the disk stuff, you've got enough others? So, the best solution for a monocore machine may well not be the best solution for a many-core box. Quote:
In many ways, its is good to know that the world still turns, as usual, and Linus can be relied upon to speak his mind. Quote:
There are indeed a lot of 'fanboi' sites out there in the gaming world, and you will note that I didn't suggest that they should be followed slavishly, just that they did better, in this one aspect. And even the non-fanboi gaming sites get deluged with hate mail from fanbois of one team or the other wanting to scream about the fact that their preferred system didn't get praised to the roof in the latest testing, claiming that only a fanboi of the opposing team could have done this. Ho, hum. |
Quote:
Quote:
Quote:
I dont think that its really a fair comapison with phoronix, there are only so many variables with windows. You've got more variables with linux distros. Most of the time when I've looked at phononix I've seen all info needed. Yeah, they tend to just use 'defualt' values for filesystems with whatever distro they are using for the test, but IMO thats no great issue. Most people just use the defaults. |
Thanks for the info and links.
The first thing I tried for this benchmark was phoronix test suite, but it was hard to use, and the disk benchmarks didn't seem close enough to real life to be accurate. I don't use SQlite, for example. I don't how the other benchmarks work, so I decided to just do some real life tests. I based them on: http://linuxgazette.net/122/piszcz.html even tho the author never released the commands used or more details. As for noatime, I would have to agree with Linus T and Ingo Molnar, that it is definitely noticeable, and probably the best single option that will speed up your disk access. |
first, i did promise some links; these are from Linux Magazine; the guy who wrote these seems to have had a particular thing about metadata performance and also about doing benchmarking correctly, so these are worth reading, if just for the ride. OTOH, I think you have to register to see the actual articles, if you haven't already. (If you are serious about this, these articles are worth reading, whether you are going to end up agreeing with them, or not - I'm not normally much of a fan of unnecessary registration, but it is not that painful.)
The whole tone of the article is less 'pop' than some of the Phoronix ones, but that doesn't necessarily make it better or worse. Lies, Damn Lies and File System Benchmarks Harping on Metadata Performance: New Benchmarks Metadata Performance of Four Linux File Systems Metadata Performance Exploration Part 2: XFS, JFS, ReiserFS, ext2, and Reiser4 Quote:
Well, a lot of the recent tests are done on a laptop, which defines things to an extent, but do you know what memory timings they run, whther that's the standard hard disk (and whether that laptop manufacturer multi-sourced its hard disks, so whether there is only one option in that size) and they could have even overclocked it. They probably haven't, but a list wouldn't be a bad thing, would it? |
Thanks again, those are very good articles. I did have to register tho, even tho I don't like to.
EDIT: btrfs still looks like the winner even from these. |
Quote:
|
You pass in the name of the filesystem, that way you can tell what filesystem the test was performed on.
|
Quote:
|
All times are GMT -5. The time now is 01:52 PM. |