Linux - GeneralThis Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
So, here's the deal. We're running a linux machine to run a specific arcade game. When it comes up, the software takes FOREVER to load 1000+ small files. However, it always loads these files in the same order. So....
I'd like to know if there's a tool or a filesystem which allows me to manually reorder the files on the hard drive, so I can put them in the order they're loaded.
I can think of other situations where this would be useful too, primarily for servers, and maybe for the files loaded at boot on a desktop system. I would think there's a way to do it.
Any of you gurus have a suggestion?
Oh, as far as our arcade game goes, we could run pretty much any distro, and pretty much any filesystem, so if one of these tools is distro-or-filesystem specific, no big deal.
Changing the order of files on the hard drive won't change anything, since hard drives are random-access media (as opposed to tape drives which are sequential-access). Stated another way: it doesn't matter where they are on the disk; they will still load in the same order.
The order in which the files are loaded is determined by the game code. If you have the source code, you may be able to edit that to change the order in which files are loaded, then re-compile the game.
However, be advised that some of those files may require that others be loaded beforehand. If not loaded, you will end up throwing runtime errors instead of loading and running the game. Think in terms of dependencies: some of the files depend on others being loaded first. If not loaded, error reporting says "oops, this is missing, I can't do my job, I quit".
Last edited by bigrigdriver; 10-16-2007 at 06:33 AM.
it doesn't matter where they are on the disk; they will still load in the same order.
Nope.
That's why we have a VFS layer, and (at 2.6 anyway) a selection of I/O elevators. Requests are delayed and sorted so mechanical movement is reduced. Then there is read-ahead, the page cache, and the cache on the device itself. A whole predictive science trying to get what you are likely to ask for next into storage before you ask for it.
Putting it in order certainly makes some sense if you can't drive the (direct) I/O yourself. Personally I'd format a new partition and copy them in the order I wanted - probably shouldn't matter what filesystem.
Then play with the I/O schedulers - maybe deadline might be best; even noop if you get things in the (sector) order you want.
Some filesystems perform better than others with the task of reading a lot of small files. IIRC, ReiserFS beats ext2/3 hands down on this one.
If the speed is too slow even using a different filesystem, or you cannot / do not want to change the filesystem you are using, I think it would be easier to put all these small files into a single larger one (or a few larger ones).
There are some options with some filesystem types to control the underlying structure, or at least request some attributes for the data. For example the Concurrent Multi-File Data Streams features of XFS. I am not aware of a mechanism to do exactly what you want though.
Then you copy per script the files during some idle time after boot to the RAM-(disc) and set a link from the real data directory to the files in RAM. After startup of the game don't forget to delete the files(, the RAM-disc) and the link.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.