LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 10-15-2007, 11:02 PM   #1
Tyco
LQ Newbie
 
Registered: Jun 2003
Posts: 10

Rep: Reputation: 0
Filesystem which allows manual file positioning?


So, here's the deal. We're running a linux machine to run a specific arcade game. When it comes up, the software takes FOREVER to load 1000+ small files. However, it always loads these files in the same order. So....
I'd like to know if there's a tool or a filesystem which allows me to manually reorder the files on the hard drive, so I can put them in the order they're loaded.
I can think of other situations where this would be useful too, primarily for servers, and maybe for the files loaded at boot on a desktop system. I would think there's a way to do it.

Any of you gurus have a suggestion?

Oh, as far as our arcade game goes, we could run pretty much any distro, and pretty much any filesystem, so if one of these tools is distro-or-filesystem specific, no big deal.

Thanks!
-T
 
Old 10-16-2007, 06:26 AM   #2
bigrigdriver
LQ Addict
 
Registered: Jul 2002
Location: East Centra Illinois, USA
Distribution: Debian stable
Posts: 5,908

Rep: Reputation: 356Reputation: 356Reputation: 356Reputation: 356
Changing the order of files on the hard drive won't change anything, since hard drives are random-access media (as opposed to tape drives which are sequential-access). Stated another way: it doesn't matter where they are on the disk; they will still load in the same order.

The order in which the files are loaded is determined by the game code. If you have the source code, you may be able to edit that to change the order in which files are loaded, then re-compile the game.

However, be advised that some of those files may require that others be loaded beforehand. If not loaded, you will end up throwing runtime errors instead of loading and running the game. Think in terms of dependencies: some of the files depend on others being loaded first. If not loaded, error reporting says "oops, this is missing, I can't do my job, I quit".

Last edited by bigrigdriver; 10-16-2007 at 06:33 AM.
 
Old 10-16-2007, 08:00 AM   #3
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,103

Rep: Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117
Quote:
Originally Posted by bigrigdriver View Post
it doesn't matter where they are on the disk; they will still load in the same order.
Nope.
That's why we have a VFS layer, and (at 2.6 anyway) a selection of I/O elevators. Requests are delayed and sorted so mechanical movement is reduced. Then there is read-ahead, the page cache, and the cache on the device itself. A whole predictive science trying to get what you are likely to ask for next into storage before you ask for it.

Putting it in order certainly makes some sense if you can't drive the (direct) I/O yourself. Personally I'd format a new partition and copy them in the order I wanted - probably shouldn't matter what filesystem.
Then play with the I/O schedulers - maybe deadline might be best; even noop if you get things in the (sector) order you want.
 
Old 10-16-2007, 10:58 AM   #4
matthewg42
Senior Member
 
Registered: Oct 2003
Location: UK
Distribution: Kubuntu 12.10 (using awesome wm though)
Posts: 3,530

Rep: Reputation: 65
Some filesystems perform better than others with the task of reading a lot of small files. IIRC, ReiserFS beats ext2/3 hands down on this one.

If the speed is too slow even using a different filesystem, or you cannot / do not want to change the filesystem you are using, I think it would be easier to put all these small files into a single larger one (or a few larger ones).

There are some options with some filesystem types to control the underlying structure, or at least request some attributes for the data. For example the Concurrent Multi-File Data Streams features of XFS. I am not aware of a mechanism to do exactly what you want though.
 
Old 10-17-2007, 06:45 AM   #5
JZL240I-U
Senior Member
 
Registered: Apr 2003
Location: Germany
Distribution: openSuSE Tumbleweed-KDE, Mint 21, MX-21, Manjaro
Posts: 4,629

Rep: Reputation: Disabled
Just an idea...

If you have plenty of RAM you could do this: Create a RAM-disc or use tmpfs
Code:
{in /etc/fstab insert:}
none   /tmp   tmpfs size=512m,noexec,noatime,notail   0   0
Then you copy per script the files during some idle time after boot to the RAM-(disc) and set a link from the real data directory to the files in RAM. After startup of the game don't forget to delete the files(, the RAM-disc) and the link.

That might help (I hope ).

Last edited by JZL240I-U; 10-17-2007 at 06:47 AM.
 
Old 10-18-2007, 11:36 PM   #6
Tyco
LQ Newbie
 
Registered: Jun 2003
Posts: 10

Original Poster
Rep: Reputation: 0
Some good ideas, and I think that's all I'll need. Thanks!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Manual file system fixing with e2fsck -- are my files permanently lost? the theorist Linux - Software 5 04-04-2007 01:18 AM
winxp reinstall w/grub manual install on ubuntu causes missing hal.dll file for xp pulper Linux - Software 6 09-13-2006 02:26 PM
DISCUSSION: Virtual Filesystem: Building a Linux Filesystem from an Ordinary File mchirico LinuxAnswers Discussion 0 10-28-2004 10:35 PM
is there manual on networking and file sharing on fedora core 2? krt47 Linux - Networking 4 08-06-2004 03:57 PM
Xvnc Positioning neophile Linux - Games 0 10-07-2003 09:12 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 10:33 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration