Linux - SoftwareThis forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Introduction to Linux - A Hands on Guide
This guide was created as an overview of the Linux Operating System, geared toward new users as an exploration tour and getting started guide, with exercises at the end of each chapter.
For more advanced trainees it can be a desktop reference, and a collection of the base knowledge needed to proceed with system and network administration. This book contains many real life examples derived from the author's experience as a Linux system and network administrator, trainer and consultant. They hope these examples will help you to get a better understanding of the Linux system and that you feel encouraged to try out things on your own.
Click Here to receive this Complete Guide absolutely free.
I need to choose a file system for a specific tasks I need to perform to collect data for some research.
The data are raw images, each 0.75MiB coming in at a rate of 30Hz. They are raw images (i.e. completely uncompressed raw coding from the camera) and need to remain as such - we have tried some on-the-fly decoding but it far too intensive for the embedded system that needs to capture the images (PM-1.6GHz EBX board).
I am experimenting with several ways to capturing the data. I have a SATA card, or have external USB HDDs, and I can successfully capture said data (formatted FAT) albeit with a fairly high CPU usage (especially on USB).
I want to lower the CPU usage for capturing the data, and the loading tends to increase when the images are being written to disk.
What file system do you think is optimal for this sort of task? Low-CPU is the prime candidate, and I need little in the way of other features.
when I quickly calculate the rate at which you want it to be captured, I come at 22,5Mb/s. Which is not quiet the maximum throughput of usb 2.0 but close enough; you must have a pretty good system to keep up with that. Personally I would not do such a task on a machine with those specs, if it is reliability/performance you want. I find fire wire (IEEE1394) worked seamless for me on a P4 2.4Ghz 512Mb ram writing to a FAT32 (baring in mind the 2Gb max file size limit) storage partition. the system remained responsive and my rate was 25Hz.
when I quickly calculate the rate at which you want it to be captured, I come at 22,5Mb/s. Which is not quiet the maximum throughput of usb 2.0 but close enough;
Ah yes, I should comment on this a little.
The camera capture programs (corander) actually saves to a ramdisk. Another process actually moves the files from the ramdisk to **two** USB2 drives, alternating between them. It writes up to 900 images per directory, then creates another directory.
In an experiment, I had a SATA (1.5Gbs) cardbus card that I plugged into the system and was able to save directly to same, although it was dropping the off frame (haven't tried yet doing the same thing via the ramdisk as described above).
I can actually do that with two cameras (one at 15Hz, the other at 30Hz), with two instances of coriander, each saving to a different SATA drive as a proof of comcept.
It was formatted FAT32 so that (at the time) it could be directly read by anyone with Windows, but that's no longer the issue, so I'm too see if there is anything better for the same purpose.
These cameras can run for an hour or more at time and can capture enormous amounts of data - it was close to 200GB in our last field test.
I hear ext4 is promising in its capabilities like stability, reliability and performance. I also read it is quicker than its predecessor ext3 and it will be the default fs for the upcoming new release of Ubuntu which should be due for download soon.
and yes, capturing full uncompressed frames needs a lot of hard drive space and very consistent and high bus speeds accompanied with a fair amount of continues cpu use. imho you would not be able to do much else with that system while it does this unless you have multiple cores.