Linux - SoftwareThis forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
The only way to insert data into the beginning or middle of a file is to copy or rewrite the file. Copy-replace is trivial, and almost always the best choice:
Code:
cat header file > newfile && mv -f newfile file
One could write a "prepend" utility to rewrite the file in place, at least in C99 or GNU C, using low-level I/O routines (from unistd.h). The trick is to use an output buffer large enough to fit both the input buffer and the prepended data, and to always refill the input buffer to the brim before flushing the output buffer, so you will never overwrite yet-unread data in the file. It would not be faster than using cat, but it would not need/use the extra disk space, not even temporarily. The problem is that if you happen to have an I/O error midway through, the file will be mangled. Using cat, you still have the original file.
Location: Northeastern Michigan, where Carhartt is a Designer Label
Distribution: Slackware 32- & 64-bit Stable
Posts: 3,541
Rep:
The only way I can think of that you can do that is if your file is marked in some way that allows you to search for a beginning mark and copy to an ending mark. That means you'd have to section that file using unique patterns that can be found by whatever utility you want or roll your own C, C++, whatever, whatever program to do the extraction. Even if you knew the beginning and ending addresses in the file, you could extract that way (but you'd have to write a program to do it).
Another way might be a DBMS, such as MySQL, PostgreSQL, etc. Put your header stuff in a table with an index so you can extract it on-the-fly using a where ind_val = some_pattern.
This is only one approach, perhaps you can think of others.
In some cases it may be acceptable to generate the concatenated stream and pass it directly to the receiving program without storing the data on disk again. For example, to pass a stream with two concatenated files to a program called "zorb":
Code:
zorb <(cat headerFile reallyBigFile)
Note that this is not replacing the command's standard input. The argument list will contain the name of a pseudo-file in /proc, which the program will need to open by name to see the content, as it would with an ordinary file.
Disadvantages include
Since input is coming from a pipe, the program won't be able to do seeks or see the metadata for the actual input files.
If the program wants to complain about something in the file, the reported file name will be something useless like, "/proc/self/fd/63".
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.