[SOLVED] Concatinate files by just manipulating cluster chains
Linux - SoftwareThis forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Concatinate files by just manipulating cluster chains
Hi,
Is there any way for concatenating (particularly large video files) files without having to copying one after the other. But, manipulating the file entries in the directory and joining the second file's cluster chain to the first file's chain.
This facility allows us to fetch a large (movie) file from any offset and write into a chunk file and when all chunks are complete we can quickly join them together into one file.Or even when two consecutive chunks are fully fetched we can join them into one chunk.
Isn't this pretty much how torrents work? They pre-allocate filespace on your filesystem, and then as each chunk is downloaded, it is inserted into the empty file at the correct offset until the file is complete.
So, maybe take a look at the sourcecode of any torrent up/downloading program, and you'll see how it's done.
You probably won't want to do directory-entry manipulations directly to the disk, because different disks may be using different filesystems, which will have very different methods for handling directory structures and it'll all get very complicated very quickly.
The idea was not to pre-allocate space as movie files are around 6-8 GB size. Moreover user may or may not completely download and watch it.He may just see portions of it (streaming VOD contents & and ability to play from any chapter like in DVD, but without fetching complete file). Therefore, we would like to create chunk files during random access downloads and join them later.
Linux supports many types of filesystems. Even if you could do this at the filesystem level for one, you won't be able for another.
Unix systems are also inode based. That means you might have deleted files which still occupy disk space as long as a process has it open. This is something to keep in mind in writing your app. Look at `lsof +L1' for examples.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.