What is the best way to copy data from one FILE stream to another...

trist007 · 07-03-2020, 05:14 PM

This is in C

I'm talking like a 300GB zip file, I was just using

Code:

FILE *fp1, *fp2;
char c;

        fp1 = fopen("/opt/test.zip", "rb");
        fp2 = fopen("/home/user/dest.zip", "w+b");


        c = fgetc(fp1);
        while (c != EOF)
        {
                fputc(c, fp2);
                c = fgetc(fp1);
        }

fclose(fp1);
fclose(fp2);

The Squash · 07-03-2020, 06:20 PM

I would not use the standard I/O library for that. I would use POSIX system calls like read() and write() for that.

I would copy the file in multi-megabyte chunks (if you have the RAM for that).

I might even go so far as to use mmap() on the files; see

Code:

man 2 mmap

Tell me if you need some sample code for the above recommendations.

trist007 · 07-03-2020, 06:53 PM

No that's perfect just needed the function, thank you!

trist007 · 07-03-2020, 06:57 PM

Actually yea some sample code would be nice.

NevemTeve · 07-04-2020, 12:08 AM

In Linux, you can try sendfile(2).

SoftSprocket · 07-04-2020, 09:01 AM

Neither mmap or sendfile use streams - they use file descriptors. If your question was meant to be specific to streams then you can use fread and fwrite with a larger buffer size for efficiency.

If you weren't wanting only streams (FILE*), and are on Linux and not worried about portability, then sendfile is very efficient (and easy to use).

pan64 · 07-04-2020, 09:07 AM

Quote:

Originally Posted by trist007

Actually yea some sample code would be nice.

sample code for what?
for example here you can find some ideas.

trist007 · 07-04-2020, 11:57 AM

Yea fread and fwrite was what I figured when using FILE streams. Thank you!

NevemTeve · 07-04-2020, 12:22 PM

Note: there is also fileno(3) is FILE/fopen is mandatory.

dugan · 07-04-2020, 04:02 PM

Quote:

Originally Posted by The Squash

I would not use the standard I/O library for that. I would use POSIX system calls like read() and write() for that.

I would copy the file in multi-megabyte chunks (if you have the RAM for that).

Tell me if you need some sample code for the above recommendations.

Looks like the people who wrote coreutils agree with you.

https://github.com/coreutils/coreuti...ter/src/copy.c