LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 07-03-2020, 05:14 PM   #1
trist007
Senior Member
 
Registered: May 2008
Distribution: Slackware
Posts: 1,044

Rep: Reputation: 70
What is the best way to copy data from one FILE stream to another...


This is in C

I'm talking like a 300GB zip file, I was just using

Code:
FILE *fp1, *fp2;
char c;

        fp1 = fopen("/opt/test.zip", "rb");
        fp2 = fopen("/home/user/dest.zip", "w+b");


        c = fgetc(fp1);
        while (c != EOF)
        {
                fputc(c, fp2);
                c = fgetc(fp1);
        }

fclose(fp1);
fclose(fp2);

Last edited by trist007; 07-03-2020 at 05:17 PM.
 
Old 07-03-2020, 06:20 PM   #2
The Squash
LQ Newbie
 
Registered: Apr 2020
Distribution: Gentoo GNU/Linux x86 (32 bit)
Posts: 24

Rep: Reputation: Disabled
I would not use the standard I/O library for that. I would use POSIX system calls like read() and write() for that.

I would copy the file in multi-megabyte chunks (if you have the RAM for that).

I might even go so far as to use mmap() on the files; see
Code:
man 2 mmap
Tell me if you need some sample code for the above recommendations.
 
Old 07-03-2020, 06:53 PM   #3
trist007
Senior Member
 
Registered: May 2008
Distribution: Slackware
Posts: 1,044

Original Poster
Rep: Reputation: 70
No that's perfect just needed the function, thank you!
 
Old 07-03-2020, 06:57 PM   #4
trist007
Senior Member
 
Registered: May 2008
Distribution: Slackware
Posts: 1,044

Original Poster
Rep: Reputation: 70
Actually yea some sample code would be nice.
 
Old 07-04-2020, 12:08 AM   #5
NevemTeve
Senior Member
 
Registered: Oct 2011
Location: Budapest
Distribution: Debian/GNU/Linux, AIX
Posts: 4,067

Rep: Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472
In Linux, you can try sendfile(2).
 
Old 07-04-2020, 09:01 AM   #6
SoftSprocket
Member
 
Registered: Nov 2014
Posts: 315

Rep: Reputation: Disabled
Neither mmap or sendfile use streams - they use file descriptors. If your question was meant to be specific to streams then you can use fread and fwrite with a larger buffer size for efficiency.

If you weren't wanting only streams (FILE*), and are on Linux and not worried about portability, then sendfile is very efficient (and easy to use).
 
Old 07-04-2020, 09:07 AM   #7
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 14,872

Rep: Reputation: 4887Reputation: 4887Reputation: 4887Reputation: 4887Reputation: 4887Reputation: 4887Reputation: 4887Reputation: 4887Reputation: 4887Reputation: 4887Reputation: 4887
Quote:
Originally Posted by trist007 View Post
Actually yea some sample code would be nice.
sample code for what?
for example here you can find some ideas.
 
Old 07-04-2020, 11:57 AM   #8
trist007
Senior Member
 
Registered: May 2008
Distribution: Slackware
Posts: 1,044

Original Poster
Rep: Reputation: 70
Yea fread and fwrite was what I figured when using FILE streams. Thank you!
 
Old 07-04-2020, 12:22 PM   #9
NevemTeve
Senior Member
 
Registered: Oct 2011
Location: Budapest
Distribution: Debian/GNU/Linux, AIX
Posts: 4,067

Rep: Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472Reputation: 1472
Note: there is also fileno(3) is FILE/fopen is mandatory.
 
Old 07-04-2020, 04:02 PM   #10
dugan
LQ Guru
 
Registered: Nov 2003
Location: Canada
Distribution: distro hopper
Posts: 9,685

Rep: Reputation: 4324Reputation: 4324Reputation: 4324Reputation: 4324Reputation: 4324Reputation: 4324Reputation: 4324Reputation: 4324Reputation: 4324Reputation: 4324Reputation: 4324
Quote:
Originally Posted by The Squash View Post
I would not use the standard I/O library for that. I would use POSIX system calls like read() and write() for that.

I would copy the file in multi-megabyte chunks (if you have the RAM for that).

Tell me if you need some sample code for the above recommendations.
Looks like the people who wrote coreutils agree with you.

https://github.com/coreutils/coreuti...ter/src/copy.c
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Best way to manually copy all network settings from one computer to another? grumpyskeptic Linux - Networking 11 04-22-2020 01:18 PM
Monitoring data passing on USB bus & copying data (dumping) to hard drive (forking an identical data stream) cilbuper Programming 4 12-11-2017 01:58 PM
best way to move data from one disk to another unix1adm Red Hat 2 01-30-2014 09:52 AM
How to copy one line from one file to another file on line number.. gajananh999 Linux - Newbie 1 08-13-2012 05:27 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 06:02 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration