LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Programming (https://www.linuxquestions.org/questions/programming-9/)
-   -   copy files from one source to multiple destination simultaneously (https://www.linuxquestions.org/questions/programming-9/copy-files-from-one-source-to-multiple-destination-simultaneously-824349/)

mdfakkeer 08-05-2010 08:31 AM

copy files from one source to multiple destination simultaneously
 
i want to copy one or more files or directory from one drive to multiple drive simultaneously. It like a cloning a disk. But i dont like clone entire disk. i want to copy/clone only certine files or folder. if any one can know how to copy one source to multiple destination simultaneously. Is any tools available? how i can implement in perl?

jschiwal 08-05-2010 08:44 AM

To send to more than one directory simultaneously, you will need to use tee, and fifos.

mkfifo destdir1.fifo
tar -C sourcebasedir -cf - dir1 dir2 dir3 | tee destdir1.fifo | tar -C destdir2 -xf - &
tar -C destdir1 -xf destdir1.fifo

mdfakkeer 08-05-2010 08:50 AM

can you explian the same with example?. I am not understand. if i want to copy dir1 to destdir1, destdir2 and destdir3. how to implement

Sergei Steshenko 08-05-2010 08:53 AM

Quote:

Originally Posted by mdfakkeer (Post 4056758)
can you explian the same with example?. ...

You need to know how 'bash' (the interactive shell you are most likely using) works, especially, what is a job in background.

So, enter into your favorite WEB search engine

bash tutorial
.

You also need to understand how 'tar' works - for that read

man tar
.

jschiwal 08-05-2010 08:58 AM

I already gave an example.

For three destination directories, you would create another fifo pipe and have to consecutive tee commands.
... | tee dir1.fifo | tee dir2.fifo | ...

And two tar commands extracting from the fifo pipes, the first one running in the background.

It would be better to use a loop and copy to one destination directory at a time. Otherwise, an error in one would cause all to fail.
Also, if the destination directories are on the same filesystem, you will cause disk thrashing. The end resullt is that it will take much longer to accomplish.

Here is something I have used in the past:
tar -C basedir -czf - . | tee /mnt/backup/backup.tar.gz | ssh user@host 'tar -C destdir -xzvf -'

It saved a tar backup to an external drive, and replicated the files to a second computer.

An example of using tar to copy directories of files is explained in Section 4.6 of the tar info manual.

crts 08-05-2010 12:37 PM

Quote:

Originally Posted by jschiwal (Post 4056748)
mkfifo destdir1
tar -C sourcebasedir -cf - dir1 dir2 dir3 | tee destdir1 | tar -C destdir2 -xf - &
tar -C destdir1 -xf -

Hi jschiwal,

I tried your solution and it did not work for me. Note, that I have not used named pipes before - haven't had a need for them.
So I did some very basic reading on this concept and experimented a bit.
Here is my directory structure:
Code:

./sourcebasedir/
./sourcebasedir/dir1/
./sourcebasedir/dir2/
./sourcebasedir/dir3/
./destdir1/
./destdir2/

When I executed your code I first got an error that destdir1 is a directory. So I renamed the pipe to 'pipe1'. And finally this slightly altered code worked:
Code:

mkfifo pipe1
tar -C sourcebasedir -cf - dir1  dir2 dir3 | tee pipe1 | tar -C destdir2 -xf - &
tar -C destdir1 -xf pipe1

Did I miss something?

jschiwal 08-05-2010 09:14 PM

I must have cut and pasted the wrong text from the terminal, because I remember naming the fifo differently when I tested it out. I just did this example:
Code:

mkfifo bin1.fifo
tar -C bin -cf - . | tee bin1.fifo | tar -C bin2 -xf - &
tar -C bin1 -xf bin1.fifo

It copied my ~/bin/ directory contents to both ~/bin1 and ~/bin2.

mdfakkeer 08-06-2010 08:05 AM

Thanks jschiwal. one source to muliple destination working fine.


All times are GMT -5. The time now is 02:38 PM.