LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 05-22-2016, 10:27 PM   #1
KenJackson
Member
 
Registered: Jul 2006
Location: Maryland, USA
Distribution: Fedora and others
Posts: 756

Rep: Reputation: 144Reputation: 144
Shell script: How do I do things simultaneously and rejoin at the end?


I need to do something like this:
Code:
#!/bin/sh
cp bigfile /mnt/diska
cp bigfile /mnt/diskb
cp bigfile /mnt/diskc
rm -f bigfile
Each copy onto a slow thumb drive takes several minutes but my PC's fast hard drive is quite capable of supplying the data for all three at once, so how can I do them simultaneously?

If I add an ampersand, &, at the end of each cp command, bigfile will get deleted before the copies are done. So how can I make the rm wait for the copies to complete?
 
Old 05-22-2016, 10:58 PM   #2
dunne
Member
 
Registered: May 2014
Distribution: OpenBSD
Posts: 67

Rep: Reputation: 36
Something like this should do the trick:
Code:
#!/bin/sh
trap 'rm -f done1 done2 done3; exit 0' 0 1 2 15
(cp bigfile /mnt/diska && touch done1) &
(cp bigfile /mnt/diskb && touch done2) &
(cp bigfile /mnt/diskc && touch done3) &
while [ ! -e done1 -o ! -e done2 -o ! -e done3 ]
do
    sleep 1
done
rm -f bigfile
"[" and "]" in the above are syntactic sugar for test(1).
 
2 members found this post helpful.
Old 05-22-2016, 11:41 PM   #3
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 4,237

Rep: Reputation: 1656Reputation: 1656Reputation: 1656Reputation: 1656Reputation: 1656Reputation: 1656Reputation: 1656Reputation: 1656Reputation: 1656Reputation: 1656Reputation: 1656
dunne's subshell method is perfect for this. I would like to point out a couple of concerns with OP's whole 'cp' idea though.

- cp doesn't do checksumming to ensure the copy was done correctly
- removing the file before checking that the copy was good, is a bad idea

Solution: (This is assuming that these files are worth the effort) use a combination of rsync and md5sum. rsync can continue where it left off if there is a copy error, and md5sum can compare the original to the copy to ensure that they are the same. Tie that into the delete file and you have a bit more of a safety net.
 
Old 05-23-2016, 04:22 AM   #4
cnamejj
Member
 
Registered: Mar 2015
Distribution: Ubuntu
Posts: 37

Rep: Reputation: Disabled
Minor tweak to the previously posted script that simplifies things.

Code:
#!/bin/bash
cp bigfile /mnt/diska &
cp bigfile /mnt/diskb &
cp bigfile /mnt/diskc &
wait
rm -f bigfile
 
4 members found this post helpful.
Old 05-23-2016, 04:29 AM   #5
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 15,608

Rep: Reputation: 5120Reputation: 5120Reputation: 5120Reputation: 5120Reputation: 5120Reputation: 5120Reputation: 5120Reputation: 5120Reputation: 5120Reputation: 5120Reputation: 5120
tee bigfile /mnt/diska/bigfile /mnt/diskb/bigfile > /mnt/diskc/bigfile
rm -f bigfile

and obviously check it before destroying anything...

Quote:
Originally Posted by KenJackson View Post
If I add an ampersand, &, at the end of each cp command, bigfile will get deleted before the copies are done. So how can I make the rm wait for the copies to complete?
as far as I know the directory entry will be deleted only but the copy processes will still keep that file alive (as long as it is in use). Therefore the 3 copy commands still will work.

and obviously check it before destroying anything...

Last edited by pan64; 05-23-2016 at 04:33 AM.
 
Old 05-23-2016, 07:15 AM   #6
KenJackson
Member
 
Registered: Jul 2006
Location: Maryland, USA
Distribution: Fedora and others
Posts: 756

Original Poster
Rep: Reputation: 144Reputation: 144
Quote:
Originally Posted by cnamejj View Post
wait
Wait!! I've seen that but could never think of a use for it. It fits perfectly here. That's actually the kind of thing I was thinking must exist.

Quote:
Originally Posted by dunne View Post
while [ ! -e done1 -o ! -e done2 -o ! -e done3 ]
I started to think of something along this line but decided there must be a better way.

Quote:
Originally Posted by pan64 View Post
tee bigfile /mnt/diska/bigfile /mnt/diskb/bigfile > /mnt/diskc/bigfile
I used to use tee for something like this but it seemed to take forever. I was left with the impression it was doing the copies sequentially, not simultaneously.

Quote:
Originally Posted by pan64 View Post
as far as I know the directory entry will be deleted only but the copy processes will still keep that file alive (as long as it is in use). Therefore the 3 copy commands still will work.
I didn't think of that, but I think you're right. But I wouldn't want to depend on it. And I may indeed add an md5sum check, even though that means taking much longer.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Unexpected end of file error when running a shell script Laughing_Man Linux - Newbie 8 09-14-2009 06:43 PM
Shell script-How to call tail then exit when 'END timestamp' appears in log? Mountain Linux - Software 4 05-01-2009 03:29 PM
how to Design Front end Using Shell Script ?? shivarajM Linux - Software 3 04-27-2009 10:29 AM
Sntax error : unexpected end of file while executing shell script priyankarathi Linux - Newbie 5 04-21-2009 02:33 AM
How to design a fron end to run different shell script? sharp859 Linux - Newbie 8 05-05-2008 08:23 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 09:14 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration