LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 09-17-2009, 03:45 PM   #1
rjo98
Senior Member
 
Registered: Jun 2009
Location: US
Distribution: RHEL, CentOS
Posts: 1,685

Rep: Reputation: 48
mv in a .sh script


There is a .sh script that scp's all files in a folder to another server, then the next line is a mv for * to another folder, which actually doesn't exist anymore.

1) will that mv to a non-existing folder just delete the file?
2) assuming #1 is true, if a file won't delete until the person who put it there deletes it manually, would there be a log file somewhere that tells me how/why he's dropping the file in there with the wrong permissions, or what the problem really is when this script runs?
 
Old 09-17-2009, 03:46 PM   #2
MS3FGX
LQ Guru
 
Registered: Jan 2004
Location: NJ, USA
Distribution: Slackware, Debian
Posts: 5,852

Rep: Reputation: 356Reputation: 356Reputation: 356Reputation: 356
If the target doesn't exist, the files won't go anywhere. It will show an error saying the target destination isn't valid.
 
Old 09-17-2009, 03:48 PM   #3
rjo98
Senior Member
 
Registered: Jun 2009
Location: US
Distribution: RHEL, CentOS
Posts: 1,685

Original Poster
Rep: Reputation: 48
where would I see that error though, since the .sh is ran by crontab every few minutes?

something is deleting the files, so it must be another cron job somewhere else if that's the case.
 
Old 09-17-2009, 04:30 PM   #4
lutusp
Member
 
Registered: Sep 2009
Distribution: Fedora
Posts: 835

Rep: Reputation: 102Reputation: 102
Quote:
Originally Posted by rjo98 View Post
where would I see that error though, since the .sh is ran by crontab every few minutes?

something is deleting the files, so it must be another cron job somewhere else if that's the case.
Typically, if a shell script run by cron emits any text on stdout or stderr, the text will be either logged onto the system log or e-mailed to the script owner. People normally deal with this by suppressing text output from scripts run by cron. Example:

Code:
0 * * * * root /path/script > /dev/null 2>&1
This suppression could of course be internal to the script as well.

So the answer to your question is that you won't necessarily see any log entries from a cron script, even if it encounters an error.

One danger is that a script writer has done something like this:

Code:
cp * /dest
rm -f *
This is very bad practice, because it might remove files that haven't been successfully copied, which "mv" won't do. I mention this because according to your account, the files are disappearing without being copied, and this is one of the few ways this can happen.

On a modern Linux sysem, cron scripts can be located in /etc/cron.d, /etc/cron.hourly, /etc/cron.daily, /etc/cron.weekly, and /etc/cron.monthly. Must likely the offending script is located either in /etc/cron.d or /etc/cron.hourly.

But it is possible that a daemon process is erasing the files -- it doesn't have to be a cron job. Example:

Code:
whle true
do
   sleep 60
   cp /path/* /dest
   rm -f /path/*
done
 
Old 09-17-2009, 04:34 PM   #5
rjo98
Senior Member
 
Registered: Jun 2009
Location: US
Distribution: RHEL, CentOS
Posts: 1,685

Original Poster
Rep: Reputation: 48
I looked in my crontab and the script or the mv command in the script aren't piped to nothing.

also, maybe i misspoke earlier, the files are getting deleted by the mv (I'm assuming), but its just when a certain user puts files in the folder that the files don't get deleted, so they keep getting scp'd over and over, creating duplicate copies.
 
Old 09-17-2009, 05:16 PM   #6
MS3FGX
LQ Guru
 
Registered: Jan 2004
Location: NJ, USA
Distribution: Slackware, Debian
Posts: 5,852

Rep: Reputation: 356Reputation: 356Reputation: 356Reputation: 356
It would make things a lot easier if you posted the script.
 
Old 09-17-2009, 05:23 PM   #7
lutusp
Member
 
Registered: Sep 2009
Distribution: Fedora
Posts: 835

Rep: Reputation: 102Reputation: 102
Quote:
Originally Posted by rjo98 View Post
I looked in my crontab and the script or the mv command in the script aren't piped to nothing.

also, maybe i misspoke earlier, the files are getting deleted by the mv (I'm assuming), but its just when a certain user puts files in the folder that the files don't get deleted, so they keep getting scp'd over and over, creating duplicate copies.
You mean "mv" successfully copies the files but then cannot delete them? This problem would be solved if the cron job were run with root authority. Modern cron jobs accept a user name along with the time schedule. If more than one user is using the target directory, the obvious solution is to run the cron job as root. It's also somewhat dangerous.

A side effect of the situation you describe is that if run under user authority, "mv" can copy files it is allowed to read, but the destination copies may have different ownership. This doesn't happen when the copy is performed by root.

One more thing. This business of moving and deleting files on a cron schedule is very bad practice. A user might be editing a file on either the source or destination and the file might simply disappear, or have its contents replaced instantly and without warning.

Very bad system policy.
 
Old 09-18-2009, 08:22 AM   #8
rjo98
Senior Member
 
Registered: Jun 2009
Location: US
Distribution: RHEL, CentOS
Posts: 1,685

Original Poster
Rep: Reputation: 48
Quote:
Originally Posted by MS3FGX View Post
It would make things a lot easier if you posted the script.
scp -r /data1/send2joe/* user@10.11.12.13:/joe/
mv /data1/send2joe/* /data1/sent2joe/


the sent2joe folder reference in the mv command does not exist, so i'm assuming this is what's deleting the file.
 
Old 09-18-2009, 08:25 AM   #9
rjo98
Senior Member
 
Registered: Jun 2009
Location: US
Distribution: RHEL, CentOS
Posts: 1,685

Original Poster
Rep: Reputation: 48
Quote:
Originally Posted by lutusp View Post
You mean "mv" successfully copies the files but then cannot delete them? This problem would be solved if the cron job were run with root authority. Modern cron jobs accept a user name along with the time schedule. If more than one user is using the target directory, the obvious solution is to run the cron job as root. It's also somewhat dangerous.

A side effect of the situation you describe is that if run under user authority, "mv" can copy files it is allowed to read, but the destination copies may have different ownership. This doesn't happen when the copy is performed by root.

One more thing. This business of moving and deleting files on a cron schedule is very bad practice. A user might be editing a file on either the source or destination and the file might simply disappear, or have its contents replaced instantly and without warning.

Very bad system policy.
actually the scp successfully copies the file, then i'm assuming the mv is what's deleting them. see my last post with the 2 commands the script runs.

the files end up in the joe folder on the other server, and majority of the time get deleted from the send2joe folder. I'm going to have the user who has problems test it out today so i can look at the file when he puts it in there. maybe he's putting a read only file in there so the mv command can't delete it (if the mv command above is even what's doing the deleting?)
 
Old 09-18-2009, 08:50 AM   #10
forrestt
Senior Member
 
Registered: Mar 2004
Location: Cary, NC, USA
Distribution: Fedora, Kubuntu, RedHat, CentOS, SuSe
Posts: 1,288

Rep: Reputation: 99
Do you really need to keep track of what has/hasn't been sent? If you are only doing that part to limit what you are sending/making the script easier to write, you may want to consider rsync instead:

Code:
rsync -ave ssh /data1/send2joe user@10.11.12.13:/joe
This will keep all the files in the "send2joe" folder however, rsync will keep track of what needs to be transferred (provided Joe leaves the files you've already sent in his /joe folder.

Just a thought,

Forrest
 
Old 09-18-2009, 08:52 AM   #11
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978
Just a note regarding the job checking/debugging: you can edit the cron job putting an echo in front of the mv command and redirecting the output to a file. In this way you can find out what actually the shell tries to run. Better redirect both the standard output and the standard error of the job, as suggested in a previous post:
Code:
30 9 * * * /path/to/script.sh > $HOME/cron.log 2>&1

Last edited by colucix; 09-18-2009 at 08:53 AM.
 
Old 09-18-2009, 08:55 AM   #12
forrestt
Senior Member
 
Registered: Mar 2004
Location: Cary, NC, USA
Distribution: Fedora, Kubuntu, RedHat, CentOS, SuSe
Posts: 1,288

Rep: Reputation: 99
colucix, I thought the same thing at first, but on closer examination, I saw it was send2joe and sent2joe. Good call on the log though.

Forrest
 
Old 09-18-2009, 08:58 AM   #13
rjo98
Senior Member
 
Registered: Jun 2009
Location: US
Distribution: RHEL, CentOS
Posts: 1,685

Original Poster
Rep: Reputation: 48
The joe folder actually is a watch folder, so once the file gets there it gets imported into another system them deleted. So truly all I need to do is 1) get the file to the joe folder on the other server then 2) delete it from the send2joe folder so it doesn't keep getting put in the joe folder and reimported over and over into the other system.
 
Old 09-18-2009, 08:59 AM   #14
rjo98
Senior Member
 
Registered: Jun 2009
Location: US
Distribution: RHEL, CentOS
Posts: 1,685

Original Poster
Rep: Reputation: 48
yeah, its kinda confusing with the names being so close
 
Old 09-18-2009, 09:12 AM   #15
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978
Quote:
Originally Posted by forrestt View Post
colucix, I thought the same thing at first, but on closer examination, I saw it was send2joe and sent2joe. Good call on the log though.

Forrest
I saw that immediately after submitting my replay and I edited my post accordingly. You've been too fast!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Need help with script to organise files into folders as part of DVD backup script jasybee2000 Linux - Newbie 5 06-15-2009 07:29 PM
How to get full path to script file inside script itself? And in case of sym links? maggus Linux - Newbie 3 05-28-2009 08:40 AM
I want to run script on the server from client machine in windows in a perl script vpradeep Linux - Newbie 2 09-01-2008 03:29 AM
ssh - using variables in call to start remote script from local script babag Linux - Networking 2 06-03-2008 04:50 PM
send automatic input to a script called by another script in bash programming jorgecab Programming 2 04-01-2004 12:20 AM


All times are GMT -5. The time now is 06:08 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration