LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 07-20-2015, 12:37 AM   #1
aristosv
Member
 
Registered: Dec 2014
Posts: 261

Rep: Reputation: 3
cron and lftp issue


I have the following scripts running on raspbian.

start
Code:
#!/bin/bash
/root/config/commands 2>&1 > /root/log/$HOSTNAME.log
commands
Code:
#!/bin/bash
source /root/config/variables
echo TestLine
/usr/bin/lftp ftp://user:pass@host.com -e "set ftp:ssl-allow no ; set net:reconnect-interval-base 5 ; set net:max-retries 2 ; mirror -e /testgroup/music /root/media ; quit"
/usr/bin/mail -s "$HOSTNAME Report" $emailaddress < $logdir/$HOSTNAME.log
When I run "start" manually I receive the email below

Code:
TestLine
Total: 1 directory, 4 files, 0 symlinks
When I run it using cron, I get the following email

Code:
TestLine
Any idea why would cron strip the lftp output from the email?

Thanks.
 
Old 07-20-2015, 07:09 AM   #2
wpeckham
LQ Guru
 
Registered: Apr 2010
Location: Continental USA
Distribution: Debian, Ubuntu, RedHat, DSL, Puppy, CentOS, Knoppix, Mint-DE, Sparky, VSIDO, tinycore, Q4OS,Manjaro
Posts: 5,166

Rep: Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502
by the numbers....

1. try adding 2>&1 at the end of the lftp line.
It may be sending something to stderr that is not being captured.

2. When it runs from cron, is the path the same? Try explicitly setting the path, or calling lftp with the full path. ENV differences have cropped up as a cause for cron script issues often.
 
Old 07-20-2015, 11:46 AM   #3
aristosv
Member
 
Registered: Dec 2014
Posts: 261

Original Poster
Rep: Reputation: 3
thanks for the suggestion. but it didn't work
I use the full path in cron. Like below.

crontab -e
Code:
* * * * * /root/start
 
Old 07-21-2015, 06:43 AM   #4
wpeckham
LQ Guru
 
Registered: Apr 2010
Location: Continental USA
Distribution: Debian, Ubuntu, RedHat, DSL, Puppy, CentOS, Knoppix, Mint-DE, Sparky, VSIDO, tinycore, Q4OS,Manjaro
Posts: 5,166

Rep: Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502
So when run under cron (ONLY under cron) the lftp line generates NO OUTPUT!

There is clearly something missing here. I can test a very similar case on my machine and it performs as expected, yet on yours it does not. We are missing something.

Let me suggest using >> rather than > to accumlate the log, otherwise it will be cleared and overwritten every 60 seconds.

Questions: the start script is really only one command. Why did you not put that one command directly into your crontab file?
Do you really mean to run this process once per minute, every single minute, forever? If the command takes over 60 seconds, you will stack processes with unfortunate results. One of those results may be trashing your log file (see suggestion above re: >> ).

I would use a more sophisticated script that would check only continue if no other copy was already running.

Have you considered setting up openssh and using rsync in mirror mode to serve the function without lftp?
 
Old 07-21-2015, 07:18 AM   #5
aristosv
Member
 
Registered: Dec 2014
Posts: 261

Original Poster
Rep: Reputation: 3
- There are other commands in the script also. But to replicate the issue I have modified the script to remove anything that could be causing the problem. The script now is as shown in the original post. Just one line.
- I only run the script every one minute when I am trying to resolve the issue, and I am testing the script. It will normally run @daily. The command usually takes less than 25 seconds to complete.
- I have considered other options including rsync and wget. I really like lftp though and I would prefer to use that.

But the question remains, why would it not provide any output when run using cron? What's changing there?
On what system did you test it, and got the expected results? Was it raspbian? Could it be something in the configuration of cron on my raspberry pi's?
 
Old 07-21-2015, 01:18 PM   #6
JaseP
Senior Member
 
Registered: Jun 2002
Location: Eastern PA, USA
Distribution: K/Ubuntu 18.04-14.04, Scientific Linux 6.3-6.4, Android-x86, Pretty much all distros at one point...
Posts: 1,802

Rep: Reputation: 157Reputation: 157
As wpeckham indicated, it's probably the environment.

To give you a working example, I had a script to deactivate DPMS on a machine running KDE, that I had to run on a regular basis (KDE has a bug in terms of DPMS, that when it's turned off, it gets turned back on, and with terribly short default settings)...

The original script (just the line I was trying to run) was as follows:
Code:
xset dpms 0 0 0 && xset -dpms
I had to change it to the following:
Code:
env DISPLAY=:0 && xset dpms 0 0 0 && env DISPLAY=:0 && xset -dpms
When you use cron,... the scripts are kind of run in "Never-never-land," for lack of a better term. You need to tell output where to go, otherwise it doesn't get directed to anywhere. After all, your cron scripts aren't being run from the standard output, they are jut being run
 
Old 07-21-2015, 01:36 PM   #7
aristosv
Member
 
Registered: Dec 2014
Posts: 261

Original Poster
Rep: Reputation: 3
I don't understand...how should my command change? I don't use KDE, so I'm guessing the additions to your command wont work on mine.
 
Old 07-21-2015, 01:40 PM   #8
aristosv
Member
 
Registered: Dec 2014
Posts: 261

Original Poster
Rep: Reputation: 3
well, this didn't work

Code:
env DISPLAY=:0 /root/config/commands 2>&1 > env DISPLAY=:0 /root/log/$HOSTNAME.log
 
Old 07-21-2015, 01:54 PM   #9
JaseP
Senior Member
 
Registered: Jun 2002
Location: Eastern PA, USA
Distribution: K/Ubuntu 18.04-14.04, Scientific Linux 6.3-6.4, Android-x86, Pretty much all distros at one point...
Posts: 1,802

Rep: Reputation: 157Reputation: 157
KDE has nothing to do with it (it was just why I needed to run the xset command). I wasn't suggesting that you use "env DISPLAY-:0," as that would just direct the output to the display (and besides, wouldn't you need to use the double ampersands, anyway?!). What I was trying to point out is that you have to direct the output to where you want it to go...

Are you trying to append a log file there??? In that case, shouldn't you be using the cat command to append to the log, and direct the output to that log file?!?!
 
Old 07-21-2015, 02:10 PM   #10
aristosv
Member
 
Registered: Dec 2014
Posts: 261

Original Poster
Rep: Reputation: 3
I know you're trying to tell me something, and I'm really trying to understand what, but I don't think I do.

Isn't this ">" enough to write the output of the command to a log file? Do I also need to use "cat"? And if I do, then how? I know "cat" reads a file, not appends to it.
 
Old 07-21-2015, 09:30 PM   #11
wpeckham
LQ Guru
 
Registered: Apr 2010
Location: Continental USA
Distribution: Debian, Ubuntu, RedHat, DSL, Puppy, CentOS, Knoppix, Mint-DE, Sparky, VSIDO, tinycore, Q4OS,Manjaro
Posts: 5,166

Rep: Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502
Check the $HOSTNAME variable under cron and in an interactive shell to see if they match, or if it is even SET under cron. Some systems it is, others not.

I know, reaching a bit, but if you shoot enough arrows into the air one MUST hit something that looks like a target SOMEDAY! ;-)

Last edited by wpeckham; 07-21-2015 at 09:31 PM.
 
Old 07-22-2015, 01:27 AM   #12
aristosv
Member
 
Registered: Dec 2014
Posts: 261

Original Poster
Rep: Reputation: 3
So I put this in crontab -e "* * * * * env > /tmp/env.output" and this is the output.

Code:
HOME=/root
LOGNAME=root
PATH=/usr/bin:/bin
LANG=en_GB.UTF-8
SHELL=/bin/sh
PWD=/root
I also put this in crontab -e "* * * * * /bin/echo $HOSTNAME > /tmp/hostname.txt" and the output was an empty line.

To compare, below is the root environment export

Code:
TERM=xterm
SHELL=/bin/bash
SSH_TTY=/dev/pts/0
USER=root
MAIL=/var/mail/root
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/root
LANG=en_GB.UTF-8
SHLVL=1
HOME=/root
LOGNAME=root
_=/usr/bin/env
OLDPWD=/tmp
Could the problem be caused by the difference in the environment variables? And if so, how do I add root's environment variables in cron?
 
Old 07-22-2015, 06:48 AM   #13
wpeckham
LQ Guru
 
Registered: Apr 2010
Location: Continental USA
Distribution: Debian, Ubuntu, RedHat, DSL, Puppy, CentOS, Knoppix, Mint-DE, Sparky, VSIDO, tinycore, Q4OS,Manjaro
Posts: 5,166

Rep: Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502Reputation: 2502
Either:
1. stop using environment variables that you have not set and that do not exist under cron by default, or
2. set them Explicitly In your script. If you set them in the first script, make sure to export them so that they will carry into the script it calls.

If you had a line
Code:
export HOSTNAME="wumpus.hunt.org"
in the script then the behavior would not change under cron due to the value of $HOSTNAME being missing.
 
Old 07-22-2015, 08:23 AM   #14
aristosv
Member
 
Registered: Dec 2014
Posts: 261

Original Poster
Rep: Reputation: 3
As you can see from the script, the only variables used are $HOSTNAME and $emailaddress.
I receive the email so that variable works. What you are saying is that if I define $HOSTNAME in the script, I should be OK?
 
Old 07-22-2015, 08:38 AM   #15
aristosv
Member
 
Registered: Dec 2014
Posts: 261

Original Poster
Rep: Reputation: 3
If I have to define the variable $HOSTNAME then I'm in trouble. I have 45 raspberry pi's and I will have to set the $HOSTNAME variable on each and every one of them.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
lftp cron and output to file Danik Linux - Server 6 07-04-2015 08:44 AM
lftp issue shazgaurav Linux - Newbie 28 12-21-2013 01:09 AM
cron issue rjb2013 Linux - Newbie 6 12-02-2013 05:20 AM
[SOLVED] cron issue dushyantgohil Linux - Server 9 10-04-2012 06:55 AM
cron issue abdul_zu Linux - Newbie 8 08-01-2005 12:39 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 11:35 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration