LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 05-02-2007, 10:46 AM   #1
sleepyhomme
LQ Newbie
 
Registered: Jan 2006
Posts: 28

Rep: Reputation: 15
Redirecting FTP Server responses to file


I wrote a bash shell script which will upload files in a directory one by one.

Since I have to log the upload result of EACH file, I would like to redirect the responses from FTP server to a file (like vsftpd's xferlog). In the shell script, I have my own redirection like this

echo "Some times Someone did something..." >> daily.log

Is it possible to redirect the FTP Server response to that file too??

Many thanks. If any information needed, please let me know!!!

Code:
Connected to 192.168.80.194.
220 "Welcome to MSL Fax Image FTP service."
331 Please specify the password.
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
200 Switching to Binary mode.
200 Switching to Binary mode.
local: pantos-edibkg-manual.odt remote: pantos-edibkg-manual.odt
229 Entering Extended Passive Mode (|||48837|)
553 Could not create file.
local: pantos-sr-manual.odt remote: pantos-sr-manual.odt
229 Entering Extended Passive Mode (|||22316|)
553 Could not create file.
221 Goodbye.
Above is the sample FTP responses I would like to redirect to a file.
 
Old 05-02-2007, 11:44 AM   #2
osor
HCL Maintainer
 
Registered: Jan 2006
Distribution: (H)LFS, Gentoo
Posts: 2,450

Rep: Reputation: 77
Did you try something as simple as:
Code:
$ ./my_bash_script >> daily.log
 
Old 05-02-2007, 08:02 PM   #3
sleepyhomme
LQ Newbie
 
Registered: Jan 2006
Posts: 28

Original Poster
Rep: Reputation: 15
Tried nearly all kinds of redirection i know... but still fail.
they juz output a blank file.
 
Old 05-02-2007, 08:11 PM   #4
osor
HCL Maintainer
 
Registered: Jan 2006
Distribution: (H)LFS, Gentoo
Posts: 2,450

Rep: Reputation: 77
Quote:
Originally Posted by sleepyhomme
they juz output a blank file.
Does that mean the output from your ftp client still goes on the screen?

If so, perhaps your ftp client writes such informational messages to stderr instead of stdout. What you can do is redirect stderr (file descriptor number 2) to stdout (fd 1), and then redirect stdout (which contains bout the original stdout and the original stderr) to another file. E.g.,
Code:
$ ./my_bash_script 2>&1 >> daily.log
 
Old 05-04-2007, 08:18 PM   #5
sleepyhomme
LQ Newbie
 
Registered: Jan 2006
Posts: 28

Original Poster
Rep: Reputation: 15
ya, still a blank file.
I have tried all these before.
redirecting stdout, stderr all together.
redirecting only stdout
redirecting only stderr

Use variable to hold the execution result then echo it.

all blank

here goes my script. it is like

Code:
FTP='/usr/bin/ftp'

$FTP -i ftp://usrname:pwd@host:port/ << YOYO
     binary
     put something
     bye
YOYO
 
Old 05-04-2007, 09:14 PM   #6
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 678Reputation: 678Reputation: 678Reputation: 678Reputation: 678Reputation: 678
Would this work?
Code:
FTP='/usr/bin/ftp'

$FTP -i ftp://usrname:pwd@host:port/ << YOYO | tee ftpsession
     binary
     put something
     bye
YOYO
It should record both your entries and the responces.

Last edited by jschiwal; 05-04-2007 at 09:16 PM.
 
Old 05-04-2007, 10:13 PM   #7
sleepyhomme
LQ Newbie
 
Registered: Jan 2006
Posts: 28

Original Poster
Rep: Reputation: 15
Thank ya'll for giving me tips and suggestions!!
but it still not working...another blank file is made.

Is it no way to capture it? hahaa. I start wondering...
 
Old 05-04-2007, 11:33 PM   #8
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 678Reputation: 678Reputation: 678Reputation: 678Reputation: 678Reputation: 678
I was able to do:
ftp <ipaddress> | tee ftpsession

It recorded the session. Can you try it manually once. I wonder if you get any response at all.
( This was using cygwin. Running "ftp ftp://user:password@<address>" <ftpscript
doesn't work on cygwin or dos, but I have done it on Linux to the same equipment.

I will use "telnet <address> | tee devicesession" to get an inventory listing from some devices. It works in Cygwin on XP but not in Cygwin on Win2000.
I'm not entering commands from a HERE script but I am producing a listing from the session, to commpare with a master device.
 
Old 05-05-2007, 12:01 AM   #9
HowDoIProgramIt
Member
 
Registered: Nov 2006
Location: East Coast, USA (in "the great northeast")
Distribution: Custom / from source; Fedora, Debian, CentOS, Scientific; LFS.
Posts: 94

Rep: Reputation: 15
Funny you should ask this, I was just finishing up some maintainence work on some C code that does what you're looking for.

I can put some commands in a file; say it's named "commands.txt":
open mirror.mcs.anl.gov
ftp

cd /pub/fedora-linux-core/updates/6/SRPMS
get kernel-2.6.20-1.2948.fc6
bye

and then run the program (call it "doit"):
./doit -i commands.txt -o log.txt ftp

"ftp" is the command that I want the program to run (in the background, as a separate session, with no controlling terminal); as you've probably guessed, my program exec()s ftp, with commands.txt set up as ftp's "stdin" and log.txt set up as ftp's stdout & stderr.

It's a pretty common *NIX system programming task & a great way to learn about processes, zombies, redirection, etc.; the key is to call fork(), have the first child call fork() again, have the first parent wait for the child that will complete (remember, one of them is calling exec so it never does come back) - you should be able to use "wait(-1,NULL,0);" - the second child does the exec, after:
* calling setsid()
* setting up the program-to-be-exec()'d stdin, stdout and
stderr to be either
= the files you want to use (eg., parse the command line
for -i, -o, etc.) or
= open /dev/null - for reading (stdin) and again for
writing (stdout, stderr)

The way you set up the files goes something like this:

int fdOut;
int fdIn;

fdIn = open( ... );
close(0); /* close current stdin */
dup(fdIn); /* now stdin == fdIn */

fdOut = open( ... );
close(1); /* close current stdout */
dup(fdOut); /* now stdout == fdOut */

close(2); /* close current stderr */
dup(fdOut); /* now stderr == stdout */
/* which == fdOut. */

If you *really* want to do I/O redirection, have a go at writing an app that uses fork(), exec() && dup(); it's a terrific excercise & quite useful, too... A hint: watch ps - and print out the pid's and sid's of each process && session ( getsid(), getpid() ) - you'll probably be surprised to see how the system goes about creating what it creates.

If you decide to do this & you get really stuck I'll gladly send code your way.

- Larry
 
Old 05-14-2007, 08:14 AM   #10
skohrs
LQ Newbie
 
Registered: May 2007
Posts: 1

Rep: Reputation: 1
Try including the verbose flag (-v) to the ftp command.

From the man page:

-v Enable verbose and progress. This is the default if output is
to a terminal (and in the case of progress, ftp is the fore*
ground process). Forces ftp to show all responses from the
remote server, as well as report on data transfer statistics.
 
1 members found this post helpful.
Old 05-14-2007, 11:30 AM   #11
HowDoIProgramIt
Member
 
Registered: Nov 2006
Location: East Coast, USA (in "the great northeast")
Distribution: Custom / from source; Fedora, Debian, CentOS, Scientific; LFS.
Posts: 94

Rep: Reputation: 15
The main reason you're having problems with automating this is that the FTP program (on the server) is detecting that what's sending it a password is a script; or, to be more precise, that it's not getting the password from a terminal. It's fairly common for a program to perform a check like that. Try "man isatty", you should see documentation for a function that's usually in "unistd.h".

Try this:
Code:
ftp some.anonymous.ftpserver <<HERE
ftp

ls
bye
HERE
After you type the command, hit enter; then, hit enter again. It isn't until after you hit enter the 2nd time that you get output from the "ls" command in the sequence of FTP commands above. That's because FTP (on the other server) is waiting for input from a "real user".

The only way I've ever "fooled" another app into thinking a script, keyboard macro, etc. was a "real user" was by putting the terminal in raw or cbreak mode and sending whatever it was that I needed to send. If you've got the time, it's interesting stuff; W. Richard Stevens' "Advanced Programming in the UNIX Environment" book contains a lot of info about that and many other things. It's definitely a non-trivial task though; you're going to pretty much have to do everything I mentioned in my first post and then some.

Hope this makes sense; good luck!
Larry

Quote:
Originally Posted by sleepyhomme
ya, still a blank file.
I have tried all these before.
redirecting stdout, stderr all together.
redirecting only stdout
redirecting only stderr

Use variable to hold the execution result then echo it.

all blank
 
Old 05-15-2007, 10:56 AM   #12
sleepyhomme
LQ Newbie
 
Registered: Jan 2006
Posts: 28

Original Poster
Rep: Reputation: 15
Thanks for the tip!!! I will find the book in lib~
Many thanks!!
 
Old 05-29-2007, 12:26 PM   #13
HowDoIProgramIt
Member
 
Registered: Nov 2006
Location: East Coast, USA (in "the great northeast")
Distribution: Custom / from source; Fedora, Debian, CentOS, Scientific; LFS.
Posts: 94

Rep: Reputation: 15
Quote:
Originally Posted by sleepyhomme
Thanks for the tip!!! I will find the book in lib~
Many thanks!!
By now you've probably solved your problem and learned a lot more about *NIX termio calls than you ever wanted to know; in any event, while doing some housekeeping the other day, I ran accross my "samples" directory, and thought I'd follow up w you though: I have some sample code here that does (more or less) what you're trying to do; would you like me to send it to you? I'd like to do it in whatever way is going to make this thread the most useful to those-who-come-after-us; I'm open to ideas as far as that goes...

I wrote a daemon that can log in to FTP sites, etc.; it's quite short, though without a good roadmap you may go nuts trying to figure out how it works (I wrote it, and I went about half batty while attempting to debug it. If you want it, I'll send it && you'll see what I mean in relatively short order). It's one of many different ways to skin that cat; this one falls into the category of "simple quasi-shells", for lack of a better name / description.

Also - I noticed the other PM that the Stevens book (APUE) is out in its 2nd edition; about dropped my teeth when I saw that the retail price was USD $75. Supposedly it's been updated to address Linux, FreeBSD, etc.; personally, I don't think it needed much in the way of updating, and should you happen to run accross a copy of the first edition - for hopefully a lot less than seventy-five bucks - my personal feeling is that you would be well served by it.

Best Regards,

Larry
 
Old 05-30-2007, 03:50 AM   #14
p_s_shah
Member
 
Registered: Mar 2005
Location: India
Distribution: RHEL 3/4, Solaris 8/9/10, Fedora 4/8, Redhat Linux 9
Posts: 237
Blog Entries: 1

Rep: Reputation: 34
This worked for me :
Code:
$cat ftp.sh
ftp -n -d -v 10.1.0.22 << END | tee temp.txt
user ftpuser ftppass
ls
bye
END
I am able to grab all output of ftp command in temp.txt

Try out and reply back if u find any problem.
 
1 members found this post helpful.
Old 05-30-2007, 10:50 AM   #15
HowDoIProgramIt
Member
 
Registered: Nov 2006
Location: East Coast, USA (in "the great northeast")
Distribution: Custom / from source; Fedora, Debian, CentOS, Scientific; LFS.
Posts: 94

Rep: Reputation: 15
Quote:
Originally Posted by p_s_shah
This worked for me :
Code:
$cat ftp.sh
ftp -n -d -v 10.1.0.22 << END | tee temp.txt
user ftpuser ftppass
ls
bye
END
I am able to grab all output of ftp command in temp.txt

Try out and reply back if u find any problem.
Didn't work for me; won't work with anonymous ftp sites either (user==ftp, password=(no password)).

What ftp program are you using && what OS?

I uploaded the solution I was referring to earlier to:

http://www.cslimits.org/rq.zip

You should be able to retrieve it without any difficulties using either a browser or my personal favorite for downloading things like that, wget:

[CODE]
wget http://www.cslimits.org/rq.zip
[CODE]

I documented the code pretty thoroughly; any questions though, please let me know.

- Larry
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
In plesk , I wish to have a backup cron job, ftp back up file to another ftp server? muskiediver Linux - General 6 07-16-2009 03:13 AM
backup server ; network file system / ftp-server ? markus1982 Linux - Security 5 06-06-2007 05:15 PM
redirecting to other pc's ftp roopunix Linux - Networking 2 05-05-2007 10:32 AM
why when redirecting output to a file any file extension seems to be fine? dr_zayus69 Linux - General 1 05-21-2005 04:09 AM
help with redirecting to ftp SlipAway172 Linux - Networking 1 03-06-2005 08:47 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 03:33 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration