Suggestion on coding this kind of requirement

gn00kie · 09-13-2006, 01:43 AM

Hi,
I would like to ask suggestion from you guys on how can I achieve this requirement. I would like to connect a a public ftp server, I already have the code to connect to it via script however, my requirement is to download only files that are listed on a txt file. I could do a command such as mget file1 file2 file3...etc. But how do I feed to the ftp program the list of file names that it should download. Here is the snippet of my code. TIA.

#!/usr/bin/bash

ftp -i -n << "END"
open publicserver.com
user anonymous 123
bin
cd loc1
mget file1.asc file2.asc
bye
END
echo "ftp complete"

chrism01 · 09-13-2006, 01:48 AM

Add this cmd
$file_list=`cat file_list.txt`
before the ftp stuff

then amend the mget
mget $file_list

gn00kie · 09-13-2006, 02:00 AM

can i output an ls command such that it would fit the requirements of the mget of ftp? For example, ls /dir > file_list.txt, where in the output would be file1 file2 file3?

chrism01 · 09-13-2006, 09:20 PM

If you just want the entire dir, use
mget *
or
mget *.*

gn00kie · 09-13-2006, 10:30 PM

nope, I am not going to get all the files, only the files that are included on the list.

chrism01 · 09-14-2006, 12:07 AM

That's what my 1st suggestion was for ...

gn00kie · 09-14-2006, 12:33 AM

have already manually edited the file. thanks for your suggestion.

ntubski · 09-14-2006, 11:59 AM

Quote:

Originally Posted by gn00kie

can i output an ls command such that it would fit the requirements of the mget of ftp? For example, ls /dir > file_list.txt, where in the output would be file1 file2 file3?

If you meant to only download files with the same name as the files in /dir on your local machine, then the answer is yes.

You could also do $files_list=`ls /dir` which amounts to the same thing.

theYinYeti · 09-14-2006, 12:33 PM

Something like that maybe:

Code:

#!/usr/bin/bash

ftp -i -n << "END"
open publicserver.com
user anonymous 123
bin
cd loc1
mget $(cat /path/to/text/file.txt)
bye
END
echo "ftp complete"

And in case the filenames appear in the file one per line, and ftp does not like this, just replace the "cat ...txt" command with: "tr '\n' ' ' </path/to/text/file.txt"

Yves.

gn00kie · 09-18-2006, 12:49 AM

Hi,
When I try to run the script, I get a $file_list.txt: no such file or directory error. I think ftp can not translate the value of my cat command...

ghostdog74 · 09-18-2006, 01:56 AM

That's where sometimes, the functionality of some commands doesn't let the programmer have control of.
Here's a simple in Perl too where there's more "control" ...

Code:

use Net::FTP;
...
...
$ftpobj = Net::FTP -> new ($server,Timeout=>20) or
        die "Cannot access $server via FTP\n";
$ftpobj -> login($server,$pwd);
$ftpobj -> cwd ("$dir");
foreach $files (@filearray) {
   $ftpobj -> get ($files);
}
$ftpobj -> quit;

gn00kie · 09-18-2006, 03:36 AM

Thanks for suggesting perl but is there another way for bash shell to do it?

theYinYeti · 09-19-2006, 04:59 AM

Quote:

Originally Posted by gn00kie

$file_list.txt: no such file

What exactly did you write?

ftp is not evaluating $(...) constructs, bash is. But you have to write it exactly so, with '$(' at the start of the command, and ')' at the end.

Yves.

gn00kie · 09-19-2006, 11:54 PM

this my code:

#!/usr/bin/bash

ftp -i -n << "END"
open www.publicserver.com
user anonymous 123
bin
cd public
mget $(cat /export/home/noc/vsftpd-2.0.4/bin/files1.txt)
bye
END
echo "ftp complete"

When I try to run it, I get this error: "$(cat: No such file or directory.
/export/home/noc/vsftpd-2.0.4/bin/files1.lst): No such file or directory.
can't find list of remote files, oops
ftp complete"

I agree that bash can translate variables with $ but on my script, ftp was invoked on bash and the way I see it, ftp is the one evaluating the variables enclosed on the << END portion of the program.

theYinYeti · 09-20-2006, 03:57 AM

OK I get it.
It's because you put quotes around END. Try without the quotes, like this: ftp -i -n <<END

Yves.