How do I do secure transfers?

koobi · 01-17-2008, 02:52 AM

I need to grab some CSV files from a remote location.
I will probably either get HTTP or SSH access.

If I'm only granted HTTP access, what would be the most secure way to grab the file?
I figure I can use wget using HTTPS. But what if I only get SSH access? I could use 'scp' but how do I get around the password prompt because this will be cronjob?

Also, how can I send an email to more than one user via command line once the process of downloading these files are over?

Thanks.

prudnikov · 01-17-2008, 04:28 AM

What language you are using?
In any case I think Curl will be helpful.

nautilus · 01-17-2008, 04:38 AM

Hi Koobi

You can use keys in order to avoid scp asking password. The procedure in two words is to create a public and private key and copy the public key to the remote machine in the ~/.ssh directory.

Now, about how you can send an email with multiple recipients, that depends what MTA is available on your machine. Just check the man pages or look it up.

Nathanael · 01-17-2008, 04:38 AM

i would prefere getting ssh or https access rather than http if security was my concern

for scp you can have public key logins (see man page for ssh/scp command line option -i)
also perhaps look at ssh-agent

chrism01 · 01-17-2008, 05:16 PM

Here's an example of sending mail in bash. You just need to put a loop around it and feed in a list of email addresses eg from a file
usr1@somewhere
usr2@somewhereelse
etc
http://theos.in/shell-scripting/send-mail-bash-script/

koobi · 01-21-2008, 01:58 AM

Quote:

Originally Posted by Nathanael

i would prefere getting ssh or https access rather than http if security was my concern

for scp you can have public key logins (see man page for ssh/scp command line option -i)
also perhaps look at ssh-agent

Sorry, what I meant to say was HTTPS and not HTTP.
I read up a bit about SCP and the identity file, it seems as though this will create two key files and one has to be placed in the remote server? The thing is, I can only read from the remote server, I don't have the authority to write anything to it.
It looks as though HTTPS is my next safest bet, right?

One more thing, assuming I do go with SCP, what if the file I'm attempting to access does not exist one day, can I get some soft of signal that will indicate this so that I can halt the rest of the processes that depend on the existence of this file?

Quote:

Originally Posted by chrism01

Here's an example of sending mail in bash. You just need to put a loop around it and feed in a list of email addresses eg from a file
usr1@somewhere
usr2@somewhereelse
etc
http://theos.in/shell-scripting/send-mail-bash-script/

Thanks, so there's no way to send an email to multiple users using the `mail` command in one go without using a loop?

I googled a bit and found that the following ways will also work to send emails:

Code:

mail -s "My Subject" user@example.org < /path/to/message/body
echo "My message body." | mail -s "My Subject" user@example.org

How would I loop to include multiple emails? I'm relatively new to shell scripting.

Thanks.

chrism01 · 01-21-2008, 11:31 PM

In re your last, make 'user' a bash var and assign to it in a loop, somehting like

Code:

for user in 'a b c d e'
do
    mail -s "My Subject" ${user}@example.org < /path/to/message/body
    or
    echo "My message body." | mail -s "My Subject" ${user}@example.org
done

See http://www.tldp.org/LDP/abs/html/ & http://rute.2038bug.com/index.html.gz

koobi · 01-25-2008, 12:50 AM

Quote:

Originally Posted by chrism01

In re your last, make 'user' a bash var and assign to it in a loop, somehting like

Code:

for user in 'a b c d e'
do
    mail -s "My Subject" ${user}@example.org < /path/to/message/body
    or
    echo "My message body." | mail -s "My Subject" ${user}@example.org
done

See http://www.tldp.org/LDP/abs/html/ & http://rute.2038bug.com/index.html.gz

Thanks.
But this brings me to another question.

I know the following works this way:

Code:

do_something () {
    echo $1; # Hello
    echo $2; #  Linux
    echo $3; #  world!
}

$var_1="Hello"
$var_2=" Linux"
$var_3=" world!"

do_something $var_1 $var_2 $var_3

But what if I wanted the second variable to be looped, can I make it an array?

Thanks and sorry for the silly questions

I've worked with PHP all my life, rarely worked with the shell.

chrism01 · 01-25-2008, 01:03 AM

You really ned to read those links I gave you, it'll be quicker than posting qns (although come back if you really get stuck).
There are many ways of generating/using lists, but explicit arrays are explained here: http://www.tldp.org/LDP/abs/html/arrays.html

koobi · 01-25-2008, 01:08 AM

Quote:

Originally Posted by chrism01

You really ned to read those links I gave you, it'll be quicker than posting qns (although come back if you really get stuck).
There are many ways of generating/using lists, but explicit arrays are explained here: http://www.tldp.org/LDP/abs/html/arrays.html

well i have been reading a few bash tutorials over the past week or more but i wanted to know if using an array is the best way because there are usually more than one way to do things. i just wanted to know which was best.

Thanks.

koobi · 01-28-2008, 03:35 AM

I have another problem that I'm not sure how to solve because I don't know what to look for:

Code:

file_exists () {
# if file doesn't exist, email with custom mail parameters else, return.
# $1 = the file
# $2 = subject
# $3 = message (can be file or text)
# $4 = list of email addresses to send to

    if [ ! -f $1 ]; then
        if [ ! -f $3 ]; then
            for user_email in $4
            do
                echo $3 | mail -s $2 ${user_email}@example.org
            done
        else
            for user_email in $4
            do
                mail -s $2 ${user_email}@example.org < $3
            done
        fi
    else
        return
    fi
}


file_exists "/path/to/file" "File missing" "The required file does not exist." ${ADMIN_EMAILS}

I want to send an email either with an existing template file as the body text or using the input as the body text, depending on how I input my parameters. But when I run this command, it tells me there are too many parameters...I'm guessing this is because it tries to stat the $3 argument which won't be possible if it's a string input with spaces in it.

How can I work around this? At least tell me what to look for.

Thanks.