LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Programming (https://www.linuxquestions.org/questions/programming-9/)
-   -   Including a space taken from string in a var (https://www.linuxquestions.org/questions/programming-9/including-a-space-taken-from-string-in-a-var-893848/)

buee 07-26-2011 08:44 AM

Including a space taken from string in a var
 
I have a tsv file that I'm trying to break down, line by line, then column by column so to speak. The file is a list of exported usernames, passwords, and display names from an email server, I need each separate to be used in other commands to provision the new server. My problem is, each display name contains a space, as usual with people's names.

Here is the script for reference:
Code:

while read line
do
        uname=`echo $line | awk '{ print $1 }' >> works`
        pass=`echo $line | awk '{ print $2 }' >> works`
        display=`echo $line | >> works`
done < /opt/tmp/test.tsv

cat works
rm -rf works
exit 0

The test.tsv file is the following repeated hundreds of times:
Code:

[email][tab][password][tab][display name]
[jdoe@example.com][tab][test123][tab][John Doe]

I'm sending the output to 'works' because I need to test it obviously before I throw it on to the real server. My issue again is getting the output of the 3 column (or all data following the second tab/whitespace) to include the space to be used in the variable $display.

I've tried:
Code:

display=`echo $line | awk '{ print $3 }' >> works` #of course, I just get the first name
display=`echo $line | awk '{ print $3 $4 }' >> works` #This seems to work the best, but I get firstnamelastname, no space between the two
display=`echo $line | awk '{ print $3\ $4 }' >> works` #Doesn't like the escape character
display=`echo $line | cut -f3 >> works` #Outputs the whole line
display=`echo $line | awk '{ print $3 } { print $4 }' >> works` #This prints firstname on one line, then lastname on the following line

So what can I do here?

MTK358 07-26-2011 10:23 AM

Quote:

Originally Posted by buee (Post 4425515)
Code:

[email][tab][password][tab][display name]
[jdoe@example.com][tab][test123][tab][John Doe]


I don't understand. Is that second line really what it looks like in the file, including the square brackets and the words "tab"? Or is it actually like this:

Code:

jdoe@example.com    test123 John Doe
(using tab characters between the fields)

MTK358 07-26-2011 10:28 AM

Assuming tabs as separators, this should work:

Code:

while read line
do
    uname="$(echo "$line" | cut -f1)"
    pass="$(echo "$line" | cut -f2)"
    name="$(echo "$line" | cut -f3)"

    # do stuff

done < /opt/tmp/test.tsv

Note that all varaibles are in double quotes to prevent splitting.

Also, use $(command) instead of backticks. It's easy to nest and can't be confused with quotes.

crts 07-26-2011 10:37 AM

Hi,

try using the tab as field separator in awk
Code:

awk -F '\t' '{print $3}'

buee 07-26-2011 11:25 AM

Quote:

Originally Posted by MTK358 (Post 4425615)
I don't understand. Is that second line really what it looks like in the file, including the square brackets and the words "tab"? Or is it actually like this:

Code:

jdoe@example.com    test123 John Doe
(using tab characters between the fields)

Your code is the accurate depiction of what a line of the file looks like. There are no brackets.

Quote:

Assuming tabs as separators, this should work:

Code:

while read line
do
    uname="$(echo "$line" | cut -f1)"
    pass="$(echo "$line" | cut -f2)"
    name="$(echo "$line" | cut -f3)"

    # do stuff

done < /opt/tmp/test.tsv

Note that all varaibles are in double quotes to prevent splitting.

Also, use $(command) instead of backticks. It's easy to nest and can't be confused with quotes.
This has the same output as with backticks. It just prints the entire line to the file, email address, password, and display name, as if there was no attempt at text selection.

Quote:

Hi,

try using the tab as field separator in awk

Code:

awk -F '\t' '{print $3}'

This output a blank line where the display name normally is.

MTK358 07-26-2011 11:41 AM

Quote:

Originally Posted by buee (Post 4425688)
This has the same output as with backticks. It just prints the entire line to the file, email address, password, and display name, as if there was no attempt at text selection.

Unless you added some code, that example shouldn't print anything. Try this to see if it works:

Code:

while read line
do
    uname="$(echo "$line" | cut -f1)"
    pass="$(echo "$line" | cut -f2)"
    name="$(echo "$line" | cut -f3)"
    echo "Name: $name"
    echo "Email: $uname"
    echo "Password: $pass"
    echo ""
done < /opt/tmp/test.tsv

EDIT: I've tested it and it does work.

David the H. 07-26-2011 11:57 AM

Just to be clear, what shell are we talking about here? Is it bash, another shell, or do you need this to be posix-compliant? It may affect what commands can be used.

If you're using bash or another bourne-based shell, then you can probably skip awk entirely and use some other technique instead. bash's read command can be used to set arrays, for example, with IFS controlling the delimiter:
Code:

IFS=$'\t'
while read -a line ; do

    uname="${line[0]}"
    pass="${line[1]}"
    display="${line[2]}"

done < /opt/tmp/test.tsv


David the H. 07-26-2011 12:02 PM

Also, I just noticed this:
Code:

uname=`echo $line | awk '{ print $1 }' >> works`
This will never do what you want it to. You can't both redirect the output to file and set the variable at the same time. In this case, the output of awk goes straight into "works", and the variable never sees it.

Set the variable first, then echo it into the file with a separate command.

MTK358 07-26-2011 12:03 PM

Quote:

Originally Posted by David the H. (Post 4425717)
you can probably skip awk entirely

My suggestions didn't use awk.

Quote:

Originally Posted by David the H. (Post 4425717)
Code:

read -a

Where are the options for read all documeanted? When I try "man read", it says that it has only one option (-r).

David the H. 07-26-2011 12:18 PM

read is a bash built-in keyword, so it's found in the bash man page. You can also get a quick description with "help read". The man page you're reading is likely from a separate application of the same name. The man page for read on my system is for the C library function.

I said awk because that's what the last post was talking about when I started typing, and I didn't see yours until after I posted. But since cut is also an external program, my comment applies to it as well; this can be done entirely in-shell, no external tools necessary. :)

MTK358 07-26-2011 12:23 PM

Quote:

Originally Posted by David the H. (Post 4425741)
read is a bash built-in keyword, so it's found in the bash man page. You can also get a quick description with "help read". The read man page you're reading is likely from a separate application of the same name. The man page for read on my system is for the C library function.

Mine is about the actual command, but it says "POSIX Programmer's Manual", so I guess that it documents only the standard options.

Quote:

Originally Posted by David the H. (Post 4425741)
I said awk because that's what was the last post was talking about when I started typing, and I didn't see yours until after I posted. But since cut is also an external program, my comment applies to it as well; this can be done entirely in-shell, no external tools necessary. :)

I think that "cut" is still way less overkill than an interpreted programming language like awk, but I agree that doing it in the shell is best. I just didn't know whether it was possible/how to do it.

David the H. 07-26-2011 12:37 PM

True, cut is a lighter and more efficient choice than awk for something like this.

But when it comes to scripting input that's field-delimited, or otherwise should be treated as separate-but-related strings, arrays are the first thing you should turn to. The other tools are mostly there for getting the data into the array. ;)

crts 07-26-2011 12:45 PM

Quote:

Originally Posted by buee (Post 4425688)
This output a blank line where the display name normally is.

Without some more info on what you actually executed it is hard to tell what went wrong. My guess is that you did not quote the variable $line and therefore the tabs got "swallowed".
This
Code:

while read line; do echo "${line}" | awk -F '\t' '{print $3}'; done < /path/to/file
does work for me.

If you do not need to do more complex String manipulations than in your example then I would also recommend that you use
David's solution from post #7.

buee 07-26-2011 12:51 PM

Quote:

Originally Posted by MTK358 (Post 4425703)
Unless you added some code, that example shouldn't print anything. Try this to see if it works:

Code:

while read line
do
    uname="$(echo "$line" | cut -f1)"
    pass="$(echo "$line" | cut -f2)"
    name="$(echo "$line" | cut -f3)"
    echo "Name: $name"
    echo "Email: $uname"
    echo "Password: $pass"
    echo ""
done < /opt/tmp/test.tsv

EDIT: I've tested it and it does work.

This did the trick. The reason I wanted to output to a file is to make sure that the variables were properly formatted.
Once this is all settled, I have to issue a command like:

Code:

provision $uname $pass displayName $name
, but it's gonna cycle a few hundred times, I don't want to have to revert and go back.

Now, I have to examine $pass to see if it's less than 6 characters. Any recommendations?

buee 07-26-2011 12:57 PM

Quote:

Originally Posted by buee (Post 4425778)
This did the trick. The reason I wanted to output to a file is to make sure that the variables were properly formatted.
Once this is all settled, I have to issue a command like:

Code:

provision $uname $pass displayName $name
, but it's gonna cycle a few hundred times, I don't want to have to revert and go back.

Now, I have to examine $pass to see if it's less than 6 characters. Any recommendations?

Nevermind, I think I found it, kinda lengthy tho.

Code:

chars=$(echo $pass | wc -c)
chars=$(( $chars - 1 ))

Seems odd that wc -c would return a count of +1 than the actual number of characters. Counting a whitespace or something?


All times are GMT -5. The time now is 11:12 AM.