LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 07-28-2009, 03:11 PM   #1
wakatana
Member
 
Registered: Jul 2009
Location: Slovakia
Posts: 141

Rep: Reputation: 16
Question shell variables advanced


Hi all, i have more questions but its all about variables so lets begin

1st, is possible to list all variables ?
Command env display only shell variables, but what if i declared another variable? Command set display more variables but not defined by me.

2nd, what difference is between set and env ?

3rd, lets say i declared variable which contains whitespaces, or i have variable which contains whitespaces for example IFS. When i type echo $IFS i see nothing, so i dont know if is declared or if contain white spaces.
Is possible to write white spaces in C format (\n \r \t ...) ?

4th what is difference between sourcing and exporting variable ?




PS: i saw declaration of IFS in set command (IFS=$' \t\n')


Thank you very much
 
Old 07-28-2009, 04:50 PM   #2
blackhole54
Senior Member
 
Registered: Mar 2006
Posts: 1,896

Rep: Reputation: 61
Quote:
Originally Posted by wakatana View Post
1st, is possible to list all variables ?
Command env display only shell variables, but what if i declared another variable? Command set display more variables but not defined by me.[]
env lists all the environmental variables. This include those that the shell inherited and those that you have exported.

set lists all variables. The variables you see that are not environmental variables and have not been defined by you are probably variables that bash always defines (if that is the shell you are using). You can read about them in the bash man page.

Quote:
2nd, what difference is between set and env ?
See above.

Quote:
3rd, lets say i declared variable which contains whitespaces, or i have variable which contains whitespaces for example IFS. When i type echo $IFS i see nothing, so i dont know if is declared or if contain white spaces.
You can see all the characters by piping the output of the echo command to something like od or hexdump. Be aware that unless you supply -n to the echo command, it will add on an extra newline (0x0a) which is not part of the variable's value.

Quote:
Is possible to write white spaces in C format (\n \r \t ...) ?
If you are talking about assigning values to variables, to my knowledge, no. But some commands do understand at least a subset of those. The bash built-in printf command works similar to the C statement by that name. So for example, in bash you could (to use a ridiculous example):

Code:
variable="$(printf "ab\ncd\tef\r")"
Try it, and then:

Code:
echo -n "$variable" | hexdump -C
to see the result.

Quote:
4th what is difference between sourcing and exporting variable ?
I assume you are talking about sourcing a file? When you souce a file, its contents are treated as if they were part of the script (or shell) which sourced it. As such all variables and functions are available to it and any variables it creates or modifies and any functions it defines will subsequently be available to the script/shell that sourced it.

When you call a script, (or other non-builtin command) it is run as a subprocess. It will inherit all of the environmental variables but no other variables, and any changes it makes to variables are not seen by the script/shell that called it. So if you create a variable that you wish a command or script to see, if it is not already an environmental variable, you must export it.

Quote:
PS: i saw declaration of IFS in set command (IFS=$' \t\n')
That's news to me. But it seemed to work! Looks like I need to study the bash man page a little more!

Last edited by blackhole54; 07-28-2009 at 04:54 PM. Reason: grammar :)
 
Old 07-29-2009, 12:27 PM   #3
wakatana
Member
 
Registered: Jul 2009
Location: Slovakia
Posts: 141

Original Poster
Rep: Reputation: 16
Thank you for you reply its much more brighter .

I tried followings

Code:
$ printf "one\two\tthree\n\r" | hexdump -C
00000000  6f 6e 65 09 77 6f 09 74  68 72 65 65 0a 0d     |one.wo.three..|
0000000e
seems OK.

Code:
$ numbers="$(printf "one\two\tthree\n\r")"
$ echo $numbers | hexdump -C
00000000  6f 6e 65 20 77 6f 20 74  68 72 65 65 20 0d 0a |one wo three ..|
0000000f
seems like in variable cannot have whitespaces, simpy they are replaced by space

also if i type
Code:
$ echo $IFS | hexdump -C
00000000  0a                                                |.|
00000001
im am trying to do for loop in bash script throught *.mp3 files in directory. But mp3 files contains white spaces so i found this tip
http://www.unix.com/shell-programmin...p-bash-sh.html
where is advised to back up and then set the "delimiter of for loop" to IFS=$’\n’. But i dont know if my $IFS is declared or not, or how can i back it up if i cannot check it ?
 
Old 07-29-2009, 12:38 PM   #4
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,578
Blog Entries: 31

Rep: Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198
Quote:
Originally Posted by wakatana View Post
Code:
$ numbers="$(printf "one\two\tthree\n\r")"
$ echo $numbers | hexdump -C
00000000  6f 6e 65 20 77 6f 20 74  68 72 65 65 20 0d 0a |one wo three ..|
0000000f
seems like in variable cannot have whitespaces, simpy they are replaced by space

also if i type
Code:
$ echo $IFS | hexdump -C
00000000  0a                                                |.|
00000001
A variable can contain whitespace but when the shell evaluates an expression containing whitespace characters it replaces all the adjacent whitespace characters with a single space. These will do what you intended to do:
Code:
echo "$IFS" | hexdump -C
$ numbers="$(printf "one\two\tthree\n\r")"
$ echo "$numbers" | hexdump -C
Quote:
Originally Posted by wakatana View Post
im am trying to do for loop in bash script throught *.mp3 files in directory. But mp3 files contains white spaces so i found this tip
http://www.unix.com/shell-programmin...p-bash-sh.html
where is advised to back up and then set the "delimiter of for loop" to IFS=$\n. But i dont know if my $IFS is declared or not, or how can i back it up if i cannot check it ?
By default IFS is set to <space><tab><newline>. You can back it up by
Code:
old_IFS="$IFS"
but you don't need to; recent discussion on LQ "discovered" that if IFS is unset or empty the bash behaves as if it were <space><tab><newline> so you can do something like
Code:
IFS=$(\n)
<stuff>
unset IFS
 
Old 07-29-2009, 07:02 PM   #5
blackhole54
Senior Member
 
Registered: Mar 2006
Posts: 1,896

Rep: Reputation: 61
Quote:
Originally Posted by catkin View Post
You can back it up by
Code:
old_IFS="$IFS"
but you don't need to; recent discussion on LQ "discovered" that if IFS is unset or empty the bash behaves as if it were <space><tab><newline> so you can do something like
Code:
IFS=$(\n)
<stuff>
unset IFS
Is that documented (i.e. man page, etc) behavior? If it's not I wonder if it is a good idea to assume it will do that. I.e. I worry that you may have a script that works but you find when you run it under a different version of bash that it breaks.

Could you provide a link to that "recent discussion on LQ?" Thanks.
 
Old 07-30-2009, 02:41 AM   #6
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,578
Blog Entries: 31

Rep: Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198
Quote:
Originally Posted by blackhole54 View Post
Is that documented (i.e. man page, etc) behavior? If it's not I wonder if it is a good idea to assume it will do that. I.e. I worry that you may have a script that works but you find when you run it under a different version of bash that it breaks.

Could you provide a link to that "recent discussion on LQ?" Thanks.
Yes, it's documented -- explicitly for POSIX shell and implicitly for bash, based on POSIX conformance and listed POSIX/non-POSIX mode differences. See this LQ thread for references.
 
Old 08-01-2009, 04:51 AM   #7
wakatana
Member
 
Registered: Jul 2009
Location: Slovakia
Posts: 141

Original Poster
Rep: Reputation: 16
So if i dont need to manipulate IFS how can i do this:

input:

Code:
-rw-r--r-- 1 marek marek 0 2009-07-31 20:37 01 - album - track001.mp3
-rw-r--r-- 1 marek marek 0 2009-07-31 20:37 02 - album - track002.mp3
-rw-r--r-- 1 marek marek 0 2009-07-31 20:37 03 - album - track003.mp3
-rw-r--r-- 1 marek marek 0 2009-07-31 20:37 04 - album - track004.mp3
-rw-r--r-- 1 marek marek 0 2009-07-31 20:37 05 - album - track005.mp3
-rw-r--r-- 1 marek marek 0 2009-07-31 20:37 06 - album - track006.mp3
-rw-r--r-- 1 marek marek 0 2009-07-31 20:37 07 - album - track007.mp3
-rw-r--r-- 1 marek marek 0 2009-07-31 20:37 08 - album - track008.mp3
-rw-r--r-- 1 marek marek 0 2009-07-31 20:37 09 - album - track009.mp3
desiret output

trackXXX.mp3

i foud this can be done with this but did not work for me

Code:
for jmeno in *.mp3
do
  cp $jmeno /tmp/`echo $i | sed "s/\([0-9]\{2\}\).*/track0\1.mp3/"`
done
is this possible using bash and for loop, i am trying to learn something about bash scripting this is what i am intersted in
 
Old 08-01-2009, 11:58 AM   #8
wakatana
Member
 
Registered: Jul 2009
Location: Slovakia
Posts: 141

Original Poster
Rep: Reputation: 16
I am trying something more bit complicated:
Code:
#!/bin/bash
#IFS=$'\n'
echo default IFS:
echo -n "$IFS" | od -a -b
IFS=$'\n'
echo new IFS:
echo -n "$IFS" | od -a -b
for x  in `ls *.mp3`
do
 echo "`sed -ne 's/\([0-9]\{2\}\) - .* - .*[0-9]\{3\}\.mp3/track\1.mp3/gp' $x`"
done
echo new IFS:
echo -n "$IFS" | od -a -b
unset IFS
echo standard IFS:
echo -n "$IFS" | od -a -b
but alsdo did not work, IFS is set to newline, sed command must be OK because if i try it outside script everything works. Now i tryed just echo output of sed but nothing appears.

also tried replace echo with
Code:
cp "$x" "`sed -ne 's/\([0-9]\{2\}\) - .* - .*[0-9]\{3\}\.mp3/track\1.mp3/gp' $x`"
but according expectation it do nothing.
Also i am noticed something strange
Code:
numbers="$(printf "\n")"; echo -n "$numbers" | od -a -b
0000000

numbers="$(printf "\t")"; echo -n "$numbers" | od -a -b
0000000  ht
        011
0000001
Can someone explain that ???


PS i am total bash scripting newbie, sorry for dummmy questions
 
Old 08-02-2009, 05:59 AM   #9
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,578
Blog Entries: 31

Rep: Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198
Hello wakatana
Quote:
Originally Posted by wakatana View Post
i foud this can be done with this but did not work for me

Code:
for jmeno in *.mp3
do
  cp $jmeno /tmp/`echo $i | sed "s/\([0-9]\{2\}\).*/track0\1.mp3/"`
done
is this possible using bash and for loop, i am trying to learn something about bash scripting this is what i am intersted in
I'm confused! Did it work or did it not work?!

*.mp3 should expand to track001.mp3 track002.mp3 etc.

In your script, where is $i given a value for the echo $i?

For debugging, it is very useful to show values so you can see if the script is doing what you think it is doing.

set -xv is good except when using `<commands>` or (better) $(<commands>) when it changes the normal output from the commands.

echo is quick and easy, too
Code:
for jmeno in *.mp3
do
  echo "'$jmeno'"
  echo "'$i'"
  echo `echo $i | sed "s/\([0-9]\{2\}\).*/track0\1.mp3/"`
  cp $jmeno /tmp/`echo $i | sed "s/\([0-9]\{2\}\).*/track0\1.mp3/"`
done
The single quotes inside the double quotes will reveal any whitespace at the beginning or end of the value.

Best

Charles
 
Old 08-02-2009, 06:20 AM   #10
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,578
Blog Entries: 31

Rep: Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198
Quote:
Originally Posted by wakatana View Post
Also i am noticed something strange
Code:
numbers="$(printf "\n")"; echo -n "$numbers" | od -a -b
0000000

numbers="$(printf "\t")"; echo -n "$numbers" | od -a -b
0000000  ht
        011
0000001
Can someone explain that ???
First $numbers is set to line end which the od command takes as end of record (?) and does not show. Second $numbers is set to (horizontal) tab which od shows.
 
Old 08-02-2009, 06:24 AM   #11
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,578
Blog Entries: 31

Rep: Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198
Quote:
Originally Posted by wakatana View Post
Code:
[snip]
unset IFS
echo standard IFS:
echo -n "$IFS" | od -a -b
When IFS is unset it is not reset to the default value but, er, unset. Bash behaviour with an unset IFS is the same as when IFS has the default value.
 
Old 08-02-2009, 09:02 AM   #12
wakatana
Member
 
Registered: Jul 2009
Location: Slovakia
Posts: 141

Original Poster
Rep: Reputation: 16
Hello catkin

the
Code:
for jmeno in *.mp3
do
  cp $jmeno /tmp/`echo $i | sed "s/\([0-9]\{2\}\).*/track0\1.mp3/"`
done
did not work. Seems strange exatly for $i variabale, so I post it there.

Yes i use echo like a "debugger" otherwise, can you please me explain or ad some reference what set -xv is doing, man is not clear for me.


Quote:
The single quotes inside the double quotes will reveal any whitespace at the beginning or end of the value.
Hmm I thing single quotes in double quotes does not have special meaning, iterpreting white characters is done because of double quotes.
Single quotes simply "turn of" variable interpretation, but i think they are in double quotes they act like any other character.
Maybe i am not righ can you please post some reference for that.

in
Code:
numbers="$(printf "\n")"; echo -n "$numbers" | od -a -b
0000000

numbers="$(printf "\t")"; echo -n "$numbers" | od -a -b
0000000  ht
        011
0000001
i am pointing to that, if you assing to variable whitespace \n (other whitespaces works \t, \r) with echo or printf command. Variable seems did not containt it!
So if you want to assign \n whitespace to variable you must do that
Code:
variable=$'\n'; echo -ne "$variable" | od -a -b
0000000  nl
        012
0000001
Code:
variable=`echo -ne "\n"`; echo -ne "$variable" | od -a -b
0000000

variable=$(printf "\n"); echo -ne "$variable" | od -a -b
0000000

but:

variable=$(printf "\t"); echo -ne "$variable" | od -a -b
0000000  ht
        011
0000001

variable=`echo -ne "\t"`; echo -ne "$variable" | od -a -b
0000000  ht
        011
0000001
That looks strange to me. What is $'\n' it is something like $(command) or I am confusing it. I read somewhere that older version of bash had IFS='\n', newest IFS=$'\n'.... what is this magic $ there. Thanks
 
Old 08-02-2009, 11:20 AM   #13
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,578
Blog Entries: 31

Rep: Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198
Hello wakatana
Quote:
Originally Posted by wakatana View Post
Hello catkin

the
Code:
for jmeno in *.mp3
do
  cp $jmeno /tmp/`echo $i | sed "s/\([0-9]\{2\}\).*/track0\1.mp3/"`
done
did not work. Seems strange exatly for $i variabale, so I post it there.
I don't understand what you are doing with $i or why you are doing it.
Quote:
Originally Posted by wakatana View Post
Yes i use echo like a "debugger" otherwise, can you please me explain or ad some reference what set -xv is doing, man is not clear for me.
http://www.gnu.org/software/bash/man...he-Set-Builtin
Quote:
Originally Posted by wakatana View Post
Hmm I thing single quotes in double quotes does not have special meaning, iterpreting white characters is done because of double quotes.
Single quotes simply "turn of" variable interpretation, but i think they are in double quotes they act like any other character.
Maybe i am not righ can you please post some reference for that.
You are right except with "iterpreting white characters is done because of double quotes". Inside both single and double quotes, whitespace characters are not changed. http://www.gnu.org/software/bash/man...#Single-Quotes
Quote:
Originally Posted by wakatana View Post

in
Code:
numbers="$(printf "\n")"; echo -n "$numbers" | od -a -b
0000000

numbers="$(printf "\t")"; echo -n "$numbers" | od -a -b
0000000  ht
        011
0000001
i am pointing to that, if you assing to variable whitespace \n (other whitespaces works \t, \r) with echo or printf command. Variable seems did not containt it!
So if you want to assign \n whitespace to variable you must do that
Code:
variable=$'\n'; echo -ne "$variable" | od -a -b
0000000  nl
        012
0000001
Code:
variable=`echo -ne "\n"`; echo -ne "$variable" | od -a -b
0000000

variable=$(printf "\n"); echo -ne "$variable" | od -a -b
0000000

but:

variable=$(printf "\t"); echo -ne "$variable" | od -a -b
0000000  ht
        011
0000001

variable=`echo -ne "\t"`; echo -ne "$variable" | od -a -b
0000000  ht
        011
0000001
That looks strange to me. What is $'\n' it is something like $(command) or I am confusing it. I read somewhere that older version of bash had IFS='\n', newest IFS=$'\n'.... what is this magic $ there. Thanks
Yes -- variable=$(printf "\n") and variable=`echo -ne "\n"` assign nothing to $variable. Strange! Both should (TM) assign line-end, according to my understanding.

$'\n' is the syntax for bash' ANSI-C quoting.

IFS='\n' cannot be correct because it would be exactly that, "\" followed by "n".

Best

Charles
 
Old 08-02-2009, 07:40 PM   #14
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 7.7 (?), Centos 8.1
Posts: 17,873

Rep: Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600Reputation: 2600
You may want to bookmark these:

http://rute.2038bug.com/index.html.gz
http://tldp.org/LDP/Bash-Beginners-G...tml/index.html
http://www.tldp.org/LDP/abs/html/

They discuss bash in increasing depth....
 
Old 10-13-2009, 07:56 AM   #15
wakatana
Member
 
Registered: Jul 2009
Location: Slovakia
Posts: 141

Original Poster
Rep: Reputation: 16
thanks , sorry for long response
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
help with shell variables/commands grndfl Linux - Newbie 2 07-25-2008 05:06 PM
Advanced Maintenance Mode limited shell Doomhammer AIX 1 06-21-2007 04:17 AM
moving files in shell - advanced trscookie Linux - Software 4 04-25-2007 04:25 PM
Shell variables Milosevic Linux - Newbie 2 12-10-2005 10:32 PM
Shell Variables. ducka Programming 5 09-03-2003 05:38 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 08:34 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration