LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 08-02-2013, 10:17 AM   #1
Lucien Lachance
Member
 
Registered: May 2013
Posts: 82

Rep: Reputation: Disabled
Bash - simplifying an array


Is there any possible way I can shorten this array in Bash? I noticed that Bash didn't have a way of expanding symbols, so I was forced to hard code each individual symbol. I need to have A-Z, a-z, 0-9, and symbols.

Code:
pass_wd=(
  {0..9}
  {a..z}
  {A..Z}
  '!'
  '"'
  '#'
  '$'
  '%'
  '&'
  '\'
  '('
  ')'
  '*'
  '+'
  ','
  '-'
  '.'
  '/'
  ':'
  ';'
  '<'
  '='
  '>'
  '?'
  '@'
  '['
  '\'
  '\'
  ']'
  '^'
  '_'
  '`'
  '{'
  '|'
  '}'
  '~'                                                                                                                                                                    
)
 
Old 08-02-2013, 10:46 AM   #2
olau
LQ Newbie
 
Registered: Jun 2006
Distribution: Debian Sarge ( lol I should generalize to: Debian Testing )
Posts: 15

Rep: Reputation: 1
Hmm,
I think we need to understand what you want to do with such an array.
It seems you are looking for a regex representing all this characters.
When this assumption is correct you may look for the dot sign? ( '.' )
Cheers
 
Old 08-02-2013, 11:09 AM   #3
Lucien Lachance
Member
 
Registered: May 2013
Posts: 82

Original Poster
Rep: Reputation: Disabled
I've written a script that randomly shuffles this array. As of now it just produces passwords with 0-9, a-z, and A-Z. I want to incorporate symbols inside of this array. It's just an array of alphanumeric characters. In python, I can easily encapsulate these characters by saying
Code:
c = string.ascii_letters + string.punctuation
print c

Output:
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!"#$%&'()*+,-./:;<=>?@[\]^_`{|}~
I just want to know is there a way of storing all of these characters inside of array that doesn't produce all of this clutter.

Last edited by Lucien Lachance; 08-02-2013 at 11:11 AM.
 
Old 08-02-2013, 12:41 PM   #4
firstfire
Member
 
Registered: Mar 2006
Location: Ekaterinburg, Russia
Distribution: Debian, Ubuntu
Posts: 709

Rep: Reputation: 428Reputation: 428Reputation: 428Reputation: 428Reputation: 428
Hi.

It is easy to obtain such a string in bash:
Code:
$ x="$(printf '\\x%x ' {33..126})"
$ echo -e $x
! " # $ % & ' ( ) * + , - . / 0 1 2 3 4 5 6 7 8 9 : ; < = > ? @ A B C D E F G H I J K L M N O P Q R S T U V W X Y Z [ \ ] ^ _ ` a b c d e f g h i j k l m n o p q r s t u v w x y z { | } ~
But to put each character into an array separately one should disable pathname expansion (because of `*' character), while keep word splitting:
Code:
$ set -o noglob    # disable pathname expansion
$ x="$(printf '\\x%x ' {33..126})"
$ A=( $(echo -e "$x") )
$ set +o noglob    # enable pathname expansion
$ echo "${A[9]}"
*
$ echo "${A[*]}"
! " # $ % & ' ( ) * + , - . / 0 1 2 3 4 5 6 7 8 9 : ; < = > ? @ A B C D E F G H I J K L M N O P Q R S T U V W X Y Z [ \ ] ^ _ ` a b c d e f g h i j k l m n o p q r s t u v w x y z { | } ~
$ declare -p A
declare -a A='([0]="!" [1]="\"" [2]="#" [3]="\$" [4]="%" [5]="&" [6]="'\''" [7]="(" [8]=")" [9]="*" [10]="+" [11]="," [12]="-" [13]="." [14]="/" [15]="0" [16]="1" [17]="2" [18]="3" [19]="4" [20]="5" [21]="6" [22]="7" [23]="8" [24]="9" [25]=":" [26]=";" [27]="<" [28]="=" [29]=">" [30]="?" [31]="@" [32]="A" [33]="B" [34]="C" [35]="D" [36]="E" [37]="F" [38]="G" [39]="H" [40]="I" [41]="J" [42]="K" [43]="L" [44]="M" [45]="N" [46]="O" [47]="P" [48]="Q" [49]="R" [50]="S" [51]="T" [52]="U" [53]="V" [54]="W" [55]="X" [56]="Y" [57]="Z" [58]="[" [59]="\\" [60]="]" [61]="^" [62]="_" [63]="\`" [64]="a" [65]="b" [66]="c" [67]="d" [68]="e" [69]="f" [70]="g" [71]="h" [72]="i" [73]="j" [74]="k" [75]="l" [76]="m" [77]="n" [78]="o" [79]="p" [80]="q" [81]="r" [82]="s" [83]="t" [84]="u" [85]="v" [86]="w" [87]="x" [88]="y" [89]="z" [90]="{" [91]="|" [92]="}" [93]="~")'
Of course you may also iterate over $x (it contains character codes) and put each character into an array separately. Or you may process `*' separately.

Hope this helps.

EDIT: here is the ASCII code table http://www.ascii-code.net/ . It will (hopefully) make clear why I use that {33..126} range.

Last edited by firstfire; 08-02-2013 at 01:01 PM.
 
Old 08-02-2013, 02:56 PM   #5
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,248
Blog Entries: 8

Rep: Reputation: 235Reputation: 235Reputation: 235
Good point by firstfire. I've been looking at printf but never saw a trick like it, which I probably can't unless I searched. Anyway you don't need to disable pathname expansion. You could just use read and process substitution:
Code:
IFS=$'\xFF' read -ra A < <(echo -e "$(printf '\\x%x\\xFF' {33..126})")
Of course since we are converting a string to an array and could only have done it with a separator, we'll have to use a character besides 0x00 which doesn't seem to work, like 0xFF.

I might find this useful to my scripts someday as well thanks.

For convenience we could use a function like this to generate characters to an array:
Code:
# syntax: generate_chars <avar> <first ascii code> <last ascii code>
#
function generate_chars {
    IFS=$'\xFF' read -ra "$1" < <(echo -e "$(eval "printf '\\\\x%x\\\\xFF' {$2..$3}")")
}
Or
Code:
# syntax: generate_chars <avar> <ascii code> [<ascii code 2> ...]
#
function generate_chars {
    IFS=$'\xFF' read -ra "$1" < <(echo -e "$(printf '\\x%x\\xFF' "${@:2}")")
}

Last edited by konsolebox; 08-02-2013 at 03:11 PM.
 
Old 08-02-2013, 03:55 PM   #6
firstfire
Member
 
Registered: Mar 2006
Location: Ekaterinburg, Russia
Distribution: Debian, Ubuntu
Posts: 709

Rep: Reputation: 428Reputation: 428Reputation: 428Reputation: 428Reputation: 428
konsolebox, nice trick with reading directly into an array!

But I'd like to stress that in this particular case the SPACE character (decimal 32) is outside of the range in question (passwords with spaces is a bad idea?), so your command may be simplified to
Code:
read -a A <<< $(echo -e "$(printf '\\x%x ' {33..126})")
Default IFS works just fine (in this case).
 
Old 08-02-2013, 04:19 PM   #7
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,248
Blog Entries: 8

Rep: Reputation: 235Reputation: 235Reputation: 235
@firstfire Yes, I did expect the obvious and also tried to quote the possibility of spaces at first and that they would be ignored if another range is used, but finding another tweak I completely discarded the note and suggested the use of another IFS value if needed. If someone would hit 0x20, 0x09 and even 0x0A, then they would be discarded. Also, creating a general function for that would not be possible.
 
Old 08-02-2013, 04:19 PM   #8
Lucien Lachance
Member
 
Registered: May 2013
Posts: 82

Original Poster
Rep: Reputation: Disabled
Some good stuff floating around, I'll take a look at some your solutions in just a minute! In the mean time you can see my script here: https://github.com/Demet19/bash_scri...master/pass_wd
 
Old 08-02-2013, 04:33 PM   #9
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,248
Blog Entries: 8

Rep: Reputation: 235Reputation: 235Reputation: 235
Quote:
Originally Posted by firstfire View Post
Code:
read -a A <<< $(echo -e "$(printf '\\x%x ' {33..126})")
And yes using <<< $() over < <() is a good preference since you no longer have to summon a named pipe, though -r for read is still necessary since using it would not interpret \ as another element and quote the space after instead. Notice that doing echo "${A[@]}" after it, \ is no longer included.
 
Old 08-02-2013, 04:40 PM   #10
Lucien Lachance
Member
 
Registered: May 2013
Posts: 82

Original Poster
Rep: Reputation: Disabled
Wow, I'm blown away as to how you've set that up ^ using printf. Can you step by step explain what this is doing? I'd really appreciate that.

Code:
read -a A <<< $(echo -e "$(printf '\\x%x ' {33..126})")
 
Old 08-02-2013, 04:55 PM   #11
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,248
Blog Entries: 8

Rep: Reputation: 235Reputation: 235Reputation: 235
If we go step by step printf would generate this:
Code:
\x21 \x22 \x23 \x24 \x25 \x26 \x27 \x28 \x29 \x2a \x2b \x2c \x2d \x2e \x2f \x30 \x31 \x32 \x33 \x34 \x35 \x36 \x37 \x38 \x39 \x3a \x3b \x3c \x3d \x3e \x3f \x40 \x41 \x42 \x43 \x44 \x45 \x46 \x47 \x48 \x49 \x4a \x4b \x4c \x4d \x4e \x4f \x50 \x51 \x52 \x53 \x54 \x55 \x56 \x57 \x58 \x59 \x5a \x5b \x5c \x5d \x5e \x5f \x60 \x61 \x62 \x63 \x64 \x65 \x66 \x67 \x68 \x69 \x6a \x6b \x6c \x6d \x6e \x6f \x70 \x71 \x72 \x73 \x74 \x75 \x76 \x77 \x78 \x79 \x7a \x7b \x7c \x7d \x7e
And echo -e would interpret that and convert to actual ascii characters.
Code:
! " # $ % & ' ( ) * + , - . / 0 1 2 3 4 5 6 7 8 9 : ; < = > ? @ A B C D E F G H I J K L M N O P Q R S T U V W X Y Z [ \ ] ^ _ ` a b c d e f g h i j k l m n o p q r s t u v w x y z { | } ~
After that, read -a A would convert the string to an array and save it to A through which each element is separated by the values of IFS respectively. As I had quoted, -r is need to parse backslashes as well.
Code:
read -ra A <<< $(echo -e "$(printf '\\x%x ' {33..126})")
If we are to read a character which is part of IFS, we have to change IFS too and use a different separator like what I did above.
Code:
IFS=$'\xFF' read -ra A < <(echo -e "$(printf '\\x%x\\xFF' {33..126})")
$'\0' with \\x00 doesn't work so I chose xFF which is rarely used as a character. This could be made configurable as well for the extremes.

Btw, make sure to give the credit for echo -e and printf to firstfire for it was his idea. I just added the easier way to convert it to arrays.

Last edited by konsolebox; 08-02-2013 at 04:58 PM.
 
Old 08-02-2013, 05:39 PM   #12
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Arch + Xfce
Posts: 6,852

Rep: Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037
I'd even take it one step further.
Code:
mapfile -t chars < <( printf '%b\n' \\0{4..7}{0..7} \\1{0..7}{0..7} )

unset chars[0] chars[${#chars[@]}-1]

printf '%s\n' "${chars[@]}"
%b directly expands escape sequences, just like echo -e. All we have to do is make sure the input strings are already in \0nn (octal) or \xnn (hex) format. I went with octal, and I also found it easier to just unset the first and last characters of the range (040 and 177, space and delete) afterwards rather than further modifying the brace expansions. Finally, mapfile gives us a convenient way to read the input into the array that doesn't require fiddling with IFS or anything.

Edit: Actually, the hex values are easier to generate:
Code:
printf '%b\n' \\x{2..7}{{0..9},{A..F}}

Last edited by David the H.; 08-02-2013 at 05:54 PM.
 
Old 08-02-2013, 05:41 PM   #13
Lucien Lachance
Member
 
Registered: May 2013
Posts: 82

Original Poster
Rep: Reputation: Disabled
Thanks so much for that, I didn't know printf was so powerful. Also, thanks @fistfire for explaining the meaning behind the ascii values. I was thinking about doing that with a while loop and then appending those characters to it, but you've given me a better look at the problem.

Last edited by Lucien Lachance; 08-02-2013 at 05:43 PM.
 
Old 08-02-2013, 05:54 PM   #14
firstfire
Member
 
Registered: Mar 2006
Location: Ekaterinburg, Russia
Distribution: Debian, Ubuntu
Posts: 709

Rep: Reputation: 428Reputation: 428Reputation: 428Reputation: 428Reputation: 428
Quote:
Originally Posted by konsolebox View Post
though -r for read is still necessary since using it would not interpret \ as another element and quote the space after instead. Notice that doing echo "${A[@]}" after it, \ is no longer included.
Yes, you are right. Nice catch!

David, thanks for noticing the %b format specifier. I managed to overlook it in man printf, though read it many times today. The mapfile is also a handy thing.
 
Old 08-02-2013, 06:12 PM   #15
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Arch + Xfce
Posts: 6,852

Rep: Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037
%b is a bash/ksh-only thing, and not in /usr/bin/printf, so it's no wonder you didn't find it in that command's man page.

mapfile, too is a fairly recent (v.4) addition to bash, so it's not portable or back-compatible, but it certainly is convenient. It's one of my favorite tools.
 
  


Reply

Tags
bash, linux, shell scripting, unix


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
BASH-Adding array element: Naming issue using array[${#array[*]}]=5 calvarado777 Programming 8 07-26-2013 10:48 PM
[SOLVED] reference bash array values, using a variable with a value of the array name gusthecat Programming 5 03-07-2012 04:41 PM
[SOLVED] Bash: Calculating in array and storing result in array ghantauke Linux - Newbie 6 12-02-2010 01:28 PM
[bash] indirect array reference to array with values containing spaces Meson Linux - Software 9 06-04-2010 10:38 PM
bash: use file as input into array, parse out other variables from array using awk beeblequix Linux - General 2 11-20-2009 11:07 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 06:57 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration