[SOLVED] Bash

Lucien Lachance · 08-02-2013, 09:17 AM

Is there any possible way I can shorten this array in Bash? I noticed that Bash didn't have a way of expanding symbols, so I was forced to hard code each individual symbol. I need to have A-Z, a-z, 0-9, and symbols.

Code:

pass_wd=(
  {0..9}
  {a..z}
  {A..Z}
  '!'
  '"'
  '#'
  '$'
  '%'
  '&'
  '\'
  '('
  ')'
  '*'
  '+'
  ','
  '-'
  '.'
  '/'
  ':'
  ';'
  '<'
  '='
  '>'
  '?'
  '@'
  '['
  '\'
  '\'
  ']'
  '^'
  '_'
  '`'
  '{'
  '|'
  '}'
  '~'                                                                                                                                                                    
)

olau · 08-02-2013, 09:46 AM

Hmm,
I think we need to understand what you want to do with such an array.
It seems you are looking for a regex representing all this characters.
When this assumption is correct you may look for the dot sign? ( '.' )
Cheers

Lucien Lachance · 08-02-2013, 10:09 AM

I've written a script that randomly shuffles this array. As of now it just produces passwords with 0-9, a-z, and A-Z. I want to incorporate symbols inside of this array. It's just an array of alphanumeric characters. In python, I can easily encapsulate these characters by saying

Code:

c = string.ascii_letters + string.punctuation
print c

Output:
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!"#$%&'()*+,-./:;<=>?@[\]^_`{|}~

I just want to know is there a way of storing all of these characters inside of array that doesn't produce all of this clutter.

firstfire · 08-02-2013, 11:41 AM

Hi.

It is easy to obtain such a string in bash:

Code:

$ x="$(printf '\\x%x ' {33..126})"
$ echo -e $x
! " # $ % & ' ( ) * + , - . / 0 1 2 3 4 5 6 7 8 9 : ; < = > ? @ A B C D E F G H I J K L M N O P Q R S T U V W X Y Z [ \ ] ^ _ ` a b c d e f g h i j k l m n o p q r s t u v w x y z { | } ~

But to put each character into an array separately one should disable pathname expansion (because of `*' character), while keep word splitting:

Code:

$ set -o noglob    # disable pathname expansion
$ x="$(printf '\\x%x ' {33..126})"
$ A=( $(echo -e "$x") )
$ set +o noglob    # enable pathname expansion
$ echo "${A[9]}"
*
$ echo "${A[*]}"
! " # $ % & ' ( ) * + , - . / 0 1 2 3 4 5 6 7 8 9 : ; < = > ? @ A B C D E F G H I J K L M N O P Q R S T U V W X Y Z [ \ ] ^ _ ` a b c d e f g h i j k l m n o p q r s t u v w x y z { | } ~
$ declare -p A
declare -a A='([0]="!" [1]="\"" [2]="#" [3]="\$" [4]="%" [5]="&" [6]="'\''" [7]="(" [8]=")" [9]="*" [10]="+" [11]="," [12]="-" [13]="." [14]="/" [15]="0" [16]="1" [17]="2" [18]="3" [19]="4" [20]="5" [21]="6" [22]="7" [23]="8" [24]="9" [25]=":" [26]=";" [27]="<" [28]="=" [29]=">" [30]="?" [31]="@" [32]="A" [33]="B" [34]="C" [35]="D" [36]="E" [37]="F" [38]="G" [39]="H" [40]="I" [41]="J" [42]="K" [43]="L" [44]="M" [45]="N" [46]="O" [47]="P" [48]="Q" [49]="R" [50]="S" [51]="T" [52]="U" [53]="V" [54]="W" [55]="X" [56]="Y" [57]="Z" [58]="[" [59]="\\" [60]="]" [61]="^" [62]="_" [63]="\`" [64]="a" [65]="b" [66]="c" [67]="d" [68]="e" [69]="f" [70]="g" [71]="h" [72]="i" [73]="j" [74]="k" [75]="l" [76]="m" [77]="n" [78]="o" [79]="p" [80]="q" [81]="r" [82]="s" [83]="t" [84]="u" [85]="v" [86]="w" [87]="x" [88]="y" [89]="z" [90]="{" [91]="|" [92]="}" [93]="~")'

Of course you may also iterate over $x (it contains character codes) and put each character into an array separately. Or you may process `*' separately.

Hope this helps.

EDIT: here is the ASCII code table http://www.ascii-code.net/ . It will (hopefully) make clear why I use that {33..126} range.

konsolebox · 08-02-2013, 01:56 PM

Good point by firstfire. I've been looking at printf but never saw a trick like it, which I probably can't unless I searched. Anyway you don't need to disable pathname expansion. You could just use read and process substitution:

Code:

IFS=$'\xFF' read -ra A < <(echo -e "$(printf '\\x%x\\xFF' {33..126})")

Of course since we are converting a string to an array and could only have done it with a separator, we'll have to use a character besides 0x00 which doesn't seem to work, like 0xFF.

I might find this useful to my scripts someday as well thanks.

For convenience we could use a function like this to generate characters to an array:

Code:

# syntax: generate_chars <avar> <first ascii code> <last ascii code>
#
function generate_chars {
    IFS=$'\xFF' read -ra "$1" < <(echo -e "$(eval "printf '\\\\x%x\\\\xFF' {$2..$3}")")
}

Or

Code:

# syntax: generate_chars <avar> <ascii code> [<ascii code 2> ...]
#
function generate_chars {
    IFS=$'\xFF' read -ra "$1" < <(echo -e "$(printf '\\x%x\\xFF' "${@:2}")")
}

firstfire · 08-02-2013, 02:55 PM

konsolebox, nice trick with reading directly into an array!

But I'd like to stress that in this particular case the SPACE character (decimal 32) is outside of the range in question (passwords with spaces is a bad idea?), so your command may be simplified to

Code:

read -a A <<< $(echo -e "$(printf '\\x%x ' {33..126})")

Default IFS works just fine (in this case).

konsolebox · 08-02-2013, 03:19 PM

@firstfire Yes, I did expect the obvious and also tried to quote the possibility of spaces at first and that they would be ignored if another range is used, but finding another tweak I completely discarded the note and suggested the use of another IFS value if needed. If someone would hit 0x20, 0x09 and even 0x0A, then they would be discarded. Also, creating a general function for that would not be possible.

Lucien Lachance · 08-02-2013, 03:19 PM

Some good stuff floating around, I'll take a look at some your solutions in just a minute! In the mean time you can see my script here: https://github.com/Demet19/bash_scri...master/pass_wd

konsolebox · 08-02-2013, 03:33 PM

Quote:

Originally Posted by firstfire

Code:

read -a A <<< $(echo -e "$(printf '\\x%x ' {33..126})")

And yes using <<< $() over < <() is a good preference since you no longer have to summon a named pipe, though -r for read is still necessary since using it would not interpret \ as another element and quote the space after instead. Notice that doing echo "${A[@]}" after it, \ is no longer included.

Lucien Lachance · 08-02-2013, 03:40 PM

Wow, I'm blown away as to how you've set that up ^ using printf. Can you step by step explain what this is doing? I'd really appreciate that.

Code:

read -a A <<< $(echo -e "$(printf '\\x%x ' {33..126})")

konsolebox · 08-02-2013, 03:55 PM

If we go step by step printf would generate this:

Code:

\x21 \x22 \x23 \x24 \x25 \x26 \x27 \x28 \x29 \x2a \x2b \x2c \x2d \x2e \x2f \x30 \x31 \x32 \x33 \x34 \x35 \x36 \x37 \x38 \x39 \x3a \x3b \x3c \x3d \x3e \x3f \x40 \x41 \x42 \x43 \x44 \x45 \x46 \x47 \x48 \x49 \x4a \x4b \x4c \x4d \x4e \x4f \x50 \x51 \x52 \x53 \x54 \x55 \x56 \x57 \x58 \x59 \x5a \x5b \x5c \x5d \x5e \x5f \x60 \x61 \x62 \x63 \x64 \x65 \x66 \x67 \x68 \x69 \x6a \x6b \x6c \x6d \x6e \x6f \x70 \x71 \x72 \x73 \x74 \x75 \x76 \x77 \x78 \x79 \x7a \x7b \x7c \x7d \x7e

And echo -e would interpret that and convert to actual ascii characters.

Code:

! " # $ % & ' ( ) * + , - . / 0 1 2 3 4 5 6 7 8 9 : ; < = > ? @ A B C D E F G H I J K L M N O P Q R S T U V W X Y Z [ \ ] ^ _ ` a b c d e f g h i j k l m n o p q r s t u v w x y z { | } ~

After that, read -a A would convert the string to an array and save it to A through which each element is separated by the values of IFS respectively. As I had quoted, -r is need to parse backslashes as well.

Code:

read -ra A <<< $(echo -e "$(printf '\\x%x ' {33..126})")

If we are to read a character which is part of IFS, we have to change IFS too and use a different separator like what I did above.

Code:

IFS=$'\xFF' read -ra A < <(echo -e "$(printf '\\x%x\\xFF' {33..126})")

$'\0' with \\x00 doesn't work so I chose xFF which is rarely used as a character. This could be made configurable as well for the extremes.

Btw, make sure to give the credit for echo -e and printf to firstfire for it was his idea. I just added the easier way to convert it to arrays.

David the H. · 08-02-2013, 04:39 PM

I'd even take it one step further.

Code:

mapfile -t chars < <( printf '%b\n' \\0{4..7}{0..7} \\1{0..7}{0..7} )

unset chars[0] chars[${#chars[@]}-1]

printf '%s\n' "${chars[@]}"

%b directly expands escape sequences, just like echo -e. All we have to do is make sure the input strings are already in \0nn (octal) or \xnn (hex) format. I went with octal, and I also found it easier to just unset the first and last characters of the range (040 and 177, space and delete) afterwards rather than further modifying the brace expansions. Finally, mapfile gives us a convenient way to read the input into the array that doesn't require fiddling with IFS or anything.

Edit: Actually, the hex values are easier to generate:

Code:

printf '%b\n' \\x{2..7}{{0..9},{A..F}}

Lucien Lachance · 08-02-2013, 04:41 PM

Thanks so much for that, I didn't know printf was so powerful. Also, thanks @fistfire for explaining the meaning behind the ascii values. I was thinking about doing that with a while loop and then appending those characters to it, but you've given me a better look at the problem.

firstfire · 08-02-2013, 04:54 PM

Quote:

Originally Posted by konsolebox

though -r for read is still necessary since using it would not interpret \ as another element and quote the space after instead. Notice that doing echo "${A[@]}" after it, \ is no longer included.

Yes, you are right. Nice catch!

David, thanks for noticing the %b format specifier. I managed to overlook it in man printf, though read it many times today. The mapfile is also a handy thing.

David the H. · 08-02-2013, 05:12 PM

%b is a bash/ksh-only thing, and not in /usr/bin/printf, so it's no wonder you didn't find it in that command's man page.

mapfile, too is a fairly recent (v.4) addition to bash, so it's not portable or back-compatible, but it certainly is convenient. It's one of my favorite tools.