LinuxQuestions.org - [SOLVED] How to break array and extract individual components in a shell script

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - How to break array and extract individual components in a shell script (https://www.linuxquestions.org/questions/programming-9/how-to-break-array-and-extract-individual-components-in-a-shell-script-4175649522/)

How to break array and extract individual components in a shell script

Hey Guys,

I have a shell script below, basically, the intent of the script is to enable firewall rule on remote Windows servers for WinRM so that ANsible can perform configuration management.

Code:



#!/bin/bash



set -x

PROCESS_ID=$$



LOGFILE=/tmp/"${PROCESS_ID}"



az vm list --output=table |tee -a >> "${LOGFILE}"



az login --service-principal -u $APP_ID --password $PASSWORD --tenant $TENANT_ID



# declare an array variable

declare -a host_list=$(cat "${LOGFILE}"|grep NXT|awk '{print $1}')

echo $host_list

## now loop through the above array

for i in "${host_list[@]}"

do

case ${i} in

*AD* )

  export resource_group="rg-ad"

  ;;

*AOS* | *WEB* )

  export resource_group="rg-app"

  ;;

*SQL* | *FS* )

  export resource_group="rg-sql"

  ;;

*APP* | *STG* )

  export resource_group="rg-remote"

  ;;

*BI* | *SSRS* )

  export resource_group="rg-bi"

  ;;

*)

  Message="Invalid hostname , please check"

  ;;

esac

  echo "Installing Windows Remote Management $i with resource group $resource_group "





  az vm run-command invoke --command-id RunPowerShellScript --resource-group "${resource_group}" --name "${i}" --scripts '(New-Object -TypeName System.Net.WebClient).DownloadFile("https://raw.githubusercontent.com/ansible/ansible/devel/examples/scripts/ConfigureRemotingForAnsible.ps1", "$env:temp\ConfigureRemotingForAnsible.ps1")'



  az vm run-command invoke --command-id RunPowerShellScript --resource-group "${resource_group}" --name "${i}" --scripts 'powershell.exe -ExecutionPolicy ByPass -File "$env:temp\ConfigureRemotingForAnsible.ps1"'

  az vm run-command invoke --command-id RunPowerShellScript --resource-group "${resource_group}" --name "${i}" --scripts 'netsh advfirewall firewall add rule name="Allow WinRM (Http)" dir=in localport=5985 protocol=tcp action=allow enable=yes'

  az vm run-command invoke --command-id RunPowerShellScript --resource-group "${resource_group}" --name "${i}" --scripts 'netsh advfirewall firewall add rule name="Allow WinRM (Https)" dir=in localport=5986 protocol=tcp action=allow enable=yes'



done



exit 0

When it is executed

Code:

./temp.sh

+ PROCESS_ID=13879

+ LOGFILE=/tmp/13879

+ az vm list --output=table

+ tee -a

+ az login "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"

[

  {

    "cloudName": "AzureCloud",

      "type": "servicePrincipal"

    }

  }

]

++ cat /tmp/13879

++ awk '{print $1}'

++ grep NXT

+ declare -a 'host_list=NXTPREPAOS-1

NXTPREPAOS-2

NXTPREPAOS-3

NXTPREPAOS-4

NXTPREPAOS-5

NXTPREPSQL-1

NXTPREPSQL-2'

+ echo NXTPREPAOS-1 NXTPREPAOS-2 NXTPREPAOS-3 NXTPREPAOS-4 NXTPREPAOS-5 NXTPREPSQL-1 NXTPREPSQL-2

NXTPREPAOS-1 NXTPREPAOS-2 NXTPREPAOS-3 NXTPREPAOS-4 NXTPREPAOS-5 NXTPREPSQL-1 NXTPREPSQL-2

+ for i in '"${host_list[@]}"'

+ case ${i} in

+ export resource_group=rg-app

+ resource_group=rg-app

+ echo 'Installing Windows Remote Management NXTPREPAOS-1

NXTPREPAOS-2

NXTPREPAOS-3

NXTPREPAOS-4

NXTPREPAOS-5

NXTPREPSQL-1

NXTPREPSQL-2 with resource group rg-app '

Installing Windows Remote Management NXTPREPAOS-1

NXTPREPAOS-2

NXTPREPAOS-3

NXTPREPAOS-4

NXTPREPAOS-5

NXTPREPSQL-1

NXTPREPSQL-2 with resource group rg-app

+ az vm run-command invoke --command-id RunPowerShellScript --resource-group rg-app --name 'NXTPREPAOS-1

NXTPREPAOS-2

NXTPREPAOS-3

NXTPREPAOS-4

NXTPREPAOS-5

NXTPREPSQL-1

NXTPREPSQL-2' --scripts '(New-Object -TypeName System.Net.WebClient).DownloadFile("https://raw.githubusercontent.com/ansible/ansible/devel/examples/scripts/ConfigureRemotingForAnsible.ps1", "$env:temp\ConfigureRemotingForAnsible.ps1")'

Operation failed with status: 'Bad Request'. Details: 400 Client Error: Bad Request for url: https://management.azure.com/subscriptions/xxxxxxxxxxxxxxxxxxxxxxxxx/resourceGroups/rg-app/providers/Microsoft.Compute/virtualMachines/NXTPREPAOS-1%0ANXTPREPAOS-2%0ANXTPREPAOS-3%0ANXTPREPAOS-4%0ANXTPREPAOS-5%0ANXTPREPSQL-1%0ANXTPREPSQL-2/runCommand?api-version=2018-10-01

I want to dynamically get the no of VM's as it could change as per requirement. I know the problem is a whitespace character in the array, how to solve?

Made it simpler, all what we want is individual hostnames from an array.

Code:

#!/bin/bash

set -x

PROCESS_ID=$$



LOGFILE=/tmp/"${PROCESS_ID}"



az vm list --output=table |tee -a >> "${LOGFILE}"



az login "xxxxxxxxxxxxxx"



# declare an array variable

declare -a host_list=$(cat "${LOGFILE}"|grep NXT|awk '{print $1}')

echo $host_list



## now loop through the above array

for i in "${host_list[@]}"

do

targetedhost=$(echo $i|cut -d ' ' -f1)

case ${targetedhost} in

*AD* )

  export resource_group="rg-ad"

  ;;

*AOS* | *WEB* )

  export resource_group="rg-app"

  ;;

*SQL* | *FS* )

  export resource_group="rg-sql"

  ;;

*APP* | *STG* )

  export resource_group="rg-remote"

  ;;

*BI* | *SSRS* )

  export resource_group="rg-bi"

  ;;

*)

  Message="Invalid hostname , please check"

  ;;

esac

  echo "Installing Windows Remote Management $targetedhost with resource group $resource_group "



done



exit 0

Output: Looks good but it not going for next hostname until end of the array:

Code:

./temp.sh

+ PROCESS_ID=15311

+ LOGFILE=/tmp/15311

+ az vm list --output=table

+ tee -a

+ az login "xxxxxxxxxxxxxxxxxxxxxxxxxxx"

[

  {

    "cloudName": "AzureCloud",

    "name": "Visual Studio Enterprise",

    }

  }

]

++ cat /tmp/15311

++ grep NXT

++ awk '{print $1}'

+ declare -a 'host_list=NXTPREPAOS-1

NXTPREPAOS-2

NXTPREPAOS-3

NXTPREPAOS-4

NXTPREPAOS-5

NXTPREPSQL-1

NXTPREPSQL-2'

+ echo NXTPREPAOS-1 NXTPREPAOS-2 NXTPREPAOS-3 NXTPREPAOS-4 NXTPREPAOS-5 NXTPREPSQL-1 NXTPREPSQL-2

NXTPREPAOS-1 NXTPREPAOS-2 NXTPREPAOS-3 NXTPREPAOS-4 NXTPREPAOS-5 NXTPREPSQL-1 NXTPREPSQL-2

+ for i in '"${host_list[@]}"'

++ echo NXTPREPAOS-1 NXTPREPAOS-2 NXTPREPAOS-3 NXTPREPAOS-4 NXTPREPAOS-5 NXTPREPSQL-1 NXTPREPSQL-2

++ cut -d ' ' -f1

+ targetedhost=NXTPREPAOS-1

+ case ${targetedhost} in

+ export resource_group=rg-app

+ resource_group=rg-app

+ echo 'Installing Windows Remote Management NXTPREPAOS-1 with resource group rg-app '

Installing Windows Remote Management NXTPREPAOS-1 with resource group rg-app

+ exit 0

As you would see above we got the hostname and it finished it need to loop through, why it is not going through?

Your quoted array is not splitting, is this intentional?

For example.

Code:

$ foo='this is a test'

$ for i in ${foo[@]}; do echo $i; done

this

is

a

test

$ for i in "${foo[@]}"; do echo $i; done

this is a test

@orbea

Thanks, heaps for that, I cannot believe that double quotes around the array were causing the grief! I am now confused, normally it a good practice to reference a variable like "${env}", so this does not hold true for an array?

Please enlighten

Thank you

It depends your goal: it you don't wish your variable space-splitted, use qutes: "$env", otherwise don't: $env or ${env}

Aah! Got it, Thanks @Nevem Teve!

This is an extremely bad example.

You should in fact double quote if it was actually an array.
As it is a simple string the ${[@]} expander is not being used correctly hence the output issues.
Here is a quick demo using orbea's example:

Code:

foo='this is a test'

for i in ${foo[@]}; do echo $i; done

for i in "${foo[@]}"; do echo $i; done





fooa=(this 'is a' test)



for i in ${fooa[@]}; do echo $i; done

for i in "${fooa[@]}"; do echo $i; done

The additional quotes inside the array are to prove why you need quotes around an array

It would be nice to see an mcve.
https://stackoverflow.com/help/mcve

Quote:

Originally Posted by grail (Post 5970214)

The additional quotes inside the array are to prove why you need quotes around an array

Thanks for the additional examples, this is what I get for almost never using bash specific arrays. :)

In this case, it probably makes more sense to use a while read loop (and drop useless use of cat and grep):

Code:

awk '/NXT/ {print $1}' "$LOGFILE" | while read i