LinuxQuestions.org - [SOLVED] help explain my error in this array for BASH

- Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)

- - help explain my error in this array for BASH (https://www.linuxquestions.org/questions/linux-newbie-8/help-explain-my-error-in-this-array-for-bash-4175427053/)

help explain my error in this array for BASH

Continuing from the work of yesterday, btw thanks again for all of the great help, I am onto the next step.

Here is what I attempted, but i ended with some odd results.

Code:

[103236_rx30@rx30 ~]$ sh -xvv DEA-MOVE

#!/bin/bash



[ ! -d "/tmp/DEA" ] && mkdir -p /tmp/DEA

+ '[' '!' -d /tmp/DEA ']'



filemove ()



{

for f in deaopts.*.orig

do

        newf=$(echo $f | tr '[:lower:]' '[:upper:]')

        mv "$f" "$newf"

done

}





deaclean()



{



for f in deaopts.*

do

        set - `cat $f`

        find ${1:3:3}.* -mtime +15 -exec mv '{}' /tmp/DEA \;

done



}





filemove

+ filemove

+ for f in 'deaopts.*.orig'

echo $f | tr '[:lower:]' '[:upper:]'

++ echo 'deaopts.*.orig'

++ tr '[:lower:]' '[:upper:]'

+ newf='DEAOPTS.*.ORIG'

+ mv 'deaopts.*.orig' 'DEAOPTS.*.ORIG'

mv: cannot stat `deaopts.*.orig': No such file or directory



deaclean

+ deaclean

+ for f in 'deaopts.*'

cat $f

++ cat deaopts.NY

+ set - $'\005ND1234567' XXXXXXXX234 '!NNYB' $'YN@\246PPY\001NN' N 'l▒`▒'

find: 123.*: No such file or directory

find: 123.*: No such file or directory

find: FN1.*: No such file or directory

find: ./cnv37db: No such file or directory

find: ./rx30rem: No such file or directory

find: ./.macromedia: No such file or directory

find: ./posout/failed: No such file or directory



[103236_rx30@rx30 ~]$

[103236_rx30@rx30 ~]$ ls -la deaopts.*

-rwxrwxr-x 1 rx30 group 183 Sep 12 11:12 deaopts.NY

-rwxrwxr-x 1 rx30 group 183 May 16 10:28 deaopts.OK

-rwxrwxr-x 1 rx30 group 183 Sep 12 12:35 deaopts.cnf

-rwxrwxr-x 1 rx30 group  23 Mar  8  2012 deaopts.fil

I also would like to ignore the deaopts.fil if i can in this array.

the first 3 results are as expected, but why did it continue to try to move everything.... I only want to move the files from the form inside of the deaopts.NY/OK/cnf on this computer.

Code:

[103236_rx30@rx30 ~]$ cat DEA-MOVE

#!/bin/bash



[ ! -d "/tmp/DEA" ] && mkdir -p /tmp/DEA



filemove ()



{

for f in deaopts.*.orig

do

        newf=$(echo $f | tr '[:lower:]' '[:upper:]')

        mv "$f" "$newf"

done

}





deaclean()



{



for f in deaopts.*

do

        set - `cat $f`

        find ${1:3:3}.* -mtime +15 -exec mv '{}' /tmp/DEA \;

done



}





filemove



deaclean

thanks in advance for the help and guidance.

You're not actually using any arrays (unless I'm missing something major)...

If you're referring to the 'find' functionality, you're not passing anything to 'deaclean' (is this your own code?) so ${1:3:3} expands to the empty string, and so find is matching '.*' - that is, anything which is hidden.

deaopts.* is a list of files that contain data. when you set - `cat deaopts.foo` then you can echo out the string i am using and it will grab the first item in the file and print out the 3rd, - 5th char of that item.

So I seem to be a little lost with what this process is trying to do, so I figure I will just look at the errors and advise:

Code:

++ echo 'deaopts.*.orig'

The above indicates that you have no files of the above format and hence the asterisk has been left intact which means that mv has nothing to move ... hence the error.
As solutions I would do the following 2 things:

1. Add shopt -s noglob to the start of the script and that way if the glob expands to nothing you will not enter the loop

2. Instead of wasting a call to tr, simply use parameter substitution:

Code:

mv "$f" "${f^^}"

Next, assuming the following files exist as per your ls:

Code:

deaopts.NY

deaopts.OK

deaopts.cnf

deaopts.fil

I do not follow the error you are showing (unless you have somehow edited it):

Code:

++ cat deaopts.NY

+ set - $'\005ND1234567' XXXXXXXX234 '!NNYB' $'YN@\246PPY\001NN' N 'l▒`▒'

find: 123.*: No such file or directory

find: 123.*: No such file or directory

find: FN1.*: No such file or directory

find: ./cnv37db: No such file or directory

find: ./rx30rem: No such file or directory

find: ./.macromedia: No such file or directory

find: ./posout/failed: No such file or directory

The errors related to '123.*' - 123 is not in the position you refer to with ${1:3:3} (this would actually give you '005'), so I do not understand how this would ever be issued.

As for the rest, apparently no other file is passed to set via cat and none of the other entries are in your output data???

Snark's explanation does hold some validity in explaining the last 4 entries which do match the glob of '.*', however I again do not follow why you would get to the last
entry using the previous argument as it is not in the current directory.

Maybe you should try to explain the purpose of the script before we try to assist in the errors?

Quote:

Originally Posted by grail (Post 4779482)

So I seem to be a little lost with what this process is trying to do, so I figure I will just look at the errors and advise:

Code:

++ echo 'deaopts.*.orig'

Code:

mv "$f" "${f^^}"

this is a nice simple little tool... Ill use that over the tr. neither should be heavy on the system as at the most there will be 50 files that get moved from lower to upper case. one for each of the states.

Quote:

Next, assuming the following files exist as per your ls:

Code:

deaopts.NY

deaopts.OK

deaopts.cnf

deaopts.fil

I do not follow the error you are showing (unless you have somehow edited it):

Code:

++ cat deaopts.NY

+ set - $'\005ND1234567' XXXXXXXX234 '!NNYB' $'YN@\246PPY\001NN' N 'l▒`▒'

find: 123.*: No such file or directory

find: 123.*: No such file or directory

find: FN1.*: No such file or directory

find: ./cnv37db: No such file or directory

find: ./rx30rem: No such file or directory

find: ./.macromedia: No such file or directory

find: ./posout/failed: No such file or directory

the error was that it was looking for files that are not in the list. the:

Code:

find: 123.*: No such file or directory

find: 123.*: No such file or directory

find: FN1.*: No such file or directory

were expected as they have been moved, or are not old enough to be touched by the find command, but the last set of entries i have to ctrl+c to stop the script from trying to move those files. That was my error.

What I have come up with is the following:

Code:

#!/bin/bash



[ ! -d "/tmp/DEA" ] && mkdir -p /tmp/DEA

rm -rf deaopts.fil >/dev/null



filemove ()



{

for f in deaopts.*.orig

do

        newf=$(echo $f | tr '[:lower:]' '[:upper:]')

        mv "$f" "$newf"

done

}





deaclean()



{



for f in deaopts.*

do

#      set - `cat $f`

        LINE=`head -n 1 $f`

        find ${LINE:3:3}.* -mtime +15 -exec mv '{}' /tmp/DEA \;

done



}





filemove



deaclean

It only has 1 little error. at the end of the script i get this:

Code:

[103236_rx30@rx30 ~]$ sh -xvv DEA-MOVE

#!/bin/bash



[ ! -d "/tmp/DEA" ] && mkdir -p /tmp/DEA

+ '[' '!' -d /tmp/DEA ']'

rm -rf deaopts.fil >/dev/null

+ rm -rf deaopts.fil



filemove ()



{

for f in deaopts.*.orig

do

        newf=$(echo $f | tr '[:lower:]' '[:upper:]')

        mv "$f" "$newf"

done

}





deaclean()



{



for f in deaopts.*

do

#      set - `cat $f`

        LINE=`head -n 1 $f`

        find ${LINE:3:3}.* -mtime +15 -exec mv '{}' /tmp/DEA \;

done



}





filemove

+ filemove

+ for f in 'deaopts.*.orig'

echo $f | tr '[:lower:]' '[:upper:]'

++ echo 'deaopts.*.orig'

++ tr '[:lower:]' '[:upper:]'

+ newf='DEAOPTS.*.ORIG'

+ mv 'deaopts.*.orig' 'DEAOPTS.*.ORIG'

mv: cannot stat `deaopts.*.orig': No such file or directory



deaclean

+ deaclean

+ for f in 'deaopts.*'

head -n 1 $f

++ head -n 1 deaopts.NY

+ LINE='ND1234567            NNYNNN234 !NNYB  YN@▒PPYNN                N          l▒`▒'

PuTTY+ find '123.*' -mtime +15 -exec mv '{}' /tmp/DEA ';'

find: 123.*: No such file or directory

+ for f in 'deaopts.*'

head -n 1 $f

++ head -n 1 deaopts.OK

+ LINE='ND1234567            NYDNNN234 %NNNB  YNNNN                N            '

+ find '123.*' -mtime +15 -exec mv '{}' /tmp/DEA ';'

find: 123.*: No such file or directory

+ for f in 'deaopts.*'

head -n 1 $f

++ head -n 1 deaopts.cnf

+ LINE='NDFN1236644          NNDYYN234 %NNYB  YN▒▒PPYNN                NPHY44957  l▒`▒'

+ find 'FN1.*' -mtime +15 -exec mv '{}' /tmp/DEA ';'

find: FN1.*: No such file or directory

[103236_rx30@rx30 ~]$ PuTTY

Now I can hit ESC or just backspace to clear up the PuTTY put on the screen by the script, but why is that showing on the screen? Is it due to not having an exit; at the end of the script?

To answer question as to the goal of the script.

I have a bunch of servers out there that are running some bad code (from long before i came to this company) that leaves the stores DEA reports in their $HOME directory. These clutter up the directory as well as make it a royal PITA for data transfer and backups.

My goal is to clean up the $HOME directory of these stores and the file names used are as you see above. Yes i have obfuscated some of the above data, but the output you are seeing is from a test server without any real data from customers (they are all made up without real data) and the DEA#s are not real either. Thus semi safe to post :D>

Each state has its own requirements for the file format, thus the deaopts.XX (NY, FL, etc... as you see above) ((side note, turns out the .fil is no longer used and should be removed thus my rm now at the top of the script)) the need for the 3rd - 5th chars of the first line in the deaopts.XX is the file format for that state. the first 2 chars are meaningless junk for my script and thus can be ignored.

We do need to keep files that are younger then 15 days as several pharmacies will need to either reprint or retransmit that exact report. After 15days the reports are worthless to the store.

Again thanks for all of the great help.

i must of messed something up with the f^^

Code:

filemove ()



{

for f in deaopts.*.orig

do

        #newf=$(echo $f | tr '[:lower:]' '[:upper:]')

        #mv "$f" "$newf"

        mv "$f" "${f^^}"

done

}





deaclean()



{



for f in deaopts.*

do

#      set - `cat $f`

        LINE=`head -n 1 $f`

        find ${LINE:3:3}.* -mtime +15 -exec mv '{}' /tmp/DEA \;

done



}





filemove

+ filemove

+ for f in 'deaopts.*.orig'

DEA-MOVE: line 13: ${f^^}: bad substitution

^^ was added in version 4 of bash, are you perhaps using an earlier version?

I see you've marked the thread as 'SOLVED' - well done for solving it, could you post something explaining to anyone who might have had a similar problem exactly what you did to solve it?

Thanks,

the final working code:

Code:

[103236_rx30@rx30 ~]$ cat DEA-MOVE

#!/bin/bash



license=`showlic | grep LICENSE | awk '{print $6}'`

DIR=/tmp/DEA-${license}

VAR=`cat deafile.out`



[ ! -d "${DIR}" ] && mkdir -p ${DIR}



if [ -f deaopts.fil ]

then

        rm -rf deaopts.fil

fi



if [ -f PMP*.dat ]

then

        find PMP*.dat -mtime +15 -exec mv '{}' ${DIR} \;

fi



if [ -f 20[0-9]*.dat ]

then

        find 20[0-9]*.dat -mtime +15 -exec mv '{}' ${DIR} \;

fi



if [ -f 20[0-9]*.txt ]

then

        find 20[0-9]*.txt -mtime +15 -exec mv '{}' ${DIR} \;

fi



filemove ()



{

for f in deaopts.*.orig

do

        newf=$(echo $f | tr '[:lower:]' '[:upper:]')

        mv "$f" "$newf"

        #mv "$f" "${f^^}"

done

}





deaclean()



{



for f in deaopts.*

do

        LINE=`head -n 1 $f`

        find ${LINE:3:3}*.* -mtime +15 -exec mv '{}' ${DIR} \;



        find ${VAR} -mtime +15 -exec mv '{}' ${DIR} \;

done



}





filemove



deaclean



exit

We could perhaps make it a little simpler:

Code:

#!/bin/bash



shopt -s nullglob



DIR=/tmp/DEA-$(showlic | awk '/LICENSE/{print $6}')

VAR=$(< deafile.out)



move_files=( PMP*.dat 20[0-9]*.dat 20[0-9]*.txt )



[[ -d "$DIR" ]] || mkdir -p "$DIR"



find "${move_files[@]}" -mtime +15 -exec mv '{}' "$DIR" \;



filemove()

{

    for f in deaopts.*.orig

    do

        newf=$(echo "$f" | tr a-z A-Z)

        mv "$f" "$newf"

    done

}



deaclean()

{



    for f in deaopts.*

    do

        LINE=$(head -n 1 "$f")

    

        find "${LINE:3:3}*.*" -mtime +15 -exec mv '{}' "$DIR" \;



        find ${VAR} -mtime +15 -exec mv '{}' ${DIR} \;

    done

}





filemove



deaclean



exit

You may also need to think about what happens if the first line of the file is empty??

very interesting ways you were able to tie stuff together. i did not know you could do that. very nice.

what is this bit of code:

Code:

shopt -s nullglob

nm, man shopt showed me what i needed to see on the nullglob. ty

i like that should help clean things up a bit.

question and problem with the code...

Code:

[103236_rx30@rx30 ~]$ vi DEA2-MOVE

#!/bin/bash



shopt -s nullglob



DIR=/tmp/DEA-$(showlic | awk '/LICENSE/{print $6}')

VAR=$(< deafile.out)



move_files=( PMP*.dat 20[0-9]*.dat 20[0-9]*.txt )



[[ -d "$DIR" ]] || mkdir -p "$DIR"



find "${move_files[@]}" -mtime +15 -exec mv '{}' "$DIR" \;



filemove()

{

    for f in deaopts.*.orig

    do

        newf=$(echo "$f" | tr a-z A-Z)

        mv "$f" "$newf"

    done

}



deaclean()

{



    for f in deaopts.*

    do

        LINE=$(head -n 1 "$f")



        find "${LINE:3:3}*.*" -mtime +15 -exec mv '{}' "$DIR" \;



        find ${VAR} -mtime +15 -exec mv '{}' ${DIR} \;

    done

}





filemove



deaclean



exit

~

~

~

~

~

~

~

~

~

[103236_rx30@rx30 ~]$ VAR=$(< deafile.out)

[103236_rx30@rx30 ~]$ echo $VAR

FN1236644*.dat*

[103236_rx30@rx30 ~]$ DIR=/tmp/DEA-$(showlic | awk '/LICENSE/{print $6}')

[103236_rx30@rx30 ~]$ echo $DIR

/tmp/DEA-103236

[103236_rx30@rx30 ~]$ [[ -d "$DIR" ]] || mkdir -p "$DIR"

[103236_rx30@rx30 ~]$ move_files=( PMP*.dat 20[0-9]*.dat 20[0-9]*.txt )

[103236_rx30@rx30 ~]$ find "${move_files[@]}" -mtime +15 -exec mv '{}' "$DIR" \;

find: PMP*.dat: No such file or directory

find: 20[0-9]*.dat: No such file or directory

find: 20[0-9]*.txt: No such file or directory

[103236_rx30@rx30 ~]$ for f in deaopts.*

> do

> LINE=$(head -n 1 "$f")

> find "${LINE:3:3}*.*" -mtime +15 -exec mv '{}' "$DIR" \;

> find ${VAR} -mtime +15 -exec mv '{}' ${DIR} \;

> done

find: 123*.*: No such file or directory

find: FN1236644*.dat*: No such file or directory

find: 123*.*: No such file or directory

find: FN1236644*.dat*: No such file or directory

find: FN1*.*: No such file or directory

find: FN1236644*.dat*: No such file or directory

[103236_rx30@rx30 ~]$ sh -xvv DEA2-MOVE

#!/bin/bash



shopt -s nullglob

+ shopt -s nullglob



DIR=/tmp/DEA-$(showlic | awk '/LICENSE/{print $6}')

showlic | awk '/LICENSE/{print $6}'

++ showlic

++ awk '/LICENSE/{print $6}'

+ DIR=/tmp/DEA-103236

VAR=$(< deafile.out)

< deafile.out

+ VAR='FN1236644*.dat*'



move_files=( PMP*.dat 20[0-9]*.dat 20[0-9]*.txt )

+ move_files=(PMP*.dat 20[0-9]*.dat 20[0-9]*.txt)



[[ -d "$DIR" ]] || mkdir -p "$DIR"

+ [[ -d /tmp/DEA-103236 ]]



find "${move_files[@]}" -mtime +15 -exec mv '{}' "$DIR" \;

+ find -mtime +15 -exec mv '{}' /tmp/DEA-103236 ';'

find: ./Rx30POS: No such file or directory

find: ./posin: No such file or directory



[103236_rx30@rx30 ~]$ sh -xvv DEA2-MOVE

#!/bin/bash



shopt -s nullglob

+ shopt -s nullglob



DIR=/tmp/DEA-$(showlic | awk '/LICENSE/{print $6}')

showlic | awk '/LICENSE/{print $6}'

++ showlic

++ awk '/LICENSE/{print $6}'

+ DIR=/tmp/DEA-103236

VAR=$(< deafile.out)

< deafile.out

+ VAR='FN1236644*.dat*'



move_files=( PMP*.dat 20[0-9]*.dat 20[0-9]*.txt )

+ move_files=(PMP*.dat 20[0-9]*.dat 20[0-9]*.txt)



[[ -d "$DIR" ]] || mkdir -p "$DIR"

+ [[ -d /tmp/DEA-103236 ]]



find "${move_files[@]}" -mtime +15 -exec mv '{}' "$DIR" \;

+ find -mtime +15 -exec mv '{}' /tmp/DEA-103236 ';'

mv: inter-device move failed: `./fax' to `/tmp/DEA-103236/fax'; unable to remove target: Is a directory

find: ./jwc_lab: No such file or directory

find: ./tdsinstall: No such file or directory

find: ./logs: No such file or directory

find: ./modem: No such file or directory

find: ./dispill: No such file or directory

find: ./docustore: No such file or directory

find: ./emar/lap: No such file or directory

mv: setting attribute `security.selinux' for `security.selinux': Operation not permitted

mv: setting attribute `security.selinux' for `security.selinux': Operation not permitted

if i run the parts from the $ prompt they work, but in the script the array is not building properly... is that due to the order of the || mkdir being between the array and the for loop?

Code:

in the script the array is not building properly

The current format for building the array assumes you run it from the directory where those files are located, which from your previous example code
would seem to be the logic you were following.

If this is not the case or you are running from there and still getting errors, you will need to provide more detail on what error you are seeing / receiving?

it is run from the same directory, but if you look at the full code i posted above i ran the script in sh -xvv so you could see the output, why is is looking to move all of the files in that directory?

Code:

find "${move_files[@]}" -mtime +15 -exec mv '{}' "$DIR" \;

+ find -mtime +15 -exec mv '{}' /tmp/DEA-103236 ';'

mv: inter-device move failed: `./fax' to `/tmp/DEA-103236/fax'; unable to remove target: Is a directory

find: ./jwc_lab: No such file or directory

find: ./tdsinstall: No such file or directory

find: ./logs: No such file or directory

find: ./modem: No such file or directory

find: ./dispill: No such file or directory

find: ./docustore: No such file or directory

find: ./emar/lap: No such file or directory

mv: setting attribute `security.selinux' for `security.selinux': Operation not permitted

mv: setting attribute `security.selinux' for `security.selinux': Operation not permitted

small snippet of the above code. it is trying to move every file and directory in the working directory instead of only the files im after.