BASH Issue

TBotNik · 07-07-2023, 03:22 PM

All,

Have a file full of scanned directories so trying to read it into an array but always get an error saying

This is a directory not a file. Thought the code for reading lines from a file was generic but guess not! So what code is needed to read each line in the file generically or as directories?

TBNK

astrogeek · 07-07-2023, 03:29 PM

Without an actual example of the file contents, the script code used to read it into an array and the exact error being produced, the question is really without meaning.

Please provide enough actual detail to allow us to understand the question and try to provide a useful answer without the need to guess about every piece of the puzzle.

MadeInGermany · 07-08-2023, 01:16 AM

read and readarray process a text file.
A directory is not a text file, not even a binary file (though there are some similarities).
In bash use a for loop:

Code:

for fn in * .*
do
  [ -e "fn" ] || continue
  echo "do sth with $fn"
done

The .* lists the hidden files. (Or use "shopt -s dotglob" to include hidden files in the * )
You can prefix a dirpath: $dir/* $dir/.*
And you can filter: $dir/*.txt $dir/*.php (or $dir/*.{txt,php} )

pan64 · 07-08-2023, 03:40 AM

also you can use:

Code:

[[ -d $fn ]] to check if $fn is a dir
[[ -f $fn ]] to check if $fn is a file

but as it was already mentioned without details hard to say more

TBotNik · 07-08-2023, 01:48 PM

All,

OK! Well the file am reading is one created by KFind and dumped into a .txt file so nothing but directories in this file. So no need to even think any kind of ls cmd. My code reading this is:

Quote:

while IFS=read -r line; do
if [[ -n ”$line" && "$line" == "'/3T'" ]]; then
$oray+="$line"
fi
done < $srcfil

I'm thinking the IFS means the line must qualify as a file, which a directory will never do. Am I right?

TBNK

teckk · 07-08-2023, 04:13 PM

Quote:

so nothing but directories in this file.

What does that mean? You have directories in a file. You have file paths and directory paths listed in a file.txt?

Code:

txt="
/usr/lib/libargon2.so.1
/usr/lib/libasm-0.189.so
/usr/lib/libassuan.so.0.8.5
/usr/lib/vdpau
/usr/lib/udev
"

while read line; do 
    if [ -d "$line" ]; then
        echo ""$line" is a directory path."
    fi
    if [ -f "$line" ]; then
        echo ""$line" is a regular file path"
    fi
done <<< "$txt"

/usr/lib/libargon2.so.1 is a regular file path
/usr/lib/libasm-0.189.so is a regular file path
/usr/lib/libassuan.so.0.8.5 is a regular file path
/usr/lib/vdpau is a directory path.
/usr/lib/udev is a directory path.

pan64 · 07-09-2023, 03:16 AM

Quote:

Originally Posted by TBotNik

I'm thinking the IFS means the line must qualify as a file, which a directory will never do. Am I right?

TBNK

No. you need to read the man page of bash and check IFS, do not assume anything [else].
https://www.baeldung.com/linux/ifs-shell-variable

TBotNik · 07-10-2023, 01:40 PM

All,

So banging through this slowly. Put in the code

Code:

readarray pray < $srcfil

Which totally works! The directories I want to process are all on the 3Tb drive, so added this code:

Code:

nray()
for i in ${pray[@]}; do
if [ "$i" == "/3T/"* ]; then
$nray+=”$i"
fi

But this array stays empty so my logic is somehow wrong.

Is my error here obvious?

TBNK

MadeInGermany · 07-10-2023, 03:33 PM

Fixes:

Code:

nray=( )
for i in "${pray[@]}"; do
  if [[ "$i" == "/3T/"* ]]; then
    nray+=( "$i" )
  fi
done

pan64 · 07-11-2023, 12:09 AM

would be nice to use shellcheck, that can find those kind of issues easily

TBotNik · 07-11-2023, 05:24 PM

Quote:

Originally Posted by MadeInGermany

Fixes:

Code:

nray=( )
for i in "${pray[@]}"; do
  if [[ "$i" == "/3T/"* ]]; then
    nray+=( "$i" )
  fi
done

MadeInGermany,

Thanks! The right syntax always works!

Now working on the next MySQL section to put the array into the DB.

Have no book on that so trying to find online examples. The DB has to be checked for each entry to make sure to eliminate any duplicates.

Thanks again!

TBNK

TBotNik · 07-15-2023, 01:13 PM

MadeInGermany,

Got a novel thought so ran a KFind on all .sh files looking for the word "mysql" and found serveral examples.

Now working on editing this, but per my PHP Config post not able to see results in PPHPmyAdmin, but have MySQL Workbench installed so will have to find results there.

Will let you know where I get with this, hopefully over the weekend.

TBNK

michaelk · 07-15-2023, 02:57 PM

You can always use the mysql command line client to insert values.
mysql_query="insert into table x (column1...) values(data1...);"
mysql -u username -e "$mysql_query"

If the table is created with an unique index on the value you will not have any duplicate rows. Instead of an array you can create a CSV file and import that into mysql.

TBotNik · 07-16-2023, 04:36 PM

Quote:

Originally Posted by michaelk

You can always use the mysql command line client to insert values.
mysql_query="insert into table x (column1...) values(data1...);"
mysql -u username -e "$mysql_query"

If the table is created with an unique index on the value you will not have any duplicate rows. Instead of an array you can create a CSV file and import that into mysql.

michaelk,

The insert is not the problem but getting a return or error from the select query looking for duplicates is the issue!

TBNK

michaelk · 07-16-2023, 06:40 PM

Without knowing anything about the structure of the table here is one way to query for duplicate rows. Replace column with an actual column name.

Code:

select column,count(column) as duprows from table group by column having count(column)>1;