[SOLVED] Directories and files without duplicates

Faki · 05-21-2022, 01:20 PM

I would be pass directory names or file names as arguments to a bash function. But want to do a check existence and remove duplicates before storing in array fdir. Currently I can handle directory names, but want to extend the code to include and check file names in addition to directory names.

The command [[ ! -d $dpa ]] && continue fails to take account of files.

Code:

declare -A tag
for dpa in "$@"; do
  [[ ! -d $dpa ]] && continue
  [[ ${tag[comint:$dpa]} ]] && continue
  fdir+=("$dpa")
  tag[comint:$dpa]=1
done

teckk · 05-21-2022, 01:48 PM

"$@" will expand to "$1" "$2"...
If there are no positional parameters, "$@" expands to nothing.

So, you are running that with args? Need some info.

I ran that. Doesn't do anything except echo the dirs that you enter as args.

test.sh

Code:

#!/usr/bin/bash

declare -A tag

for dpa in "$@"; do
  [[ ! -d $dpa ]] && continue
  [[ ${tag[comint:$dpa]} ]] && continue
  fdir+=("$dpa")
  tag[comint:$dpa]=1
done

echo "${fdir[@]}"
echo "${tag[@]}"

You want to tell what you are trying to achieve, or just ignore the forum and keep on going?

Faki · 05-21-2022, 01:53 PM

Yes, I would pass directory names or file names as arguments to a bash function. But want to do a check existence and remove duplicates before storing in array fdir.

teckk · 05-21-2022, 01:56 PM

If you want to find duplicate file names in subdirectories

Code:

find . -maxdepth 2 -printf "%f\n" | sort -f | uniq -di

pan64 · 05-21-2022, 02:10 PM

you need to use associative array if you want to catch duplicated arguments.
you can have not only -d but -f too to check, see man test (for example)

MadeInGermany · 05-21-2022, 02:11 PM

Test for existence:

Code:

[[ ! -e $dpa ]] && continue

Explicitly test for directory or file (and not a special file):

Code:

[[ -d $dpa ]] || [[ -f $dpa ]] || continue

--
The following is correct - it tests the value; not existing members are substituted by an empty value (empty string).

Code:

[[ ${tag[comint:$dpa]} ]] && continue

More efficient is to directly test for existence:

Code:

[[ ${tag[comint:$dpa]+X} ]] && continue

Then the defined members may have an empty value:

Code:

tag[comint:$dpa]=

teckk · 05-21-2022, 02:35 PM

This will spit out duplicate files in tree. Might do it better.
Example:

Code:

for f in *; do
    find . -maxdepth 2 -iname "$f" | wc -l | while read count; do
        [ $count -gt 1 ] && find -iname "$f"
    done
done

Faki · 05-22-2022, 06:09 AM

Having defined members with empty values does not seem relevant. Because arguments are being passed, composed of directories and files (with possibility of user-errors).

I do not quite understand what `[[ ${tag[comint:$dpa]+X} ]] && continue` does. Would I still keep `tag[comint:$dpa]=1`? Some examples would help me understand your plan.

MadeInGermany · 05-22-2022, 07:13 AM

${tag[comint:$dpa]+X}
or generally
${var+string}
If $var exists then substitute "string".
If it does not exist then substitute nothing (empty string "").

[[ ]] without a further operator is true for a non-empty string.

The && runs the following command if the previous result was true. You can as well do

Code:

if [[ ${tag[comint:$dpa]+X} ]]; then continue; fi

Faki · 05-22-2022, 09:15 AM

I understand. This means that I should have

Code:

[[ ${tag[comint:$dpa]+X} ]] && continue

followed by

Code:

tag[comint:$fda]=1

How is your solution an improvement over

Code:

[[ ${tag[comint:$fda]} ]] && continue

MadeInGermany · 05-22-2022, 09:49 AM

Not much.
It takes a few CPU cycles less, and allows
tag[comint:$fda]= that is a few bytes less.