compare 2 filenames & move one to other's directory

ericlindellnyc · 11-04-2019, 03:00 AM

I have code that compares two files, & if equal, touches one with the other, so it has the same date.

This time, I'd like to compare the file names only, without the extension, and if equal, move one to the other's subdirectory.

This process makes the comparison for all files in two directories, iterating through one directory to find a match for each in the other directory, which it also iterates through, for a nested loop.

FILES=$( find ./nu/Mrj -type f ); for f in $FILES; do for g in ./ol/Mrj/*; do echo "f is $f g is $g"; if cmp "$f" "$g"; then echo "match"; touch -r $g $f; else echo "no match"; fi; done; done

Thanks in advance

Turbocapitalist · 11-04-2019, 05:07 AM

test is probably what is needed.

Code:

FILES=$( find ./nu/Mrj -type f ); 
for f in $FILES; do 
        for g in ./ol/Mrj/*; do 
                echo "f is $f g is $g"; 
                if test "$f" = "$g"; then 
                        echo "match"; 
                        touch -r $g $f; 
                else echo "no match"; 
                fi; 
         done; 
done

See "man test"

ericlindellnyc · 11-04-2019, 09:01 PM

Quote:

Originally Posted by Turbocapitalist

test is probably what is needed.

Code:

FILES=$( find ./nu/Mrj -type f ); 
for f in $FILES; do 
        for g in ./ol/Mrj/*; do 
                echo "f is $f g is $g"; 
                if test "$f" = "$g"; then 
                        echo "match"; 
                        touch -r $g $f; 
                else echo "no match"; 
                fi; 
         done; 
done

See "man test"

Great.
Now I need to extract the filename independent of the extension to make the comparison.
If the FN's match (regardless of extension),
then move (or copy, doesn't matter) one file to the same directory as the other.

This code separates FN from EXT.

Code:


	Code:
	NAME=`echo "$FILE" | cut -d'.' -f1`
EXTENSION=`echo "$FILE" | cut -d'.' -f2`

I can try to extract one file's path so I can move the other file to its directory,
which seems cumbersome.
I could try a simple mv, but I need to keep both files.

Got it !! .. Duplicate the directory, then use mv to overwrite the (let's say) TXT files in one directory with MP3 files in the other.
Then MERGE the directory of TXT files with the directory of MP3 files !!
Perfect, thank you.

BW-userx · 11-04-2019, 09:07 PM

another way to get extension using bash internals

Code:

$ filename=hommer.txt

$ ext=${filename##*.} ; echo $ext
txt

so yours would be

Code:

ext=${FILE##*.}
or
ext=${f##*.}

depending on when you grab the extension.
to get file name w/o ext

Code:

$ filename=/path/to/file/hommer.txt

$ fname=$(basename $filename) ; echo $fname
hommer.txt

$ prefix=${fname%.*} ; echo $prefix
hommer

ericlindellnyc · 11-05-2019, 08:47 PM

THIS WORKS PERFECTLY, EXCEPT FOR THE LINE

Code:

mv $g $f;

Code:

#!/bin/bash
FILES=$( find ./txtMp3s/txts -type f ); 
for f in $FILES; do 
    for g in ./txtMp3s/mp3s/*; do 
        echo "f is $f g is $g"; 
        fPrefix=${f%.*} ; echo $fPrefix;
        gPrefix=${g%.*} ; echo $gPrefix;
        if test "$fPrefix” = "$gPrefix”; then 
                echo "match"; 
                mv $g $f;
        else echo "no match"; 
        fi; 
    done;
done

If I don't have that line there, everything works perfectly, all files are properly echoed, and match is printed, perfectly.
But then I add the

Code:

"mv $g $f;"

and it says

Code:

mv: ./txtMp3s/mp3s/*: No such file or directory

And when I check the original directories, the one that had the mp3 file (called g) is empty, and nothing new has been moved to directory containing txt file (called f), though at least f is still there.

Thoughts much appreciated !!

Firerat · 11-05-2019, 09:36 PM

Code:

FILES=$( find ./txtMp3s/txts -type f )

is bad!
https://mywiki.wooledge.org/BashGuide/Arrays
that explains why

Code:

mv $g $f

since you are not wrapping the vars with " you are subject to splitting if the path/filename has spaces in it

do you really need xPrefix vars?

Code:

#!/bin/bash
while read f; do 
    for g in ./txtMp3s/mp3s/*; do 
        printf "f is %s g is %s\n%s\n%s\n" $f $g ${f%.*} ${g%.*}  
        # do you really need that ?
        filename="${f##*/}" # strips path
        [[ ${g%.*} =~ /${filename%.*}$ ]] \
            && mv -v "$g" "${f%/*}/" \
            || echo "no match"
    done
done < <( find ./txtMp3s/txts -type f )

https://mywiki.wooledge.org/BashGuid...ndConditionals

https://mywiki.wooledge.org/BashGuid...eter_Expansion
see that page to understand the ${f%.*} and ${f%/*}/

# Edit, Doh, this will never match, since path is different
#[[ ${f%.*} == ${g%.*} ]] \

filename="${f##*/}" # strips path
# Edit2, this is not great either
#[[ ${g%.*} =~ ${filename%.*}$ ]]
it is only checking the end matches
tune_1.mp3 remix_tune_1.mp3
tune_1 could overwrite the remix version
adding a leading slash should fix it
[[ ${g%.*} =~ /${filename%.*}$ ]]

see the section on Regular Expressions
https://mywiki.wooledge.org/BashGuide/Patterns
to understand the =~ bit

BW-userx · 11-06-2019, 07:33 AM

It strips files down to just the name, then compares the two, then creates subdir within the same place the file is found, then moves dup file into same dir match in the subdir of the path. keeps a running total of match, no match, and a running list to where each individual file was moved to.

Even though you do not specify between dirs what goes where and from where. Not good programing logic. It is an incomplete thought.
"This time, I'd like to compare the file names only, without the extension, and if equal, move one to the other's subdirectory."
which one goes where??????

Code:

#!/usr/bin/env bash
#set -xv 

#allows script to be ran outside
#of working directory
noMatch=0
match=0
#Keep is control loop, or outside loop
#move or delete file is the inner loop
SAVEFILEARRAY=()
#Set path to base directory's
KeepFile=/home/userx/linux
MoveFile=/home/userx/test2

#Name of the place to put
#dup filenames
SubDirName=Dups

script_dir="$HOME/scripts"
#keep loop
while read KFile 
	do
	#chop off both ends to get just the filename w/o extension
	  KeepFileName=${KFile##*/}
	  KeepFileName=${KeepFileName%.*}
		
	 while read MFile
		do
			#chop off both ends to get just the filename w/o extension
			MoveFileName=${MFile##*/}
			MoveFileName=${MoveFileName%.*}
			if [[ "${KeepFileName}"  = "${MoveFileName}" ]] ; then
				#prepare path to move the file to
				
				MFileP=${MFile%/*}
				Mv2Dir="$MFileP"/"$SubDirName"
				mkdir -p "$Mv2Dir"
				
			echo "	mv -v "$MFile" "$Mv2Dir"  "
			
			#keep track of where they went to
				SAVEFILEARRAY+=( "$Mv2Dir"/"${MFile##*/}" )
				
				#keep a count?????
				((match++))
			else
				((noMatch++))
			 fi
			 
	done < <(find "$MoveFile" -type f -name "*.*")
			#clear the var to prevent 
			#adding on to the string
	        unset Mv2Dir
done < <(find "$KeepFile" -type f -name "*.*" )

echo "
Files Matched is $match, Not matched $noMatch
Files where moved to.
"

printf "%s\n" "${SAVEFILEARRAY[@]}"

Just swap out the MFileP=${MFile%/*} to MFileP=${KFile%/*} to get a reverse save/move to path.

for loops vs while loops:
So between you, Firerat and mine you see that there is more than one way to get this done. But it is all still the same basic logical flow.

If you have more than one dup of the "same file" then as suggest just add a number to the end of the file. Though you'd have to check save dir for a match to that file before moving it into the same dir, if match, then add a number to the file, then move the file into the same save dir to prevent over writing of the two files. Which would also require that you get last number saved so you do not over write that part of the file in the process. Instead increment one more than last number of same file saved.

This way you can then run diff on the matching files to see if contents within the files are too dups or not to determine what to then do with them.

output sample

Code:

Files Matched is 98, Not matched 36914
Files where moved to.

/home/userx/test2/scripts/Dups/schema.pl
/home/userx/test2/scripts/Dups/schema.pl
/home/userx/test2/scripts/Dups/HandBreakCLI.3
/home/userx/test2/scripts/Dups/HandBreakCLI.2
/home/userx/test2/scripts/Dups/HandBreakCLI.1
/home/userx/test2/scripts/perl/Dups/changeFileNames.pl
/home/userx/test2/scripts/perl/Dups/perl-find-missing-numbers.pl
/home/userx/test2/scripts/perl/Dups/perl-Dir-number-list.pl

if you wanted to put all dups in one central location, then I do suppose you'd already know how to redirect path -> ../sameDir/subdir/dupfile or ../saveDir/dups/DupFile or ../saveDir/DupFile

ericlindellnyc · 11-06-2019, 03:25 PM

Thanks, Firerat & BW-userx, for detailed comments,
which will take me some time to work through.
I like the back-to-basics bootcamp approach.

Some of my results are just bizarre. I end up with a target directory
whose size doesn't add up to its contents.
Directory has only TXT files with kilobyte-range sizes,
but the total directory size seems to reflect the presence of megabyte-size MP3's,
which I tried to MV there but that do not show up when I look at directory contents.

Moreover, these MP3s are not in their original directory, in the recycle bin,
or anywhere in the entire system, wtf, as far as I can tell.

BW-userx · 11-06-2019, 03:45 PM

I do not fully understand your situation in what now are you saying, mp3 size files everywhere including the recyle bin. are your find compare paths the entire system?

you show two dirs

Code:

/txtMp3s/mp3s/
which indicates it has mp3 in it.

/txtMp3s/txts
which indicates it has txt files in it.

I merely created a script that looks at the filename without an extension if match put it into a subdir of one of the files base directory.

which would in your case either be in
/txtMp3s/mp3s/subdir
/txtMp3s/txts/subdir

what relationship does the txt files have with the mp3 files?

did you know you can get meta data off of mp3s and sort them out using that?
providing they have the metadata written to them.
artist,title,album etc...