[SOLVED] Correcting an SFTP backup gone wrong.

gs10411 · 03-23-2010, 04:13 PM

I need a script that can undo a complete backed-up file system restructuring that took place after an upgrade to the latest version of SyncBackPro. Basically a windows machine running SyncBackPro runs a backup every morning to a SuSE 11.2 SFTP file server and after the monthly update of SyncBack I had some issues getting the existing profiles to run. Suffice it to say I did some tinkering and thought I reset the backup paths to their correct locale but the next day I found that the program took everything in the parent SFTP user directory on the SuSE server and did the backup in the main user directory. Since the user directory is not the backup directory, SyncBackPro took all of the files in the normal user directories and moved them into versioned $SBV$ directories and inserted a unique 12-digit numerical value to the front of every file below ~/ in every directory (recursively). I found a program named krename to correct the renaming issue and it is still adding the ~3 million files to the rename structure (tests on smaller folders worked flawlessly) to correct the 100*********.'s that have been appended to the beginning of every file. My problem is that I need to take all files that have been moved into these $SBV$ directories and move them up one level to their original directory and then remove the $SBV$ directory after the move. I do have a little experience with bash/shell scripts as I have made and use them for extracting, timestamping and moving jobs that execute through TORQUE and G03 but fall far short of having a programming degree. Chemistry, now that's a different story...

I have found another post which appears to move the files in a specified directory up one level but I need a solution that can search recursively for the $SBV$ folder and then move the files and delete the then empty $SBV$ folder. http://www.linuxquestions.org/questi...43#post1217543

Any assistance would be greatly appreciated. Also, if a perl or any script other than bash would do the job better I am fair game to broaden my horizons.

grail · 03-23-2010, 08:15 PM

Just to confirm, so you had a directory called ~/music and now have ~/music/$SBV$music?
Based on the above being correct, I would use a simple for loop:

Code:

for d in $(find ~ -type d -name "\$SBV\$*")
do
    mv $d/* $d/..
    rmdir $d
done

The issue I forsee here is the case where you have ~/music/ACDC becomes ~/music/$SBV$music/$SBV$ACDC as it will find the top one first and try to remove it,
however, if it becomes ~/music/$SBVmusic ~/music/ACDC/$SBV$ACDC this could be alright but you would need to change the rmdir to rm -rf

As usual, I would test on a small sample set first.

gs10411 · 03-23-2010, 09:47 PM

Actually, the scenario is as follows:
For a given directory, let's say ~/Documents/Backups/, there are files and folders in this folder named "Backups". SyncBackPro took all of the files in Backups/ and put them into a folder named $SBV$/ so the new path looks similar to ~/Documents/Backups/$SBV$/ which all of the loose files which were in ~/Documents/Backups are moved to and then renamed with a prefix of 100***********.whateverthefilenameis.(I took care of the prefix issue with krename (wonderful utility by the way).) The folders that are in ~/Documents/Backups/ were altered recursively as the ~/Documents/Backups was scanned and files moved to /$SBV$/. So each folder in Backups/ also has a folder named $SBV$ if there were files in the next directory level so on and so forth... So I need a script that can scan for basically the $SBV$ folder, moved the files to the parent directory which contains the $SBV$ directory and then remove the $SBV$ directory hence restoring the original file location and directory structure. I will give the script you listed a try and see how it works on a small batch, thanks.

gs10411 · 03-23-2010, 11:50 PM

Code:

#!/bin/bash
for d in $(find . -type d -name "\$SBV\$")
do
        mv $d/* $d/..
        rmdir $d
done

The above script works correctly unless the directory name has a space in it such as "Node Reinstall" or has files in the $SBV$ directory that start with a "." such as .directory or .bashrc_bak. Since these files/directories aren't processed by the script they cannot be removed by rmdir as there is still data in them. I did replace the mv $d/* with $d/*.* which produced the same results.

grail · 03-24-2010, 06:58 AM

Take 2

Code:

#!/bin/bash
for d in $(find . -type d -name "\$SBV\$")
do
    for f in $(find "$d" -type f -maxdepth 1)
    do
        mv "$f" "${d}/.."
    done

    rmdir "$d"
done

The inverted commas will preserve any spacing issues.

gs10411 · 03-24-2010, 07:32 AM

For some reason I am now getting an error via find:

Code:

mv: cannot stat `./TrueCrypt/$SBV$/TrueCrypt': No such file or directory
mv: cannot stat `Format.exe': No such file or directory
rmdir: failed to remove `./TrueCrypt/$SBV$': Directory not empty
find: warning: you have specified the -maxdepth option after a non-option argument -type, but options are not positional (-maxdepth affects tests specified before it as well as those specified after it).  Please specify options before other arguments.

For a file located in folder TrueCrypt/$SBV$/ which is named Truecrypt Format.exe (that's with a space). It seems that we are past the names of folders with spaces but not the files.

grail · 03-24-2010, 10:06 AM

ruh roh ... I think that is my bad, please change the inner for loop to:

Code:

for f in "$(find Documents/ -maxdepth 1 -type f)"

gs10411 · 03-29-2010, 12:39 AM

Here is what I have run into over the last few days in my spare time. With the code:

Code:

#!/bin/bash
for d in $(find . -type d -name "\$SBV\$")
do
        for f in "$(find . -maxdepth 1 -type f)"
                        do
                                        mv "$f" "${d}"/..
                                            done

                                                rmdir "$d"
                                        done

I end up with the following:

Code:

mv: cannot move `./orig.sh' to `./Virtual/..': No such file or directory
rmdir: failed to remove `./Virtual': No such file or directory
mv: cannot move `./orig.sh' to `Server/..': No such file or directory
rmdir: failed to remove `Server': No such file or directory
mv: cannot move `./orig.sh' to `2005/Linux/..': No such file or directory
rmdir: failed to remove `2005/Linux': No such file or directory
mv: cannot move `./orig.sh' to `Additions/..': No such file or directory
rmdir: failed to remove `Additions': No such file or directory
mv: cannot move `./orig.sh' to `for/..': No such file or directory
rmdir: failed to remove `for': No such file or directory
mv: cannot move `./orig.sh' to `Virtual/..': No such file or directory
rmdir: failed to remove `Virtual': No such file or directory
mv: cannot move `./orig.sh' to `Server/$SBV$/..': No such file or directory
rmdir: failed to remove `Server/$SBV$': No such file or directory
mv: cannot move `./orig.sh' to `./Virtual/..': No such file or directory
rmdir: failed to remove `./Virtual': No such file or directory
mv: cannot move `./orig.sh' to `Server/..': No such file or directory
rmdir: failed to remove `Server': No such file or directory
mv: cannot move `./orig.sh' to `2005/$SBV$/..': No such file or directory
rmdir: failed to remove `2005/$SBV$': No such file or directory

for the script named orig.sh located in tessst/ with a directory named Virtual Server 2005 as a subdirectory.

I have tried an absolute path such as ~/Documents/Data/SBVBakCorrections/tessst for

Code:

for f in "$(find ~/ABUNCHOFDIRECTORIES/tessst -maxdepth 1 -type f)"

and receive the following:

Code:

mv: cannot move `/ABUNCHOFDIRECTORIES/tessst/orig.sh' to `2005/$SBV$/..': No such file or directory
rmdir: failed to remove `2005/$SBV$': No such file or directory

I was able to do a great deal of manual cleanups via the following three commands:

Code:

for i in $(find . -type d -name "\$SBV\$"); do mv "$i"/* "$i"/..; done
for i in $(find . -type d -name "\$SBV\$"); do mv "$i"/.* "$i"/..; done
for i in $(find . -type d -name "\$SBV\$"); do rmdir "$i"; done

The only down side is that there are soooo many directories and subdirectories that have spaces in them that it would take weeks of spare time just fishing through them to get them all. It would be much better if there was a way to recursively follow through the directory structure from a given starting point and find all $SBV$ folder, move the files in this folder up one level, then remove the $SBV$ folder in one swath. I have also looked into using the find -print0 w/ xargs argument but have been unable to get the /bin/mv to accept the piped in info.

Code:

find . -name "\$SBV\$" -type d -print0 | xargs -0 /bin/mv *.* ..

As I get the output:

Code:

/bin/mv: cannot move `..' to `./Virtual Server 2005/$SBV$/..': Device or resource busy

Once again, any input would be greatly appreciated and I will continue in my spare time to continue reading, testing and banging my head on the desk...

grail · 03-29-2010, 02:29 AM

Man I cannot I forget some of this stuff sometimes . . . really sorry about that.
Let us see if third time is the charm:

Code:

OFS=$IFS
IFS=$(echo -en "\n\b")

for d in $(find . -type d -name "\$SBV\$")
do
    for f in $(find "$d" -type f -prune)
    do
        mv "$f" "${d}/.."
    done

    rmdir "$d"
done

IFS=$OFS

Hopefully sorted now :$

gs10411 · 03-29-2010, 11:46 PM

That pretty much did it. Thank you much...