LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 11-06-2011, 01:49 PM   #1
Valpskott
LQ Newbie
 
Registered: Nov 2011
Posts: 4

Rep: Reputation: Disabled
Bash Script Automation for new Subdirectories.


I'm producing game-related videos, and I've made 2 scripts to make my work much easier. Lets call them Script A and Script B.

Script A run several copies of Script B (one in each subdirectory). And every time I have new directories, I manually copy Script B into them, and then edit Script A so that the pathways for the new Script Bs are correct.

Sooo, I'd like help in remaking Script A so that it doesn't need editing each time new subdirectories are added.

Right now Script A looks something like this.

Code:
#!/bin/bash
cd Directory-01
./Script-B.sh
cd ..

cd Directory-02
./Script-B.sh
cd ..

cd Directory-03
./Script-B.sh
cd ..

#etc...

It's a simple script that just changes directory, run Script-B.sh in that directory, then goes to the parent directory and up the next subdirectory and runs Script-B.sh in that directory, and so on. Worth mentioning, the actual directories have more unique names, and each time I run the script the amount of subdirectories varies.

So what I need for my new Script A, is that it copies Script-B.sh into all the current subdirectories(I normally do this manually), then executes Script-B.sh in each of these subdirectories.


I'd also like help with Script B, right now, it outputs a standardised name as "Final-Video.mov" in the subdirectory the script is ran from. I'd really appreciate it if I could end the script with something like

Code:
mv Final-Video.mov ../$1.mov
rm -r -f $1
where $1 is the name of the current subdirectory, essentially naming the file after the Subdirectory it's in, moving it out of the subdirectory and then removing the subdirectory once the script is done. But how to get $1 to hold the name of the current child-directory?
 
Old 11-06-2011, 01:51 PM   #2
corp769
LQ Guru
 
Registered: Apr 2005
Location: /dev/null
Posts: 5,818

Rep: Reputation: 1005Reputation: 1005Reputation: 1005Reputation: 1005Reputation: 1005Reputation: 1005Reputation: 1005Reputation: 1005
As far as all of your subdirectories, you can use a for loop, and loop through all file (*), and then do what you need to do. As far as "script B," what exactly does it do? Maybe you can share it with us, and possible find a completely better way to tackle what you are trying to accomplish.

Cheers,

Josh
 
Old 11-06-2011, 03:10 PM   #3
Nominal Animal
Senior Member
 
Registered: Dec 2010
Location: Finland
Distribution: Xubuntu, CentOS, LFS
Posts: 1,723
Blog Entries: 3

Rep: Reputation: 948Reputation: 948Reputation: 948Reputation: 948Reputation: 948Reputation: 948Reputation: 948Reputation: 948
Quote:
Originally Posted by Valpskott View Post
But how to get $1 to hold the name of the current child-directory?
pwd will tell you the path to the current working directory. Therefore, basename `pwd` will tell you the name of the current directory, and dirname `pwd` the path to the parent directory (logically the same as .. ).

Quote:
Originally Posted by Valpskott View Post
Script A run several copies of Script B (one in each subdirectory). And every time I have new directories, I manually copy Script B into them, and then edit Script A so that the pathways for the new Script Bs are correct.
Don't do that. Replace them with Script C, that takes one or more (target) directory names as parameters. Then you can do
Code:
find tree-containing-directories -mindepth 1 -depth -type d -print0 | xargs -r0 path-to-Script-C
or, if you have a very powerful machine, and want to do all directories in parallel,
Code:
find tree-containing-directories -mindepth 1 -depth -type d -print0 | bash -c 'while read -d "" DIR ; do path-to-Script-C "$DIR" & ; done ; wait'
No need to do any copying or setup by hand. Note that the -depth parameter above tells find to supply the deepest directories first; this way you'll always get all child directories before their parents, and you can freely nest the directories.

The basic structure of Script C could be something like the following:
Code:
#!/bin/bash

while [ $# -gt 0 ]; do
    dir="$1"
    name=$(basename "$1")
    shift 1

    if (
        cd "$dir" || exit $?
 
        # Current working directory is the one to work on.
        # "$name" is its name (not path).
        # Use 'exit N' to cancel work.

        # Does the final movie exist yet?
        if [ ! -e "final.ogg" ]; then

            # ... Work ...

        fi

        # Verify we now have "final.ogg" in the current directory.
        [ -e "final.ogg" ] || exit 1

        # Make sure nobody changed the directory name from under us.
        [ "$dir" -ef . ] || exit 1

        # Copy the final movie to parent directory, using the
        # name of the current directory, but with .ogg suffix.
        mv -f "final.ogg" "../$name.ogg" || exit $?

        # Remove this child directory.
        cd ..
        rm -rf "$name"
    ) ; then
        echo "$dir: Replaced with '$name.ogg' successfully." >&2
    else
        echo "$dir: Processing aborted due to an error." >&2
        exit 1
    fi
done
with # ... Work ... obviously replaced with the code you use to convert your input files into the final movie.

If you change a directory name while the script is still working, it will abort. You can then safely rerun the command. If the final movie already exists, the script will not regenerate it, just move (and rename) it.

Last edited by Nominal Animal; 11-06-2011 at 03:14 PM.
 
Old 11-07-2011, 12:09 AM   #4
Valpskott
LQ Newbie
 
Registered: Nov 2011
Posts: 4

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by corp769 View Post
As far as all of your subdirectories, you can use a for loop, and loop through all file (*), and then do what you need to do. As far as "script B," what exactly does it do? Maybe you can share it with us, and possible find a completely better way to tackle what you are trying to accomplish.

Cheers,

Josh
Currently, script B looks like this.

Code:
#!/bin/bash

#Merge videos
mencoder -oac copy -ovc copy -o 01-Merged.mov *.avi

#Extract Sound
ffmpeg -i 01-Merged.mov -vn -ar 48000 -ac 2 -ab 192000 -f flac 01-Audio-Unfixed.flac

#Noise Reduction, Compressor, Normalize
sox 01-Audio-Unfixed.flac 02-Audio-Fixed.flac noisered noiseprofil.sox 0.35 gain -l 9 compand 0.0,0.3 -60,-60,-30,-15,-20,-12,-4,-8,-2,-7 -2 norm -0.1

#Join Sound with Video and force video to 1280x720
ffmpeg -i 01-Merged.mov -i 02-Audio-Fixed.flac -map 0:0 -map 1:0 -f mp4 -vcodec mpeg4 -qscale 1 -s 1280x720 -qmin 1 -intra -threads 2 -acodec libmp3lame -ab 160k -ar 44100 -ac 2 02-Merged-Audiofix.mov

#Join Intro with Video
mencoder -oac copy -ovc copy -o 03-compress-me.mov ../intro.mov 02-Merged-Audiofix.mov

#Compress For Youtube
ffmpeg -i 03-compress-me.mov -f mp4 -acodec copy -pix_fmt yuv420p -vcodec libx264 -minrate 0 -b 5000k -b_strategy 1 -subcmp 2 -cmp 2 -coder 1 -flags +loop -flags2 dct8x8 -qmax 51 -subq 7 -qmin 10 -qcomp 0.6 -qdiff 4 -trellis 1 -aspect 16:9 Final-Video.mov

#Remove all files except original (in case of error) and final.
rm 01-Audio-Unfixed.flac
rm 01-Merged.mov
rm 02-Audio-Fixed.flac
rm 02-Merged-Audiofix.mov
rm 03-compress-me.mov
rm noiseprofil.sox
rm Script-B.sh

So, when script B is done... the Subdirectory is still there, and it contains the original videofiles + Final-Video.mov. So, if I have like 10 subdirectories with videos, I will have 10 videos with the name Final-Video.mov (one in each Subdirectory). The name of the Subdirectory is descriptive of what the video inside it, so thats why I'd like to name "Final-Video.mov" after the directory it's in.
 
Old 11-07-2011, 12:27 AM   #5
Valpskott
LQ Newbie
 
Registered: Nov 2011
Posts: 4

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by Nominal Animal View Post
pwd will tell you the path to the current working directory. Therefore, basename `pwd` will tell you the name of the current directory, and dirname `pwd` the path to the parent directory (logically the same as .. ).


Don't do that. Replace them with Script C, that takes one or more (target) directory names as parameters. Then you can do
Code:
find tree-containing-directories -mindepth 1 -depth -type d -print0 | xargs -r0 path-to-Script-C
or, if you have a very powerful machine, and want to do all directories in parallel,
Code:
find tree-containing-directories -mindepth 1 -depth -type d -print0 | bash -c 'while read -d "" DIR ; do path-to-Script-C "$DIR" & ; done ; wait'
No need to do any copying or setup by hand. Note that the -depth parameter above tells find to supply the deepest directories first; this way you'll always get all child directories before their parents, and you can freely nest the directories.

The basic structure of Script C could be something like the following:
Code:
#!/bin/bash

while [ $# -gt 0 ]; do
    dir="$1"
    name=$(basename "$1")
    shift 1

    if (
        cd "$dir" || exit $?
 
        # Current working directory is the one to work on.
        # "$name" is its name (not path).
        # Use 'exit N' to cancel work.

        # Does the final movie exist yet?
        if [ ! -e "final.ogg" ]; then

            # ... Work ...

        fi

        # Verify we now have "final.ogg" in the current directory.
        [ -e "final.ogg" ] || exit 1

        # Make sure nobody changed the directory name from under us.
        [ "$dir" -ef . ] || exit 1

        # Copy the final movie to parent directory, using the
        # name of the current directory, but with .ogg suffix.
        mv -f "final.ogg" "../$name.ogg" || exit $?

        # Remove this child directory.
        cd ..
        rm -rf "$name"
    ) ; then
        echo "$dir: Replaced with '$name.ogg' successfully." >&2
    else
        echo "$dir: Processing aborted due to an error." >&2
        exit 1
    fi
done
with # ... Work ... obviously replaced with the code you use to convert your input files into the final movie.

If you change a directory name while the script is still working, it will abort. You can then safely rerun the command. If the final movie already exists, the script will not regenerate it, just move (and rename) it.

I found that "basename `pwd`" didn't handle spaces too well.

Code:
filer@nellie:~/Video Edit/custom$ basename `pwd`
Video
Also, will the following line run Script C from within the Subdirectories, even if the path to Script C is in the parent? (no need to copy it to subdirectory?)

Code:
find tree-containing-directories -mindepth 1 -depth -type d -print0 | xargs -r0 path-to-Script-C
 
Old 11-07-2011, 02:34 AM   #6
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,578
Blog Entries: 31

Rep: Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198Reputation: 1198
Quote:
Originally Posted by Valpskott View Post
I found that "basename `pwd`" didn't handle spaces too well.
Use ...
Code:
basename "`pwd`"
... instead to prevent the shell from tokenising the pwd output into words where there is whitespace (space, tab, newline).

Or you may prefer
Code:
basename "$( pwd )"
for reasons explained here.
 
Old 11-07-2011, 01:55 PM   #7
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian + kde 4 / 5
Posts: 6,849

Rep: Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024
Instead of basename+pwd, you can use bash's already supplied PWD variable and parameter expansion.

Code:
path="${PWD%/*}"
name="${PWD##*/}"
 
Old 11-08-2011, 01:53 AM   #8
Valpskott
LQ Newbie
 
Registered: Nov 2011
Posts: 4

Original Poster
Rep: Reputation: Disabled
Nominal Animal, you are bash script crazy!
Tried your solution and it worked. Thank you very much!

Also, thank you all for taking your time to assist me.

I'll mark this thread as solved, but I'm still curious of how the PWD thing works and how variables work, and what the correct syntax is between the two.

Would this be correct

Code:
$mydir=basename "`pwd`"
echo $mydir
Would that print the name of the directory I'm in? Or what is a way to load variable $mydir with the name of the current dir(not complete path) I'm in?
 
Old 11-08-2011, 09:06 AM   #9
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian + kde 4 / 5
Posts: 6,849

Rep: Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024Reputation: 2024
Quote:
Originally Posted by Valpskott View Post
Would this be correct

Code:
$mydir=basename "`pwd`"
echo $mydir
Would that print the name of the directory I'm in? Or what is a way to load variable $mydir with the name of the current dir(not complete path) I'm in?
No, that's incorrect.

Let's go step by step, starting with basename. And as an example we'll even use a filename with, not one, but three spaces in it.

Code:
$ basename /home/david/test/my   file.txt
my
Hmm, didn't work. What happened is that the shell broke the filename up into two arguments "/home/david/test/my" and "file.txt". Since basename only accepts one argument, the second part was dropped.

So we quote the filename:
Code:
$ basename "/home/david/test/my   file.txt"
my   file.txt
This forces the shell to treat the entire string as a single unit, rather than breaking it up at whitespace. See the links below for details.

Now, we want to create a variable to store that value in. First of all, the syntax for setting a variable is very simply name=value. You don't put a $ in front of the name when setting it, only when retrieving the value. And there are no spaces around the equals sign either.

Let's try to set it directly first:
Code:
$ name=my    file.txt
bash: file.txt: command not found
Whoops! We had the same word-splitting problem again, and it thinks "file.txt" is meant to be a second command. And of course the solution is again to quote the string.

Code:
$ name="my    file.txt"
Now if we want to insert the output of one command into another command we use command substitution. This is either `..` or $(..), with the latter being much preferred.

In this case we insert the output of basename into the variable setting.

Code:
$ name="$( basename "/home/david/test/my   file.txt" )"
Notice how I put a second level of quotes around the "$(..)". This ensures that the output is treated as a single unit, just as with the literal string.

(Actually in this specific situation quoting isn't needed, as substitutions get copied over without word-breaking, but it doesn't hurt, and they are necessary elsewhere).

Now let's echo back the variable to check its value.

Code:
$ echo $name
my file.txt
What happened to the extra spaces? Well, without quotes the shell again broke the output up into separate arguments, and echo got passed two separate strings, "my" and "file.txt", which it printed separated by a single space.

Code:
$ echo "$name"
my   file.txt
That's better.

In fact, let's do something funky and try passing the output of basename directly to echo.

Code:
$ echo $( basename "/home/david/test/my   file.txt" )
my file.txt

$ echo "$( basename "/home/david/test/my   file.txt" )"
my   file.txt
Just as with the variable, the quotes protect the string formatting.

So as you can see, it's vital to understand how the shell handles arguments and whitespace. Read and absorb these three links.

http://mywiki.wooledge.org/Arguments
http://mywiki.wooledge.org/WordSplitting
http://mywiki.wooledge.org/Quotes


And don't forget the suggestion in my last post. $PWD is a variable automatically set by the shell that holds the full path to the current directory, in the same format as the pwd command.

Code:
$ pwd
/home/david/test
$ echo "$PWD"
/home/david/test
And built-in parameter substitution rules can take the place of the external basename and pathname commands.

Code:
$ name="/home/david/test/my   file.txt"
$ echo "$name"
/home/david/test/my   file.txt

$ echo "${name%/*}"	# strips off everything from the end to the last "/"
/home/david/test

$ echo "${name##*/}"	# strips off everything from the beginning to the last "/"
my   file.txt
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Bash script to rename photos in directory & all subdirectories shy_guest Linux - Software 7 09-02-2009 01:40 PM
Bash script to rename photos in directory & all subdirectories shy_guest Linux - Software 2 09-02-2009 05:53 AM
Bash script to rename photos in directory & all subdirectories shy_guest Linux - Software 1 09-02-2009 05:28 AM
Using Bash, Find script files in a directory or subdirectories within... ray5_83 Programming 4 10-10-2008 07:42 PM
Bash script - executing a script through subdirectories bubkus_jones Programming 5 04-24-2006 05:05 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 08:36 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration