LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 12-30-2009, 10:05 AM   #1
No_one_knows_me
LQ Newbie
 
Registered: Dec 2009
Posts: 20

Rep: Reputation: 0
Bash script needed to move files to another folder.


Greetings One and All,

This is my first post in this forum. Let me tell you what I am needing. I need a bash script that will look at the 6th and 7th characters of a filename and if it is equal to a set number, then it will be moved to a specified folder. Here's an example of what I'm talking about:

If I have a listing of files

2009-12-18.fictitious.gz
2009-11-01.notreal.gz
2009-02-22.fake.gz

and I want to move 2009-12-18.fictitious.gz to the December folder, since the 6th and 7th digits of the filename = 12, then I would like it to create the December folder if it doesn't exist and move the file to that directory. Likewise, I want to move 2009-11-01.notreal.gz to the November folder and have it create the folder if it doesn't exist, etc. Anyone up to the challenge?

Also, can all the .gz files for the month go into one big tarball?
 
Old 12-30-2009, 10:14 AM   #2
rweaver
Senior Member
 
Registered: Dec 2008
Location: Louisville, OH
Distribution: Debian, CentOS, Slackware, RHEL, Gentoo
Posts: 1,833

Rep: Reputation: 163Reputation: 163
Well you can get the sixth and seventh character using cut, awk, sed, etc...

Code:
#!/bin/bash
for i in `ls /directory`; do 
  SS=`echo $i | cut -c6,7`
  if [ $SS -eq 1 ]; then
    mkdir -p /sorted/january
    mv $i /sorted/january
  elsfi [ $SS -eq 2 ]; then
    mkdir -p /sorted/febuary
    mv $i /sorted/febuary
  fi
done
There's the basics of it. You'd need to add the remainder of the logic of course. Also keep in mind I didn't test this, but if it's off it should be simple to fix it. If you need to gzip each month just add the logic to the end of the script after the if, basically for each folder tar cfvz month.tgz /sorted/month

Last edited by rweaver; 12-30-2009 at 10:17 AM.
 
Old 12-30-2009, 10:25 AM   #3
tredegar
LQ 5k Club
 
Registered: May 2003
Location: London, UK
Distribution: Debian "Jessie"
Posts: 6,017

Rep: Reputation: 369Reputation: 369Reputation: 369Reputation: 369
As usual with linux, there are many possible solutions.

I think it might be easier to create the January ... December folders yourself instead of writing a script to do it.

Then you could just do
Code:
mv 2009-12* /path/to/December
mv 2009-11* /path/to/November
Etcetera.

You could then copy your command history to a file with history > foo
and then edit foo to turn it into a script you can run next year to save yourself creating all those monthly directories again.

Quote:
Also, can all the .gz files for the month go into one big tarball?
Yes, they can.

Welcome to LQ!
 
Old 12-30-2009, 10:27 AM   #4
No_one_knows_me
LQ Newbie
 
Registered: Dec 2009
Posts: 20

Original Poster
Rep: Reputation: 0
rweaver,

Thank you for such a quick turn! It seems pretty straight forward and simple enough. I will have to make a modification since all the files in that directory don't begin with the date, but a simple find statement should be enough. I will try to finish writing it, and test it for my December files later on today to see if it works. Thanks once again!
 
Old 12-30-2009, 10:32 AM   #5
No_one_knows_me
LQ Newbie
 
Registered: Dec 2009
Posts: 20

Original Poster
Rep: Reputation: 0
tredegar,

I had contemplated just creating the monthly folders myself, too, but I have like 10 servers that this script will need to run on, so I was hoping to put the script in a cron job and let it work its magic.
 
Old 12-30-2009, 10:48 AM   #6
tredegar
LQ 5k Club
 
Registered: May 2003
Location: London, UK
Distribution: Debian "Jessie"
Posts: 6,017

Rep: Reputation: 369Reputation: 369Reputation: 369Reputation: 369
Quote:
I was hoping to put the script in a cron job and let it work its magic.
You still can, but realise that at some stage you are going to have to type mkdir January February March ... and you might as well do this only once, then extract that (and anything else useful) from your history, and use it to help you write your script without too much re-typing.

Test your script.

When you are happy with it, copy the script to your servers.
 
Old 12-30-2009, 11:29 AM   #7
No_one_knows_me
LQ Newbie
 
Registered: Dec 2009
Posts: 20

Original Poster
Rep: Reputation: 0
rweaver,

I have completed the proposed script you gave me, and I am getting a

Code:
./monthly_logs.sh: line 52: syntax error near unexpected token `done'
./monthly_logs.sh: line 52: `done'
I went to line 52, and all that is there is:

Code:
done
I am thinking that I will need more 'fi' statements since I had so many else if statements. Here is my script in its entirety, let me know of any suggestions:

Code:
#!/bin/bash
#Script to move daily log files to one big tar.gz at the end of the month
#and place them in their respective folders.

DIR=/var/log/
#TMP=$DIR/month/tmp

#mkdir $TMP

cd $DIR

for i in `ls $DIR`; do
         SS=`echo $i | cut -c6,7`
           if [ $SS -eq 01 ]; then
           mkdir -p /month/January
           mv $i /month/January
         else if [ $SS -eq 02]; then
           mkdir -p /month/February
           mv $i /month/February
          else if [ $SS -eq 03]; then
           mkdir -p /month/March
           mv $i /month/March
         else if [ $SS -eq 04]; then
           mkdir -p /month/April
           mv $i /month/April
         else if [ $SS -eq 05]; then
           mkdir -p /month/May
           mv $i /month/May
         else if [ $SS -eq 06]; then  
           mkdir -p /month/June
           mv $i /month/June
         else if [ $SS -eq 07]; then
           mkdir -p /month/July
           mv $i /month/July
          else if [$SS -eq 08]; then
           mkdir -p /month/August
           mv $i /month/August
         else if [$SS -eq 09]; then
           mkdir -p /month/September
           mv $i /month/September
         else if [$SS -eq 10]; then
           mkdir -p /month/October
           mv $i /month/October
         else if [$SS -eq 11]; then
           mkdir -p /month/November
           mv $i /month/November
         else if [$SS -eq 12]; then
           mkdir -p /month/December
           mv $i /month/December
     fi
done
 
Old 12-30-2009, 11:49 AM   #8
tredegar
LQ 5k Club
 
Registered: May 2003
Location: London, UK
Distribution: Debian "Jessie"
Posts: 6,017

Rep: Reputation: 369Reputation: 369Reputation: 369Reputation: 369
Quote:
I had so many else if statements....
How about:

Code:
case $SS in
    01)
           mkdir -p /month/January
           mv $i /month/January
;;
    02)           
           mkdir -p /month/February
           mv $i /month/February
;;
                  .
                  .
                  .
                  .
    12)           
           mkdir -p /month/December
           mv $i /month/December
;;
     *)
           Echo Bad month number: $SS; exit 1
;; 
esac
Not tested, but easier to read / understand /debug too

Edit: I think if you are going to use "-p" with mkdir, you should not have the leading slash before /month as otherwise the dir month will be made at /.
you'll have to experiment!

Last edited by tredegar; 12-30-2009 at 11:54 AM.
 
1 members found this post helpful.
Old 12-30-2009, 11:56 AM   #9
No_one_knows_me
LQ Newbie
 
Registered: Dec 2009
Posts: 20

Original Poster
Rep: Reputation: 0
tredegar,

I like your approach to this. It is clean and should eliminate the else if statements that seem to be causing me a little grief in scripting correctly. I will go back to the drawing board and see what happens. Thanks for your help!
 
Old 12-30-2009, 11:57 AM   #10
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949
There's no need to use external tools like cut or awk to extract the substring. All you need is bash's built-in parameter substitution.

Also $() is recommended over ``.

Code:
for i in $(ls $DIR); do
         if [ ${i:5:2} -eq 01 ]; then
           mkdir -p /month/January
           mv $i /month/January

           ...etc...
The error messages you're getting are probably because you need to have spaces on both sides of the test brackets

Code:
if [ value to test ]; then
    ^             ^
And actually, I think a case structure would be more appropriate here than if.

Code:
for i in $(ls $DIR); do

     case "${i:5:2}" in

          01) mkdir -p /month/January
           mv $i /month/January
          ;;

          02) mkdir -p /month/February
           mv $i /month/February
          ;;

          ...etc...

          *) echo "$i is not a valid filename."   #error message and exit if
             exit 1                               #filename is invalid.
          ;;
     esac
done
Edit: Beaten by tredegar.

Last edited by David the H.; 12-30-2009 at 12:03 PM. Reason: Added additional comments.
 
Old 12-30-2009, 12:02 PM   #11
tredegar
LQ 5k Club
 
Registered: May 2003
Location: London, UK
Distribution: Debian "Jessie"
Posts: 6,017

Rep: Reputation: 369Reputation: 369Reputation: 369Reputation: 369
Quote:
Beaten by tredegar.
- Not at all, you always have something to contribute, in this case parameter substitution.
bash-scripting is not my strong point, so I like to learn too
 
Old 12-30-2009, 12:51 PM   #12
No_one_knows_me
LQ Newbie
 
Registered: Dec 2009
Posts: 20

Original Poster
Rep: Reputation: 0
David the H.,

Quote:
Also $() is recommended over ``.
Is this true for the find command string or not?

So where I would normally do a:

Code:
for i in $DIR ; do `find . 2* -name -type f -print`
as a recommendation it should be:

Code:
$(find . 2* -name -type f -print)
I'm just trying to get up to speed, as it appears my ways may be somewhat antiquated.
 
Old 12-30-2009, 01:04 PM   #13
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949
That's true. I did have more to add. I was mostly just complaining about all the time I took writing up the case statement, only to find someone beat me to it.

But this stuff happens all the time, so no worries.


I see that I also failed to point out that there's no need for the embedded ls command (use of ls in loops is considered by many to be very ungraceful). Bash can read the directory listing directly; just use a * wildcard in the directory path. You can use variable substitution to provide the path itself.

Code:
for i in $DIR/*; do     # or specify a matching pattern
                        # with something like "$DIR/*.gz"
...commands...

done
 
Old 12-30-2009, 01:16 PM   #14
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949Reputation: 1949
Embedded commands work similarly to variable substitution. Any commands placed inside will be executed first, and their output substituted before the line containing it is executed.

http://www.tldp.org/LDP/abs/html/commandsub.html

Backticks (``) are a very old way to use embedded commands in bourne-type shells.

$() is a newer form of the same thing. I say newer, but it's been around long enough now that, unless you work on very old legacy systems, you shouldn't have any problems using it.

This page enumerates the reasons the newer form is preferred.
http://mywiki.wooledge.org/BashFAQ/082
 
Old 12-30-2009, 01:24 PM   #15
No_one_knows_me
LQ Newbie
 
Registered: Dec 2009
Posts: 20

Original Poster
Rep: Reputation: 0
David the H.,

Thank you for clarifying that for me, and I will bear that in mind in future scripting efforts. As it stands, everything is working as it should, but I neglected to create a find statement to grab just the '2*.gz' files in the directory and then utilize the parameter substitution you pointed out earlier. So I've just created that statement, and here's what I have:

Code:
for i in $DIR ; $(find . '2*.gz' -name -type f -print); do case "${i:5:2}" in
01) mkdir -p month/January
    mv $i month/January
;;

....etc.....
I'm getting
Code:
syntax error near unexpected token `$(find . '2*.gz' -name -type f -print)'
Now, I'm not testing this in a script, I'm just running it from the command-line. That shouldn't matter, though, right?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
how to tell bash to move files to another folder? hq4ever Linux - Newbie 10 12-30-2010 03:15 AM
Bash script: Move files, ignore case Fredde87 Linux - Software 3 09-03-2008 03:34 AM
how to merge files in different folder using bash script kkpal Programming 5 04-21-2008 02:35 AM
bash script, for all files in folder do ............ Simon_6162 Programming 2 07-18-2007 02:40 PM
making a script that will move a file or files in a trash folder Paxmaster Programming 5 12-12-2004 06:00 PM


All times are GMT -5. The time now is 07:16 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration