LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 04-28-2014, 04:21 PM   #1
Ruffe42
LQ Newbie
 
Registered: Apr 2014
Posts: 3

Rep: Reputation: Disabled
Smile Script to copy files, rsync?


Hi,

I need some help to make a script to copy files from a local folder to a network drive.
First of all I want to tell you this is my first time in linux environment, so be kind and clear. tongue.gif

My setup: Raspberry Pi with Raspbian wheezy, with a local connected USB-drive and a network drive. I have mounted both drives so they are available from the pi.

/home/pi DOWNLOAD > completed (local USB-drive)
MOVIES > HD (Network Drive part.1)
TV > SERIES > X (Network Drive part.2)
Y (Network Drive part.2)
Z (Network Drive part.2)

I want a script that copy all specific files (.avi etc) from /home/pi/DOWNLOAD/completed which dosen't already exist on the destination and put those in the predetermined folder.

etc. /home/pi/DOWNLOAD/completed/xxxxxYxx.avi goes to /home/pi/TV/SERIES/Y
/home/pi/DOWNLOAD/completed/xxxxxXxx.avi goes to /home/pi/TV/SERIES/X
/home/pi/DOWNLOAD/completed/xxxxxZxx.avi goes to /home/pi/TV/SERIES/Z
/home/pi/DOWNLOAD/completed/xxxxxxxx.avi goes to /home/pi/MOVIES/HD


I've struggled with rsync for a few days now and I think I got a good structure but somehow nothing happens when I run the .sh-script and when I try to run the rsync command manually it just say it is a (dry-run) even though I don't have the parameter -n in the command.

here is my "script" ^^'

#log-file
LOG_FILE=/home/pi/TV/rsynclogg.log

#the header for log-file
echo '-----------------------------------------------------------------------------' >> $LOG_FILE
date >> $LOG_FILE

#rsync commands
rsync -av --include='*/' --include='*X*.avi' --exclude '*' -prune-emty-dirs /home/pi/DOWNLOAD/completed /home/pi/TV/SERIES/X &>> $LOG_FILE
rsync -av --include='*/' --include='*Y*.avi' --exclude '*' -prune-emty-dirs /home/pi/DOWNLOAD/completed /home/pi/TV/SERIES/Y &>> $LOG_FILE
rsync -av --include='*/' --include='*Z*.avi' --exclude '*' -prune-emty-dirs /home/pi/DOWNLOAD/completed /home/pi/TV/SERIES/Z &>> $LOG_FILE
rsync -av --include='*/' --include='*.avi' --exclude '*' --exclude='*X*.avi' --exclude='*Y*.avi' --exclude='*Z*.avi' -prune-emty-dirs /home/pi/DOWNLOAD/completed /home/pi/MOVIES/HD &>> $LOG_FILE



I would really appreciate some help, maybe there is another tool I can use? I'm stuck and really desperate..


Please help me!
br J
 
Old 04-28-2014, 05:47 PM   #2
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,353
Blog Entries: 55

Rep: Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541
Something like this?
Code:
#!/bin/bash --
# Unset debug when done testing:
set -vxe
# Set default behaviour:
LANG=C; LC_ALL=C; export LANG LC_ALL
RSYNC_ARGS="--dry-run --checksum --itemize-changes --human-readable --stats --log-file=/home/pi/TV/rsync.log"

# Throw in some ways to nice things:
# which nice >/dev/null 2>&1 && nice -n +20 $$ >/dev/null 2>&1
# which ionice >/dev/null 2>&1 && ionice -c3 -p $$ >/dev/null 2>&1
# RSYNC_ARGS="$RSYNC_ARGS --bwlimit=1m"

for ITEM in X Y Z; do
 _MYTMPFILE=`mktemp XXXXXXXXXX` && {
 find /home/pi/DOWNLOAD/completed/ -type f -name \*${ITEM}\*.avi > "${_MYTMPFILE}"
 rsync $RSYNC_ARGS --files-from="${_MYTMPFILE}" "/home/pi/TV/SERIES/${ITEM}/"
 }; rm -f "${_MYTMPFILE}"
done

_MYTMPFILE=`mktemp XXXXXXXXXX` && {
find /home/pi/DOWNLOAD/completed/ -type f \( -not -name \*X\*.avi -a -not -name \*Y\*.avi -a -not -name \*Z\*.avi \)
rsync $RSYNC_ARGS --files-from="${_MYTMPFILE}" /home/pi/MOVIES/HD/
}; rm -f "${_MYTMPFILE}"

exit 0
See 'man rsync' for explanation of rsync options.
Remove the "--dry-run" when you think it looks OK.
 
Old 04-28-2014, 06:33 PM   #3
Ruffe42
LQ Newbie
 
Registered: Apr 2014
Posts: 3

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by unSpawn View Post
Something like this?
Code:
#!/bin/bash --
# Unset debug when done testing:
set -vxe
# Set default behaviour:
LANG=C; LC_ALL=C; export LANG LC_ALL
RSYNC_ARGS="--dry-run --checksum --itemize-changes --human-readable --stats --log-file=/home/pi/TV/rsync.log"

# Throw in some ways to nice things:
# which nice >/dev/null 2>&1 && nice -n +20 $$ >/dev/null 2>&1
# which ionice >/dev/null 2>&1 && ionice -c3 -p $$ >/dev/null 2>&1
# RSYNC_ARGS="$RSYNC_ARGS --bwlimit=1m"

for ITEM in X Y Z; do
 _MYTMPFILE=`mktemp XXXXXXXXXX` && {
 find /home/pi/DOWNLOAD/completed/ -type f -name \*${ITEM}\*.avi > "${_MYTMPFILE}"
 rsync $RSYNC_ARGS --files-from="${_MYTMPFILE}" "/home/pi/TV/SERIES/${ITEM}/"
 }; rm -f "${_MYTMPFILE}"
done

_MYTMPFILE=`mktemp XXXXXXXXXX` && {
find /home/pi/DOWNLOAD/completed/ -type f \( -not -name \*X\*.avi -a -not -name \*Y\*.avi -a -not -name \*Z\*.avi \)
rsync $RSYNC_ARGS --files-from="${_MYTMPFILE}" /home/pi/MOVIES/HD/
}; rm -f "${_MYTMPFILE}"

exit 0
See 'man rsync' for explanation of rsync options.
Remove the "--dry-run" when you think it looks OK.
Looks neat, would love to get that working.
Would you mind explain more? I don't really know what to replace to make it run, if you could highlight or something. ITEM, MYTMPFILE
 
Old 04-28-2014, 08:37 PM   #4
Ruffe42
LQ Newbie
 
Registered: Apr 2014
Posts: 3

Original Poster
Rep: Reputation: Disabled
Got some nice ideas from your code, thx!

I'm looking for something lighter and came up with this:

Code:
sudo find /home/pi/DOWNLOAD -type f \( -not -iname '*sample*.mkv' -a -iname '*X*.mkv' \) -exec cp -i -n {} "/home/pi/TV/SERIES/X" \;
sudo find /home/pi/DOWNLOAD -type f \( -not -iname '*sample*.mkv' -a -iname '*Y*.mkv' \) -exec cp -i -n {} "/home/pi/TV/SERIES/Y" \;
sudo find /home/pi/DOWNLOAD -type f \( -not -iname '*sample*.mkv' -a -iname '*Z*.mkv' \) -exec cp -i -n {} "/home/pi/TV/SERIES/Z" \;
sudo find /home/pi/DOWNLOAD -type f \( -not -iname '*sample*.mkv' -a -iname '*W*.mkv' \) -exec cp -i -n {} "/home/pi/TV/SERIES/W" \;
sudo find /home/pi/DOWNLOAD -type f \( -not -iname '*sample*.mkv' -a -iname '*V*.mkv' \) -exec cp -i -n {} "/home/pi/TV/SERIES/V" \;

sudo find /home/pi/DOWNLOAD -type f \( -name '*.mkv' -a -not -iname '*X*.mkv' -a -not -iname '*Y*.mkv' -a -not -iname '*Z*.mkv' -a -not -iname '*W*.mkv' -a -not -iname 'V*.mkv' -a -not -iname '*sample*.mkv' \) -exec cp -i -n {} "/home/pi/MOVIES/HD" \;
Each line runs smooth and good in the cmd, but when I run the .sh file, nothing happens.
How do I turn the code above to a working script?

br J
 
Old 04-30-2014, 02:51 PM   #5
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,353
Blog Entries: 55

Rep: Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541
Quote:
Originally Posted by Ruffe42 View Post
I don't really know what to replace to make it run, if you could highlight or something. ITEM, MYTMPFILE
If you save the code as script, say "/usr/local/bin/rsync_movies.sh" you can run it as '/bin/bash /usr/local/bin/rsync_movies.sh'.
The output will show you what it does (how it works) but it will essentially move no files.
If you no longer want to see what it does comment out the "set -vxe" line (should look like "#set -vxe" then). You should still see output in "/home/pi/TV/rsync.log".
If you think it's what you want then you have to delete the "--dry-run" switch from the RSYNC_ARGS= line (looks like RSYNC_ARGS="--checksum --itemize-changes --human-readable --stats --log-file=/home/pi/TV/rsync.log" then).
 
Old 04-30-2014, 02:55 PM   #6
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,353
Blog Entries: 55

Rep: Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541Reputation: 3541
Quote:
Originally Posted by Ruffe42 View Post
How do I turn the code above to a working script?
Save this:
Code:
#!/bin/bash

find /home/pi/DOWNLOAD -type f \( -not -iname '*sample*.mkv' -a -iname '*X*.mkv' \) -exec cp -i -n {} "/home/pi/TV/SERIES/X" \;
find /home/pi/DOWNLOAD -type f \( -not -iname '*sample*.mkv' -a -iname '*Y*.mkv' \) -exec cp -i -n {} "/home/pi/TV/SERIES/Y" \;
find /home/pi/DOWNLOAD -type f \( -not -iname '*sample*.mkv' -a -iname '*Z*.mkv' \) -exec cp -i -n {} "/home/pi/TV/SERIES/Z" \;
find /home/pi/DOWNLOAD -type f \( -not -iname '*sample*.mkv' -a -iname '*W*.mkv' \) -exec cp -i -n {} "/home/pi/TV/SERIES/W" \;
find /home/pi/DOWNLOAD -type f \( -not -iname '*sample*.mkv' -a -iname '*V*.mkv' \) -exec cp -i -n {} "/home/pi/TV/SERIES/V" \;
find /home/pi/DOWNLOAD -type f \( -name '*.mkv' -a -not -iname '*X*.mkv' -a -not -iname '*Y*.mkv' -a -not -iname '*Z*.mkv' -a -not -iname '*W*.mkv' -a -not -iname 'V*.mkv' -a -not -iname '*sample*.mkv' \) -exec cp -i -n {} "/home/pi/MOVIES/HD" \;

exit 0
as say "/usr/local/bin/move_movies.sh" and make it executable ('chmod 0755 /usr/local/bin/move_movies.sh'). Then if you execute '/usr/local/bin/move_movies.sh' (shouldn't need sudo if your home is /home/pi/ else use 'sudo /usr/local/bin/move_movies.sh') files should be moved according to your script. IIRC the curly braces need to be escaped BTW.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Files seem to take up more space in destination after rsync copy Karderio Linux - Newbie 6 09-22-2014 07:41 PM
[SOLVED] copy files with rsync or dd vnc Linux - Newbie 4 02-07-2013 05:51 AM
RSYNC Help - Copy Only Changed Files to New Directory euphoricrhino Linux - Software 1 11-23-2011 03:29 PM
Rsync can't copy certain files with unusual characters while cp can smithaa02 Linux - Software 7 11-15-2010 02:26 PM
[SOLVED] shell script to copy newly changed files with rsync genderbender Programming 15 04-22-2010 07:27 AM


All times are GMT -5. The time now is 07:43 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration