LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 06-10-2010, 04:02 PM   #1
mattydee
Member
 
Registered: Dec 2006
Location: Vancouver, BC
Distribution: Debian,Ubuntu,Slackware
Posts: 479

Rep: Reputation: 48
rsnapshot and ntfs


Hello,

I am backing up data from a remote server onto a local ntfs partition. It seems that the rm -rf and cp -a commands are taking a long time to complete in what should be short, incremental backups.

Has anyone had similar problems when backing up to an ntfs partition? Thanks.

Here is my rsnapshot.conf:
Code:
#################################################
# rsnapshot.conf - rsnapshot configuration file #
#################################################
#                                               #
# PLEASE BE AWARE OF THE FOLLOWING RULES:       #
#                                               #
# This file requires tabs between elements      #
#                                               #
# Directories require a trailing slash:         #
#   right: /home/                               #
#   wrong: /home                                #
#                                               #
#################################################

#######################
# CONFIG FILE VERSION #
#######################

config_version	1.2

###########################
# SNAPSHOT ROOT DIRECTORY #
###########################

# All snapshots will be stored under this root directory.
#
snapshot_root	/lablab/backups

# If no_create_root is enabled, rsnapshot will not automatically create the
# snapshot_root directory. This is particularly useful if you are backing
# up to removable media, such as a FireWire or USB drive.
#
#no_create_root	1

#################################
# EXTERNAL PROGRAM DEPENDENCIES #
#################################

# LINUX USERS:   Be sure to uncomment "cmd_cp". This gives you extra features.
# EVERYONE ELSE: Leave "cmd_cp" commented out for compatibility.
#
# See the README file or the man page for more details.
#
cmd_cp		/bin/cp

# uncomment this to use the rm program instead of the built-in perl routine.
#
#cmd_rm		/bin/rm

# rsync must be enabled for anything to work. This is the only command that
# must be enabled.
#
cmd_rsync	/usr/bin/rsync

# Uncomment this to enable remote ssh backups over rsync.
#
cmd_ssh	/usr/bin/ssh

# Comment this out to disable syslog support.
#
cmd_logger	/usr/bin/logger

# Uncomment this to specify the path to "du" for disk usage checks.
# If you have an older version of "du", you may also want to check the
# "du_args" parameter below.
#
#cmd_du		/usr/bin/du

# Uncomment this to specify the path to rsnapshot-diff.
#
#cmd_rsnapshot_diff	/usr/local/bin/rsnapshot-diff

# Specify the path to a script (and any optional arguments) to run right
# before rsnapshot syncs files
#
#cmd_preexec	/path/to/preexec/script

# Specify the path to a script (and any optional arguments) to run right
# after rsnapshot syncs files
#
#cmd_postexec	/path/to/postexec/script

#########################################
#           BACKUP INTERVALS            #
# Must be unique and in ascending order #
# i.e. hourly, daily, weekly, etc.      #
#########################################

interval	hourly	4
interval	daily	7
interval	weekly	4
interval	monthly	3

############################################
#              GLOBAL OPTIONS              #
# All are optional, with sensible defaults #
############################################

# Verbose level, 1 through 5.
# 1     Quiet           Print fatal errors only
# 2     Default         Print errors and warnings only
# 3     Verbose         Show equivalent shell commands being executed
# 4     Extra Verbose   Show extra verbose information
# 5     Debug mode      Everything
#
verbose		2

# Same as "verbose" above, but controls the amount of data sent to the
# logfile, if one is being used. The default is 3.
#
loglevel	3

# If you enable this, data will be written to the file you specify. The
# amount of data written is controlled by the "loglevel" parameter.
#
logfile	/home/matty/logs/rsnapshot

# If enabled, rsnapshot will write a lockfile to prevent two instances
# from running simultaneously (and messing up the snapshot_root).
# If you enable this, make sure the lockfile directory is not world
# writable. Otherwise anyone can prevent the program from running.
#
# lockfile	/var/run/rsnapshot.pid

# Default rsync args. All rsync commands have at least these options set.
#
rsync_short_args	-rlt
#rsync_long_args	--delete --numeric-ids --relative --delete-excluded

# ssh has no args passed by default, but you can specify some here.
#
ssh_args	-p 6958

# Default arguments for the "du" program (for disk space reporting).
# The GNU version of "du" is preferred. See the man page for more details.
# If your version of "du" doesn't support the -h flag, try -k flag instead.
#
#du_args	-csh

# If this is enabled, rsync won't span filesystem partitions within a
# backup point. This essentially passes the -x option to rsync.
# The default is 0 (off).
#
#one_fs		0

# The include and exclude parameters, if enabled, simply get passed directly
# to rsync. If you have multiple include/exclude patterns, put each one on a
# separate line. Please look up the --include and --exclude options in the
# rsync man page for more details on how to specify file name patterns. 
# 
#include	???
#include	???
exclude	data/*


# The include_file and exclude_file parameters, if enabled, simply get
# passed directly to rsync. Please look up the --include-from and
# --exclude-from options in the rsync man page for more details.
#
#include_file	/path/to/include/file
#exclude_file	/path/to/exclude/file

# If your version of rsync supports --link-dest, consider enable this.
# This is the best way to support special files (FIFOs, etc) cross-platform.
# The default is 0 (off).
#
#link_dest	0

# When sync_first is enabled, it changes the default behaviour of rsnapshot.
# Normally, when rsnapshot is called with its lowest interval
# (i.e.: "rsnapshot hourly"), it will sync files AND rotate the lowest
# intervals. With sync_first enabled, "rsnapshot sync" handles the file sync,
# and all interval calls simply rotate files. See the man page for more
# details. The default is 0 (off).
#
#sync_first	0

# If enabled, rsnapshot will move the oldest directory for each interval
# to [interval_name].delete, then it will remove the lockfile and delete
# that directory just before it exits. The default is 0 (off).
#
#use_lazy_deletes	0

###############################
### BACKUP POINTS / SCRIPTS ###
###############################

# LOCALHOST
#backup	/home/		localhost/
#backup	/etc/		localhost/
#backup	/usr/local/	localhost/
#backup	/var/log/rsnapshot		localhost/
#backup	/etc/passwd	localhost/
#backup	/home/foo/My Documents/		localhost/
#backup	/foo/bar/	localhost/	one_fs=1, rsync_short_args=-urltvpog
#backup_script		/usr/local/bin/backup_pgsql.sh	localhost/postgres/

# EXAMPLE.COM
#backup_script	/bin/date "+ backup of example.com started at %c"	unused1
#backup	root@example.com:/home/	example.com/	+rsync_long_args=--bwlimit=16,exclude=core
#backup	root@example.com:/etc/	example.com/	exclude=mtab,exclude=core
#backup_script	ssh root@example.com "mysqldump -A > /var/db/dump/mysql.sql"
#backup	root@example.com:/var/db/dump/	example.com/
#backup_script	/bin/date "+ backup of example.com ended at %c"	unused2

# CVS.SOURCEFORGE.NET
#backup_script	/usr/local/bin/backup_rsnapshot_cvsroot.sh	rsnapshot.cvs.sourceforge.net/

# RSYNC.SAMBA.ORG
#backup	rsync://rsync.samba.org/rsyncftp/	rsync.samba.org/rsyncftp/

backup	matty@xxxx.xxx.xxx.ca:/z/	lablab/
 
Old 06-11-2010, 01:32 PM   #2
mattydee
Member
 
Registered: Dec 2006
Location: Vancouver, BC
Distribution: Debian,Ubuntu,Slackware
Posts: 479

Original Poster
Rep: Reputation: 48
Clarification: By "long time" I mean 3 hours or so. The NTFS drive is the destination. Could it be a hard links issue?

Ex:
Code:
[11/Jun/2010:06:00:01] /usr/bin/rsnapshot -c /home/matty/rsnapshot-lablab.conf hourly: started
[11/Jun/2010:06:00:01] rm -rf /lablab/backups/hourly.3/
[11/Jun/2010:07:19:14] mv /lablab/backups/hourly.2/ /lablab/backups/hourly.3/
[11/Jun/2010:07:19:14] mv /lablab/backups/hourly.1/ /lablab/backups/hourly.2/
[11/Jun/2010:07:19:14] /bin/cp -al /lablab/backups/hourly.0 /lablab/backups/hourly.1
[11/Jun/2010:08:53:09] /usr/bin/rsync -rlt --delete --numeric-ids --relative --delete-excluded --exclude=data/* --rsh=/usr/bin/ssh -p xx matty@xxx.xxx.xxx.ca:/z /lablab/backups/hourly.0/lablab/
[11/Jun/2010:09:24:20] touch /lablab/backups/hourly.0/
[11/Jun/2010:09:24:20] /usr/bin/rsnapshot -c /home/matty/rsnapshot-lablab.conf hourly: completed successfully

Last edited by mattydee; 06-11-2010 at 01:50 PM.
 
Old 06-11-2010, 03:35 PM   #3
mostlyharmless
Senior Member
 
Registered: Jan 2008
Distribution: Arch/Manjaro, might try Slackware again
Posts: 1,859
Blog Entries: 14

Rep: Reputation: 285Reputation: 285Reputation: 285
I've had a similar issue with plain old rsync and ntfs. I can't remember the exact issue, but particularly for larger files it was much, much faster to simply copy the files over rather than rsync them. You might want to check out "rsync ntfs" on google; e.g.

http://ubuntuforums.org/showthread.php?t=820425

to see the kinds of problems people deal with. Hope that helps a bit.

Last edited by mostlyharmless; 06-11-2010 at 03:38 PM.
 
Old 06-12-2010, 01:27 PM   #4
mattydee
Member
 
Registered: Dec 2006
Location: Vancouver, BC
Distribution: Debian,Ubuntu,Slackware
Posts: 479

Original Poster
Rep: Reputation: 48
Thanks for the tip. I did have a simple rsync script working properly before, but I wanted rsnapshot for the extra features. Like I said the rsync part works well; it's the cp, mv and rm commands that take hours.

Also, du takes a really long time on ntfs partitions... don't know if that's related. Maybe I'll look into mount options in ntfs-3g... ?

Thanks
 
Old 06-20-2010, 03:57 PM   #5
mattydee
Member
 
Registered: Dec 2006
Location: Vancouver, BC
Distribution: Debian,Ubuntu,Slackware
Posts: 479

Original Poster
Rep: Reputation: 48
bump
 
Old 12-24-2010, 04:35 AM   #6
przepiorkowski
LQ Newbie
 
Registered: Dec 2010
Posts: 1

Rep: Reputation: 0
rsnapshot to NTFS and "link_dest 1" -- hardlinks problems

For people coming across this thread (like I did, looking for a solution to my problem), this bug report -- concerning hardlinks problems with rsnapshot to NTFS with the option "link_dest 1" set -- may be relevant:

https://sourceforge.net/tracker/?fun...group_id=88546
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Backup VM using rsnapshot cccc Linux - Virtualization and Cloud 3 02-05-2010 08:35 AM
Rsnapshot du automation steven19782007 Linux - Newbie 2 06-11-2009 01:42 AM
rsnapshot require Lchown ok4life Linux - Software 3 01-11-2008 07:22 PM
Backups with rsnapshot jeremy LQ Articles Discussion 0 07-30-2007 12:58 PM
rsync and rsnapshot madman100 Linux - Newbie 4 10-10-2006 09:33 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 03:15 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration