Linux - Server This forum is for the discussion of Linux Software used in a server related context. |
Notices |
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
Are you new to LinuxQuestions.org? Visit the following links:
Site Howto |
Site FAQ |
Sitemap |
Register Now
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
|
 |
06-10-2010, 04:02 PM
|
#1
|
Member
Registered: Dec 2006
Location: Vancouver, BC
Distribution: Debian,Ubuntu,Slackware
Posts: 479
Rep:
|
rsnapshot and ntfs
Hello,
I am backing up data from a remote server onto a local ntfs partition. It seems that the rm -rf and cp -a commands are taking a long time to complete in what should be short, incremental backups.
Has anyone had similar problems when backing up to an ntfs partition? Thanks.
Here is my rsnapshot.conf:
Code:
#################################################
# rsnapshot.conf - rsnapshot configuration file #
#################################################
# #
# PLEASE BE AWARE OF THE FOLLOWING RULES: #
# #
# This file requires tabs between elements #
# #
# Directories require a trailing slash: #
# right: /home/ #
# wrong: /home #
# #
#################################################
#######################
# CONFIG FILE VERSION #
#######################
config_version 1.2
###########################
# SNAPSHOT ROOT DIRECTORY #
###########################
# All snapshots will be stored under this root directory.
#
snapshot_root /lablab/backups
# If no_create_root is enabled, rsnapshot will not automatically create the
# snapshot_root directory. This is particularly useful if you are backing
# up to removable media, such as a FireWire or USB drive.
#
#no_create_root 1
#################################
# EXTERNAL PROGRAM DEPENDENCIES #
#################################
# LINUX USERS: Be sure to uncomment "cmd_cp". This gives you extra features.
# EVERYONE ELSE: Leave "cmd_cp" commented out for compatibility.
#
# See the README file or the man page for more details.
#
cmd_cp /bin/cp
# uncomment this to use the rm program instead of the built-in perl routine.
#
#cmd_rm /bin/rm
# rsync must be enabled for anything to work. This is the only command that
# must be enabled.
#
cmd_rsync /usr/bin/rsync
# Uncomment this to enable remote ssh backups over rsync.
#
cmd_ssh /usr/bin/ssh
# Comment this out to disable syslog support.
#
cmd_logger /usr/bin/logger
# Uncomment this to specify the path to "du" for disk usage checks.
# If you have an older version of "du", you may also want to check the
# "du_args" parameter below.
#
#cmd_du /usr/bin/du
# Uncomment this to specify the path to rsnapshot-diff.
#
#cmd_rsnapshot_diff /usr/local/bin/rsnapshot-diff
# Specify the path to a script (and any optional arguments) to run right
# before rsnapshot syncs files
#
#cmd_preexec /path/to/preexec/script
# Specify the path to a script (and any optional arguments) to run right
# after rsnapshot syncs files
#
#cmd_postexec /path/to/postexec/script
#########################################
# BACKUP INTERVALS #
# Must be unique and in ascending order #
# i.e. hourly, daily, weekly, etc. #
#########################################
interval hourly 4
interval daily 7
interval weekly 4
interval monthly 3
############################################
# GLOBAL OPTIONS #
# All are optional, with sensible defaults #
############################################
# Verbose level, 1 through 5.
# 1 Quiet Print fatal errors only
# 2 Default Print errors and warnings only
# 3 Verbose Show equivalent shell commands being executed
# 4 Extra Verbose Show extra verbose information
# 5 Debug mode Everything
#
verbose 2
# Same as "verbose" above, but controls the amount of data sent to the
# logfile, if one is being used. The default is 3.
#
loglevel 3
# If you enable this, data will be written to the file you specify. The
# amount of data written is controlled by the "loglevel" parameter.
#
logfile /home/matty/logs/rsnapshot
# If enabled, rsnapshot will write a lockfile to prevent two instances
# from running simultaneously (and messing up the snapshot_root).
# If you enable this, make sure the lockfile directory is not world
# writable. Otherwise anyone can prevent the program from running.
#
# lockfile /var/run/rsnapshot.pid
# Default rsync args. All rsync commands have at least these options set.
#
rsync_short_args -rlt
#rsync_long_args --delete --numeric-ids --relative --delete-excluded
# ssh has no args passed by default, but you can specify some here.
#
ssh_args -p 6958
# Default arguments for the "du" program (for disk space reporting).
# The GNU version of "du" is preferred. See the man page for more details.
# If your version of "du" doesn't support the -h flag, try -k flag instead.
#
#du_args -csh
# If this is enabled, rsync won't span filesystem partitions within a
# backup point. This essentially passes the -x option to rsync.
# The default is 0 (off).
#
#one_fs 0
# The include and exclude parameters, if enabled, simply get passed directly
# to rsync. If you have multiple include/exclude patterns, put each one on a
# separate line. Please look up the --include and --exclude options in the
# rsync man page for more details on how to specify file name patterns.
#
#include ???
#include ???
exclude data/*
# The include_file and exclude_file parameters, if enabled, simply get
# passed directly to rsync. Please look up the --include-from and
# --exclude-from options in the rsync man page for more details.
#
#include_file /path/to/include/file
#exclude_file /path/to/exclude/file
# If your version of rsync supports --link-dest, consider enable this.
# This is the best way to support special files (FIFOs, etc) cross-platform.
# The default is 0 (off).
#
#link_dest 0
# When sync_first is enabled, it changes the default behaviour of rsnapshot.
# Normally, when rsnapshot is called with its lowest interval
# (i.e.: "rsnapshot hourly"), it will sync files AND rotate the lowest
# intervals. With sync_first enabled, "rsnapshot sync" handles the file sync,
# and all interval calls simply rotate files. See the man page for more
# details. The default is 0 (off).
#
#sync_first 0
# If enabled, rsnapshot will move the oldest directory for each interval
# to [interval_name].delete, then it will remove the lockfile and delete
# that directory just before it exits. The default is 0 (off).
#
#use_lazy_deletes 0
###############################
### BACKUP POINTS / SCRIPTS ###
###############################
# LOCALHOST
#backup /home/ localhost/
#backup /etc/ localhost/
#backup /usr/local/ localhost/
#backup /var/log/rsnapshot localhost/
#backup /etc/passwd localhost/
#backup /home/foo/My Documents/ localhost/
#backup /foo/bar/ localhost/ one_fs=1, rsync_short_args=-urltvpog
#backup_script /usr/local/bin/backup_pgsql.sh localhost/postgres/
# EXAMPLE.COM
#backup_script /bin/date "+ backup of example.com started at %c" unused1
#backup root@example.com:/home/ example.com/ +rsync_long_args=--bwlimit=16,exclude=core
#backup root@example.com:/etc/ example.com/ exclude=mtab,exclude=core
#backup_script ssh root@example.com "mysqldump -A > /var/db/dump/mysql.sql"
#backup root@example.com:/var/db/dump/ example.com/
#backup_script /bin/date "+ backup of example.com ended at %c" unused2
# CVS.SOURCEFORGE.NET
#backup_script /usr/local/bin/backup_rsnapshot_cvsroot.sh rsnapshot.cvs.sourceforge.net/
# RSYNC.SAMBA.ORG
#backup rsync://rsync.samba.org/rsyncftp/ rsync.samba.org/rsyncftp/
backup matty@xxxx.xxx.xxx.ca:/z/ lablab/
|
|
|
06-11-2010, 01:32 PM
|
#2
|
Member
Registered: Dec 2006
Location: Vancouver, BC
Distribution: Debian,Ubuntu,Slackware
Posts: 479
Original Poster
Rep:
|
Clarification: By "long time" I mean 3 hours or so. The NTFS drive is the destination. Could it be a hard links issue?
Ex:
Code:
[11/Jun/2010:06:00:01] /usr/bin/rsnapshot -c /home/matty/rsnapshot-lablab.conf hourly: started
[11/Jun/2010:06:00:01] rm -rf /lablab/backups/hourly.3/
[11/Jun/2010:07:19:14] mv /lablab/backups/hourly.2/ /lablab/backups/hourly.3/
[11/Jun/2010:07:19:14] mv /lablab/backups/hourly.1/ /lablab/backups/hourly.2/
[11/Jun/2010:07:19:14] /bin/cp -al /lablab/backups/hourly.0 /lablab/backups/hourly.1
[11/Jun/2010:08:53:09] /usr/bin/rsync -rlt --delete --numeric-ids --relative --delete-excluded --exclude=data/* --rsh=/usr/bin/ssh -p xx matty@xxx.xxx.xxx.ca:/z /lablab/backups/hourly.0/lablab/
[11/Jun/2010:09:24:20] touch /lablab/backups/hourly.0/
[11/Jun/2010:09:24:20] /usr/bin/rsnapshot -c /home/matty/rsnapshot-lablab.conf hourly: completed successfully
Last edited by mattydee; 06-11-2010 at 01:50 PM.
|
|
|
06-11-2010, 03:35 PM
|
#3
|
Senior Member
Registered: Jan 2008
Distribution: Arch/Manjaro, might try Slackware again
Posts: 1,859
|
I've had a similar issue with plain old rsync and ntfs. I can't remember the exact issue, but particularly for larger files it was much, much faster to simply copy the files over rather than rsync them. You might want to check out "rsync ntfs" on google; e.g.
http://ubuntuforums.org/showthread.php?t=820425
to see the kinds of problems people deal with. Hope that helps a bit.
Last edited by mostlyharmless; 06-11-2010 at 03:38 PM.
|
|
|
06-12-2010, 01:27 PM
|
#4
|
Member
Registered: Dec 2006
Location: Vancouver, BC
Distribution: Debian,Ubuntu,Slackware
Posts: 479
Original Poster
Rep:
|
Thanks for the tip. I did have a simple rsync script working properly before, but I wanted rsnapshot for the extra features. Like I said the rsync part works well; it's the cp, mv and rm commands that take hours.
Also, du takes a really long time on ntfs partitions... don't know if that's related. Maybe I'll look into mount options in ntfs-3g... ?
Thanks
|
|
|
06-20-2010, 03:57 PM
|
#5
|
Member
Registered: Dec 2006
Location: Vancouver, BC
Distribution: Debian,Ubuntu,Slackware
Posts: 479
Original Poster
Rep:
|
bump
|
|
|
12-24-2010, 04:35 AM
|
#6
|
LQ Newbie
Registered: Dec 2010
Posts: 1
Rep:
|
rsnapshot to NTFS and "link_dest 1" -- hardlinks problems
For people coming across this thread (like I did, looking for a solution to my problem), this bug report -- concerning hardlinks problems with rsnapshot to NTFS with the option "link_dest 1" set -- may be relevant:
https://sourceforge.net/tracker/?fun...group_id=88546
|
|
|
All times are GMT -5. The time now is 03:15 PM.
|
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
|
Latest Threads
LQ News
|
|