LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 05-26-2011, 09:03 AM   #1
Skaperen
Senior Member
 
Registered: May 2009
Location: center of singularity
Distribution: Xubuntu, Ubuntu, Slackware, Amazon Linux, OpenBSD, LFS (on Sparc_32 and i386)
Posts: 2,684
Blog Entries: 31

Rep: Reputation: 176Reputation: 176
verifying files from svn


Is there a way in svn to verify if an already existing file tree that "should" be at a given revision is in fact correct at that revision?

Suppose I had an earlier revision and obtains patches via svn diff for subsequent revisions, and applied them up to a given revision (or head). Now consider that perhaps some error happened during the application of these patches, or perhaps a file just got corrupted. What I would want to do is just verify if the tree I have now at this revision is the exactly tree I would get at this revision ... without actually pulling down the whole revision.

Note, I want to do this for exports, not for checkouts. That may make it simpler since the result of svn diff seems to only be for the actual data files.
 
Old 05-27-2011, 03:35 AM   #2
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,417

Rep: Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985
doing that on an export is much harder as what you have locally is NOT SVN, they're just standard files so svn can not have any view about it at all. if it was a checkout then the "svn update" command does exactly what you want.
 
Old 05-27-2011, 04:42 PM   #3
Skaperen
Senior Member
 
Registered: May 2009
Location: center of singularity
Distribution: Xubuntu, Ubuntu, Slackware, Amazon Linux, OpenBSD, LFS (on Sparc_32 and i386)
Posts: 2,684

Original Poster
Blog Entries: 31

Rep: Reputation: 176Reputation: 176
Quote:
Originally Posted by acid_kewpie View Post
doing that on an export is much harder as what you have locally is NOT SVN, they're just standard files so svn can not have any view about it at all. if it was a checkout then the "svn update" command does exactly what you want.
I understand that. What I'm looking for is something like how rsync works, but with a perspective of SVN. So could checksums work? All I need to do, per file, is to avoid the transfer of files I already have that are identical (if they are different, then the transfer is needed).
 
Old 05-28-2011, 02:49 AM   #4
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,417

Rep: Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985
well maybe you should use rsync then. svnfs will let you mount a repo as a filesystem and then you can rsync on top of that, but exclude the .svn dirs.
 
Old 05-28-2011, 11:20 PM   #5
Tinkster
Moderator
 
Registered: Apr 2002
Location: earth
Distribution: slackware by choice, others too :} ... android.
Posts: 23,067
Blog Entries: 11

Rep: Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928
Moved: This thread is more suitable in <Linux-Software> and has been moved accordingly to help your thread/question get the exposure it deserves.
 
Old 05-31-2011, 12:14 PM   #6
Skaperen
Senior Member
 
Registered: May 2009
Location: center of singularity
Distribution: Xubuntu, Ubuntu, Slackware, Amazon Linux, OpenBSD, LFS (on Sparc_32 and i386)
Posts: 2,684

Original Poster
Blog Entries: 31

Rep: Reputation: 176Reputation: 176
Quote:
Originally Posted by acid_kewpie View Post
well maybe you should use rsync then. svnfs will let you mount a repo as a filesystem and then you can rsync on top of that, but exclude the .svn dirs.
Unfortunately, I don't have control over the SVN server. I have only read access to SVN and no other access to the machine at all. In fact, what I would download via SVN I was planning to re-export via rsync, anyway.
 
Old 05-31-2011, 12:26 PM   #7
jamesf
Member
 
Registered: Dec 2004
Location: USA
Distribution: Slackware 12, Slackware64 14.2
Posts: 236
Blog Entries: 1

Rep: Reputation: 57
Quote:
Originally Posted by Skaperen View Post
I understand that. What I'm looking for is something like how rsync works, but with a perspective of SVN. So could checksums work? All I need to do, per file, is to avoid the transfer of files I already have that are identical (if they are different, then the transfer is needed).
Think about that. Where would the checksum of the svn-hosted file come from? You'd retrieve the file and compute the checksum, then compute the local checksum, compare, and then if there's a difference you'd retrieve the file from svn again to overwrite the local copy. You've just retrieved the file at least once, and at most twice, per file in your local copy of the revision for which you are checking.

And what about those files that might have been deleted from your exported copy? How will you know that they exist in the revision and not locally? What about files added to your local export, but never imported into subversion? What if those files would have been handled by the svn exclude list?

I work with subversion every day for multiple projects. I am not claiming to be an expert, but I'm familiar with what we do. I have to ask, though, why spend so much time crafting a solution that is solved either with 'svn update' for a working copy or with 'svn export' for a clean (no .svn dirs)?

good luck!
 
Old 05-31-2011, 01:04 PM   #8
Skaperen
Senior Member
 
Registered: May 2009
Location: center of singularity
Distribution: Xubuntu, Ubuntu, Slackware, Amazon Linux, OpenBSD, LFS (on Sparc_32 and i386)
Posts: 2,684

Original Poster
Blog Entries: 31

Rep: Reputation: 176Reputation: 176
Quote:
Originally Posted by jamesf View Post
Think about that. Where would the checksum of the svn-hosted file come from? You'd retrieve the file and compute the checksum, then compute the local checksum, compare, and then if there's a difference you'd retrieve the file from svn again to overwrite the local copy. You've just retrieved the file at least once, and at most twice, per file in your local copy of the revision for which you are checking.
I did think about that. That's why I was asking if there was a way to do it through the server, so the file would not have to be transferred. I was thinking about how rsync did it. What rsync does is compute the checksum at the server, and transmits just the checksum over the network, when the -c option is used. Unless -W is used, it transmits checksums of blocks of data, too.

Maybe the SVN server already has checksums cached and ready to use. Or maybe it's OK to have it recompute.

Quote:
Originally Posted by jamesf View Post
And what about those files that might have been deleted from your exported copy? How will you know that they exist in the revision and not locally? What about files added to your local export, but never imported into subversion? What if those files would have been handled by the svn exclude list?
If the server sends a list of files, the clients detects a file not in the list and deletes it if the list was transmitted in sorted order or finished. There would be no files added in the normal sense, as this isn't a checkout.

Quote:
Originally Posted by jamesf View Post
I work with subversion every day for multiple projects. I am not claiming to be an expert, but I'm familiar with what we do. I have to ask, though, why spend so much time crafting a solution that is solved either with 'svn update' for a working copy or with 'svn export' for a clean (no .svn dirs)?
To minimize network traffic.

I suppose if "svn update" really will minimize traffic, I might be able to live with the extra wasted space in a staging area and do a 2nd stage re-export via rsync with --exclide to take out .svn directories.
 
Old 05-31-2011, 01:28 PM   #9
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,417

Rep: Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985
Quote:
Originally Posted by Skaperen View Post
Unfortunately, I don't have control over the SVN server. I have only read access to SVN and no other access to the machine at all. In fact, what I would download via SVN I was planning to re-export via rsync, anyway.
you don't need access to the server, svnfs is just a fuse filesystem and svn client.
 
Old 05-31-2011, 01:42 PM   #10
Skaperen
Senior Member
 
Registered: May 2009
Location: center of singularity
Distribution: Xubuntu, Ubuntu, Slackware, Amazon Linux, OpenBSD, LFS (on Sparc_32 and i386)
Posts: 2,684

Original Poster
Blog Entries: 31

Rep: Reputation: 176Reputation: 176
Quote:
Originally Posted by acid_kewpie View Post
you don't need access to the server, svnfs is just a fuse filesystem and svn client.
How would it compute a checksum of a file without transferring that file over the network (avoiding this is the whole objective)?
 
Old 05-31-2011, 01:47 PM   #11
jamesf
Member
 
Registered: Dec 2004
Location: USA
Distribution: Slackware 12, Slackware64 14.2
Posts: 236
Blog Entries: 1

Rep: Reputation: 57
Quote:
Originally Posted by Skaperen View Post
I did think about that. That's why I was asking if there was a way to do it through the server, so the file would not have to be transferred. I was thinking about how rsync did it. What rsync does is compute the checksum at the server, and transmits just the checksum over the network, when the -c option is used. Unless -W is used, it transmits checksums of blocks of data, too.


Maybe the SVN server already has checksums cached and ready to use. Or maybe it's OK to have it recompute.
I'm an idiot. Sorry, now I see what you were thinking about. I'm not aware of any way to get svn to send you a list of files and their checksums. Doesn't mean there isn't one, but if there is I don't know it.


Quote:
[snippage]
I suppose if "svn update" really will minimize traffic, I might be able to live with the extra wasted space in a staging area and do a 2nd stage re-export via rsync with --exclide to take out .svn directories.
Now that sounds like a pretty good solution! ;v)

The only way you might have a problem is for locally edited files in your working copy. 'svn update' won't revert those changes, but only bring up-to-date files changed on the server (committed) but at a lower revision in your checkout.

So, something like this?
Code:
cd /path/to/local/working/copy
# Make sure to discard local edits
svn revert --recursive .
# Pick up changes from the server
svn update
then either
Code:
# Discard any changes to rsync area
rmdir -r -f /path/to/rsync/staging
# repopulate rsync area
svn export . /path/to/rsync/staging
or your rsync excluding svn dirs. I like your way better.
 
Old 05-31-2011, 01:48 PM   #12
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,417

Rep: Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985
well if you're rsyncing you'd have to make do with size and date really.
 
Old 05-31-2011, 02:32 PM   #13
jamesf
Member
 
Registered: Dec 2004
Location: USA
Distribution: Slackware 12, Slackware64 14.2
Posts: 236
Blog Entries: 1

Rep: Reputation: 57
Quote:
Originally Posted by acid_kewpie View Post
well if you're rsyncing you'd have to make do with size and date really.
Unless the rsync daemon is running, no? Won't it help with calculating checksums on whatever machine is running the daemon?
 
Old 05-31-2011, 02:56 PM   #14
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,417

Rep: Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985Reputation: 1985
Quote:
Originally Posted by jamesf View Post
Unless the rsync daemon is running, no? Won't it help with calculating checksums on whatever machine is running the daemon?
well the point raised is that to get the checksum you'd need to read the whole file, and in the context of a mount of an svn repo would mean downloading the entire file each time.
 
Old 05-31-2011, 03:12 PM   #15
jamesf
Member
 
Registered: Dec 2004
Location: USA
Distribution: Slackware 12, Slackware64 14.2
Posts: 236
Blog Entries: 1

Rep: Reputation: 57
Quote:
Originally Posted by acid_kewpie View Post
well the point raised is that to get the checksum you'd need to read the whole file, and in the context of a mount of an svn repo would mean downloading the entire file each time.
Ah, yes. But I had thought that the poster was now suggesting to use svn update and then using rsync (excluding .svn dirs) to perform the equivalent of an 'svn export' on the local working copy.

So, svn to handle transferring on changes from svn, rsync to handle local working copy to rsync 'staging' area (if it is still necessary).

That is, of course, if I understood his suggestions correctly. ;v)
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
copy svn working dir without svn hidden dirs and files? erinspice Linux - Software 4 03-29-2011 05:58 PM
How to update files with svn danndp Linux - Newbie 1 12-13-2010 06:56 PM
svn: show who edited what files? trey85stang Linux - General 1 06-05-2009 07:01 PM
viewing checked out files using SVN girish_hilage Programming 2 06-14-2008 06:16 AM
Verifying Rsync Backups of Large Volumes of Files mcgirvanmedia Linux - Server 2 06-03-2008 11:30 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 12:16 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration