LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 11-05-2007, 11:54 AM   #1
tmbigrigg
LQ Newbie
 
Registered: Nov 2007
Location: Canton, Ohio
Distribution: Ubuntu, Mepis, Damn Small Linux,
Posts: 10

Rep: Reputation: 0
question about shell script


I hope this is the right place for this post...

I have a shell script (I have forgotten the original author) that will search recursively through directories looking for duplicate files, It then generates a second script to remove the duplicates with one file per line. Each file is commented out. I would like to be able to automatically remove the comment from one of the lines. Then I can run script to delete the duplicate files, in this case they are .jpg files.

I will include part of the script as an example:
I would like to remove the comment from one file leave the other (oldest date) commented out. In the following example I would like to remove the comment from the lines that have been placed in bold. Any help would be greatly appreciated, the actual file is huge somewhere around 2072 lines.

#! /bin/sh
#rm ./2006-05-09/100_4375.jpg
#rm ./2006-06-25/100_4375.jpg

#rm ./2006-08-04/100_4906.jpg
#rm ./2006-08-26/100_4906.jpg
#rm ./2006-10-21/100_4906.jpg

#rm ./2004-11-25/100b0270.jpg
#rm ./2004-12-24/100b0270.jpg


tmbigrigg
 
Old 11-05-2007, 12:13 PM   #2
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,335
Blog Entries: 55

Rep: Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535Reputation: 3535
Wouldn't it be better to post the (D/L location of) the script? With all due respect but writing a kludge for this instead of the source seems a bit of a waste of time.
 
Old 11-05-2007, 12:18 PM   #3
tmbigrigg
LQ Newbie
 
Registered: Nov 2007
Location: Canton, Ohio
Distribution: Ubuntu, Mepis, Damn Small Linux,
Posts: 10

Original Poster
Rep: Reputation: 0
Here is where I found the original script: http://elonen.iki.fi/code/misc-notes...les/index.html

OUTF=rem-duplicates.sh;
echo "#! /bin/sh" > $OUTF;
find "$@" -type f -print0 |
xargs -0 -n1 md5sum |
sort --key=1,32 | uniq -w 32 -d --all-repeated=separate |
sed -r 's/^[0-9a-f]*( )*//;s/([^a-zA-Z0-9./_-])/\\\1/g;s/(.+)/#rm \1/' >> $OUTF;
chmod a+x $OUTF; ls -l $OUTF
 
Old 11-05-2007, 12:47 PM   #4
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738
to selectively modify certain lines in a file, you would have to be able to define some common attribute. In your example, it appears that the common attribute is that you want to modify based on folder names which are dates later than the first one in the list. This might be messy.

An easier algorithm would be to remove everything except the first entry in a group. Then you can simply loop on the file name, removing all instances after the first one.
 
Old 11-05-2007, 02:35 PM   #5
David1357
Senior Member
 
Registered: Aug 2007
Location: South Carolina, U.S.A.
Distribution: Ubuntu, Fedora Core, Red Hat, SUSE, Gentoo, DSL, coLinux, uClinux
Posts: 1,302
Blog Entries: 1

Rep: Reputation: 107Reputation: 107
Re: question about shell script

You are probably not going to find an easy way to do what you want to do in a bash script.

If you are determined to use a script, you will probably have to use Perl, PHP, Python, etc.

I am a programmer, so I would write a C program to do this, but that option is not available to everyone.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
shell script Question Whiteghost Programming 4 09-25-2005 10:05 PM
Another shell script question J_Szucs Programming 4 09-10-2005 02:47 PM
Shell script question. pete1234 Programming 10 09-06-2005 02:46 AM
Shell script question... defa0009 Linux - General 7 04-26-2005 08:16 PM
Shell Script Question swinchen Programming 1 08-20-2004 02:09 PM


All times are GMT -5. The time now is 08:23 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration