LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 06-24-2013, 05:50 PM   #1
mreff555
Member
 
Registered: Sep 2011
Location: Philly
Distribution: Gentoo
Posts: 473

Rep: Reputation: Disabled
sed help


I have a <newline> separated list of files at various websites. eg.

Code:
http://blah.com/thing1.tar.gz
http://blah.com/thing2.tar.gz
I want to write a script to read this list and determine if all these files are in a directory.

I think I can figure out the rest, but I'm having trouble working out a sed statement to take a line of text and remove everything before and including the last forward slash. So I'm just left with a filename.
 
Old 06-24-2013, 06:09 PM   #2
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,284

Rep: Reputation: 2347Reputation: 2347Reputation: 2347Reputation: 2347Reputation: 2347Reputation: 2347Reputation: 2347Reputation: 2347Reputation: 2347Reputation: 2347Reputation: 2347
How about param expansion
Code:
t='http://blah.com/thing1.tar.gz'

 echo ${t##*/}
thing1.tar.gz
http://wiki.bash-hackers.org/syntax/...string_removal
 
1 members found this post helpful.
Old 06-24-2013, 06:27 PM   #3
linuxCode
Member
 
Registered: Jun 2013
Location: New York City
Distribution: Arch
Posts: 33

Rep: Reputation: 43
Try this script

Code:
#/bin/bash

while read line
do
if [ -f $(echo ${line##*/}) ]; then echo "${line##*/} exists"
else
echo "${line##*/} does not exists" 
fi
done < list
Replace list with the file of http links

This script will extract the filename from the http link and check the current directory for existence of file or not

Bonus Code with Awk:

Code:
awk -F "/" '{print $NF }' list

Last edited by linuxCode; 06-24-2013 at 06:58 PM.
 
Old 06-25-2013, 09:41 AM   #4
mreff555
Member
 
Registered: Sep 2011
Location: Philly
Distribution: Gentoo
Posts: 473

Original Poster
Rep: Reputation: Disabled
Thanks guys, it worked great.
 
Old 06-25-2013, 03:53 PM   #5
linuxCode
Member
 
Registered: Jun 2013
Location: New York City
Distribution: Arch
Posts: 33

Rep: Reputation: 43
You're welcome mreff555

Mark the thread solved from the threads tool if you feel it was solved.
 
Old 06-28-2013, 05:06 AM   #6
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958
Quote:
Originally Posted by linuxCode View Post
Code:
if [ -f $(echo ${line##*/}) ]; then echo "${line##*/} exists"
Useless Use Of Echo. Just evaluate the expansion directly.

Also, when using advanced shells like bash or ksh, it's recommended to use [[..]] for string/file tests, and ((..)) for numerical tests. Avoid using the old [..] test unless you specifically need POSIX-style portability.

http://mywiki.wooledge.org/BashFAQ/031
http://mywiki.wooledge.org/ArithmeticExpression

If you use single brackets you have to make sure to enclose the string in quotes so that it's properly protected from word-splitting and glob expansion.

And finally, the formatting could be a bit cleaner.

I'd personally write it more like this:

Code:
#/bin/bash

listfile=/path/to/list.txt

while read -r line || [[ -n $line ]]; do

    line=${line##*/}

    if [[ -f $line ]]; then
        echo "$line exists"
    else
        echo "$line does not exist" 
    fi

done < "$listfile"
The extra [[ -n $line ]] test protects against the possibility of the input having no final newline.
 
Old 06-28-2013, 05:45 AM   #7
linuxzilla.com
LQ Newbie
 
Registered: May 2013
Distribution: CentOS, Ubuntu, Debian
Posts: 22

Rep: Reputation: Disabled
Quote:
Originally Posted by linuxCode View Post
Try this script

Code:
#/bin/bash

while read line
do
if [ -f $(echo ${line##*/}) ]; then echo "${line##*/} exists"
else
echo "${line##*/} does not exists" 
fi
done < list
Replace list with the file of http links

This script will extract the filename from the http link and check the current directory for existence of file or not

Bonus Code with Awk:

Code:
awk -F "/" '{print $NF }' list
Well that is great & I've saved it with me. Very smart script you've written. Thanks ))
 
Old 06-28-2013, 08:05 PM   #8
linuxCode
Member
 
Registered: Jun 2013
Location: New York City
Distribution: Arch
Posts: 33

Rep: Reputation: 43
Quote:
Originally Posted by David the H. View Post
Useless Use Of Echo. Just evaluate the expansion directly.

Also, when using advanced shells like bash or ksh, it's recommended to use [[..]] for string/file tests, and ((..)) for numerical tests. Avoid using the old [..] test unless you specifically need POSIX-style portability.

http://mywiki.wooledge.org/BashFAQ/031
http://mywiki.wooledge.org/ArithmeticExpression

If you use single brackets you have to make sure to enclose the string in quotes so that it's properly protected from word-splitting and glob expansion.

And finally, the formatting could be a bit cleaner.

I'd personally write it more like this:

Code:
#/bin/bash

listfile=/path/to/list.txt

while read -r line || [[ -n $line ]]; do

    line=${line##*/}

    if [[ -f $line ]]; then
        echo "$line exists"
    else
        echo "$line does not exist" 
    fi

done < "$listfile"
The extra [[ -n $line ]] test protects against the possibility of the input having no final newline.
Thanks David the H. for the bash tips.

+1

Last edited by linuxCode; 06-28-2013 at 08:09 PM.
 
Old 06-28-2013, 08:07 PM   #9
linuxCode
Member
 
Registered: Jun 2013
Location: New York City
Distribution: Arch
Posts: 33

Rep: Reputation: 43
Quote:
Originally Posted by linuxzilla.com View Post
Well that is great & I've saved it with me. Very smart script you've written. Thanks ))
You're Welcome
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Multipal line edited using sed, how to make sed specific coolpraz Programming 4 01-05-2013 01:14 PM
[Cygwin, sed] Using filenames as both files and search strings within sed lingh Linux - Newbie 5 10-20-2012 10:38 AM
[SOLVED] sed help to run sed command against multiple different file names bkone Programming 2 04-16-2012 12:27 PM
[SOLVED] sed 's/Tb05.5K5.100/Tb229/' alone but doesn't work in sed file w/ other expressions Radha.jg Programming 6 03-03-2011 07:59 AM
Insert character into a line with sed? & variables in sed? jago25_98 Programming 5 03-11-2004 06:12 AM


All times are GMT -5. The time now is 06:26 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration