LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 08-16-2006, 06:11 AM   #1
walkingman
LQ Newbie
 
Registered: Aug 2006
Posts: 6

Rep: Reputation: 0
Program that can sort out multimedia links from html page


Hi !

Anybody know if there is any script or batch program out that can give you the links from a given URL ?

Thank you
 
Old 08-16-2006, 11:53 PM   #2
bulliver
Senior Member
 
Registered: Nov 2002
Location: Edmonton AB, Canada
Distribution: Gentoo x86_64; Gentoo PPC; FreeBSD; OS X 10.9.4
Posts: 3,760
Blog Entries: 4

Rep: Reputation: 78
Code:
lynx --dump http://whatever.com | grep "http://"

Last edited by bulliver; 08-16-2006 at 11:56 PM.
 
Old 08-18-2006, 04:52 AM   #3
walkingman
LQ Newbie
 
Registered: Aug 2006
Posts: 6

Original Poster
Rep: Reputation: 0
Lynx

Thanks for answer about using Lynx. But a multimedia URL has so complicated URL inside that a grep will not function.

I willo give you an example:

nettv.aftenposten.no/player/player.php?id=1730&category_id=6

Regards
walkingman
 
Old 08-18-2006, 05:08 AM   #4
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400Reputation: 2400
Hi,

Why wouldn't grep work, I just tried bulliver's solution and it works. You end up with a (numbered) list of (complicated) URL's.
 
Old 08-18-2006, 06:54 AM   #5
theYinYeti
Senior Member
 
Registered: Jul 2004
Location: France
Distribution: Arch Linux
Posts: 1,897

Rep: Reputation: 66
Code:
lynx ... | sed 's/href="\([^"]*\)"/\n=LINK=\1\n/g' | sed -n 's/^=LINK=//p' | sort -u
Yves.
 
Old 08-22-2006, 06:14 AM   #6
walkingman
LQ Newbie
 
Registered: Aug 2006
Posts: 6

Original Poster
Rep: Reputation: 0
Why Lynx is not enough

Quote:
Originally Posted by druuna
Hi,

Why wouldn't grep work, I just tried bulliver's solution and it works. You end up with a (numbered) list of (complicated) URL's.
If you try the command on the given URL then one can see that it want work!

Code:
ann@hegel:~$ lynx -dump nettv.aftenposten.no/player/player.php?id=1730&category_id=6. | grep "http:"
[2] 31340
ann@hegel:~$
   [EMBED]


[2]-  Done                    lynx -dump nettv.aftenposten.no/player/player.php?id=1730
The reason is that a simple cut of http will not catch javascript variables or compounded references. Some examples of what should be the outcome from the same URL:

...nettv.aftenposten.no/player/player.php?id=1730&category_id=6
...nettv.aftenposten.no/player/player.php?id=1662&category_id=6
...nettv.aftenposten.no/player/player.php?id=1653&category_id=6

Also the other solution ( lynx ... | sed 's/href="\([^"]*\)"/\n=LINK=\1\n/g' | sed -n 's/^=LINK=//p' | sort -u ) will not work for the same reasons.

Regards
walkingman
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
html relative links slzckboy Programming 4 05-03-2006 08:35 AM
html hyper links slzckboy Programming 3 04-06-2006 04:49 PM
html links from various sources rblampain Programming 2 11-24-2005 10:32 PM
HTML links in signatures? BajaNick LQ Suggestions & Feedback 3 12-20-2004 02:32 AM
index.html in apache with multimedia embedded htm Linux - Software 3 05-27-2004 10:05 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 08:55 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration