Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back > Forums > Linux Forums > Linux - General
User Name
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.


  Search this Thread
Old 12-14-2004, 02:59 AM   #1
Registered: Oct 2004
Distribution: Fedora 7
Posts: 232

Rep: Reputation: 30
Question wget + amazon search engine?

I'm writing a bash script that (is supposed to) take in info (artist, title) and insert these as a search string into amazon's "popular music" search engine.
I know this sounds dumb, but I think it's genious, let me explain what my program will really do.
It prompts you for artist and album, and if this album is found on amazon's site, It will rename the .mp3 files of the album accourding to what's on Amazon's site.
I have already wrote the entire script, but I can't seem to get the Amazon engine working.
I looked at source codes and other websites like this , but I couldn't find anything (Keep in mind that I'm not that big of a html coder).

Anyone know how to do this, or does anyone have any ideas?


Sorry ....
I should've specified this before, but I'm looking for just a URL format so it can be useful with wget.

Last edited by bendeco13; 12-14-2004 at 03:03 AM.
Old 12-14-2004, 09:34 AM   #2
LQ Newbie
Registered: Dec 2004
Posts: 1

Rep: Reputation: 0
I don't thing wget can handle POST data, but my favorite app for this sort of thing, curl, can do just about anything that a browser can do. I haven't looked at, but I can give you an outline on how to approach things:

First find out what data is sent to amazon by your browser. I usually use netcat for this.
- open the search site at amazon
- have netcat listen to some local port
- configure your browser to user<port> as proxy
- enter search data and click send

Now the data should be in the console window where you run netcat. The first line will tell you weather it is a GET or a POST request, and the last line contains the search data: opt1=arg1&opt2=arg2...

Finally, test curl (use -d <data> to specify the search data)

If you are lucky this works. If not, you have to make curl do more advanced simulation of your browser. (e.g. handle cookies, set referrer address, ...)

For more information on using curl, look at:
Old 12-14-2004, 12:48 PM   #3
Registered: Oct 2004
Distribution: Fedora 7
Posts: 232

Original Poster
Rep: Reputation: 30
Thanks Alot ....I'll Check into It !!
Old 12-18-2004, 03:39 AM   #4
Registered: Sep 2004
Location: Santa Cruz, CA, USA
Distribution: Redhat 9.0
Posts: 104

Rep: Reputation: 15
If you are planning on distributing this program to other people then Amazon will be none too pleased. They don't mind automated interfaces but they want to get their fair share of exposure and credit.
Old 01-27-2005, 02:11 PM   #5
Registered: Oct 2004
Distribution: Fedora 7
Posts: 232

Original Poster
Rep: Reputation: 30
Thanks for the comment, I wasn't planning on distributing it, but now I have a definite reason not to...
Yes I do feel like I'm cheating Amazon/whoever Elses websites I use for this script, but hey if it's just me and maybe a couple of friends, they're probably not going to miss a tiny bit of bandwidth.


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
new search engine? markhod General 3 04-08-2005 04:44 PM
search engine jean-michel LQ Suggestions & Feedback 2 04-09-2004 12:40 PM
search all book *contents* @ jimveta General 2 10-24-2003 12:37 AM
LQ search engine Tinkster LQ Suggestions & Feedback 6 03-16-2003 01:57 PM
Search Engine? phek Linux - General 1 09-20-2001 07:00 PM > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 01:12 PM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration