web scraper suggestions

zpimp · 02-25-2016, 06:38 AM

do you need a web scraping program?
(get data from a specific site)
i am looking for ideas for scrapers

i use perl so it works on linux,windows and android

i have done torrents, online stores, wallpapers, car prices, sports odds resulsts, domain names availability, facebook pictures

worked in the past with xpath, but to maintain android compatibility, im using regex now, i know its more difficult, but is also faster

i want to make something that people really need, and use
waiting for your suggestions

ugjka · 02-25-2016, 07:26 AM

Curl, wget is fine for my use

zpimp · 02-25-2016, 10:35 AM

i already have the tools, and some knowledge
maybe some of you can make this yourselfes, but some cant
im looking for ideas, and to learn something on the way, by solving realworld problems

PACMANchasingme · 02-25-2016, 02:28 PM

Why exactly do you need ideas for web scraping, there's infinite amount of things you can do with the data you pull off any web page.

zpimp · 02-26-2016, 05:53 AM

i want to know what people need,
something that would be useful for more people