Does anyone know any good web crawler and also indexer?

I've tried a few, Larbin for example, but it only retrieves URLs. I need also an indexer for crawling the web (a small portion of it) to build an experimental search engine.

thanks

Try Grub
http://www.grub.org/html/downloads.php
Grub is a distributed internet crawler/indexer designed to run on multi-platform systems, interfacing with a central server/database. It is used by LookSmart, as well, for it's Peer to Peer search and indexing.

There are a lot of search engine projects out there. Harvest, htdig, etc. A large list of Unix based ones (commercial as well as open source) can be found on searchtools.com (specific page linked here). But if you're just looking to index your Web site for your visitors to search your site, check out siteLevel. siteLevel powers site search engines for tens of thousand of sites world wide, and they also provide very good technical support. There's a free service offering, as well as many affordable pay options. It's an ASP (application service provider) model, so there's no software to install; however, if you're looking for a server-side solution to install, check out the first link.