Ryan,

You can generate a file of FILE urls (eg)

file:///x/y/z/file1.html
file:///x/y/z/file2.html

Use find and AWK accordingly to generate this. put it in the url
directory and just set depth to 1, and change crawl_urlfilter.txt to
admit file:///x/y/z/ (note, if you dont head qualify it, it will
apparently try to index directories above the base one, by using ../
notation. (I only read this, havent tried it).

then just do the intranet crawl example.

NOTE this will NOT (as far as I can see no matter how much tweaking),
use ANCHOR TEXT or PageRank (OPIC version) for any links in these
files. The ONLY way to do this is to use a webserver as far as I can
tell. Don't understand the logic, but there you are. Note, if you
use a webserver, be aware you will have to disable IGNORE.INTERNAL
setting in Nutch-Site.xml (you'll be messing around a lot in here).

Cheers,
Winton



At 2:40 PM -0400 7/3/08, Ryan Smith wrote:
>Is there a simple way to have nutch index a folder full of other folders and
>html files?
>
>I was hoping to avoid having to run apache to serve the html files, and then
>have nutch crawl the site on apache.
>
>Thank you,
>-Ryan