Hi,

I am using one of the latest nightly build versions of Nutch (#567 to be
precise) and after crawling a dump of local file system files hosted as a
virtual directory, it is failing at segment merging stage with the following
Exception:

2008-10-07 08:52:46,000 INFO crawl.LinkDb - LinkDb: adding segment:
file:/c:/NutchDev/Nutch567Updated/KShopCrawl/segments/20081006220137
2008-10-07 08:52:46,000 INFO crawl.LinkDb - LinkDb: adding segment:
file:/c:/NutchDev/Nutch567Updated/KShopCrawl/segments/20081006220219
2008-10-07 08:52:46,000 INFO crawl.LinkDb - LinkDb: adding segment:
file:/c:/NutchDev/Nutch567Updated/KShopCrawl/segments/20081007085143
2008-10-07 08:52:49,517 WARN mapred.LocalJobRunner - job_local_8
java.io.EOFException
 at java.io.DataInputStream.readFully (DataInputStream.java:178)
 at
org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:56)
 at org.apache.hadoop.io.DataOutputBuffer.write (DataOutputBuffer.java:90)
 at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1882)
 at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1782)
 at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1828)
 at
org.apache.hadoop.mapred.SequenceFileRecordReader.next (SequenceFileRecordReader.java:79)
 at
org.apache.hadoop.mapred.SequenceFileRecordReader.next (SequenceFileRecordReader.java:31)
 at
org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:158)
 at org.apache.hadoop.mapred.MapRunner.run (MapRunner.java:45)
 at org.apache.hadoop.mapred.MapTask.run (MapTask.java:219)
 at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:157)

I have gone through the threads in the forum, and it seems, there was an
issue in Hadoop which got fixed later. Is this the reason for the above
error, and if so, how do I solve it in my Nutch installation?


--
Sent from the Nutch - User mailing list archive at Nabble.com.