Java Mailing List Archive

http://www.java2.5341.com/

Home » java-user.lucene »

Lucene Input/Output error

JulieSoko

2008-10-24

Replies: Find Java Web Hosting

Author LoginPost Reply

Hello All,
First of all I’m new to Lucene, and have written code using it to search
over 1 to man indexes, using a user defined query.
I don't have any code on this system so have to type everything in here...
I have the following design but am getting
An Input / Output error exception which I have typed in a part of it below.
My question is this? Do I have glaring flaw
In this design? I am reusing the IndexSearchers/IndexReaders and not
closing them.  The input/output error arises when 2 or
More searches occur at the same time over some of the same indexes. Can you
give me some direction where I should look
For the solution to the exception?

Here is an explanation of my
  Data:
    Up to 60 different indexes used at a time
     1 directory per 1 day of data
    Millions of documents per day
    Data is received and indexes merged on a continual bases - whole
separate process

  Index contains
       value: content   eventType: type of data  eventTime: time data
collected

  1 to many users can create individual queries containing 1 or more of
the fields and values
  searching over 1 to many indexes

  Design:
     Utilize the IndexAccessor classes to cache
IndexSearcher/IndexReaders i.e. they are made one per index and never
closed.
   

     Use a ParallelMultiSearcher - create one per request using 1 to
many of the indexes

     try {
     QueryParser parser = new QueryParser("value", new
StandardAnalyzer(0);
     parser.setDefaultOperator(QueryParser.AND_OPERATOR);
     Query query = parser.parse(queryString);

    TopDocCollector col = new TopDocCollector(MAX_NUMBER_HITS);
    multiSearcher.search(query, new
RangeFilter("eventTime",startTime,endTime,true,true),col);
    int numHits = col.getTotalHits(0);
    TopDocs docs = col.topDocs();
 
    if (numHits > 0))
      for (int i=0; i< numHits && i< MAX_NUMBER_HITS; i++)[
       Document doc = multiSearcher.doc(docs.scoreDocs[i].doc);
        ....
      }
   
  }catch(Exception e ){
     e.printStackTrace();
} finally{
   //IndexSearchers are not closed since shared by many users
}
     


     
When the second user accesses directories used by the first query then I get
the following error:

java.io.IOException: Input/output error
java.io.RandomAccessFile.readBytes(Native Method)
java.io.RandomAccessFile.read (RandomAccessFile.java:315)
at
org.apache.lucene.store.FSDirecotry$FSIndexInput.readInternal(FSDirectory.java:550)
at
org.apache.lucene.store.BufferedIndexInput.readBytes (BufferedInputInput.java:131)
at
org.apache.lucene.index.CompoundFileReader$CSIndexInput.readInternal(CompoundFileReader.java:240)
at
org.apache.lucene.instoreBufferedIndexInput.refill(BufferedIndexInput.java:
152)
at
org.apache.lucene.instoreBufferedIndexInput.readByte(BufferedIndexInput.java:
152)
at org.lucene.store.IndexInput.readVInt(IndexInput.java:76)
at org.apache.lucene.index.TermBuffer.read (TermBuffer.java:63)
at org.apache.lucene.index.SegmentTermEnum.next (SegmentTermEnum.java:123)
at org.apache.lucene.index.SegmentTermEnum.scanTo (SegmentTermEnum.java:154)
at
org.apache.lucene.index.TermInfosReader.scanEnum (TermInfosReader.java:223)
at org.apache.lucene.index.TermInfosReader.get (TermInfosReader.java:217)
at org.apache.lucene.index.SegmentReader.docFreq (SegmentReader.java:678)
at org.apache.lucene.search.IndexSearcher.docFreq (IndexSearcher.java:87)
at org.apache.lucene.search.Searcher.docFreqs (searcher.java:118)
at
org.apache.lucene.search.MultiSearcher.createWeight (MultiSearcher.java:311)
at org.apache.lucene.search.Searcher.search (Searcher.java:178)

Thanks!

--
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@(protected)
For additional commands, e-mail: java-user-help@(protected)

©2008 java2.5341.com - Jax Systems, LLC, U.S.A.