Java Mailing List Archive

http://www.java2.5341.com/

Home » nutch-user.lucene »

Is there a performance penalty for merging content segments?

charlie w

2008-05-29


Author LoginPost Reply
If I use the SegmentMerger tool to merge many fetched content segments
(segments/<timestamp>/parse_text, etc.) into a single huge segment, do
I then create a performance problem when generating page summaries for
search hits? Are there contention or other issues reading these
fetched segments?

If there is a penalty, is there a recommended maximum size for content segments?

The cumulative size of all my content segments is in the neighborhood of 70GB.

Thanks,
Charlie
©2008 java2.5341.com - Jax Systems, LLC, U.S.A.