I am doing a search using Lucene, and when I get the search results (hits),
I want to be able to get the number of tokens in a certain field.

Is this possible? Where is this sort of information stored?

I know the IndexSearcher.Explain can get some information, but it seems
mostly in free text and I cannot see the numTokens value.

Ideally what I would like to be able to do is get the NumTokens value and
the amount of hits within a field and display to the user what the "keyword
density" is as a percentage.

Any pointers would be greatly appreciated.

Thanks,

Andrew