Did a lot of refactoring...

[maymount]

I did so much refactoring that you might not want to accept the changes.

However, I did find a bug that you should definitely fix.

Line 204: good_metric = [ 1.0, other_count / pool_count ].min
Line 209: bad_metric = [ 1.0, this_count / them_count ].min

Ruby is treating those divisions as integers, and since the numerator is less than the denominator the result is always 0. The effect is that tokens always have a probability of 0.999 instead of a lower probability for tokens that appear in multiple pools.

The fix is to force the values to be floats:

good_metric = [ 1.0, Float(other_count) / Float(pool.token_count) ].min

Other than that:

1. Lots of refactoring, encapsulating logic, making things private, and removing vestigial code.
2. Corpus is generated just like the cache, and treated separately, which simplified a lot of code
3. Tests
4. RDoc comments