Generate statements for design point experiment #86

[markwhiting]

In the design point experiment (#86), we want to do something like the following:

• Generate 100 candidate statements for each design point
• Measure commonsensicality for them (presumably with some AI approach to save money (@joshnguyen99 thoughts?)
• Select 1 for each of 15 rank buckets of commonsensicality, so we get 15 statements per design point.

It is worth doing this ad hoc, e.g., doing it for one design point before moving on to the next and even starting experiments while we do it.

This is mostly in commonsense-statements but I've added it here for now because the experiment is here.

Do we need some explicit GitHub representation of experiments? e.g., a repo for each major experiment? (@amirrr thoughts?)

[markwhiting]

Implicit here is that the 100 candidates are unique and probably semantically unique, too. Also that they are unique compared to any of the previous statements for other design points, and that they are positioned reliably in the design space, so step 1 requires a bit more detail than just generating a bunch of statements.

To do this we should think about tooling, reproducibility and the pipeline and automation we have in place in commonsense-statements.