Do not expect this code to run. It was written by Paul Christiano and Buck Shlegeris to produce the results in the paper Supervising Strong Learners by Amplifying Weak Experts. It is not intended to allow other researchers to reproduce those results, and won't be maintained or improved. It is released under the MIT license.