Mark a build as failing before building, and as successful afterwards

[jonascarpay]

Some builds may crash the server. I'm mostly thinking about a build that runs out of memory, but there may be other ways.
If such builds exist, they currently allow a user to crash the server by repeatedly starting that build.

A simple way around this is to first mark the failing, and then only after it has completed successfully marking it as such.

This is very conservative since it assumes every crash during a build is that build's fault. A more sophisticated mechanism could allow a build to fail $n$ times before never trying it again.

[kayhide]

This may be a bit off-topic from this issue but I was thinking that there are two types of build failure; deterministic and non-deterministic.

The deterministic failure is caused by source code (including build instructions) and reproduces exactly the same no matter how many we retry.

The non-deterministic failure is caused by memory shortage, network malfunction, unexpected user-interaction etc. If a failure is this type, we have a chance to succeed by retrying.

I think it is worthwhile to take account of these types when we think about how to handle build failures.

[cdepillabout]

A simple way around this is to first mark the failing, and then only after it has completed successfully marking it as such.

We might have to also take into the fact that if person A requests a build for vim, and then before that finishes, person B also requests a build for vim, we don't want to send a response to person B that the build has failed (but we instead want to wait to send a response to person B until after the build of vim for person A finishes).

[kayhide]

if person A requests a build for vim, and then before that finishes, person B also requests a build for vim

This is related to the discussion on parallelism.
#16 (comment)