Huge custom dataclips will refuse to trigger a run

[josephjclark]

If you paste a dataclip of sufficient size into the custom dataclip box, you'll get a 413 (Request Entity Too Large error

I've got a dataclip here that's 400k lines long. I won't upload it here (not sure how safe the data is) but I can send it privately.

See Slack thread

Btw in prod this error doesn't render very well:

It's trying to parse the response body as JSON and failing, so throwing a quite different error

Locally on main I saw the proper error message - so either that's a bug fixed or something in the dev environment.

[josephjclark]

A sensible solution would be to impose a character limit on a custom dataclip size.

We might also explore ways to chunk the data and upload over several smaller requests

Is there any kind of limit to dataclip size in the app?

[brandonjackson]

@josephjclark I think the character limit is a good idea.

1. Do we know at what point that Entity Too Large error starts being thrown?
2. Could we display a better error message when a 413 (Request Entity Too Large) is returned?

[josephjclark]

1. I do not know - someone will have to investigate and work out a reasonable limit
2. We can display whatever error you like. Although if we impose a character limit then this part of the app should never see that error again