3 -> 2 notion requests / second

[dionjwa]

I am running from the main branch, and I am constantly getting 502s. According to the notion docs, rate limit requests should be 429 (https://developers.notion.com/reference/request-limits) but I suspect that their API is not perfect, and returns 502s instead a lot of the time.

Anecdotally, someone claims that the limit is 3/requests per second. As this value is set in docu-notion I can imagine that due to network, randomness, having the set limit be exactly their internal limit would mean that sometimes you skirt over the edge.

https://www.reddit.com/r/Notion/comments/xfufed/how_do_you_handle_request_limits_using_notion_api/

I have found that setting it to 2 greatly reduces the 502 timeout/rate limit issues.

---

This change is 

[bolt-new-by-stackblitz]

Run & review this pull request in StackBlitz Codeflow.

[hatton]

@dionjwa, thanks for doing that research. If the info in your link is correct, (2700 API calls over a 15 minute period) then we should be able to change the rate-limiting algorithm to match that?

[hatton]

Related, a blog post covering various limits: https://notionmastery.com/pushing-notion-to-the-limits/

[hatton]

In case you have time to try this out, it seems that we could use the same RateLimiter but with an different interval:

const notionLimiter = new RateLimiter({
 // the NOtion API docs say "an average of 3 / second", but reportedly, it's actually 2700 per 15 minutes. See https://notionmastery.com/pushing-notion-to-the-limits/
  tokensPerInterval: 2700,
  interval: 15*60*1000,
});

[hatton]

Later, I realized that while that change might speed up operations, it shouldn't improve anything.

I did some work making docu-notion more resilient to timeouts. That is now PR #81. Even with no timeout at all, it was able to handle docs.BloomLibrary.org.