Further explorations with calibration curve

[CodexVeritas]

This issue explored the dip in the calibration curve a little more to see if its represented right. Below are some ideas for further exploration.

Things to explore further:

• Explore why the 2024 data is causing a calibration dip and data between 2020 and 2023 does not have this (bots are a suspicious cause)
• Double check that bots are not included when creating the values in Recency Weighted AggregateForecast table rows
• Triple check that the bots are in fact filtered out post-AggregateForecast calculation
• Make unit tests to triple check calibration curve math
• Do a data dump of the forecasts in the 20-60% range and see if there are any patterns
• See if removing 'possibly spam' forecasts helps - Filter out the forecasts made by people who have not been on the site longer than 1month or who have not made a comment or that were marked spam or that are inactive (these filters might already be applied)
• Double check that the math for calculating recency weighted forecast is correct
• Make sure that the forecasts used in the calibration curve are the same as the ones used from the global leaderboard
• Once final updates are made, update the embeded picture in /introduction page that uses track record

Other useful info:

• Results found thus far and other ideas: https://metaculus.slack.com/archives/C52PD5LUS/p1733276259333379?thread_ts=1731959762.059169&cid=C52PD5LUS
• Molly said to check this out and see if we can redo this analysis: https://forum.effectivealtruism.org/posts/ozPL3mLGShqvjhiaG/some-research-ideas-in-forecasting