Logging is clogged with warnings

[rhpijnacker]

When I run a reporter (to calculate derived energy prices), this logs lots of warning lines:

server-1  | /usr/local/lib/python3.12/dist-packages/pandas/core/frame.py:706: DeprecationWarning: Passing a BlockManager to BeliefsDataFrame is deprecated and will raise in a future version. Use public APIs instead.
server-1  |   warnings.warn(
...
server-1  | /usr/local/lib/python3.12/dist-packages/timely_beliefs/beliefs/classes.py:776: DeprecationWarning: Passing a SingleBlockManager to BeliefsSeries is deprecated and will raise in a future version. Use public APIs instead.
server-1  |   super().__init__(*args, **kwargs)
...

This has the unwanted side-effect of this is that the disk space of the Docker container also filling up quickly forcing me to restart the containers reguarly.

[nhoening]

So this is something happening in Pandas. Some internal usage is deprecated. Some other teams who had to deal with this, see here and here.

I also found this issue in timely-beliefs about using a more recent pandas version which would be nice to do before, so we at least get the most recent pandas version (or at least the last 2.x.x if moving to 3 is an issue)

[nhoening]

I suggest asking copilot or some other AI what a mitigation strategy could be. At least for the places which the warnings indicate are in timely-beliefs.

I see there are warnings coming from pandas itself, maybe some would go away by upgrading pandas...

[nhoening]

@rhpijnacker can you add a log to this issue where we see affected lines?

[rhpijnacker]

These are the last 1000 lines of today's run

fm.log.gz

[nhoening]

Okay so it boils down to only the two you mentioned in the issue description. Maybe one fix in timely-beliefs and upgrading pandas can help.

This does not happen during scheduling?

I notice that this should be a timely-beliefs issue, actually.

[nhoening]

@rhpijnacker would you be able to test with timely-beliefs on branch copilot/fix-logging-warnings ?

With docker compose, it would require building your own docker image with it.