Queries reset at midnight when CSV logs roll over

[immanuelfodor]

Amazing updates in the latest release! 😍😍

One remaining annoyance is that when Blocky's CSV logs are rotated at midnight (also with csv-client), the query log resets along with the charts.

The above shows the state at 00:02. The 7/30 days settings also don't mean anything in this context, it's just more empty data.

Would it be possible to load the current day and the previous day? Or to have a date picker for the log files' dates found on the disk? Or an interval selector like in Google Analytics to set how much logs to load/show? (I don't know how much RAM it would need or what constraints Node has, I'm just giving options that could be useful.)

[coderabbitai]

📝 CodeRabbit Plan Mode

Generate an implementation plan and prompts that you can use with your favorite coding agent.

• Create Plan

Examples

• Example 1
• Example 2

---

🔗 Related PRs

#189 - feat: CSV logs provider [merged]
#211 - feat: improve query logs UI with filters and instance support [closed]

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[GabeDuarteM]

Hey! Thank you for the feedback! 😄

This is one of the constraints of using csv/csv-client at the moment, since blocky rotates it automatically... At the current state, I'm honestly not too keen on implementing something like loading multiple days due to performance concerns, since each file can grow quite a lot and the current network request already takes quite a bit to load just a single file on the csvs mode. This would also increase the complexity over the queries we need to do, which is already quite high since we're dealing with a storage type that's not designed to be queried...

I could think of something like a file selector, or some similar design, but honestly, I don't think thats a good enough UX since you still need to go back and forth between days (and switching files can also take some time to load everything).

What I could maybe do is something like an observer targeting the csv files for new content and ingesting them into an sqlite database. That way we can have proper fast querying (and not limited to the last day), but I'd need to play around with this idea to see how it would look...