Optimize socket connections from the server

[cforcomputer]

• Implement Progressive Loading:

• Optimize DOM Updates:

• Use the 24 hour client killmail store (~3000 killmails) and keep it in sync

• What should happen is that the server performs all the calculations using a single central store that is synced with the server using the current socket system.

• No separate socket for camps, roams, battles,salvage,etc.

• All use the single socket connection store established when the server starts for the calculations, which are done on the client. This means that for example, camps would be processed as the killmails arrived, but would still be in sync with all other clients because they all have the exact same killmails store.

• The server uses its cache store of killmails and spits them all over to a connected client (last 24 hours of kills), so that server and client have the same number of kills at all times.

• The server sends to client in order of newest first to oldest killmails so camps,salvagefields, and roams (all occur the in the last 2 hours) can be calculated first for the new clients.

• campManager and roamManager should be moved clientside (rip)

• server.js calculations should be moved into campManager and roamManager. Since we are keeping server and client separate, they should be moved into campManager and roamManager respectively.

[cforcomputer]

Instead of managing client cache on the server, create the enriched killmails and then push them to redis server with TTL expiry dependent on their age, so they will naturally expire after their 24 hour period. This will also stop duplicates from being sent to clients, etc.

Simplify handling in DataManager to use new redis store

[cforcomputer]

Instead of managing client cache on the server, create the enriched killmails and then push them to redis server with TTL expiry dependent on their age, so they will naturally expire after their 24 hour period. This will also stop duplicates from being sent to clients, etc.

Simplify handling in DataManager to use new redis store

This is dumb because then the server needs to fetch it anyways
so km-processor --> redis upload --> server --> redis download --> socket.io --> clients

Since clients shouldn't connect directly to the store we are using to handle session cookies (oof)

It is nice having the km processor run as a separate service though.

Since we don't need redis at all for this application and the data is small, we should use a simple cache service on the server. Like Keyv.This seems to be the most popular.

So then we separate the killmail parsing to make server.js more maintainable, but we don't need to run it as a separate service.

so then it is: Redis --> km-processor --> keyvdb_cache --> server --> socket.io --> client

So in summary:

• Remove manual cleanup
• Switch to TTL based expiry based on time of killmail and current time (24 hours)
• Remove redis for session store and keep everything within keyv within the server
• Remove duplicate kills in socket.io and fix filters to use the new single store

[cforcomputer]

This works, but now I want to save historical camps to a database to automatically calculate the probability of camps occurring on specific gates. That way I don't have to manually enter high risk systems.

• Calculate gangs and camps serverside, only stream these to the server when socket room connected for page. Record them in a database for up to 30 days for historical trend analysis.