Add sync log endpoint for desync log uploads#45
Add sync log endpoint for desync log uploads#4511EJDE11 wants to merge 2 commits intoCnCNet:mainfrom
Conversation
- SyncLog model with migrations (game_hash, player_name, slot index, map/game metadata, file, IP, 500 MB storage cap with LRU eviction) - POST /sync-logs/ - unauthenticated upload endpoint - GET /sync-logs/session/<hash>/ — lists logs for a session (IP omitted) - GET /sync-logs/compare/<hash>/view/ - HTML page for client-side side-by-side diff of any two logs using jsdiff + diff2html - Django admin with game slug column and per-game filtering - Enable Django admin login (ModelBackend, has_perm, get_by_natural_key) - nginx client_max_body_size raised to 15m to accommodate large logs - Add .vs/ to .gitignore
alexlambson
left a comment
alexlambson
left a comment
There was a problem hiding this comment.
Overall agree we need this feature, but there are some structural issues to revisit, and we need to make the endpoints compliant with DRF (Django Rest Framework.)
At the core, I think we can pretty safely assume that 99% of desync reports will relate to a map in our database, so we should use the map's foreign key instead of a bespoke hash.
I can help make the changes if you need.
As for your concern about "is this out of scope for the map DB?" No, I think this is perfectly in scope. It'd be nice to run analysis like "which INI rules are most likely to cause desync"
|
|
||
| game_hash = models.CharField( | ||
| max_length=64, | ||
| db_index=True, |
There was a problem hiding this comment.
Why the index on this? It will change for every lobby.
| game_hash = models.CharField( | ||
| max_length=64, | ||
| db_index=True, | ||
| help_text='SHA1 of "{mapSha1}|{randomSeed}|{gameSlug}" as a lowercase hex string. Same for all players in the same game session.', |
There was a problem hiding this comment.
It'd be better to store these as separate fields and use the game_id to get the slug
| ) | ||
| player_name = models.CharField(max_length=64) | ||
| sync_file_index = models.SmallIntegerField( | ||
| help_text="Player slot index from the SYNC filename (0-7). e.g. SYNC2.TXT → index 2.", |
There was a problem hiding this comment.
Let's call it player_index because that's more clear on what this integer actually represents.
| help_text="Player slot index from the SYNC filename (0-7). e.g. SYNC2.TXT → index 2.", | ||
| ) | ||
|
|
||
| map_sha1 = models.CharField(max_length=40, blank=True) |
There was a problem hiding this comment.
Use the hash to look up the map file foreign key when the logs are uploaded.
cnc_map_file = models.ForeignKey() etc
| ) | ||
|
|
||
| map_sha1 = models.CharField(max_length=40, blank=True) | ||
| map_name = models.CharField(max_length=255, blank=True) |
There was a problem hiding this comment.
Remove. It will be on the map itself.
| .order_by("sync_file_index", "uploaded_at") | ||
| ) | ||
|
|
||
| result = [ |
There was a problem hiding this comment.
DRF serializers can do this for you.
| class SyncLogCompareView(KirovyApiView): | ||
| """Return the raw text of all sync logs for a game session for side-by-side comparison.""" | ||
|
|
||
| permission_classes = [AllowAny] |
There was a problem hiding this comment.
This should be limited to admins or moderators for user privacy.
| logs: QuerySet[SyncLog] = SyncLog.objects.filter(game_hash=game_hash).order_by("sync_file_index", "uploaded_at") | ||
|
|
||
| result = {} | ||
| for log in logs: |
There was a problem hiding this comment.
We need to make a serializer for this
| </script> | ||
| </body> | ||
| </html> | ||
| """ |
There was a problem hiding this comment.
This needs to be a django template if we're doing server-side rendering
| @@ -0,0 +1,45 @@ | |||
| """Django admin registrations for Kirovy models. | |||
There was a problem hiding this comment.
I am pretty against using django admin due to prior negative experiences and database integrity issues.
I am open to discussing it, but we'd need to find a way to make it use the ladder for auth.
2 participants
/sync-logs/endpoint so xna-cncnet-client can uploadSYNC*.TXTdesync logs automatically after a desync occursgame_hash(derived client-side from map SHA1, random seed, and game slug), making it easy to group and compare logs across machines/sync-logs/compare/<hash>/view/- no server-side file comparison required, diffs entirely in JS using jsdiff + diff2htmlEndpoints
POST/sync-logs/SYNC*.TXTfileGET/sync-logs/session/<hash>/GET/sync-logs/compare/<hash>/view/