Support large file metadata via Discord attachments#26
Conversation
Discord messages have a 4000-character limit. For very large files (e.g., 10GB+), the JSON-serialized file descriptor containing hundreds of part message IDs can exceed this limit, causing uploads to fail with a 400 Bad Request error. This patch introduces a transparent overflow mechanism in MessageApi: - Payloads exceeding 4000 characters are automatically uploaded as an 'overflow.json' attachment. - The message content is set to a 'DCFS_OVERFLOW' sentinel. - MessageApi.get_text() transparently retrieves the full payload by downloading the attachment when the sentinel is detected. DCMsgFDRepository has been updated to use this mechanism, enabling support for arbitrarily large file metadata. Co-authored-by: VulcanoSoftware <113239901+VulcanoSoftware@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
Discord messages have a 4000-character limit. For very large files (e.g., 10GB+), the JSON-serialized file descriptor containing hundreds of part message IDs can exceed this limit, causing uploads to fail with a 400 Bad Request error. This patch introduces a transparent overflow mechanism in MessageApi: - Payloads exceeding 4000 characters are automatically uploaded as an 'overflow.json' attachment. - The message content is set to a 'DCFS_OVERFLOW' sentinel. - MessageApi.get_text() transparently retrieves the full payload by downloading the attachment when the sentinel is detected. DCMsgFDRepository has been updated to use this mechanism, enabling support for arbitrarily large file metadata. Includes a fix for a Mypy issue in the unit tests identified in the first CI run. Co-authored-by: VulcanoSoftware <113239901+VulcanoSoftware@users.noreply.github.com>
1 participant
Implemented a metadata overflow mechanism to handle Discord's 4000-character message limit. This allows DCFS to support files with a very large number of parts (such as 10GB+ files) by automatically switching to file attachments for descriptor storage when the serialized JSON exceeds the text limit.
Key changes:
get_text()toMessageApifor transparent large payload retrieval.MessageApi.send_textandedit_message_textto handle overflows via attachments.DiscordBotAPIto support simultaneous text and media updates.DCMsgFDRepositoryto utilize the new overflow-aware retrieval.tests/dcfs/core/api/message/test_overflow.py.PR created automatically by Jules for task 5237680782998250731 started by @VulcanoSoftware