Skip to content

feature-max-log-message-size#1284

Open
ilya-maltsev wants to merge 11 commits intoyandex:masterfrom
ilya-maltsev:max_log_size
Open

feature-max-log-message-size#1284
ilya-maltsev wants to merge 11 commits intoyandex:masterfrom
ilya-maltsev:max_log_size

Conversation

@ilya-maltsev
Copy link
Copy Markdown
Contributor

@ilya-maltsev ilya-maltsev commented Mar 6, 2026

Summary

  • Add log_max_msg_size configuration option to control the maximum size of the entire log line (default: 1024, max: 32768 bytes)

Motivation

When log_query is enabled, large SQL queries are silently truncated at 1024 bytes, making it impossible to debug issues involving long queries. This change makes the limit configurable while keeping the default behavior unchanged for backward compatibility.

Configuration

log_max_msg_size 32768
Parameter Default Range Description
log_max_msg_size 1024 1024-32768 Maximum total log line size in bytes

Invalid values (0, negative) fall back to 1024. Values exceeding 32768 are capped.

Backward compatibility

  • Default remains 1024 bytes - existing configurations work unchanged
  • No changes to external APIs or wire protocol

Comment thread sources/logger.c Outdated
int msg_len = vsnprintf(message, sizeof(message), fmt, args);
if (msg_len >= (int)sizeof(message)) {
msg_len = sizeof(message) - 1;
char *message = malloc(output_len);
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to use od_malloc\od_free here

@@ -0,0 +1,19 @@
log_format "%p %t %l [%i %s] (%c) %M\n"
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe this feature need to be tested not in separated flow, but in test/functional/tests ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@rkhapov
Copy link
Copy Markdown
Collaborator

rkhapov commented Mar 8, 2026

what about performance? this pr adds new allocation/deallocation per each log line

@ilya-maltsev
Copy link
Copy Markdown
Contributor Author

what about performance? this pr adds new allocation/deallocation per each log line

Replaced per-log-line malloc/free with thread-local reusable buffers

Eliminated heap allocation on every log call by using OD_THREAD_LOCAL buffers that are allocated once per thread and reused across all subsequent log lines. Two separate buffers are used: od_log_buf for the final formatted output (od_logger_write, od_logger_write_plain) and od_log_tmp for intermediate formatting (od_logger_escape, od_logger_format_json).

Comment thread sources/logger.c Outdated
if (*buf != NULL && *cur_size >= size) {
return *buf;
}
char *new_buf = realloc(*buf, size);
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

od_realloc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants