Add dedicated log level flag for ApiRequestLogger

[julientinguely-da]

fixes https://github.com/DACH-NY/canton-network-internal/issues/3036

logs for scratchnet test: https://console.cloud.google.com/logs/query;query=resource.labels.cluster_name%3D%22cn-scratchbnet%22%0A%22ApiRequestLogger%22%0Aseverity%3DDEBUG%0A%22GetConnectedSynchronizers%22;cursorTimestamp=2026-02-11T10:24:58.130Z;startTime=2026-02-11T09:58:00.000Z;endTime=2026-02-11T12:59:06.985Z?project=da-cn-scratchnet

Pull Request Checklist

Cluster Testing

• If a cluster test is required, comment /cluster_test on this PR to request it, and ping someone with access to the DA-internal system to approve it.
• If a hard-migration test is required (from the latest release), comment /hdm_test on this PR to request it, and ping someone with access to the DA-internal system to approve it.

PR Guidelines

• Include any change that might be observable by our partners or affect their deployment in the release notes.
• Specify fixed issues with Fixes #n, and mention issues worked on using #n
• Include a screenshot for frontend-related PRs - see README or use your favorite screenshot tool

Merge Guidelines

• Make the git commit message look sensible when squash-merging on GitHub (most likely: just copy your PR description).

[julientinguely-da]

/cluster_test

[github-actions]

Deploy cluster test triggered for Commit 7fc3f40aea825719597c5d236fd149f4e2bf0326 in , please contact a Contributor to approve it in CircleCI: https://app.circleci.com/pipelines/github/DACH-NY/canton-network-internal/51523

[julientinguely-da]

/cluster_test

[github-actions]

Deploy cluster test triggered for Commit 4e5bd4a1075aaea969a1acfdde8c982b1f6a6a57 in , please contact a Contributor to approve it in CircleCI: https://app.circleci.com/pipelines/github/DACH-NY/canton-network-internal/53446

[julientinguely-da]

/cluster_test

[github-actions]

Deploy cluster test triggered for Commit ba5bf665e3135a6f5104e27e73d2a1099fb1d941 in , please contact a Contributor to approve it in CircleCI: https://app.circleci.com/pipelines/github/DACH-NY/canton-network-internal/53495

[julientinguely-da]

/cluster_test

[github-actions]

Deploy cluster test triggered for Commit 184a340c0f44b151c3daa2adc6045be6f0962d9b in , please contact a Contributor to approve it in CircleCI: https://app.circleci.com/pipelines/github/DACH-NY/canton-network-internal/53509

[julientinguely-da]

this is working now: see the logs
but, still need to add the flag

[julientinguely-da]

the magic happens here

[moritzkiefer-da]

Lgtm, have you checked how close our logback is to the upstream one in Canton? I'm a bit wary of there being potentially more differences than the change you made as we haven't used them for Canton for a while which may come back to bite us.

Let's also upstream this change to Canton.

And now that we enabled info log on CI how about also enabling those request logs there by default?

[julientinguely-da]

have you checked how close our logback is to the upstream one in Canton?

The diffs is the following, only addition to canton-network.logback:

        <filter class="ch.qos.logback.core.filter.EvaluatorFilter">
          <evaluator>
            <matcher>
              <Name>hacoordinator_logger</Name>
              <regex>ha\.(HaCoordinator|PollingChecker)</regex>
            </matcher>
            <matcher>
              <Name>hacoordinator_message</Name>
              <regex>(Check successful|Scheduled checking|Acquiring lock)</regex>
            </matcher>
            <matcher>
              <Name>requestlogger_message</Name>
              <regex>StateService/GetConnectedDomains</regex>
            </matcher>
            <expression>
                level &lt;= INFO &amp;&amp;
                ( hacoordinator_logger.matches(logger) &amp;&amp; hacoordinator_message.matches(message) )
            </expression>
          </evaluator>
          <OnMismatch>NEUTRAL</OnMismatch>
          <OnMatch>DENY</OnMatch>
        </filter>

      <logger name="org.lfdecentralizedtrust.splice" level="DEBUG"/>
      <logger name="com.digitalasset.canton.networking.grpc" level="${LOG_LEVEL_API_REQUEST:-DEBUG}" />
      <logger name="org.lfdecentralizedtrust.splice" level="${LOG_LEVEL_CANTON:-INFO}"/>

and, all file names going from canton_xxx.clog to canton_network_xxx.clog. But that should be overwritten anyway by env vars.

And we don't have KMS-realted logs in canton_network.logback, which is problematic

[julientinguely-da]

@moritzkiefer-da I propose we clone canton.clog with our change to make it less conflicting for now, make the require change on canton, and revert once the upstream change is merge? I feel the upstream change might require more changes like adding separate flags etc.

[moritzkiefer-da]

wfm

[julientinguely-da]

And now that we enabled info log on CI how about also enabling those request logs there by default?

They are enabled by default: https://github.com/hyperledger-labs/splice/pull/3578/changes#diff-7f885255a483e1db9d0e10c531a8c4193cb3aac8fea2612d4351f83b6202ad68:~:text=%2B-,apiRequestLogLevel%3A%20DEBUG,-169

[julientinguely-da]

I'm a bit wary of there being potentially more differences than the change you made as we haven't used them for Canton for a while which may come back to bite us.

@moritzkiefer-da would be happy to double check that with you to be sure I'm not breaking anything with this PR

[moritzkiefer-da]

They are enabled by default: https://github.com/hyperledger-labs/splice/pull/3578/changes#diff-7f885255a483e1db9d0e10c531a8c4193cb3aac8fea2612d4351f83b6202ad68:~:text=%2B-,apiRequestLogLevel%3A%20DEBUG,-169

that's the cluster config not CI config. I mean logs for the Canton instance started by start-canton.sh

[moritzkiefer-da]

@moritzkiefer-da would be happy to double check that with you to be sure I'm not breaking anything with this PR

you did run a git diff against the one in the Canton repo and there were no meaningful differences right? if so I think we're good

[julientinguely-da]

@moritzkiefer-da , the only thing that was not present when doing the git diff was the appender for the KMS logs, so adding them here

[julientinguely-da]

that's the cluster config not CI config. I mean logs for the Canton instance started by start-canton.sh

Ah understood. So for that I'd need to define a new flag in upstream canton similar to --log-level-canton. But that' said when the env var is not defined it's set to DEBUG: https://github.com/hyperledger-labs/splice/pull/3578/changes#diff-b9f3371143f38a414af6eda864be54e4116effa7e0df4b3163f0f0eb27afa96c:~:text=%3Clogger%20name%3D%22com.digitalasset.canton.networking.grpc%22%20level%3D%22%24%7BLOG_LEVEL_API_REQUEST%3A%2DDEBUG%7D%22%20/%3E

[moritzkiefer-da]

So for that I'd need to define a new flag in upstream canton similar to --log-level-canton

or just set the env var?

[moritzkiefer-da]

but if the default is debug that should also be fine