Enable Address Sanitizer in specified CI tests #4725

Copilot · 2026-01-05T06:28:57Z

Description

Added -fsanitize=address to compiler and linker flags for six CI test configurations to enable memory error detection during testing.

ci-mac.yml:

video-openh264-1: Added ASan to CFLAGS, CXXFLAGS, LDFLAGS
video-openh264-2: Added ASan to CFLAGS, CXXFLAGS, LDFLAGS
video-openh264-3: Added ASan to CFLAGS, CXXFLAGS, LDFLAGS

ci-linux.yml:

default-full-bundle-1: Added ASan to CFLAGS, CXXFLAGS, LDFLAGS
default-full-bundle-2: Added ASan to CFLAGS, CXXFLAGS, LDFLAGS
default-full-bundle-3: Added ASan to CFLAGS, CXXFLAGS, LDFLAGS

Motivation and Context

Address Sanitizer detects memory errors (use-after-free, buffer overflows, memory leaks) at runtime. Enabling it in these test configurations provides early detection of memory safety issues during CI runs.

How Has This Been Tested?

YAML syntax validated for workflow file changes
Configuration changes are correct and will be tested by the CI workflows on the next run

Screenshots (if appropriate):

N/A

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

My code follows the CODING STYLE of this project.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have read the CONTRIBUTING document.
I have added tests to cover my changes.
All new and existing tests passed.

Original prompt

Use Address Sanitizer in the following CI tests:

ci-mac
video-openh264-1, video-openh264-2, video-openh264-3

ci-linux
default-full-bundle-1, default-full-bundle-2, default-full-bundle-3

Fix any issues (if any)

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: sauwming <17044930+sauwming@users.noreply.github.com>

sauwming · 2026-01-05T10:20:23Z

@copilot: Fix these leaks:

Direct leak of 7608 byte(s) in 1 object(s) allocated from:
    #0 0x7f49bccfd9c7 in malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x7f49bc8237c4 in CRYPTO_zalloc (/lib/x86_64-linux-gnu/libcrypto.so.3+0x2237c4) (BuildId: 0698e1ff610cb3c6993dccbd82c1281b1b4c5ade)
    #2 0x7f49bd319d20 in SSL_new (/lib/x86_64-linux-gnu/libssl.so.3+0x3cd20) (BuildId: 5f3b12d47114f9fbdc7765266cd0bb8f1b5ee8fc)
    #3 0x55756b9cc52b in ssl_create ../src/pj/ssl_sock_ossl.c:1688
    #4 0x55756b9bfb91 in ssock_on_accept_complete ../src/pj/ssl_sock_imp_common.c:1054
    #5 0x55756b9c18ba in asock_on_accept_complete2 ../src/pj/ssl_sock_imp_common.c:1317
    #6 0x55756b9a0c48 in ioqueue_on_accept_complete ../src/pj/activesock.c:908
    #7 0x55756b992854 in ioqueue_dispatch_read_event ../src/pj/ioqueue_common_abs.c:536
    #8 0x55756b998c71 in pj_ioqueue_poll ../src/pj/ioqueue_select.c:1093
    #9 0x55756b961950 in client_non_ssl ../src/pjlib-test/ssl_sock.c:1229
    #10 0x55756b9655b7 in ssl_sock_test ../src/pjlib-test/ssl_sock.c:1841

Indirect leak of 2 byte(s) in 1 object(s) allocated from:
    #0 0x7f49bccfd9c7 in malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x7f49bc6d542b in ASN1_STRING_set (/lib/x86_64-linux-gnu/libcrypto.so.3+0xd542b) (BuildId: 0698e1ff610cb3c6993dccbd82c1281b1b4c5ade)
    #2 0x7f49bc6cc160  (/lib/x86_64-linux-gnu/libcrypto.so.3+0xcc160) (BuildId: 0698e1ff610cb3c6993dccbd82c1281b1b4c5ade)
    #3 0x7f49bc6df0ec  (/lib/x86_64-linux-gnu/libcrypto.so.3+0xdf0ec) (BuildId: 0698e1ff610cb3c6993dccbd82c1281b1b4c5ade)
    #4 0x7f49bc6e006a  (/lib/x86_64-linux-gnu/libcrypto.so.3+0xe006a) (BuildId: 0698e1ff610cb3c6993dccbd82c1281b1b4c5ade)
    #5 0x7f49bc6e0721  (/lib/x86_64-linux-gnu/libcrypto.so.3+0xe0721) (BuildId: 0698e1ff610cb3c6993dccbd82c1281b1b4c5ade)
    #6 0x7f49bc6e09e5  (/lib/x86_64-linux-gnu/libcrypto.so.3+0xe09e5) (BuildId: 0698e1ff610cb3c6993dccbd82c1281b1b4c5ade)
    #7 0x7f49bc6dfd75  (/lib/x86_64-linux-gnu/libcrypto.so.3+0xdfd75) (BuildId: 0698e1ff610cb3c6993dccbd82c1281b1b4c5ade)
    #8 0x7f49bc6e0721  (/lib/x86_64-linux-gnu/libcrypto.so.3+0xe0721) (BuildId: 0698e1ff610cb3c6993dccbd82c1281b1b4c5ade)
    #9 0x7f49bc6dfd75  (/lib/x86_64-linux-gnu/libcrypto.so.3+0xdfd75) (BuildId: 0698e1ff610cb3c6993dccbd82c1281b1b4c5ade)
    #10 0x7f49bc6e0d11 in ASN1_item_d2i_ex (/lib/x86_64-linux-gnu/libcrypto.so.3+0xe0d11) (BuildId: 0698e1ff610cb3c6993dccbd82c1281b1b4c5ade)
    #11 0x7f49bc8f1362 in PEM_ASN1_read_bio (/lib/x86_64-linux-gnu/libcrypto.so.3+0x2f1362) (BuildId: 0698e1ff610cb3c6993dccbd82c1281b1b4c5ade)
    #12 0x55756b95e2d6 in load_cert_direct ../src/pjlib-test/ssl_sock.c:642
    #13 0x55756b95ef41 in echo_test ../src/pjlib-test/ssl_sock.c:751
    #14 0x55756b965234 in ssl_sock_test ../src/pjlib-test/ssl_sock.c:1750

Fixed memory leaks in the load_cert_direct() function in ssl_sock.c test: 1. Added proper cleanup of X509 and EVP_PKEY objects on error paths 2. For OpenSSL 3.0+, free the X509 and EVP_PKEY references after calling pj_ssl_cert_load_direct() since pj_ssl_sock_set_certificate() will call X509_up_ref() and EVP_PKEY_up_ref() to increase ref counts This fixes the leaks reported by Address Sanitizer: - Direct leak of 7608 bytes from SSL_new - Indirect leak of 2 bytes from ASN1_STRING_set (X509 certificate) Co-authored-by: sauwming <17044930+sauwming@users.noreply.github.com>

Copilot · 2026-01-05T10:33:51Z

@copilot: Fix these leaks:

Direct leak of 7608 byte(s) in 1 object(s) allocated from:

...

Fixed in commit f56b4e1. The issue was in the test code's load_cert_direct() function. When using OpenSSL 3.0+, the test wasn't freeing its references to X509 and EVP_PKEY objects after passing them to pj_ssl_cert_load_direct(). Since pj_ssl_sock_set_certificate() calls X509_up_ref() and EVP_PKEY_up_ref() for OpenSSL 3.0+, the test code needs to free its original references to avoid leaks. Also added proper cleanup on error paths.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

trac.pjsip.org
- Triggering command: ../bin/pjlib-test-x86_64-pc-linux-gnu ../bin/pjlib-test-x86_64-pc-linux-gnu ssl_sock_test /home/REDACTED/work/pjproject/pjproject/pjsip/include --gdwarf-5 --64 -o output/sample-x86_64-pc-linux-gnu/vid_streamutil.o /tmp/ccXp8HjS.s -I t.o -I -gnu/pjsua_im.o SE_O�� ude ../include /include ../../pjlib/inclas ENDIAN=0 -DPJ_IS-I ../../pjlib-util/home/REDACTED/work/pjproject/pjproject/pjlib/include N=1 -DPJMEDIA_U-I (dns block)
- Triggering command: ../bin/pjlib-test-x86_64-pc-linux-gnu ../bin/pjlib-test-x86_64-pc-linux-gnu ssl_sock_test -DPJMEDIA_USE_OL/home/REDACTED/work/pjproject/pjproject/pjlib/include lude -c clude -DPJMEDIA_HAS_OP/home/REDACTED/work/pjproject/pjproject/pjnath/include ude -DPJMEDIA_HAS_LYas /include -I/home/REDACTED/w/home/REDACTED/work/pjproject/pjproject/pjlib/include lude -Wal�� ude -g /include E_CFLAGS="-DPJ_Amkdir lude -I../include clude (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

sauwming · 2026-01-06T01:21:08Z

@copilot: revert last commit

This reverts commit f56b4e1. Co-authored-by: sauwming <17044930+sauwming@users.noreply.github.com>

Copilot · 2026-01-06T01:23:20Z

@copilot: revert last commit

Reverted in commit a924797.

…p/pjproject into copilot/use-address-sanitizer-tests

sauwming · 2026-01-07T05:26:37Z

There were a couple of issues encountered:

Leak in pjlib-test

Indirect leak of 2 byte(s) in 1 object(s) allocated from:
    #0 0x7f49bccfd9c7 in malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x7f49bc6d542b in ASN1_STRING_set (/lib/x86_64-linux-gnu/libcrypto.so.3+0xd542b) (BuildId: 0698e1ff610cb3c6993dccbd82c1281b1b4c5ade)
    #2 0x7f49bc6cc160  (/lib/x86_64-linux-gnu/libcrypto.so.3+0xcc160) (BuildId: 0698e1ff610cb3c6993dccbd82c1281b1b4c5ade)
    #10 0x7f49bc6e0d11 in ASN1_item_d2i_ex (/lib/x86_64-linux-gnu/libcrypto.so.3+0xe0d11) (BuildId: 0698e1ff610cb3c6993dccbd82c1281b1b4c5ade)
    #11 0x7f49bc8f1362 in PEM_ASN1_read_bio (/lib/x86_64-linux-gnu/libcrypto.so.3+0x2f1362) (BuildId: 0698e1ff610cb3c6993dccbd82c1281b1b4c5ade)
    #12 0x55756b95e2d6 in load_cert_direct ../src/pjlib-test/ssl_sock.c:642

This seems to be intentional:

The leak is indirect, caused by OpenSSL
OpenSSL keeps small ASN.1 objects in global caches that are not released at exit.

So, a suppression file in tests/sanitizers/lsan.supp is created for this.

Failed pjsip-test on Mac
AddressSanitizer: CHECK failed: sanitizer_posix_libcdep.cpp:209 "((0)) == ((sigaltstack(&altstack, &oldstack)))" (0x0, 0xffffffffffffffff) (tid=80453) <empty stack>

This is because ASan failed to install its alternate signal stack, most likely because app hasApplication already installed one.

There's also an issue with the usage of longjmp in sip_parser to handle error:

AddressSanitizer: CHECK failed: asan_poisoning.cpp:40 "((AddrIsInMem(addr + size - (1ULL << 3)))) != (0)" (0x0, 0x0) (tid=122826882)
    #0 0x0001059773e0 in __asan::CheckUnwind()+0x24 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x473e0)
    #1 0x0001059912d0 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long)+0x94 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x612d0)
    #2 0x0001059b6244 in __asan::PoisonShadow(unsigned long, unsigned long, unsigned char) (.cold.4)+0x20 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x86244)
    #3 0x00010596f09c in __asan::PoisonShadow(unsigned long, unsigned long, unsigned char)+0x258 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x3f09c)
    #4 0x000105970c3c in __asan::PlatformUnpoisonStacks()+0x54 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x40c3c)
    #5 0x000105977090 in __asan_handle_no_return+0x2c (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x47090)
    #6 0x00010460babc in on_syntax_error sip_parser.c:230
    #7 0x000104811260 in pj_scan_syntax_err+0x54 (pjsip-test-aarch64-apple-darwin24.5.0:arm64+0x100275260)

The issue occurs when a function is:
- marked attribute((noreturn)), or
- calls longjmp, abort, exit, etc.

static void on_syntax_error(...)
{
    ...
    PJ_THROW(PJ_EINVAL);   // longjmp-style escape
}

ASan tries to unpoison stack memory
On macOS (especially arm64), ASan sometimes cannot prove the stack range is valid
ASan aborts itself → this is an internal invariant failure

So currently we disable pjsip-test MSG_TEST.

Initial plan

5d51e8d

Copilot AI assigned Copilot and sauwming Jan 5, 2026

Copilot started work on behalf of sauwming January 5, 2026 06:29 View session

Add Address Sanitizer to CI tests for Mac and Linux

d645a17

Co-authored-by: sauwming <17044930+sauwming@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Add address sanitizer to CI tests for mac and linux~~ Enable Address Sanitizer in specified CI test workflows Jan 5, 2026

Copilot finished work on behalf of sauwming January 5, 2026 06:34

Copilot AI requested a review from sauwming January 5, 2026 06:34

sauwming added 2 commits January 5, 2026 16:48

Fixed failed timer test and OpenSSL leak

f19a884

Fixed Linux build failure

964933e

sauwming added this to the release-2.17 milestone Jan 5, 2026

sauwming added type: task component: devops labels Jan 5, 2026

Copilot started work on behalf of sauwming January 5, 2026 10:20 View session

Copilot AI changed the title ~~Enable Address Sanitizer in specified CI test workflows~~ Enable Address Sanitizer in CI tests and fix memory leaks Jan 5, 2026

Copilot finished work on behalf of sauwming January 5, 2026 10:34

Copilot started work on behalf of sauwming January 6, 2026 01:21 View session

Revert "Fix memory leaks in SSL test when using OpenSSL 3.0+"

a924797

This reverts commit f56b4e1. Co-authored-by: sauwming <17044930+sauwming@users.noreply.github.com>

Copilot AI changed the title ~~Enable Address Sanitizer in CI tests and fix memory leaks~~ Enable Address Sanitizer in specified CI tests Jan 6, 2026

Copilot finished work on behalf of sauwming January 6, 2026 01:24

sauwming added 5 commits January 6, 2026 10:05

Support sanitizers options for unit tests and add suppression file

bf19d44

Merge branch 'copilot/use-address-sanitizer-tests' of github.com:pjsi…

a773a03

…p/pjproject into copilot/use-address-sanitizer-tests

Disable LSan for pjlib test Linux only

92dba05

Disable LSan for pjlib test Linux only

a4f1da7

Use suppression file

823488e

Fixed memory leak in pjlib-util-test encryption_benchmark

5b6b3f9

sauwming marked this pull request as ready for review January 6, 2026 04:18

trengginas approved these changes Jan 6, 2026

View reviewed changes

Disable pjsip message test under ASan

9cf0e75

sauwming requested a review from nanangizz January 7, 2026 05:26

sauwming approved these changes Jan 8, 2026

View reviewed changes

nanangizz approved these changes Jan 8, 2026

View reviewed changes

sauwming merged commit eab6c19 into master Jan 8, 2026
50 checks passed

sauwming deleted the copilot/use-address-sanitizer-tests branch January 8, 2026 04:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Enable Address Sanitizer in specified CI tests #4725

Enable Address Sanitizer in specified CI tests #4725

Copilot AI commented Jan 5, 2026 •

edited

Loading

Uh oh!

sauwming commented Jan 5, 2026

Uh oh!

Copilot AI commented Jan 5, 2026 •

edited

Loading

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

sauwming commented Jan 6, 2026

Uh oh!

Copilot AI commented Jan 6, 2026

Uh oh!

sauwming commented Jan 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Enable Address Sanitizer in specified CI tests #4725

Enable Address Sanitizer in specified CI tests #4725

Conversation

Copilot AI commented Jan 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

Checklist:

Uh oh!

sauwming commented Jan 5, 2026

Uh oh!

Copilot AI commented Jan 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

sauwming commented Jan 6, 2026

Uh oh!

Copilot AI commented Jan 6, 2026

Uh oh!

sauwming commented Jan 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Copilot AI commented Jan 5, 2026 •

edited

Loading

Copilot AI commented Jan 5, 2026 •

edited

Loading