Fix container build fail test #691

singhd789 · 2025-12-16T16:31:07Z

Describe your changes

was previously appending lines to createContainer.sh instead of overwriting
also addressed some pylint messages

Issue ticket number and link (if applicable)

n/a

Checklist before requesting a review

I ran my code
I tried to make my code readable
I tried to comment my code
I wrote a new test, if applicable
I wrote new instructions/documentation, if applicable
I ran pytest and inspected it's output
I ran pylint and attempted to implement some of it's feedback
No print statements; all user-facing info uses logging module

- was previously appending lines to createContainer.sh instead of overwriting - also addressed some pylint messages

codecov · 2025-12-16T16:43:50Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.32%. Comparing base (f3155e8) to head (3a40b3b).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #691   +/-   ##
=======================================
  Coverage   85.32%   85.32%           
=======================================
  Files          68       68           
  Lines        4490     4490           
=======================================
  Hits         3831     3831           
  Misses        659      659

Flag	Coverage Δ
unittests	`85.32% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

ilaflott · 2025-12-16T18:17:35Z

This is closer! Now instead of test_run_fremake_container_build_notransfer and test_run_fremake_container_build_fail failing within fre/make/tests/compilation/test_run_fremake_builds.py, i just have test_run_fremake_container_build_notransfer failing. It seems to fail with:

============================================= FAILURES 
_____________________________________________ test_run_fremake_container_build_notransfer 

    @pytest.mark.skipif(not has_podman, reason="missing podman")
    def test_run_fremake_container_build_notransfer():
        ''' checks image creation with the .sif transfer turned off '''
        if Path("createContainer.sh").exists():
            os.remove("createContainer.sh")
>       run_fremake_script.fremake_run(YAMLPATH, CONTAINER_PLATFORM, TARGET,
            nparallel=False, njobs=1, no_parallel_checkout=True,
            no_format_transfer=True, execute=True, verbose=VERBOSE)

fre/make/tests/compilation/test_run_fremake_builds.py:102: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
fre/make/run_fremake_script.py:103: in fremake_run
    dockerfile_create(yamlfile, container_platforms, target, execute, no_format_transfer)
fre/make/create_docker_script.py:116: in dockerfile_create
    subprocess.run(args=[dockerBuild.userScriptPath], check=True)

................................
... CLIPPED "USELESS" OUTPUT ...
................................


             subprocess.CalledProcessError: Command '['/home/Ian.Laflotte/Working/fre-cli/createContainer.sh']' returned non-zero exit status 1.

../../conda/envs/fre-cli/lib/python3.11/subprocess.py:571: CalledProcessError
--------------------------------------------- Captured stdout call

COMBINED YAML VALID

COMBINED YAML VALID

COMBINED YAML VALID

COMBINED YAML VALID
STEP 1/19: FROM ghcr.io/rem1776/hpc-mini:alpine3gcc14.2 AS builder
-------------------------------------------------------------------------------------------------------------- Captured stderr call --------------------------------------------------------------------------------------------------------------
+ podman build -f Dockerfile -t null_model_full:debug
time="2025-12-16T13:12:17-05:00" level=warning msg="Network file system detected as backing store.  Enforcing overlay option `force_mask=\"700\"`.  Add it to storage.conf to silence this warning"
Trying to pull ghcr.io/rem1776/hpc-mini:alpine3gcc14.2...
Getting image source signatures
Copying blob sha256:750cdedbeac62fc688a081dc47cabedc7c668c010dc571178595e7524f4fa4b7
Copying blob sha256:210a2ae1a75e9e998739647b797326a97064eb38a1557d9191ff2f0ce0bef86c
Error: creating build container: writing blob: adding layer with blob "sha256:210a2ae1a75e9e998739647b797326a97064eb38a1557d9191ff2f0ce0bef86c": processing tar file(lsetxattr /bin: operation not supported): exit status 1
--------------------------------------------------------------------------------------------------------------- Captured log call ----------------------------------------------------------------------------------------------------------------
INFO     fre.yamltools.combine_yamls_script:combine_yamls_script.py:166 initializing a compile yaml instance...
INFO     fre.yamltools.info_parsers.compile_info_parser:compile_info_parser.py:61 Combining yaml files into one dictionary: 
INFO     fre.yamltools.info_parsers.compile_info_parser:compile_info_parser.py:84    model yaml: fre/make/tests/null_example/null_model.yaml
INFO     fre.yamltools.info_parsers.compile_info_parser:compile_info_parser.py:114    compile yaml: fre/make/tests/null_example/compile.yaml
INFO     fre.yamltools.info_parsers.compile_info_parser:compile_info_parser.py:146    platforms yaml: fre/make/tests/null_example/platforms.yaml
INFO     fre.yamltools.combine_yamls_script:combine_yamls_script.py:175 Combined yaml information saved as dictionary
INFO     fre.make.run_fremake_script:run_fremake_script.py:74 Running fre make: calling checkout_create
INFO     fre.yamltools.combine_yamls_script:combine_yamls_script.py:166 initializing a compile yaml instance...
INFO     fre.yamltools.info_parsers.compile_info_parser:compile_info_parser.py:61 Combining yaml files into one dictionary: 
INFO     fre.yamltools.info_parsers.compile_info_parser:compile_info_parser.py:84    model yaml: fre/make/tests/null_example/null_model.yaml
INFO     fre.yamltools.info_parsers.compile_info_parser:compile_info_parser.py:114    compile yaml: fre/make/tests/null_example/compile.yaml
INFO     fre.yamltools.info_parsers.compile_info_parser:compile_info_parser.py:146    platforms yaml: fre/make/tests/null_example/platforms.yaml
INFO     fre.yamltools.combine_yamls_script:combine_yamls_script.py:175 Combined yaml information saved as dictionary
INFO     fre.make.create_checkout_script:create_checkout_script.py:147 
Checkout script created at tmp/hpcmini.2025/checkout.sh 

INFO     fre.make.run_fremake_script:run_fremake_script.py:79 Running fre make: calling makefile_create
INFO     fre.yamltools.combine_yamls_script:combine_yamls_script.py:166 initializing a compile yaml instance...
INFO     fre.yamltools.info_parsers.compile_info_parser:compile_info_parser.py:61 Combining yaml files into one dictionary: 
INFO     fre.yamltools.info_parsers.compile_info_parser:compile_info_parser.py:84    model yaml: fre/make/tests/null_example/null_model.yaml
INFO     fre.yamltools.info_parsers.compile_info_parser:compile_info_parser.py:114    compile yaml: fre/make/tests/null_example/compile.yaml
INFO     fre.yamltools.info_parsers.compile_info_parser:compile_info_parser.py:146    platforms yaml: fre/make/tests/null_example/platforms.yaml
INFO     fre.yamltools.combine_yamls_script:combine_yamls_script.py:175 Combined yaml information saved as dictionary
INFO     fre.make.create_makefile_script:create_makefile_script.py:118 
Makefile created at ./tmp/hpcmini.2025/Makefile

INFO     fre.make.run_fremake_script:run_fremake_script.py:102 Running fre make: calling dockerfile_create
INFO     fre.yamltools.combine_yamls_script:combine_yamls_script.py:166 initializing a compile yaml instance...
INFO     fre.yamltools.info_parsers.compile_info_parser:compile_info_parser.py:61 Combining yaml files into one dictionary: 
INFO     fre.yamltools.info_parsers.compile_info_parser:compile_info_parser.py:84    model yaml: fre/make/tests/null_example/null_model.yaml
INFO     fre.yamltools.info_parsers.compile_info_parser:compile_info_parser.py:114    compile yaml: fre/make/tests/null_example/compile.yaml
INFO     fre.yamltools.info_parsers.compile_info_parser:compile_info_parser.py:146    platforms yaml: fre/make/tests/null_example/platforms.yaml
INFO     fre.yamltools.combine_yamls_script:combine_yamls_script.py:175 Combined yaml information saved as dictionary
INFO     fre.make.create_docker_script:create_docker_script.py:109 
tmpDir created in /home/Ian.Laflotte/Working/fre-cli/tmp
INFO     fre.make.create_docker_script:create_docker_script.py:110 Dockerfile created in /home/Ian.Laflotte/Working/fre-cli

INFO     fre.make.create_docker_script:create_docker_script.py:111 Container build script created at /home/Ian.Laflotte/Working/fre-cli/createContainer.sh
============================================================================================================ short test summary info =============================================================================================================
FAILED fre/make/tests/compilation/test_run_fremake_builds.py::test_run_fremake_container_build_notransfer - subprocess.CalledProcessError: Command '['/home/Ian.Laflotte/Working/fre-cli/createContainer.sh']' returned non-zero exit status 1.

singhd789 · 2025-12-16T18:31:15Z

test_run_fremake_container_build_notransfer

Huh, ok so this one passes for me

ilaflott · 2025-12-23T18:02:52Z

@rem1776 did you test this on PPAN? my problem was visible on PPAN.

i would test it myself but i'm on a plane on my phone.

Fix container build fail test

e10c367

- was previously appending lines to createContainer.sh instead of overwriting - also addressed some pylint messages

singhd789 requested a review from ilaflott December 16, 2025 16:31

Add skip if podman not available

ca62296

Dana Singh added 2 commits December 16, 2025 14:04

Fix nparallel options - change to default number for clarity

d80e492

Add apptainer skip to test

3a40b3b

singhd789 requested review from kiihne-noaa and rem1776 December 17, 2025 21:28

ilaflott removed their request for review December 18, 2025 16:04

rem1776 approved these changes Dec 23, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix container build fail test #691

Fix container build fail test #691

Uh oh!

singhd789 commented Dec 16, 2025

Uh oh!

codecov bot commented Dec 16, 2025 •

edited

Loading

Uh oh!

ilaflott commented Dec 16, 2025

Uh oh!

singhd789 commented Dec 16, 2025

Uh oh!

ilaflott commented Dec 23, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Fix container build fail test #691

Are you sure you want to change the base?

Fix container build fail test #691

Uh oh!

Conversation

singhd789 commented Dec 16, 2025

Describe your changes

Issue ticket number and link (if applicable)

Checklist before requesting a review

Uh oh!

codecov bot commented Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ilaflott commented Dec 16, 2025

Uh oh!

singhd789 commented Dec 16, 2025

Uh oh!

ilaflott commented Dec 23, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov bot commented Dec 16, 2025 •

edited

Loading