Bootstrap overflows

[jcguu95]

bootstrap.sh seems to be overflowing my memory and killed by the system. From the system profiler, I see it is taking more than 67 GB of memory. Is there a workaround? Or should I simply get a better computer?

$ ./bootstrap.sh 
boostrap.sh: $MAL_IMPL is not set; using built-in default Mal interpreter
boostrap.sh: Compiling mal-to-llvm.mal ...
malc: Source file: /private/tmp/malc/mal-to-llvm.mal
malc: Compile mode: release
malc: Intermediate LLVM IR file: /private/tmp/malc/mal-to-llvm.ll
malc: Compiling Mal program to LLVM IR
malc: Using Mal interpreter: /private/tmp/malc/mal-interpreter/ruby/run /private/tmp/malc/mal-to-llvm.mal
malc: Optimizing LLVM IR to: /private/tmp/malc/mal-to-llvm.opt.ll
malc: Compiling LLVM IR to object file: /private/tmp/malc/mal-to-llvm.o
malc: Linking executable file: /private/tmp/malc/mal-to-llvm
malc: Cleaning up
malc: Done
boostrap.sh: Sanity check
malc: Source file: /private/tmp/malc/sanity-check.mal
malc: Compile mode: release
malc: Intermediate LLVM IR file: /private/tmp/malc/sanity-check.ll
malc: Compiling Mal program to LLVM IR
malc: Using binary compiler: /private/tmp/malc/mal-to-llvm
./malc: line 83: 37997 Killed: 9               $bindir/mal-to-llvm $compile_mode $bindir $srcfile > "${outfile}.ll"

[dubek]

Interesting. I haven't touched this in a long time. It used to work inside github actions, so definitely didn't require so much memory. I'll take a look.

[dubek]

I tried to re-run this in Github Actions, I get an error earlier: ld.lld: error: undefined symbol: main

https://github.com/dubek/malc/actions/runs/8130816059/job/22219459873#step:4:15

Maybe it doesn't work with newest ubuntu / clang.
Which OS (+version) and clang/llvm version do you have?

[jcguu95]

OS: macOS (M2)
llvm: stable 17.0.6
Homebrew clang version 17.0.6
Target: arm64-apple-darwin21.6.0
Thread model: posix

[dubek]

How did you install libgc on macOS? I tried brew install bdw-gc but I get ld: library 'gc' not found:

Run ./bootstrap.sh
  ./bootstrap.sh
  shell: /bin/bash -e {0}
  env:
    LDFLAGS: -L/opt/homebrew/opt/llvm/lib -L/opt/homebrew/Cellar/bdw-gc/8.2.6/lib
boostrap.sh: $MAL_IMPL is not set; using built-in default Mal interpreter
boostrap.sh: Compiling mal-to-llvm.mal ...
malc: Source file: /Users/runner/work/malc/malc/mal-to-llvm.mal
malc: Compile mode: release
malc: Intermediate LLVM IR file: /Users/runner/work/malc/malc/mal-to-llvm.ll
malc: Compiling Mal program to LLVM IR
malc: Using Mal interpreter: /Users/runner/work/malc/malc/mal-interpreter/python/run /Users/runner/work/malc/malc/mal-to-llvm.mal
malc: Optimizing LLVM IR to: /Users/runner/work/malc/malc/mal-to-llvm.opt.ll
malc: Compiling LLVM IR to object file: /Users/runner/work/malc/malc/mal-to-llvm.o
malc: Linking executable file: /Users/runner/work/malc/malc/mal-to-llvm
clang: warning: argument unused during compilation: '-no-pie' [-Wunused-command-line-argument]
ld: library 'gc' not found
clang: error: linker command failed with exit code 1 (use -v to see invocation)

See https://github.com/dubek/malc/actions/runs/8181464816/job/22371287583?pr=6#step:5:18

[dubek]

(btw meanwhile I did some fixes for newer Ubuntu and clang on branch main (previously master), you might want to try with the latest.)

[dubek]

I managed to reproduce this in Github Actions CI on macOS worker:
https://github.com/dubek/malc/actions/runs/8192130963/job/22402968066?pr=6

boostrap.sh: Sanity check
malc: Source file: /Users/runner/work/malc/malc/sanity-check.mal
malc: Compile mode: release
malc: Intermediate LLVM IR file: /Users/runner/work/malc/malc/sanity-check.ll
malc: Compiling Mal program to LLVM IR
malc: Using binary compiler: /Users/runner/work/malc/malc/mal-to-llvm
./malc: line 83:  7760 Killed: 9               $bindir/mal-to-llvm $compile_mode $bindir $srcfile > "${outfile}.ll"
Error: Process completed with exit code 137.

I still don't know why it doesn't work on macOS.

It does work on docker with arm64 architecture, so if you have Docker around, you can use that.

[jcguu95]

I confirmed that it does work on arm64 docker. Feel free to close this. Thanks!