Support for cross-compiling on Linux

[universeindex]

Description

I've implemented support for cross-compiling a Windows build on a Linux host using clang-cl. I added a toolchain file at cmake/LinuxCrosscompile.cmake that achieves this. I also modified many source files to not use backslashes in include statements (which error with clang-cl) and use case-sensitive paths (many Linux filesystems are case-sensitive, unlike NTFS, which causes errors). I avoided changing actual functionality as much as possible, so I hope this isn't too intrusive. Additionally I apologize for the messy commit messages, I hope this isn't a big issue, I have detailed all of the changes I made here.

Changes

Include statements use forward slashes instead of backslashes
Include statements reference files with correct capitalization
Added a CMake toolchain file for using clang-cl on Linux
Minecraft.Client/Xbox/MinecraftWindows.rc is not compiled on Linux, this causes errors otherwise, as it can't find the windows SDK and I am unsure how to specify the location here.
I also added documentation for building on Linux.

Previous Behavior

Running CMake on Linux would throw a hardcoded error stating that Linux isn't supported. When removed, and clang-cl is set to be used, many errors occur due to include paths using backslashes and inconsistent capitalization. The only way to build and run on Linux before is to use a Windows VM, and there is no way to get proper code completion on a Linux IDE.

Root Cause

Backslashes in include paths are supported on Windows, but not other systems, and NTFS is case-insensitive, so include paths do not have to match capitalization, also not on other systems.

New Behavior

The game now compiles and outputs a Windows binary on Linux using clang-cl.

Fix Implementation

See changes section

[universeindex]

P.S. I tested building on Windows, which still works fine, so hopefully there shouldn't be any issues

[universeindex]

Most commits made to upstream will probably conflict with those by overwriting the includes I changed back with backslashes, but I'll be fixing those until this is merged

[void2012]

Yes, I will be very happy to have clang-cl support and cross compilation support. Actually, clang-cl will be a test site for /fp:fast because it optimizes better than MSVC. Let me review this one.

[void2012]

Have you tested it compiling with MSVC? Both msbuild and cmake. I hope this doesn't break the MSVC compilation.

[universeindex]

I tested CMake w/ visual studio generator on Windows (with MSVC) which worked fine, but I haven't tried the normal .sln yet. I can do that now, but it might be a minute, i'm working on a really slow machine currently

[universeindex]

works fine, compiles and runs w/ visual studio build ^

[void2012]

Could you please tell, aside from #includes, what did you change? Reviewing that much files is pain.

[universeindex]

I did rewrite Minecraft.Client/iob_shim.asm in C++ as IobShim.cpp because I couldn't get MASM to work under Linux, but that's it aside from the includes I believe.

[DwifteJB]

can confirm does compile on linux w/ cmake

well done :)

[void2012]

That asm file wasn't really an asm thing rather a MSVC linker hack and it appears we don't actually need that at all.

[DwifteJB]

the cmake file assumes /opt/msvc whereas https://github.com/mstorsjo/msvc-wine recommends ~/my_msvc/opt/msvc

just a simple issue as you can syslink or run the msvc-wine installer as root & install to /opt/msvc

[void2012]

@universeindex

[universeindex]

oh, good catch, i think i might set it to CMAKE_BINARY_DIR/msvc so users can symlink one instead, or if theres a way to set options of a toolchain file that would be best
will change soon, not home atm

[DwifteJB]

msvc-wine sets itself in the path directory (although the user has to set this up manually through something like .zshrc)

could do something like

whereis msbuild

to determine the directory?

if it isn't set as CMAKE_MSVC_DIR

[DwifteJB]

opened pr on your fork that fixes this & a clang error universeindex#1

[void2012]

I think __iob_func should be implemented as __declspec(naked) + __asm.

[DwifteJB]

I think __iob_func should be implemented as __declspec(naked) + __asm.

_declspec(naked) seems to not be working since __asm seems to have been dropped for x64 (if you want to use it you have to use a .asm file, like how 4J seems to have done it)

__attribute__((naked)) & __asm__() works however

[DwifteJB]

will need someone to test compiling on windows however

[void2012]

I think __iob_func should be implemented as __declspec(naked) + __asm.

_declspec(naked) seems to not be working since __asm seems to have been dropped for x64 (if you want to use it you have to use a .asm file, like how 4J seems to have done it)

__attribute__((naked)) & __asm__() works however

Forgot about it that, as I am used to x86 development. Uncool, Microsoft. Anyway, maybe conditionally use the .asm file for MSVC assembler and .cpp with [[__gnu__::__naked__]] for Clang-cl?

[DRAGONTOS]

i dont seem to be able to compile this

[Twig6943]

hello, i've fixed the merge conflicts and also added the linux build pipeline, is there anything else i should do before this is ready to merge? @void2012

I don't see a successful run of the ci on your repo

you should do that just to be sure

[Twig6943]

hello, i've fixed the merge conflicts and also added the linux build pipeline, is there anything else i should do before this is ready to merge? @void2012

I don't see a successful run of the ci on your repo

you should do that just to be sure

It seems to fail
https://github.com/Twig6943/MinecraftConsoles/actions/runs/22791109398/job/66117716430

[Twig6943]

You might wanna look at this which does build (goes further at least)

[universeindex]

oh, thanks, do you mind if i just use this?

[Twig6943]

oh, thanks, do you mind if i just use this?

Sure xD (wait for it to get a successfull build tho)

[universeindex]

oh nevermind i just set the wrong toolchain file, lol
should work now, or build at least, maybe it will break somewhere after that now that i've just said that it should work

[Twig6943]

Seems to get stuck at the end on me (or might finish all of a sudden who knows)

Also the run on your repo seems to have errors that doesn't happen on mine

Building thrm seperately (minecraft.world & minecraft.client) might change things (for the better or worse, not sure which)

[Twig6943]

Yea also gets stuck on yours after that error
It doesn:t give an error on mine but seems to get stuck at the very end.

I'll try to look into it tomorrow but no promises

[universeindex]

it seems to be compiling without errors, but has been frozen for almost an hour now... not sure if this means it broke or not

[carterisonline]

There are some mentions of the /MP flag causing wine-msvc to hang: mstorsjo/msvc-wine#153 (comment)

I tried removing /MP from build.ninja and running ninja -j 1 instead, but it just ended up hanging even earlier in the build.

Here's the build script I'm using: https://gist.github.com/carterisonline/42a647b01beb93738fa8f8f278459d45
On nixpkgs with python3, msitools, cmake, ninja, libclang, lld

[universeindex]

that's probably referring to the actual msvc running under wine, in this case msvc-wine is only providing the windows sdk and clang is the actual compiler, so i doubt it

[carterisonline]

Yeah MSVC is weird and I'm thinking that /MP applied to more than the compiler. On my end I'm seeing 5 identical processes trying to link (?) with Lib.exe

[void2012]

Note: if /MP is broken for you, it doesn't mean we need to disable it for the whole project. It significantly boosts the compilation speed.

[carterisonline]

Yeah waiting for this to compile with only one thread made me seriously consider switching to Windows. I can't wish that on anybody

[Twig6943]

Anybody got a successful run? or know what to change?

[Twig6943]

I did everything in the .yml in a local machine and it built fine

It seems to only get stuck on github actions

[Twig6943]

Tried with fedora, same result https://github.com/Twig6943/MinecraftConsoles/actions/runs/22802811436/job/66147070210

[Twig6943]

What are the odds theres something wrong with Minecraft.Client/stubs.cpp as it gets stuck on that, not on the last file?

[xgui4]

based, if that mean i can cross-compile on my linux system without a vm that could awesome, as i could make changes easily :).

[bulkmoerls]

let it all sink in right now please

[Twig6943]

What are the odds theres something wrong with Minecraft.Client/stubs.cpp as it gets stuck on that, not on the last file?

I removed everything in it besides pragma once but yea it just got stuck earlier I'm out of ideas

[universeindex]

I feel like this PR is having some scope creep, maybe we should just move the workflow stuff to its own PR?
The actual linux crosscompilation stuff I made this PR for initially is basically ready for merging anyways (once i fix merge conflicts again)

[Twig6943]

I feel like this PR is having some scope creep, maybe we should just move the workflow stuff to its own PR? The actual linux crosscompilation stuff I made this PR for initially is basically ready for merging anyways (once i fix merge conflicts again)

Makes sense to me as nobody has an idea as to how to fix the workflow as of writing this, and this is sort of a blocker for this pr as the .yml is included.

[universeindex]

I'll just delete it for now then

[universeindex]

there are build errors right now, fixing them

[universeindex]

I don't know wtf happened, but there are huge amounts of build errors happening now, give me a bit to fix those