Script does not compile with `FSharp.Compiler.Service` while `dotnet fsi` runs it just fine

[kMutagene]

Hi there, i am new to working with the compiler directly, so please bear with me if this issue does not contain all relevant information.

My basic use case is that i want to use the FSharp.Compiler.Service nuget package to verify whether the content of a script contains valid F# code. To do that, i followed this tutorial.

this compilation fails, while just running the script via dotnet fsi works without issues (meaning that there is actually nothing wrong with the code)

Repro steps

Above tutorial basically means calling

fsc.exe -o ./test.dll -a test.fsx

wrong, see below, i just thought it is also relevant that calling fsc.exe directly also fails.

The respective F# code using FSharp.Compiler.Service looks like this (i am using this in a xUnit test project):

type Assert with

    static member ScriptCompiles (scriptPath: string) =
        let t = Path.GetTempFileName()
        let tempPath = Path.ChangeExtension(t, ".dll")
        let checker = FSharpChecker.Create()
        let errors, exitCode =
            checker.Compile([| "fsc.exe"; "-o"; tempPath; "-a"; scriptPath |]) 
            |> Async.RunSynchronously
        Assert.Empty(errors)
        Assert.Equal(0, exitCode)

try to compile a script with this content (i removed much of the original content trying to narrow this issue down):

#r "nuget: ARCExpect, 2.0.0"

open ARCExpect
open System.IO

// Input:
let _ = Directory.GetCurrentDirectory()

printfn "yay"

Expected behavior

Script compiles (exitcode is 0 and error list is empty)

Actual behavior

this results in the following error for both, compiling via cli or compiling programmatically:

error FS0193: The module/namespace 'System.IO' from compilation unit 'System.Runtime' did not contain the namespace, module or type 'Directory'

but running it via dotnet fsi just works fine (also both via cli or programmatically):

❯ dotnet fsi ./test.fsx
yay

Known workarounds

use dotnet fsi to "verify" the script, however this is not a good workaround since this actually executes the code, but i just want to know whether it could run

Related information

Provide any related information (optional):

• Win 11

• ❯ dotnet --version
  8.0.300
  

• Editing Tools (e.g. Visual Studio Version, Visual Studio): script shows no errors when opening in either of those
• <PackageReference Include="FSharp.Compiler.Service" Version="[43.8.300]" />

[vzarytovskii]

Invoking fsc.dll directly is not something which is a supported scenarip. Compiler normally needs all dlls, including runtime to be passed as references.

Also, fsx files have a special treatment in the compiler, so might do resolution differently. And lastly - I am not sure if fsc can load up packages from nuget, I'm pretty certain it's only FSI/dependency manager.

[kMutagene]

Invoking fsc.dll directly is not something which is a supported scenario

Is doing it programmatically a supported scenario? At least i had that impression reading this tutorial i followed from the docs: https://fsharp.github.io/fsharp-compiler-docs/fcs/compiler.html

And lastly - I am not sure if fsc can load up packages from nuget, I'm pretty certain it's only FSI/dependency manager

It works with other scripts that use #r nuget though. In my example, if i go back one major version in the dependency (which is targeting netstandard, while 2.0.0 targets .net 8), it works.

If this is not a supported scenario though, how would one proceed to check wether a script contains correct F# code programmatically?

[vzarytovskii]

Invoking fsc.dll directly is not something which is a supported scenario

Is doing it programmatically a supported scenario? At least i had that impression reading this tutorial i followed from the docs: https://fsharp.github.io/fsharp-compiler-docs/fcs/compiler.html

This is not exactly invoking it, it's just part of API, it's not actually calling fsc.exe/fsc.dll. Something which is not supported is to directly call fsc.exe file.fs.

And lastly - I am not sure if fsc can load up packages from nuget, I'm pretty certain it's only FSI/dependency manager

It works with other scripts that use #r nuget though. In my example, if i go back one major version in the dependency (which is targeting netstandard, while 2.0.0 targets .net 8), it works.

It is supported in scripts, when you run them with dotnet fsi, it doesn't (as far as I know) when you try to compile such script with compiler (fsc, not fsi).~~

Update: I think I know what you mean, I initially misunderstood the question/issue.

I think you need to pass all references (including runtime ones) if you run it from compiled code.

@TheAngryByrd or @baronfel might actually know better since they work much more with FCS from the consumer perspective.

[kMutagene]

I initially misunderstood the question/issue.

No worries, I see why. I tried to include all information i deemed relevant, but using fsc.exe directly is not really what this issue is about, i just thought it was relevant that that also fails.

To be fair, this line

checker.Compile([| "fsc.exe"; "-o"; tempPath; "-a"; scriptPath |]) 

is directly from the docs and confused me since it clearly looks like it is calling fsc.exe, but in the xml doc the function states that the first argument is just ignored, so this is why i thought it relevant using fsc.exe also from command line.

To come back to the issue at hand, compiling the script programmatically with FSharp.Compiler.Service fails. So i played around with the script a little and it seems like it starts failing once i reference an assembly that targets net8.0 only (specifically, this package version). When i switch the #r nuget reference back to this version that targets netstandard2.0, it compiles fine. So it seems to me like the difference in target frameworks of the referenced package here makes the difference.

I think you need to pass all references (including runtime ones) if you run it from compiled code.

How would i proceed adding these references? I do not think i can just add these assemblies via their path, as this is running in a CI job. also, it seems like #r nuget references are handled just fine in the working case, why is the necessary runtime included in that compilation?

I hope this is formulated more clearly now, sorry for the confusion

[KevinRansom]

@kMutagene

Compiling an F# application requires the developer to provide all of the required compiler inputs, these are:

1. Each required source file
2. Every referenced assembly(or it's corresponding reference assembly)
3. compile options.

For fsc.exe these inputs are gathered by the Dotnet SDK build process, so Ideally you can build an fsproj file, and run the command dotnet build yourproj.fsproj to build the project or dotnet run yourproj.fsproj.

Back in the day fsc myapp.fs used to create a runnable .exe file, that was relatively painless for the compiler to do, because most of the runtime was in the mscorlib.dll and System.XML.dll and one other whose name escapes for now, which we could discover at runtime. However, with the advent of Windows 8 and the refactoring of the runtime references got more compilated and with the coreclr even more complicated. To compile a coreclr app, the compiler is now presented with a list of 164 reference assemblies. When we run dotnet fsi we run a nasty hack(TM) which has us go and grovel through the dotnet SDK to find these assemblies, we do this mainly to remain compatible with the desktop FSI application, which mostly didn't require developers to reference desktop framework assemblies [to be clear: we regret this nasty hack it is the cause of a horrific start up penalty when executing FSI, that we are still struggling to figure out a fix for].

On day one of converting the FSC compiler to the coreclr we determined we would never want it to 'helpfully' find a framework to compile against, we would always rely on the dotnet SDK to do that. That principle is the same as that held by the CSC compiler. We did not stick to that principle for FSI, although in my opinion we should have, it would have certainly improved startup and script execution time.

My recommendation to you is you create a new project that Includes your script file and use dotnet build and dotnet run that is simplest and the most reliable. Note that the allowed language for .FS and .FSX files is slightly different, so the project file should include the scriptname.fsx file

I hope this helps

Kevin

[kMutagene]

Thanks for the interesting insights @KevinRansom !

So if i understand correctly, dotnet fsi is only able to run the script because of nasty hack, and my programmatic compilation fails because that hack is not present for that use case?

My recommendation to you is you create a new project that Includes your script file and use dotnet build and dotnet run that is simplest and the most reliable. Note that the allowed language for .FS and .FSX files is slightly different, so the project file should include the scriptname.fsx file

This will not work for me i am afraid, because then i would have to programmatically create projects and compile them, hundreds of times for each CI job. However, i found two projects that are dotnet tools that compile fsx scripts to executables: fflat and FSharpPacker, which seem to do exactly that - so i think i'll take the route of calling them and just checking if they have an exit code of 0.

On a side note, I do not understand why FSharp.Compiler.Service was even able to resolve #r nuget ... references in my old scripts. If i understand your comment correctly, that should have failed as well, because i need to supply referenced assemblies, which i did not.

[KevinRansom]

@kMutagene #r Mutagene ... productizes a different nasty hack. Or at least it's nasty hack exists in a different separate assembly from the compiler. Someone from the C# team was violently ill, when I explained how we did it, and it should be noted that C# interactive has nothing like it, although they do have a nasty hack for locating framework assemblies, otherwise it wouldn't run.

I am not at all proud of either piece of code, but I have to admit they are very useful, both have been on my "I absolutely must rewrite this better and differently list since the day they were committed".
In order to allow C# in dotnet interactive to do what we do, we re-use our F# library, but it is controlled from within notebooks.

This will not work for me i am afraid, because then i would have to programmatically create projects and compile them, hundreds of times for each CI job. However, i found two projects that are dotnet tools that compile fsx scripts to executables: fflat and FSharpPacker, which seem to do exactly that - so i think i'll take the route of calling them and just checking if they have an exit code of 0.

In order to avoid doing this once per test we build the project and cache the reference list once per target framework. And re-use the paths to the references over and over again.

It's mostly handled in our tests here:

fsharp/tests/FSharp.Test.Utilities/Utilities.fs

Line 173 in 324ba3d

module private TestReferences =

It is not well factored or has great code, but it shows you some of the ways we deal with this problem in our testing.

I hope this clears things up, please don't think Ill of me; the msbuild/nuget dependency to get framework assembly references needs a better solution that is less hacky and contextual it just hasn't bubbled to the top of anyone's consciousness.

Kevin