Expose build_web_compilers options to customize sdk and kernel paths

[schultek]

TLDR;

Adding more customization options to build_web_compilers would allow Jaspr to support flutter embedding and flutter plugins without needing to maintain a custom fork of this package. These include changing the sdk-, kernel- and library paths of ddc_builder, adding to the list of platform libraries (to allow dart:ui) and a few more. No custom logic is needed beyond configuration options.

---

When I initially added flutter embedding support to Jaspr, I was forced to fork the build_web_compilers package because of the deep changes I needed to make to the compilation pipeline. The two main problems were:

1. Compiling a dart program that uses flutter (and thereby dart:ui).
2. Correctly initializing any depended-on flutter plugin.

The compilation is solved by:

• replacing a bunch of paths like the sdk path, sdk kernel path and libraries.json path with ones pointing to the respective assets from the installed flutter framework (compare)
• adding ui and ui_web to the list of allowed platform libraries (compare)
• ignoring checks on dart:io (compare)
• copying dart_sdk.js from flutter instead of compiling from source (compare)

The plugin initialization is solved by generating a custom entrypoint script that initializes all plugins, similar to what flutter is generating when running for web.

---

In the last release I made significant changes to the framework to reduce the custom logic (including moving all the plugin generation to jaspr_builders) and bring the fork much closer to the original, to a point where I think now there is a way to do everything Jaspr needs with a few additional build options on the original builders.

The following should be all options that would need to be added (preliminary names):

• build_web_compilers:ddc:
  • platform_sdk_path to be passed to platformSdk of DevCompilerBuilder and KernelBuilder
  • sdk_kernel_path to be passed to sdkKernelPath of DevCompilerBuilder and KernelBuilder
  • libraries_path to be passed to librariesPath of DevCompilerBuilder and KernelBuilder
  • additional_web_libraries to be passed to DartPlatform.register
• build_web_compilers:entrypoint
  • skip_unsupported_dependencies_check to be passed to module.computeTransitiveDependencies(throwIfUnsupported: ...) inside dart2js_bootstrap and dart2wasm_bootstrap
  • libraries_path to be passed to dart2js_bootstrap
• build_web_compilers:sdk_js
  • dart_js_path to use for copying the dart_sdk.js file and skipping its compilation (alternatively make compilation work using a different sdk, but so far that was unsuccessful for me)

By exposing these options, the package can stay completely independent and doesn't need any Flutter or Jaspr specific logic, while still enabling this use-case.

---

I'm happy to work on a PR for this, but before I do I would love some feedback from the team if this is the right direction. Finally I have two specific questions:

1. Does someone see a way to do the sdk.js compilation work instead of copying over? The only downside rn is that this doesn't allow enabling canary features (whatever that is).
2. Any concerns regarding the new hot-reload related parts of this package? I haven't tried it out yet and am not familiar enough with the whole thing to evaluate.

CC @parlough @kevmoo @davidmorgan @Markzipan

[kevmoo]

So excited to have you cranking on this.

@Markzipan is likely the best person with whom to coordinate.

[Markzipan]

Thanks for the details @schultek - and definitely happy to coordinate!

The ui, ui_web, and dart:io are fine to me (perhaps loosely related to this discussion: dart-lang/sdk#62125).

For using flutter-compiled resourcees (and related flags), passing a special set of BuilderOptions as in _ensureSameDdcOptions is what I've been doing. jaspr's specific invocation of webdev would enable this special pathway for all of its apps via a new flag (such as 'webdev serve --use-flutter-dart-sdk ...'), which could then be passed to build_runner via --defines here. This change should be entirely invisible to existing jaspr users. I'm open to other approaches you had in mind, and let me know if I'm missing something about the existing workflow that might make this difficult.

Wrt 1), what do you mean by 'make compilation work using a different sdk'? IIRC Flutter pulls a prebuilt SDK by default (which should be the one being copied here). Do you mean you'd like to specify non-Flutter SDKs or perform cold SDK builds (in the event the user doesn't have Flutter installed)?

For 2), the context is that DDC's swapping to a different module system that enables web hot reload. This is currently what 'canary' means. Supporting this mode is important, but I can work with you on that. None of these changes would cause issues there, but this hot-reload-enabled/canary mode does require a special SDK (of which Flutter already has a copy - so your strategy should be fine).

[schultek]

Great to hear!

The ui, ui_web, and dart:io are fine to me (perhaps loosely related to this discussion: dart-lang/sdk#62125).

Good to know, though I think for now this doesn't affect us.

jaspr's specific invocation of webdev would enable this special pathway for all of its apps via a new flag (such as 'webdev serve --use-flutter-dart-sdk ...'), which could then be passed to build_runner via --define. This change should be entirely invisible to existing jaspr users. I'm open to other approaches you had in mind, and let me know if I'm missing something about the existing workflow that might make this difficult.

Yes that's what I had in mind, Jaspr is already configuring the builders through different --define options, which are transparent to the user.

However, I think it's better if the logic to locate the flutter sdk should be on Jaspr side, not inside the builder. So instead of a boolean flag like --use-flutter-dart-sdk I would let Jaspr pass the actual path to the sdk location. Because a) then the builders stay flutter-independent and can potentially also be used with sdk forks, and b) the localization of the flutter sdk is a bit finnicky and I wouldn't want to put unsafe/unstable code in the builders.

Do you mean you'd like to specify non-Flutter SDKs or perform cold SDK builds (in the event the user doesn't have Flutter installed)?

I was thinking about cold sdk builds yes, but sounds like copying is a good solution for now.

---

I'll start preparing a PR then.

[schultek]

I opened #4301 as a starting point. It should already include all things that are needed for this to work.
Of course I need to test more but to give you an idea of where I'm going with this.

[Markzipan]

However, I think it's better if the logic to locate the flutter sdk should be on Jaspr side, not inside the builder. So instead of a boolean flag like --use-flutter-dart-sdk I would let Jaspr pass the actual path to the sdk location. Because a) then the builders stay flutter-independent and can potentially also be used with sdk forks, and b) the localization of the flutter sdk is a bit finnicky and I wouldn't want to put unsafe/unstable code in the builders.

Great points - I'm onboard with passing the configs in. I'll follow up in the PR - thanks for taking this on!

[schultek]

I tested this with a local version of jaspr_cli and it works great!