Support version pinning for aliases/scripts

[quintesse]

The problem with running aliases and scripts from remote catalogs/sites is that the original author can update them and then we're stuck with several issues:

• do we update automatically (possibly breaking an existing script) or not (possibly forgoing critical bug/security fixes)?
• we can already somewhat tell JBang to update by using --fresh , but what if we don't want that and want to "pin" a specific version we already have?
• how can we notify the user that a newer version is available (if automatic updating is not available/enabled)?
• how can we support multiple versions and how do we make it possible to install a specific version?
• how can a user list the versions that are available?

Also see discussion here quarkiverse/quarkus-mcp-servers#30

Edit: my current recommendation would be something similar to #1979 (comment)

Which basically is:

• special alias syntax alias:version@catalog
• the version will be used to "patch" the script-ref, this would work for git repos and maven artifacts
• nice to have: query repos for possible versions (tags/branches for git, artifact versions for maven)

The advantage is that the catalog format stays exactly the same so it's fully backward-compatible.

Examples (added 2026-01-03):
jbang myapp:0.0.3@catalog would be:

• for a G:A:V it would become G:A:0.0.3
• a relative or absolute that resolve to a url will get "swizzled" based on its format so i.e. https://github.com/jbangdev/jbang-catalog/blob/main/jupyter/trylink.java would become https://github.com/jbangdev/jbang-catalog/blob/0.0.3/jupyter/trylink.java
• etc.

[wfouche]

@quintesse , a possible solution could be to allow "fatjars" to be registered in the jbang-catalog along with the version of the command associated with each one of them.

    "tulip-cli": {
      "script-ref": "scripts/tulip-cli.kt",
      "fatjar-template": "scripts/tulip-cli-{version}-fatjar.jar",
      "fatjar-stable-version": "2.1.5",
      "fatjar-versions":  {
        "2.1.4" : "Supports Java 11 and 17",
        "2.1.5":  "Supports Java 11, 17, and 21",
        "2.1.6-testing": "Testing Java 24"
      }
    }

Just thinking out loud.

$ jbang tulip-cli@..... # works as it does today, runs scripts/tulip-cli.kt
$ jbang tulip-cli:2.1.4@..... # would check that a version 2.1.4 fatjar is registered, and then retrieve and run it
$ jbang tulip-cli:stable@..... # would retrieve the stable version 2.1.5 fatjar and run it
$ jbang tulip-cli:versions@..... # Should display all the registered versions of fatjar and version specific message

I have not checked what is in the fatjar, but ideally the fatjar also contains the script source code file(s).

[quintesse]

Having a section for versions was indeed something I thought of as well. Although I'm not sure why you chose to go with fatjars? Committing binaries alongside code is somewhat frowned upon and it would go against the idea that jbang always compiles the code on-the-fly (taking into account the Java version you're currently using etc).

So a very simple option could for example be:

    "tulip-cli": {
      "script-ref": "scripts/tulip-cli.kt",
      "description": "Tulip CLI command",
      "versions":  {
        "2.1.4" : "scripts/tulip-cli-2.1.4.kt",
        "2.1.5":  "scripts/tulip-cli-2.1.5.kt",
        "2.1.6-testing": "scripts/tulip-cli-2.1.6.kt"
      }
    }

with an extended version if you want to provide more information, eg:

    "tulip-cli": {
      "script-ref": "scripts/tulip-cli.kt",
      "description": "Tulip CLI command",
      "versions":  {
        "2.1.4" : "scripts/old/tulip-cli-2.1.4.kt",
        "2.1.5": {
           "script-ref": "scripts/old/tulip-cli-2.1.5.kt",
           "description": "New awesome features!"
        }
        "2.1.6-testing": {
           "script-ref": "scripts/old/tulip-cli-2.1.6.kt",
           "description": "Only for testing"
        }
      }
    }

But if your catalog is already on a versioning system like GitHub people will of course say "but why do I need to keep my versions manually when they are already versioned? That doesn't make any sense!". And they'd be right.

So even though this is an option, we need something more or something different.

[quintesse]

So in case of having your catalog on Github/Gitlab/etc we could imagine extending the above with something like this:

    "tulip-cli": {
      "script-ref": "scripts/tulip-cli.kt",
      "description": "Tulip CLI command",
      "versions":  {
        "2.1.4" : {
          "tag": "v2.1.4"
        },
        "2.1.5": {
          "tag": "v2.1.5"
           "description": "New awesome features!"
        }
        "2.1.6-testing": {
           "commit": "0fef023",
           "description": "Only for testing"
        }
      }
    }

And JBang would figure out the URL to create (you could of course also use the full URL directly to the specific version of that file, but that's not very user-friendly)

[quintesse]

But I know @maxandersen doesn't like the idea of having to supply/maintain all the versions (there could be hundreds of them and the people generating those versions might not be the ones also maintaining the catalog) so it would be nice if we could get lists of versions somehow without manually defining them all.

The first thing that comes to mind is:

    "tulip-cli": {
      "script-ref": "scripts/tulip-cli.kt",
      "description": "Tulip CLI command",
      "allow-versioning": "true"
    }

Which would try to do something smart, meaning that if the catalog is on a versioning system and you ask for an alias like tulip-cli:XXX, which has a relative path in its script-ref, it will interpret the XXX to be either a branch, tag or a commit and adjust the URL to retrieve the script accordingly.

In this case though:

    "cli": {
      "script-ref": "io.quarkus:quarkus-cli:3.21.0:runner",
      "allow-versioning": "true"
    }

The code would see that it's a Maven artifact and simply replace the existing version with XXX.

In both case we can try to be equally clever if the user asks for a list of available versions (using jbang alias versions tulip-cli@somewhere?) and either return a list of tags or a list of versions obtained from the Maven repos. But that's quite a bit of work.

Edit: thinking about it some more I guess the "allow-versioning": "true" is superfluous. Basically the absence of a "versions" section would imply that we can simply override any versioning using the above mentioned "smart" system. Power to the users! :-)

[wfouche]

@quintesse,

Why did I think of fatjars as a possible solution building block?

1. Some scripts uses the //SOURCE tag to reference multiple additional source files that are part of the script program. We cannot rename these files as they are imported using their current names into the main script.
2. JBang can run JARs as described in the documentation - https://www.jbang.dev/documentation/guide/latest/usage.html#running-jars.
3. Fatjars provides a mechanism to create a point-in-time snapshot of a script and its dependent source files in a Jbang friendly way.
4. It is easy to run a fatjar, just "java -jar scriptname-fatjar.jar" it.
5. Fatjars are also (mostly) tamper-proof. Once created, they will not be modified again. Which means that a year or two from now running the same fatjar will deliver the same result.

But I know @maxandersen doesn't like the idea of having to supply/maintain all the versions

The "version number" of a fatjar could simply be a monotonically increasing positive inter starting at 1. The command to snapshot (create a fatjar from a script file) will look at the most recent fatjar created for that script and use the old fatjar's "version number"+1 for the new fatjar.

$ jbang snapshot .{java,groovy,kt} # create versioned fatjar for this script.

The jbang snapshot command should have a --description option.

jbang snapshot .... would use the script name (without file extension) as the name of the directory to store versioned fatjars in for that specific script.

The implementation of this proposed design do not require any changes to be made to the current jbang-catalog,json file ,,, that's a major win!

So how would this work from an end-user point of view?

1. jbang tulip-cli@user # Normal behaviour, just as it works today, run scripts/tulip-cli.kt
2. jbang tulip-cli:1@user # Run scripts/tulip-cli/1/tulip-cli-fatjar.jar
3. jbang tulip-cli:versions@user # list the available versions, date created together with description

But if you think fatjars is not the right way to do this, then just simply snapshot the source files into the versioned folder.

jbang snapshot .... should create a sha1 for every script file so that the integrity of the jbang catalog can be verified by anyone.

I'm now thinking that the snapshotted source files is a better idea than using fatjars.

To re-iterate, this proposal can be implemented without modifying the jbang-catalog.json fle. 🥇

Once we agree on the design, I'm happy to contribute the code changes.

• Implement jbang snapshot command
• Update catalog.json processing and (if a version number is provided) redirect to the correct set of snapshotted scripts. SHA1 will be tested and an error message produced if the the source file(s) were modified since the snapshot was created.

The design is best recorded in an ADR document. How do you feel about creating ADRs for JBang development and putting them in folder jbang/adrs?

[wfouche]

On second thought jbang snapshot functionaility can initially be implemented as a jbang script and jbang itself only need to be updated to understand the versioning folder structure (minor changes to jbang). Once the jbang snapshot script is considered stable, it can then (if required) be incorporated into jbang.

I realized that a small extension to jbang-catalog.json is required to allow default (pinned) version to be set.

 "tulip-cli": {
      "script-ref": "scripts/tulip-cli.kt",
      "description": "Tulip CLI command".

      "versioning": {
           "enabled": true,
            "version":  5
       }

}

jbang tulip-cli@user is then equivalent to jbang tulip-cli:5@user

Version numbers start at 1 for snapshots, and version number 0 is reserved for the script file itself. If you want to run the script file as it is right now (with versioing enabled) then execute jbang tulip-cli:0@user.

@quintesse and @maxandersen , do you see anything that is not addressed by this design sketch?

[wfouche]

Evolution of the design sketch. Here is an example of two commands (cmd1.kt and cmd2.kt) which both import a shared library file called slib.kt

Cmd1 has been snapshotted once, and Cmd2 has been snapshotted twice. Description.txt is replaced by 00index.json

[wfouche]

Proof-of-concept snapshot script implemented and tested.

$ python3 jbang-snapshot.py <script-to-snapshot>.{java,kt,groovy} "<snapshot-description>"

Test example:

$ python3 jbang-snapshot.py scripts/cmd2.kt "Java 21 support enabled"
Source script   : scripts/cmd2.kt
Description     : Java 21 support enabled
Snapshot folder : scripts/cmd2/3
Snapshot started.
    scripts/cmd2.kt --> scripts/cmd2/3/
    scripts/cmd2.kt --> scripts/cmd2/3/cmd2.kt.sha1
    scripts/slib.kt --> scripts/cmd2/3/
    scripts/slib.kt --> scripts/cmd2/3/slib.kt.sha1
    scripts/cmd2/3/00index.json
    scripts/cmd2/3/00index.json --> scripts/cmd2/3/00index.json.sha1
Snapshot done.

[quintesse]

Some scripts uses the //SOURCE tag to reference multiple additional source files that are part of the script program. We cannot rename these files as they are imported using their current names into the main script.

Good point, But that can easily be remedied by doing a bit of clever file management.

As long as the other source files don't change you could still do:

app.java
app-1.0.java
app-1.1.java
included_a.java
included_b.java

But this has a couple of disadvantages anyway:

1. I realized that changing the name of the file brings problems because it would mean that the filename and the classname won't match anymore making it necessary to rename the file before use.
2. The file structure becomes messy

But you could do:

app.java
included_a.java
included_b.java
old/1.0/app.java
old/1.0/included_a.java
old/1.0/included_b.java
old/1.1/app.java
old/1.1/included_a.java
old/1.1/included_b.java

[wfouche]

@quintesse , did you see the results above of POC that I implemented?

A further refinement to this script is to optionally call git -m " commit message" <list of snapshot files so that a snapshot is committed to git as soon as the snapshot is created.

[quintesse]

But the whole issue with these solutions is that it doesn't bring anything new to the table, you could literally do this right now without making any changes:

    "tulip-cli": {
      "script-ref": "scripts/tulip-cli.kt",
    },
    "tulip-cli-1": {
      "script-ref": "scripts/1/tulip-cli.kt",
    },
    "tulip-cli-2": {
      "script-ref": "scripts/2/tulip-cli.kt",
    }

We'd only need a way to list versions which would be easy enough if we simply say something like "If alias X exists and also one or more aliases with the names X:1, X:2, ... X:N" then they are all considered different versions of the same alias.

No need for making anything more complicated.

Edit: I also realized this could make things backward compatible with older JBang versions: they'd just list all the versions as separate aliases and you could run them without any problem. That's a major advantage.

[quintesse]

So all in all, given my previous message, I'm not sure it's worth do any of this at all. It brings very few advantages to create a versioning that probably very few people are even going to use.

The solution that will give us "true" versioning using already existing technologies is the one I described earlier: #1979 (comment) as it doesn't deal with copying files and/or snapshotting etc., so that has my preference for now.

If it turns out that people are really into all this versioning and they want additional options like the ones you and I have been discussing here then we could always revisit.

[wfouche]

Just for fun, I wrote a snapshot tool in Java and added it to my JBang catalog.

$ jbang snapshot-cli@wfouche scripts/cmd1.kt "First public release"

Source script   : scripts/cmd1.kt
Description     : First public release

Snapshot folder:
   scripts/cmd1/1

Sources files:
   scripts/cmd1.kt
   scripts/slib.kt

Snapshot started:
    scripts/cmd1.kt copy scripts/cmd1/1
    scripts/cmd1.kt sha1 scripts/cmd1/1/cmd1.kt.sha1
    scripts/slib.kt copy scripts/cmd1/1
    scripts/slib.kt sha1 scripts/cmd1/1/slib.kt.sha1
    scripts/cmd1/1/00index.json create
    scripts/cmd1/1/00index.json sha1 scripts/cmd1/1/00index.json.sha1

Snapshot files:
   scripts/cmd1/1/00index.json
   scripts/cmd1/1/00index.json.sha1
   scripts/cmd1/1/cmd1.kt
   scripts/cmd1/1/cmd1.kt.sha1
   scripts/cmd1/1/slib.kt
   scripts/cmd1/1/slib.kt.sha1

Snapshot index file:
   scripts/cmd1/1/00index.json
   {
       "description": "First public release",
       "timestamp": "2025-03-30T08:27:32"
   }

Snapshot done.

[wfouche]

@quintesse , I've started using the versioning folder structure created by snapshot-cli in my own jbang catalog.

{
    "tulip-cli-dev": {
      "script-ref": "scripts/tulip-cli.kt"
    },
    "tulip-cli"     : {
      "script-ref": "scripts/tulip-cli/1/tulip-cli.kt"
    }
}

• tulip-cli-dev is the current development version of the script.
• tulip-cli is the stable version that users should use, and references snapshot 1 of tulip-cli.

I've also snapshotted the snapshot-cli itself.

snapshot-cli-dev@wfouche
   https://github.com/wfouche/jbang-catalog/blob/HEAD/scripts/SnapshotCli.java
snapshot-cli@wfouche
   https://github.com/wfouche/jbang-catalog/blob/HEAD/scripts/SnapshotCli/1/SnapshotCli.java
tulip-cli-dev@wfouche
   https://github.com/wfouche/jbang-catalog/blob/HEAD/scripts/tulip-cli.kt
tulip-cli@wfouche
   https://github.com/wfouche/jbang-catalog/blob/HEAD/scripts/tulip-cli/1/tulip-cli.kt

@maxandersen , @quintesse - having been using the snapshot-cli and snapshotting process for less than an hour with my jbang-catalog, I find it tremendously useful. Maybe it is something that in the future can be promoted to the wider jbang community.

This is what my jbang-catalog repo looks like at the moment: