Skip to content

WIP: Start with the old versions.json, instead of starting from scratch#67

Draft
DilumAluthge wants to merge 2 commits into
mainfrom
dpa/reuse-existing-work
Draft

WIP: Start with the old versions.json, instead of starting from scratch#67
DilumAluthge wants to merge 2 commits into
mainfrom
dpa/reuse-existing-work

Conversation

@DilumAluthge
Copy link
Copy Markdown
Member

@DilumAluthge DilumAluthge commented Apr 17, 2026

🤖 Some parts written by Codex.

This addresses the first sub-item from #51:

  1. That process is very slow, so the overall build takes a considerable amount of time: my most recent runs as of this writing took 3.5 hours each, though earlier runs had usually been more like 1.5 hours (which is still a long time). It will continue to take more time for each new version that's added.

Prior to this PR, we build the entire versions.json from scratch every time. With this PR, we re-use as much of the existing versions.json as possible. The only parts we have to do are:

  1. New Julia versions that didn't exist previously.
  2. If any of the old info is stale, we have to re-do that.

In order to know whether the old info is good or stale, we store the ETag and Last-Modified headers, and we discard the old info if either header has changed.

Other changes

  1. Use custom structs for JSON.jl struct-mapping (instead of just using Dicts).

TODO before merging:

  • Uncomment the line marked "uncomment before merging"

🤖 Some parts written by Codex.

Co-authored-by: OpenAI Codex GPT-5 <codex@openai.com>
@DilumAluthge DilumAluthge force-pushed the dpa/reuse-existing-work branch from 3ab54f1 to 9f06959 Compare May 1, 2026 04:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant