Any possibility we could generate the charts with github directly and make them available for release automatically?

[jamez70]

On another project we generate SD card images. I am not sure about storage limits, but perhaps we could run the docker image to create a sectional tile database and it can just be downloaded directly

[N129BZ]

@jamez70 I'm not sure what you mean by "generate the charts with github directly..." I have zero experience doing anything with github other than managing source control and releases...

Regards,
Brian

[jamez70]

I understand. Basically, GitHub has runners and we could perhaps generate the charts right on here directly. However, there might be a limit to how large they can be.

[N129BZ]

Looks like this could definitely be done but it would have to be a self-hosted runner and use self-hosted file storage. The user would then (I think) need to use git LFS to access the generated file. I don't have any kind of server or public IP that could do that...

From what I'm reading, git has a 5 gb limit on LFS.

[N129BZ]

After more looking around, a public IP should not be necessary, just a machine that is available 24/7

[N129BZ]

Just a thought - individual area charts are anywhere between 35 to 200 mb, those would definitely be under the github file size limit, as opposed to the full sectional at 3-5 gb depending on image type and quality

[jamez70]

I could possibly host it if there isn't a lot of demand - meaning like 100s of thousands wouldn't be downloading it.

[N129BZ]

Since you have a web server - I have added webserver and websocket functionality to the application, which allows a remote client to request whatever chart(s) they want generated. (The command line interface is still there and working.) The next piece I'll be working on is checking if the requested chart needs to be or already has been generated and adding a finished job zip file download link upon process completion... Then also some kind of back end file archive management... Not sure if that fits in with what we're talking about here.

[gaurav85]

Looks like this could definitely be done but it would have to be a self-hosted runner and use self-hosted file storage. The user would then (I think) need to use git LFS to access the generated file. I don't have any kind of server or public IP that could do that...

From what I'm reading, git has a 5 gb limit on LFS.

I ended up setting up a pipeline to do this exactly. I used github actions to spin up an EC2 instance that processes the charts and then uploads to a cloud storage. Took a few tries but everything works and is fully automated to check for new charts when available from FAA.

[N129BZ]

Excellent!

[jamez70]

Were you able to get free cloud storage or is this just something for your own use?

[gaurav85]

Were you able to get free cloud storage or is this just something for your own use?

I ended up using Cloudflare R2. The free tier provides enough Class A operations for upload and it has free egress. I am planning on working on a few aviation apps and was wanting to host my own charts for this use.