switch to fiber/v3, add in project configuration#4
Conversation
MERGE STATUS: ALREADY MERGED ✓This PR is already merged (merged 21 hours ago). The review is post-facto, Overview106 files changed, +10,939 additions, -2,809 deletions across 46 commits The PR combines three major initiatives:
This is NOT a small feature patch. It's a fundamental restructuring of Gecko's architecture. Assessment Against Your Focus Areas1. Auth Correctness
|
Findings-- 1. API contract is incomplete for the new authorization mapping fields
|
|
feedback addressed in new branch. See #5 |
Bottom lineGecko and
Gecko is a server-side GitHub App workflow: it stages upload metadata in Postgres, requests GitHub App installation tokens through Fence, then creates GitHub trees, commits, branches, and PRs directly through the GitHub API. Direct duplicationBoth generate this same LFS pointer body:
Key architectural difference
Gecko stores Areas of conflict / risk
RecommendationCreate a shared small library/package for:
Then make a clear product decision: Either Gecko is only a PR/pointer author and |
1. Ask Fence for a write-scoped installation tokenCall siteFile: accessToken, err := service.RequestInstallationToken(
ctx,
authorizationHeader,
organization,
project,
identity,
"write",
)Source: Git service wrapperFile: func (service *GitService) RequestInstallationToken(
ctx context.Context,
authorizationHeader string,
organization string,
project string,
identity GitRepositoryIdentity,
access string,
) (string, error) {
return service.fenceAPI.RequestInstallationToken(
ctx,
authorizationHeader,
organization,
project,
identity,
access,
)
}Source: Fence broker requestFile: if err := c.requestFenceGitHubBroker(
ctx,
authorizationHeader,
map[string]any{
"action": "installation_token",
"owner": identity.Owner,
"repo": identity.Repo,
"organization": organization,
"project": strings.TrimSpace(project),
"access": requestedAccess,
},
&payload,
); err != nil {Source: HTTP call to FenceFile: req, err := http.NewRequestWithContext(
ctx,
http.MethodPost,
strings.TrimRight(c.config.BaseURL, "/")+"/credentials/github",
bytes.NewReader(requestBody),
)Source: 2. Create GitHub clientFile: client, err := service.githubClient(accessToken)Source: Client constructionFile: func (service *GitService) githubClient(accessToken string) (*github.Client, error) {
options := []github.ClientOptionsFunc{
github.WithAuthToken(accessToken),
github.WithHTTPClient(service.client),
}Source: 3. Create Git LFS pointer filesLFS pointer generation functionFile: func BuildLFSPointerContent(checksum string, size int64) string {
return fmt.Sprintf(
"version https://git-lfs.github.com/spec/v1\noid sha256:%s\nsize %d\n",
strings.ToLower(strings.TrimSpace(checksum)),
size,
)
}Source: Pointer file creation during upload finalizationFile: entries = append(entries, &github.TreeEntry{
Path: github.Ptr(file.TargetPath),
Mode: github.Ptr("100644"),
Type: github.Ptr("blob"),
Content: github.Ptr(
BuildLFSPointerContent(
file.Checksum.String,
file.Size,
),
),
})Source: Important observationGecko does not write pointer files into a local Git worktree. Instead, it creates the LFS pointer text in memory and sends it directly to GitHub as blob content inside a Git tree. 4. Create GitHub treeBuild tree entriesFile: Source: Create tree API callFile: tree, response, err := client.Git.CreateTree(
ctx,
identity.Owner,
identity.Repo,
baseCommit.GetTree().GetSHA(),
entries,
)Source: GitHub REST equivalentPOST /repos/{owner}/{repo}/git/trees5. Create commitLoad base commitFile: baseCommit, response, err := client.Git.GetCommit(
ctx,
identity.Owner,
identity.Repo,
baseCommitSHA,
)Source: Create commit API callFile: commit, response, err := client.Git.CreateCommit(
ctx,
identity.Owner,
identity.Repo,
github.Commit{
Message: github.Ptr(title),
Tree: &github.Tree{SHA: github.Ptr(tree.GetSHA())},
Parents: []*github.Commit{
{SHA: github.Ptr(baseCommitSHA)},
},
},
nil,
)Source: GitHub REST equivalentPOST /repos/{owner}/{repo}/git/commits6. Create branchFile: _, response, err = client.Git.CreateRef(
ctx,
identity.Owner,
identity.Repo,
github.CreateRef{
Ref: "refs/heads/" + branchName,
SHA: commit.GetSHA(),
},
)Source: GitHub REST equivalentPOST /repos/{owner}/{repo}/git/refs7. Open pull requestFile: pr, response, err := client.PullRequests.Create(
ctx,
identity.Owner,
identity.Repo,
&github.NewPullRequest{
Title: github.Ptr(title),
Body: github.Ptr(body),
Base: github.Ptr(baseBranch),
Head: github.Ptr(branchName),
},
)Source: GitHub REST equivalentPOST /repos/{owner}/{repo}/pullsComplete call graphThis is the complete server-side GitHub App workflow introduced in PR #4. Unlike |
2 participants
PR Notes:
feature/project-configWhat This Branch Actually Does
This branch is not a small project-config patch. It changes Gecko from a relatively flat config and vector service into a multi-surface backend that now owns:
The branch is large because it combines feature delivery with a substantial internal re-layout. The biggest review question is not any one handler. It is whether the new package boundaries, auth model, data model, and route surface still line up cleanly with the CALYPR frontend and deployment stack.
Scope by Area
git diff --dirstatagainstmainshows the branch is concentrated in these areas:internal/git/~10%internal/server/http/git/~9%internal/server/middleware/~8%gecko/legacy removal/replacement ~8%internal/server/http/config/~7%internal/db/~6%config/~5%internal/thumbnail/~5%tests/integration/~4%That matches the real shape of the work: this is mostly a Git/project backend branch, plus the refactor needed to support it.
Architecture Reorganization
Before
The old service logic lived mostly in the top-level
gecko/package with mixed concerns:After
The branch moves Gecko toward a clearer package layout:
config/internal/db/internal/git/internal/integrations/fence/internal/integrations/github/go-githubinternal/server/http/configdirectorygithealthvectorsharedinternal/server/middleware/internal/thumbnail/internal/httputil/internal/logging/internal/vectoradapter/This is the right direction structurally. The tradeoff is that the branch mixes architectural reorganization with product behavior changes, so code review needs to separate “moved” from “changed.”
Runtime / Bootstrap Changes
main.gonow builds Gecko as a composable server with optional integrations:The important runtime additions are:
--git-data-dir/GIT_DATA_DIR--fence-base-url/FENCE_BASE_URL--github-api-base-url/GITHUB_API_BASE_URLOnce DB connectivity is available, Gecko now also constructs:
git.NewGitService(...)thumbnail.NewFilesystemStore(gitDataDir)That makes
gitDataDira hard runtime dependency for the Git-enabled server path.CI impact
Because Gecko now requires a Git data directory, CI had to be updated. The branch modifies:
.github/workflows/tests.yamlto launch Gecko with:
Without that, the service exits before health checks come up.
Config Model Changes
Project config becomes first-class
This branch adds a typed
ProjectConfigmodel in:config/projectConfig.goSupported fields:
titlecontact_emailsrc_repoorg_titledescriptionproject_titleValidation includes:
host/owner/repoformThis is a real shift from using generic config blobs everywhere. Project metadata now has a stronger contract and normalization behavior.
Explorer config compatibility
This branch also carries forward explorer config compatibility work, including the richer
fileActionsshape:{ "extensions": {"ext": ["action"]}, "actions": {"action": "/path"} }That matters because the frontend expects this richer form, and Gecko has to unmarshal it correctly for CALYPR configs to load.
Database / Persistence Changes
The database layer is no longer only about simple
config_schema.<type>JSON tables.Existing typed config tables
The init path now ensures config tables for:
explorernavfile_summaryprojectprojectsNew Git-related state tables
internal/db/EnsureGitProjectStateTableexpands the DB footprint significantly. This branch adds persistence for:config_schema.git_project_stateconfig_schema.git_organization_stateconfig_schema.git_upload_sessionconfig_schema.git_upload_session_fileconfig_schema.git_pending_repositoryconfig_schema.git_setup_sessionThis is one of the biggest branch changes. Gecko is no longer stateless around Git operations; it is now persisting lifecycle and reconciliation state explicitly.
HTTP Surface Changes
The route surface is much broader than before.
Top-level registration
The new entrypoint is:
internal/server/http/register.goRegistered route families:
/health/Dir.../config.../git...Config routes
internal/server/http/config/register.gonow exposes:Generic config routes
GET /config/typesGET /config/listTyped config groups:
/config/explorer/config/nav/config/file_summary/config/projectPer-type operations include combinations of:
GET /listGET /:configIdPUT /:configIdDELETE /:configIdProject config routes
This branch adds a dedicated project config surface:
GET /config/projectsGET /config/projects/listGET /config/projects/summaryGET /config/projects/:orgTitle/:projectTitlePUT /config/projects/:orgTitle/:projectTitleDELETE /config/projects/:orgTitle/:projectTitleDELETE /config/projects/:orgTitleThis is materially different from the old “config row by arbitrary key” model. The route shape now matches organization/project semantics directly.
Git routes
internal/server/http/git/register.gois one of the largest new route families in the service.Organization-level Git routes
GET /git/projectsGET /git/organizations/statusPOST /git/organizations/reconcilePOST /git/organizations/:orgTitle/init-connectPOST /git/organizations/:orgTitle/connectGET /git/organizations/:orgTitle/statusPOST /git/organizations/:orgTitle/reconcileThese routes cover installation status, connect flows, and organization-wide repository reconciliation.
Project-level Git read routes
GET /git/projects/:orgTitle/:projectTitleGET /git/projects/:orgTitle/:projectTitle/refsGET /git/projects/:orgTitle/:projectTitle/treeGET /git/projects/:orgTitle/:projectTitle/tree/*GET /git/projects/:orgTitle/:projectTitle/file/*GET /git/projects/:orgTitle/:projectTitle/download/*GET /git/projects/:orgTitle/:projectTitle/thumbnailThese expose Gecko as a repository-backed project read service, not just a config API.
Project-level Git write/workflow routes
PUT /git/projects/:orgTitle/:projectTitle/setupPUT /git/projects/:orgTitle/:projectTitle/storagePUT /git/projects/:orgTitle/:projectTitle/thumbnailDELETE /git/projects/:orgTitle/:projectTitle/thumbnailPOST /git/projects/:orgTitle/:projectTitle/updatePOST /git/projects/:orgTitle/:projectTitle/uploads/sessionGET /git/projects/:orgTitle/:projectTitle/uploads/session/:sessionIDPOST /git/projects/:orgTitle/:projectTitle/uploads/session/:sessionID/filesPOST /git/projects/:orgTitle/:projectTitle/uploads/session/:sessionID/finalizeThis is a major expansion of product surface. Gecko is now responsible for project repository setup, update, artifact flow staging, and PR-style finalization.
Auth and Access Model
The middleware layer has been significantly expanded.
Resource path model
The branch standardizes project authorization around Arborist-style paths:
Helper functions in
internal/server/middleware/access.gonormalize and check these resource paths.Config auth model
ConfigAuthnow treats explorer configs differently from base/global config routes:Project config auth model
ProjectConfigAuthchecks direct access on:and also allows certain broader admin-like resource paths such as:
*/programs/programs/{org}/programs/{org}/projectsGit auth model
Git reads and organization reads are protected separately:
GitProjectAuthGitOrganizationAuthThe important boundary in this branch is:
Authorizationtoken and Arborist/Fence checksThat split is correct, but reviewers should confirm it stays consistent across all write routes.
Fence / GitHub Integration Model
This branch adds explicit integration clients instead of spreading ad hoc HTTP logic through handlers.
Fence integration
internal/integrations/fence/client.goturns Fence into a GitHub App broker that Gecko calls for:The request target is:
POST {FENCE_BASE_URL}/credentials/githubwith action-based payloads.
This is architecturally important. Gecko is moving away from owning GitHub App secrets directly and toward asking Fence for short-lived GitHub access on demand.
GitHub integration
internal/integrations/github/client.gouses:github.com/google/go-github/v87/githubfor GitHub API metadata reads.
At minimum, it currently centralizes repository metadata lookup:
This is the right direction and avoids more hand-written GitHub REST client code.
Git Service and Repository Semantics
The branch adds a large
internal/git/package that now owns:The key product shift is that Gecko is no longer just proxying config or metadata. It now maintains local repository state under a configured data directory and serves project Git views from there.
Reviewers should pay special attention to:
updatebehaves when the local repo is missing vs already presentUpload and Thumbnail Workflows
Two entirely new concerns land in this branch.
Upload sessions
Upload state is now explicit and persistent:
This means Gecko now participates in a staged contribution workflow rather than just reading repository state.
Thumbnails
internal/thumbnail/adds filesystem-backed thumbnail storage plus validation.Route support includes:
GET /git/projects/:orgTitle/:projectTitle/thumbnailPUT /git/projects/:orgTitle/:projectTitle/thumbnailDELETE /git/projects/:orgTitle/:projectTitle/thumbnailThat is a durable product-surface change and should be reviewed as such, not as a minor helper addition.
Legacy Code Removal
A large part of the diff is deletion of the old flat handlers from
gecko/, including legacy files such as:handleConfig.gohandleDir.gohandleVector.gomiddleware.goresponse.goserver.goThis is not dead-code cleanup alone. These deletions are paired with replacements under
internal/server/http/...,internal/server/middleware/..., and supporting packages.Build / Tooling Changes
This branch also touches:
DockerfileMakefile.dockerignorego.modgo.sumThose are not side noise. They are part of the fallout from:
Review should include a sanity pass on container build assumptions and root
go build .behavior.Testing Changes
The branch adds or updates tests across multiple layers:
Notable files include:
config/explorerConfig_test.goconfig/projectConfig_test.gointernal/git/service_test.gointernal/git/upload_test.gointernal/integrations/fence/client_test.gointernal/server/middleware/git_test.gointernal/thumbnail/store_test.gotests/integration/*Given the branch size, the main review question is coverage shape rather than raw test count:
Highest-Risk Areas
If reviewing this branch for merge readiness, focus here first:
Auth correctness
Route compatibility
DB/state lifecycle
Git/Fence boundary
Config schema compatibility
Startup behavior
git-data-dirBottom Line
This branch should be read as a backend expansion and service re-platforming branch, not as a narrow project-config feature.
The durable outcomes are: