From c2c1e599b2c1c0f9ecd994922b95f0ccd5e3b6c7 Mon Sep 17 00:00:00 2001 From: Aidar Siraev Date: Wed, 11 Feb 2026 15:28:49 +0300 Subject: [PATCH 01/11] init --- .gitignore | 5 +- .goreleaser.yml | 30 +- CLAUDE.md | 405 +-------- README.md | 189 +++-- db.go | 482 +++-------- go.mod | 12 +- go.sum | 36 +- main.go | 2112 +++++++++++++++++++++++++++------------------- priority_test.go | 1194 ++++++++++++++++++++++++++ 9 files changed, 2694 insertions(+), 1771 deletions(-) diff --git a/.gitignore b/.gitignore index c504490..8be55d8 100644 --- a/.gitignore +++ b/.gitignore @@ -1,4 +1,5 @@ -github-feed -github.db +gitlab-feed +gitlab.db gitai gitai.db +.env diff --git a/.goreleaser.yml b/.goreleaser.yml index d7e555a..f528279 100644 --- a/.goreleaser.yml +++ b/.goreleaser.yml @@ -6,8 +6,8 @@ before: - go mod tidy builds: - - id: github-feed - binary: github-feed + - id: gitlab-feed + binary: gitlab-feed env: - CGO_ENABLED=0 goos: @@ -74,28 +74,28 @@ release: ### macOS ```bash # Intel Mac - tar -xzf github-feed_{{.Version}}_Darwin_x86_64.tar.gz - chmod +x github-feed - sudo mv github-feed /usr/local/bin/ + tar -xzf gitlab-feed_{{.Version}}_Darwin_x86_64.tar.gz + chmod +x gitlab-feed + sudo mv gitlab-feed /usr/local/bin/ # Apple Silicon Mac - tar -xzf github-feed_{{.Version}}_Darwin_arm64.tar.gz - chmod +x github-feed - sudo mv github-feed /usr/local/bin/ + tar -xzf gitlab-feed_{{.Version}}_Darwin_arm64.tar.gz + chmod +x gitlab-feed + sudo mv gitlab-feed /usr/local/bin/ ``` ### Linux ```bash # x86_64 - tar -xzf github-feed_{{.Version}}_Linux_x86_64.tar.gz - chmod +x github-feed - sudo mv github-feed /usr/local/bin/ + tar -xzf gitlab-feed_{{.Version}}_Linux_x86_64.tar.gz + chmod +x gitlab-feed + sudo mv gitlab-feed /usr/local/bin/ # ARM64 - tar -xzf github-feed_{{.Version}}_Linux_arm64.tar.gz - chmod +x github-feed - sudo mv github-feed /usr/local/bin/ + tar -xzf gitlab-feed_{{.Version}}_Linux_arm64.tar.gz + chmod +x gitlab-feed + sudo mv gitlab-feed /usr/local/bin/ ``` ### Windows - Download the appropriate `.zip` file, extract it, and add the `github-feed.exe` to your PATH. + Download the appropriate `.zip` file, extract it, and add the `gitlab-feed.exe` to your PATH. diff --git a/CLAUDE.md b/CLAUDE.md index c71f12f..ef682d4 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,398 +1,43 @@ -# CLAUDE.md +# GitLab Feed (gitlab-feed) -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. - -## Project Overview - -GitHub Feed is a Go CLI tool for monitoring GitHub pull requests and issues across repositories. It tracks contributions, reviews, and assignments with colorized output and real-time progress visualization. - -The tool is also called "GitAI" in the README (branding name), but the binary is `github-feed`. +GitLab Feed is a Go CLI for monitoring GitLab merge requests and issues across a bounded set of projects. ## Build & Run ```bash -# Build the binary -go build -o github-feed . - -# Run directly (fetches from GitHub API) -./github-feed - -# Run with flags -./github-feed --time 3h # Show items from last 3 hours -./github-feed --time 2d # Show items from last 2 days -./github-feed --time 3w # Show items from last 3 weeks -./github-feed --time 6m # Show items from last 6 months (default: 1m) -./github-feed --time 1y # Show items from last year -./github-feed --debug # Show detailed API logging instead of progress bar -./github-feed --local # Use local database instead of GitHub API (offline mode) -./github-feed --links # Show hyperlinks underneath each PR/issue -./github-feed --ll # Shortcut for --local --links (offline mode with links) -./github-feed --clean # Delete and recreate the database cache -./github-feed --allowed-repos="owner/repo1,owner/repo2" # Filter to specific repos +go build -o gitlab-feed . + +./gitlab-feed +./gitlab-feed --time 3h +./gitlab-feed --debug +./gitlab-feed --local +./gitlab-feed --links +./gitlab-feed --ll +./gitlab-feed --clean +./gitlab-feed --allowed-repos "group/repo,group/subgroup/repo" ``` ## Configuration -The tool requires a GitHub Personal Access Token with `repo` and `read:org` scopes (not needed in `--local` mode). Configuration is loaded from: -1. Environment variables: `GITHUB_TOKEN` or `GITHUB_ACTIVITY_TOKEN`, and `GITHUB_USERNAME` or `GITHUB_USER` -2. Config file at `~/.github-feed/.env` (automatically created on first run) - -Database location: `~/.github-feed/github.db` (automatically created on first run) - -**First Run**: The tool automatically creates `~/.github-feed/` directory with: -- `.env` file with template for credentials (permissions: 0600) -- `github.db` database for caching GitHub data (permissions: 0666) -- Directory permissions: 0755 - -## Architecture & Key Components - -### Data Flow - -#### Online Mode (Default) -1. **Parallel API Fetching**: Six concurrent GitHub search queries for PRs (authored, mentioned, assigned, commented, reviewed, review-requested) - note: no "involved" query exists -2. **Issue Collection**: Four parallel searches for issues (authored, mentioned, assigned, commented) -3. **Database Caching** (db.go): All fetched PRs, issues, and comments are automatically saved to `~/.github-feed/github.db` BBolt database -4. **Cross-Reference Detection**: Links issues to PRs by checking PR/issue bodies and comments for references -5. **Display Rendering**: Separates items into sections by state (open/closed for PRs with merged as subset of closed) with colorized output -6. **Progress Tracking**: Dynamic progress bar that adjusts total count as pagination and additional API calls are discovered -7. **Error Handling**: Infinite retry with exponential backoff for all API calls, handling rate limits gracefully - -#### Offline Mode (`--local`) -1. **Database Loading**: Reads all PRs and issues from `~/.github-feed/github.db` instead of making API calls -2. **Data Conversion**: Converts database records to PRActivity and IssueActivity structures -3. **Display Rendering**: Same rendering logic as online mode, showing all cached data -4. **No API Calls**: Completely offline, no GitHub token required - -### Core Data Structures - -**Config**: Global configuration structure (main.go:46-57): -- Consolidates all application settings (debug mode, local mode, time range, etc.) -- Shared across the application via global `config` variable -- Includes client, database, progress, and context references -- Fields: debugMode, localMode, showLinks, timeRange, username, allowedRepos (map[string]bool), client, db, progress, ctx - -**PRActivity**: Represents a PR with metadata (main.go:22-30): -- Label: How the user is involved (e.g., "Authored", "Reviewed") -- Owner/Repo: Repository identification -- PR: GitHub PullRequest object (pointer to github.PullRequest) -- UpdatedAt: Last update timestamp (time.Time) -- HasUpdates: True if API version is newer than cached version -- Issues: Slice of linked IssueActivity that reference this PR - -**IssueActivity**: Represents an issue with similar metadata structure (main.go:32-39): -- Label: How the user is involved -- Owner/Repo: Repository identification -- Issue: GitHub Issue object (pointer to github.Issue) -- UpdatedAt: Last update timestamp -- HasUpdates: True if API version is newer than cached version - -**Progress**: Thread-safe progress tracking with colored bar display (main.go:41-44): -- Uses `atomic.Int32` for both `current` and `total` fields (no mutexes needed) -- Dynamically adjusts total as pagination and additional API calls are discovered -- Updates in real-time across all goroutines -- Provides visual feedback with color-coded completion status (red <33%, yellow <66%, green >=66%) -- Supports warning messages during retries via `displayWithWarning()` method (main.go:149-159) -- Methods: increment(), addToTotal(n int), buildBar(), display(), displayWithWarning(message string) - -**Database**: BBolt wrapper providing structured storage (db.go:20-22): -- PRWithLabel and IssueWithLabel: Wraps items with their activity labels -- Supports both old format (without labels) and new format (with labels) for backwards compatibility - -### Key Functions - -**getPRLabelPriority / getIssueLabelPriority**: Label priority functions (main.go:61-87): -- Define priority ordering for PR labels: Authored(1) > Assigned(2) > Reviewed(3) > Review Requested(4) > Commented(5) > Mentioned(6) -- Define priority ordering for issue labels: Authored(1) > Assigned(2) > Commented(3) > Mentioned(4) -- Unknown labels get priority 999 (lowest) -- Used by `shouldUpdateLabel()` to determine if a label should be replaced - -**shouldUpdateLabel**: Determines if a label should be updated (main.go:89-104): -- Takes current label, new label, and isPR flag -- Returns true if new label has higher priority (lower number) -- Empty current labels always get updated -- Ensures PRs/issues always show their most important involvement type -- Tested in priority_test.go - -**fetchAndDisplayActivity**: Main orchestration function (main.go:603-852): -- Checks GitHub API rate limits before starting (unless in local mode) -- Initial progress total: 10 (6 PR queries + 4 issue queries) in online mode, or 10 in offline mode -- In online mode, does NOT add event polling to progress (events API not used in current code) -- Launches parallel PR/issue searches with dynamic progress tracking -- Performs cross-reference detection to link issues with PRs -- Sorts and displays results by state (open PRs, closed/merged PRs, open issues, closed issues) -- Uses global `config` for all settings -- Handles both online (API) and offline (database) modes - -**retryWithBackoff**: Wraps API calls with infinite retry logic (main.go:161-247): -- Exponential backoff starting at 1s, max 30s -- Special handling for rate limit errors (longer backoff with factor 2.0, max 30s) -- General errors use shorter backoff (factor 1.5, max 5s) -- Shows countdown timer in progress bar during waits via `displayWithWarning()` -- Works seamlessly with both debug and normal modes -- Uses global `config.ctx` for cancellation support -- Detects rate limit errors by checking for "rate limit", "API rate limit exceeded", or "403" in error messages - -**areCrossReferenced**: Determines if a PR and issue reference each other (main.go:854-923): -- Checks PR/issue body text for mentions first (fast path) -- Fetching and checking PR comments for issue references only if body check fails -- Uses mentionsNumber() to detect patterns like "#123", "fixes #123", or full GitHub URLs -- Returns early if mention found in bodies to avoid API call -- Adds API call to progress total dynamically only when needed -- In local mode, loads comments from database instead of API - -**collectSearchResults**: Handles PR search with pagination (main.go:1151-1425): -- Supports both API mode (GitHub search) and local mode (database) -- In API mode: Paginates through GitHub search results, dynamically adds pages to progress -- In local mode: Filters by storedLabel matching the query label, respects timeRange cutoff -- Uses `sync.Map` for thread-safe deduplication (seenPRs and activitiesMap) -- Implements label priority system: only updates PR if new label has higher priority -- In API mode: Does NOT fetch full PR details separately - uses Issue data from search result directly -- Detects updates by comparing API timestamps with cached versions -- Caches all data to database with labels for offline mode -- Filters by allowed repos if configured - -**collectIssueSearchResults**: Handles issue search (main.go:1493-1709): -- Same pattern as PR collection but for issues -- Filters out items with PullRequestLinks (actual PRs, not issues) -- Uses `sync.Map` for thread-safe deduplication -- Implements label priority system for issues -- Stores issues with their activity labels -- Detects updates by comparing API timestamps with cached versions -- In local mode: filters by storedLabel and timeRange - -**collectActivityFromEvents**: ~~NOT CURRENTLY CALLED in the codebase~~ ✅ **REMOVED** - Dead code removed (was 186 lines) -- This function existed but was never invoked by fetchAndDisplayActivity -- Removed to reduce code complexity and maintainability burden - -**displayPR**: Renders a PR with color-coded information (main.go:1241-1267): -- Formatted date, label, username, repo, and title -- Update indicator (yellow ● icon) if item has updates since last cache -- Optional hyperlink with 🔗 icon (when `--links` flag is used) -- Uses global `config.showLinks` setting -- Format: `[●] YYYY/MM/DD LABEL USERNAME owner/repo#NUM - title` - -**displayIssue**: Renders an issue (main.go:1455-1491): -- Similar formatting to displayPR with proper indentation for nested issues -- State indicator (OPEN/CLOSED) when displayed under a PR (indented with "--") -- Update indicator (yellow ● icon) if item has updates since last cache -- Optional hyperlink with proper indentation -- Uses global `config.showLinks` setting - -**Color System**: Consistent color coding throughout: -- `getUserColor()` (main.go:267-287): FNV hash-based consistent colors per username (11 color options from fatih/color) -- `getLabelColor()` (main.go:249-265): Fixed colors for involvement types (Authored=cyan, Mentioned=yellow, Assigned=magenta, Commented=blue, Reviewed=green, Review Requested=red, Involved=hi-black, Recent Activity=hi-cyan) -- `getStateColor()` (main.go:289-300): Fixed colors for PR/issue states (open=green, closed=red, merged=magenta) - -**Helper Functions**: -- `loadEnvFile()` (main.go:302-325): Parses `.env` file and loads environment variables, skips comments and empty lines -- `parseTimeRange()` (main.go:327-357): Converts time range strings (e.g., "1h", "2d") to `time.Duration` -- `isRepoAllowed()` (main.go:549-555): Checks if a repository is in the allowed repos list -- `checkRateLimit()` (main.go:557-601): Checks GitHub API rate limits before making requests, warns when running low -- `mentionsNumber()` (main.go:925-962): Detects if text mentions a specific PR/issue number (supports patterns: #NUM, fixes #NUM, closes #NUM, resolves #NUM, GitHub URLs) - -### Label Priority System - -When a PR or issue appears in multiple search results (e.g., you both authored and reviewed a PR), the tool uses a priority system to determine which label to display: - -**PR Label Priorities** (from highest to lowest): -1. Authored - You created the PR -2. Assigned - You're assigned to the PR -3. Reviewed - You reviewed the PR -4. Review Requested - Your review was requested -5. Commented - You commented on the PR -6. Mentioned - You were mentioned in the PR +Online mode requires a GitLab Personal Access Token and `ALLOWED_REPOS`. -**Issue Label Priorities** (from highest to lowest): -1. Authored - You created the issue -2. Assigned - You're assigned to the issue -3. Commented - You commented on the issue -4. Mentioned - You were mentioned in the issue +The app loads configuration from: +1) Environment variables +2) `~/.gitlab-feed/.env` (auto-created on first run) -The system ensures that each PR/issue is displayed with its most important involvement type. When processing search results, labels are only updated if the new label has higher priority than the existing one. This prevents less important labels from overwriting more important ones. +Environment variables take precedence over values in the `.env` file. -### Concurrency Patterns +Environment variables: +- `GITLAB_TOKEN` or `GITLAB_ACTIVITY_TOKEN` +- `GITLAB_HOST` (optional host override; takes precedence over `GITLAB_BASE_URL`) +- `GITLAB_BASE_URL` (optional; default: `https://gitlab.com`) +- `ALLOWED_REPOS` (required online; comma-separated `group[/subgroup]/repo`) -The codebase uses `sync.WaitGroup` with goroutines (via `.Go()` method) for parallel API calls: -- **PR Collection**: 6 parallel search queries (authored, mentioned, assigned, commented, reviewed, review-requested) -- **Issue Collection**: 4 parallel search queries (authored, mentioned, assigned, commented) -- **Cross-Reference Detection**: Parallel checking of PR-issue relationships using WaitGroup - -All concurrent access to shared data is protected using modern Go patterns: -- **sync.Map**: `seenPRs`, `seenIssues`, `activitiesMap`, and `issueActivitiesMap` use `sync.Map` for lock-free concurrent access -- **atomic operations**: `Progress` struct uses `atomic.Int32` for `current` and `total` fields (no mutexes needed) -- **Conversion to slices**: After all goroutines complete, `sync.Map` data is converted to regular slices for sorting/display - -Progress tracking is thread-safe and updated after each API call across all goroutines. The progress bar dynamically adjusts its total as new work is discovered during execution. - -### Time Filtering - -Controlled by `--time` flag (default: `1m`): -- Shows both open and closed items updated in the specified time period -- Default: Items updated in last month (`1m` = 30 days) -- Supports flexible time ranges: - - `h` = hours (e.g., `3h` = 3 hours) - - `d` = days (e.g., `2d` = 2 days) - - `w` = weeks (e.g., `3w` = 3 weeks) - - `m` = months (e.g., `6m` = 6 months, approximated as 30 days each) - - `y` = years (e.g., `1y` = 1 year, approximated as 365 days) -- No separate state filtering - shows all states (open/merged/closed) from the time period -- Parsing happens in main() via parseTimeRange() function - -## GitHub API Integration - -Uses `google/go-github/v57` library. Key API patterns: -- **Search API** for bulk queries: `client.Search.Issues()` - used for both PRs and issues (PRs are identified by PullRequestLinks field) -- **PullRequests API**: NOT used for individual PR fetching - PR data comes directly from search results -- **Comments API** for cross-references: `client.PullRequests.ListComments()` - PR comment bodies (only when checking cross-references) -- **Rate limit checking**: `client.RateLimit.Get()` monitors both core (5000/hr) and search (30/min) limits - -**Error Handling**: All API calls wrapped with `retryWithBackoff()`: -- Infinite retries with exponential backoff -- Rate limit detection via error message inspection -- Countdown timer displayed during backoff periods -- Context-aware cancellation support - -**Progress Bar Accuracy**: The progress bar accurately tracks all API calls: -- Initial total: 10 (6 PR searches + 4 issue searches) in online mode -- Dynamically adds pagination when discovered on first page of results (lastPage - 1 additional pages) -- Dynamically adds comment fetches during cross-reference checks (only when body mentions not found) -- Each API call increments counter immediately after completion -- Does NOT track individual PR detail fetches (as they don't happen) - -## Database Module (db.go) - -**Database** structure wraps BBolt operations with three buckets: -- `pull_requests`: Stores PRs with key format `owner/repo#number` -- `issues`: Stores issues with same key format -- `comments`: Stores PR/issue comments with key format `owner/repo#number/type/commentID` - -**Data Format Evolution**: -- Old format: Direct JSON serialization of `github.PullRequest` / `github.Issue` -- New format: Wrapped in `PRWithLabel` / `IssueWithLabel` to store activity labels -- All read functions support both formats for backwards compatibility - -**Key Database Functions**: - -**Saving Data** (with labels, new format): -- `SavePullRequestWithLabel(owner, repo, pr, label, debugMode)` (db.go:90-120): Wraps PR in PRWithLabel, marshals to JSON, stores in pull_requests bucket -- `SaveIssueWithLabel(owner, repo, issue, label, debugMode)` (db.go:215-245): Same pattern for issues with IssueWithLabel -- `SavePRComment(owner, repo, prNumber, comment, debugMode)` (db.go:322-347): Stores PR review comments with key format `owner/repo#NUM/pr_review_comment/commentID` - -**Saving Data** (legacy, without labels): -- `SavePullRequest(owner, repo, pr, debugMode)` (db.go:63-88): Legacy format, still supported -- `SaveIssue(owner, repo, issue, debugMode)` (db.go:188-213): Legacy format -- `SaveComment(owner, repo, itemNumber, comment, commentType)` (db.go:308-320): Generic comment storage - -**Reading Data**: -- `GetPullRequest(owner, repo, number)` (db.go:122-146): Returns PR, attempts new format first, falls back to old format -- `GetPullRequestWithLabel(owner, repo, number)` (db.go:148-181): Returns PR and label, handles both formats -- `GetIssue(owner, repo, number)` (db.go:247-271): Returns issue, handles both formats -- `GetIssueWithLabel(owner, repo, number)` (db.go:273-306): Returns issue and label -- `GetPRComments(owner, repo, prNumber)` (db.go:573-595): Returns all PR comments using cursor-based prefix search -- `GetComment(owner, repo, itemNumber, commentType, commentID)` (db.go:349-366): Returns single comment - -**Bulk Reading**: -- `GetAllPullRequests(debugMode)` (db.go:378-418): Returns map[string]*github.PullRequest, handles both formats -- `GetAllPullRequestsWithLabels(debugMode)` (db.go:420-465): Returns PRs and labels separately, used for offline mode -- `GetAllIssues(debugMode)` (db.go:467-507): Returns map[string]*github.Issue -- `GetAllIssuesWithLabels(debugMode)` (db.go:509-554): Returns issues and labels separately -- `GetAllComments()` (db.go:556-571): Returns all comment bodies as strings - -**Utility**: -- `Stats()` (db.go:368-376): Returns counts of PRs, issues, and comments in database -- `Close()` (db.go:54-56): Closes BBolt database connection -- `OpenDatabase(path)` (db.go:24-52): Opens/creates database, creates buckets, sets permissions to 0666 - -## Command-Line Flags - -Flags are parsed manually in main() (main.go:359-410): -- `--time RANGE`: Show items from last time range (default: `1m`) - - Examples: `1h` (hour), `2d` (days), `3w` (weeks), `4m` (months), `1y` (year) -- `--debug`: Show detailed API logging instead of progress bar -- `--local`: Use local database instead of GitHub API (offline mode, no token required) -- `--links`: Show hyperlinks (🔗) underneath each PR/issue -- `--ll`: Shortcut for `--local --links` (offline mode with links) - IMPORTANT: This flag is NOT implemented in the code -- `--clean`: Delete and recreate the database cache (useful for starting fresh) -- `--allowed-repos REPOS`: Filter to specific repositories (comma-separated: `owner/repo1,owner/repo2`) - -**NOTE**: The `--ll` flag is documented in README but NOT implemented in the command-line parsing logic. Only `--local` and `--links` work as separate flags. - -## Testing Considerations - -When modifying this codebase: -- **Mode Testing**: Test with both `--debug` and default modes (progress bar vs. detailed logs) -- **Offline Mode**: Test `--local` mode to ensure database reads work correctly -- **Link Display**: Test `--links` flag to verify URLs are displayed correctly with proper indentation -- **Concurrency Safety**: Verify atomic operations and `sync.Map` usage on any new shared data structures -- **Label Priority**: Test label updates to ensure priority system works correctly (see priority_test.go) -- **Cross-Reference Patterns**: Test with various mention patterns: `#123`, `fixes #123`, `closes #123`, GitHub URLs -- **Rate Limits**: Consider GitHub API rate limits when adding new API calls (5000/hr core, 30/min search) -- **Progress Tracking**: When adding new API calls, ensure progress bar is updated: - - Add to total before making the call: `config.progress.addToTotal(1)` - - Increment after call completes: `config.progress.increment()` - - Display after each update: `config.progress.display()` (unless debug mode) -- **Error Handling**: Wrap all API calls with `retryWithBackoff()` for resilience -- **Database Errors**: Currently some database write errors are silently ignored with `_` - consider adding error logging -- **First Run**: Verify `~/.github-feed/` directory is created on first run with proper permissions (0755 for dir, 0600 for .env, 0666 for db) -- **Backwards Compatibility**: When changing database format, ensure old format can still be read -- **Global Config**: Use global `config` variable instead of passing parameters individually +Database cache: +- `~/.gitlab-feed/gitlab.db` (BBolt) ## Testing -The project includes unit tests for critical functionality: - -**priority_test.go**: Tests for label priority system -- `TestPRLabelPriority`: Validates PR label priority ordering (6 labels + unknown) -- `TestIssueLabelPriority`: Validates issue label priority ordering (4 labels + unknown) -- `TestShouldUpdateLabel_PR`: Tests PR label update logic with 7 test cases covering priority combinations -- `TestShouldUpdateLabel_Issue`: Tests issue label update logic with 7 test cases - -Run tests with: ```bash -go test -v -go test -v -run TestPRLabelPriority # Run specific test -``` - -## Known Issues & Discrepancies - -1. ~~**`--ll` flag**: Documented in README but NOT implemented in code~~ ✅ **FIXED** - Flag now properly implemented (lines 395-397) -2. ~~**collectActivityFromEvents**: Function exists but is never called~~ ✅ **FIXED** - Dead code removed (186 lines) -3. **Initial progress total**: README mentions "7 PR queries" but actual code uses 6 PR queries -4. **PR detail fetching**: Code no longer fetches individual PR details via PullRequests.Get() - uses search result data directly -5. ~~**Database write errors**: Some database Save operations ignore errors~~ ✅ **FIXED** - Error logging added with atomic counter and summary warnings - -## Refactoring Opportunities - -Based on code analysis, potential improvements include: -1. ✅ **COMPLETED: Atomic Operations** - Progress now uses `atomic.Int32` instead of mutexes -2. ✅ **COMPLETED: sync.Map** - All shared maps now use `sync.Map` for lock-free concurrency -3. ✅ **COMPLETED: Progress Bar Logic** - `buildBar()` method extracts progress bar building logic (main.go:114-136) -4. **Implement `--ll` flag**: Add parsing for combined `--local --links` shortcut -5. **Remove collectActivityFromEvents**: Delete unused code or re-integrate if needed -6. **Error Handling**: Add logging for ignored database write errors -7. **Code Reuse**: Extract common patterns between collectSearchResults and collectIssueSearchResults (both follow same structure) -8. **Display Logic**: Unify displayPR and displayIssue with generic display function -9. **Database Operations**: Extract common patterns in GetAll* functions (all do similar iteration and format handling) -10. **Channels**: Consider replacing WaitGroups with result channels for cleaner coordination - -## File Structure - -``` -github-feed/ -├── main.go # Main application (1710 lines) -├── db.go # Database operations (596 lines) -├── priority_test.go # Unit tests (106 lines) -├── go.mod # Go module definition -├── go.sum # Go module checksums -├── README.md # User documentation -├── CLAUDE.md # This file (AI assistant instructions) -├── .goreleaser.yml # GoReleaser configuration for builds -├── .gitignore # Git ignore patterns -└── .github/ - └── workflows/ - └── release.yml # GitHub Actions workflow for releases - -~/.github-feed/ # Config directory (auto-created) - ├── .env # Configuration file with credentials - └── github.db # BBolt database for caching +go test ./... -count=1 ``` diff --git a/README.md b/README.md index ae6f056..bfabdd8 100644 --- a/README.md +++ b/README.md @@ -1,16 +1,16 @@ -# GitAI - GitHub Activity Monitor +# GitAI - Activity Monitor -A fast, colorful CLI tool for monitoring GitHub pull requests and issues across repositories. Track your contributions, reviews, and assignments with real-time progress visualization. +A fast, colorful CLI tool for monitoring pull requests and issues across repositories. Track your contributions, reviews, and assignments with real-time progress visualization. ## Features -- 🚀 **Parallel API Calls** - Fetches data concurrently for maximum speed -- 🎨 **Colorized Output** - Easy-to-read color-coded labels, states, and progress -- 📊 **Smart Cross-Referencing** - Automatically links related PRs and issues -- ⚡ **Real-Time Progress Bar** - Visual feedback with color-coded completion status -- 🔍 **Comprehensive Search** - Tracks authored, mentioned, assigned, commented, and reviewed items -- 📅 **Time Filtering** - View items from the last month by default (configurable with `--time`) -- 🎯 **Organized Display** - Separates open, merged, and closed items into clear sections +- **Parallel API Calls** - Fetches data concurrently for maximum speed +- **Colorized Output** - Easy-to-read color-coded labels, states, and progress +- **Smart Cross-Referencing** - Automatically links related PRs and issues +- **Real-Time Progress Bar** - Visual feedback with color-coded completion status +- **Comprehensive Search** - Tracks authored, mentioned, assigned, commented, and reviewed items +- **Time Filtering** - View items from the last month by default (configurable with `--time`) +- **Organized Display** - Separates open, merged, and closed items into clear sections ## Installation @@ -21,37 +21,39 @@ Download the latest release for your platform from the [releases page](https://g **macOS** ```bash # Intel Mac -curl -L https://github.com/zveinn/github-feed/releases/latest/download/github-feed__Darwin_x86_64.tar.gz | tar xz -chmod +x github-feed -sudo mv github-feed /usr/local/bin/ +curl -L https://github.com/zveinn/github-feed/releases/latest/download/gitlab-feed__Darwin_x86_64.tar.gz | tar xz +chmod +x gitlab-feed +sudo mv gitlab-feed /usr/local/bin/ # Apple Silicon Mac -curl -L https://github.com/zveinn/github-feed/releases/latest/download/github-feed__Darwin_arm64.tar.gz | tar xz -chmod +x github-feed -sudo mv github-feed /usr/local/bin/ +curl -L https://github.com/zveinn/github-feed/releases/latest/download/gitlab-feed__Darwin_arm64.tar.gz | tar xz +chmod +x gitlab-feed +sudo mv gitlab-feed /usr/local/bin/ ``` **Linux** ```bash # x86_64 -curl -L https://github.com/zveinn/github-feed/releases/latest/download/github-feed__Linux_x86_64.tar.gz | tar xz -chmod +x github-feed -sudo mv github-feed /usr/local/bin/ +curl -L https://github.com/zveinn/github-feed/releases/latest/download/gitlab-feed__Linux_x86_64.tar.gz | tar xz +chmod +x gitlab-feed +sudo mv gitlab-feed /usr/local/bin/ # ARM64 -curl -L https://github.com/zveinn/github-feed/releases/latest/download/github-feed__Linux_arm64.tar.gz | tar xz -chmod +x github-feed -sudo mv github-feed /usr/local/bin/ +curl -L https://github.com/zveinn/github-feed/releases/latest/download/gitlab-feed__Linux_arm64.tar.gz | tar xz +chmod +x gitlab-feed +sudo mv gitlab-feed /usr/local/bin/ + + ``` **Windows** -Download the appropriate `.zip` file from the releases page, extract it, and add `github-feed.exe` to your PATH. +Download the appropriate `.zip` file from the releases page, extract it, and add `gitlab-feed.exe` to your PATH. ### Build from Source ```bash -go build -o github-feed . +go build -o gitlab-feed . ``` ### Release Management @@ -74,88 +76,99 @@ This will automatically: ### First Run Setup -On first run, GitAI automatically creates a configuration directory at `~/.github-feed/` with: +On first run, GitAI automatically creates a configuration directory at `~/.gitlab-feed/` with: - `.env` - Configuration file (with helpful template) -- `github.db` - Local database for caching GitHub data +- `gitlab.db` - Local database for caching activity data -### GitHub Token Setup +### API Token Setup -Create a GitHub Personal Access Token with the following scopes: -- `repo` - Access to repositories -- `read:org` - Read organization data +Create a GitLab Personal Access Token with: +- `read_api` (recommended) +- `api` (use this only if your self-managed instance requires it) -**Generate token:** https://github.com/settings/tokens +GitLab.com token page: https://gitlab.com/-/user_settings/personal_access_tokens ### Environment Setup -You can provide your token and username in two ways: +You can provide your token and optional username in two ways: **Option 1: Configuration File (Recommended)** -Edit `~/.github-feed/.env` and add your credentials: +Edit `~/.gitlab-feed/.env` and add your credentials: ```bash -# Your GitHub Personal Access Token (required) -GITHUB_TOKEN=your_token_here +# Your GitLab Personal Access Token (required for online mode) +GITLAB_TOKEN=your_token_here + +# Optional fallback token variable +# GITLAB_ACTIVITY_TOKEN=your_token_here -# Your GitHub username (required) -GITHUB_USERNAME=your_username +# Optional username (the tool can also resolve current user via API) +GITLAB_USERNAME=your_username -# Optional: Comma-separated list of allowed repos -ALLOWED_REPOS=user/repo1,user/repo2 +# Optional for self-managed GitLab (defaults to https://gitlab.com) +# Example: http://10.10.1.207/ +GITLAB_BASE_URL=https://gitlab.com + +# Required in online mode: comma-separated project paths +# Format: group[/subgroup]/repo +ALLOWED_REPOS=team/repo1,platform/backend/repo2 ``` **Option 2: Environment Variables** ```bash -export GITHUB_TOKEN="your_token_here" -export GITHUB_USERNAME="your_username" -export ALLOWED_REPOS="user/repo1,user/repo2" # Optional: filter to specific repos +export GITLAB_TOKEN="your_token_here" +export GITLAB_USERNAME="your_username" # Optional +export GITLAB_BASE_URL="https://gitlab.com" # Optional +export ALLOWED_REPOS="team/repo1,platform/backend/repo2" # Required in online mode ``` **Note:** Environment variables take precedence over the `.env` file. +If both are set, `GITLAB_HOST` takes precedence over `GITLAB_BASE_URL`. + ## Usage ### Basic Usage ```bash -# Monitor PRs and issues from the last month (default, fetches from GitHub) -github-feed +# Monitor merge requests and issues from the last month (default, fetches from API) +gitlab-feed # Show items from the last 3 hours -github-feed --time 3h +gitlab-feed --time 3h # Show items from the last 2 days -github-feed --time 2d +gitlab-feed --time 2d # Show items from the last 3 weeks -github-feed --time 3w +gitlab-feed --time 3w # Show items from the last 6 months -github-feed --time 6m +gitlab-feed --time 6m # Show items from the last year -github-feed --time 1y +gitlab-feed --time 1y # Show detailed logging output -github-feed --debug +gitlab-feed --debug -# Use local database instead of GitHub API (offline mode) -github-feed --local +# Use local database instead of API (offline mode) +gitlab-feed --local -# Show hyperlinks underneath each PR/issue -github-feed --links +# Show hyperlinks underneath each merge request/issue +gitlab-feed --links # Delete and recreate the database cache (start fresh) -github-feed --clean +gitlab-feed --clean # Filter to specific repositories only -github-feed --allowed-repos="user/repo1,user/repo2" +gitlab-feed --allowed-repos="team/repo1,platform/backend/repo2" # Quick offline mode with links (combines --local and --links) -github-feed --ll +gitlab-feed --ll # Combine flags -github-feed --local --time 2w --debug --links --allowed-repos="miniohq/ec,tunnels-is/tunnels" +gitlab-feed --local --time 2w --debug --links --allowed-repos="team/repo1,platform/backend/repo2" ``` ### Command Line Options @@ -164,11 +177,11 @@ github-feed --local --time 2w --debug --links --allowed-repos="miniohq/ec,tunnel |------|-------------| | `--time RANGE` | Show items from the last time range (default: `1m`)
Examples: `1h` (hour), `2d` (days), `3w` (weeks), `4m` (months), `1y` (year) | | `--debug` | Show detailed API call progress instead of progress bar | -| `--local` | Use local database instead of GitHub API (offline mode, no token required) | -| `--links` | Show hyperlinks (with 🔗 icon) underneath each PR and issue | +| `--local` | Use local database instead of GitLab API (offline mode, no token required) | +| `--links` | Show hyperlinks underneath each PR and issue | | `--ll` | Shortcut for `--local --links` (offline mode with links) | | `--clean` | Delete and recreate the database cache (useful for starting fresh or fixing corrupted cache) | -| `--allowed-repos REPOS` | Filter to specific repositories (comma-separated: `user/repo1,user/repo2`) | +| `--allowed-repos REPOS` | Filter to specific repositories (comma-separated: `group/repo,group/subgroup/repo`) | ### Color Coding @@ -192,26 +205,17 @@ github-feed --local --time 2w --debug --links --allowed-repos="miniohq/ec,tunnel ### Online Mode (Default) -1. **Parallel Fetching** - Simultaneously searches for: - - PRs you authored - - PRs where you're mentioned - - PRs assigned to you - - PRs you commented on - - PRs you reviewed - - PRs requesting your review - - PRs involving you - - Your recent activity events - - Issues you authored/mentioned/assigned/commented - -2. **Local Caching** - All fetched data is automatically saved to a local BBolt database (`~/.github-feed/github.db`) - - PRs, issues, and comments are cached for offline access +1. **Scoped Fetching** - Loads merge requests and issues from `ALLOWED_REPOS` project paths + - Uses GitLab identity to label authored/assigned/reviewed/commented/mentioned activity + - Applies your `--time` window to keep output focused + +2. **Local Caching** - All fetched data is automatically saved to a local BBolt database (`~/.gitlab-feed/gitlab.db`) + - Merge requests, issues, and notes are cached for offline access - Each item is stored/updated with a unique key - Database grows as you fetch more data -3. **Cross-Reference Detection** - Automatically finds connections between PRs and issues by: - - Checking PR body and comments for issue references (`#123`, `fixes #123`, full URLs) - - Checking issue body and comments for PR references - - Displaying linked issues directly under their related PRs +3. **Cross-Reference Detection** - Automatically finds connections between merge requests and issues + - Displays linked issues directly under their related merge requests 4. **Smart Filtering**: - Shows both open and closed items from the specified time period @@ -220,9 +224,9 @@ github-feed --local --time 2w --debug --links --allowed-repos="miniohq/ec,tunnel ### Offline Mode (`--local`) -- Reads all data from the local database instead of GitHub API -- No internet connection or GitHub token required -- Displays all cached PRs and issues +- Reads all data from the local database instead of GitLab API +- No internet connection or token required +- Displays all cached merge requests and issues - Useful for: - Working offline - Faster lookups when you don't need fresh data @@ -230,11 +234,8 @@ github-feed --local --time 2w --debug --links --allowed-repos="miniohq/ec,tunnel ## API Rate Limits -GitAI monitors GitHub API rate limits and will warn you when running low: -- **Search API**: 30 requests per minute -- **Core API**: 5000 requests per hour - -Rate limit status is displayed in debug mode. +GitAI monitors API limits and retries automatically when responses indicate throttling. +Rate limit and retry status is displayed in debug mode. ### Automatic Retry & Backoff @@ -242,13 +243,16 @@ When rate limits are hit, GitAI automatically retries with exponential backoff: - Detects rate limit errors (429, 403 responses) - Waits progressively longer between retries (1s → 2s → 4s → ... up to 30s max) - Continues indefinitely until the request succeeds -- Shows clear warnings: `⚠ Rate limit hit, waiting [duration] before retry...` +- Shows clear warnings: `Rate limit hit, waiting [duration] before retry...` - No manual intervention required - the tool handles rate limits gracefully ## Troubleshooting -### "GITHUB_TOKEN environment variable is required" -Set up your GitHub token as described in [Configuration](#configuration). +### "token is required for GitLab API mode" +Set `GITLAB_TOKEN` (or `GITLAB_ACTIVITY_TOKEN`) as described in [Configuration](#configuration). + +### "ALLOWED_REPOS is required for GitLab API mode" +Set `ALLOWED_REPOS` with one or more GitLab project paths in `group[/subgroup]/repo` format. ### "Rate limit exceeded" Wait for the rate limit to reset. Use `--debug` to see current rate limits. @@ -260,9 +264,9 @@ Your terminal may not support ANSI colors properly. Use `--debug` mode for plain ### Project Structure ``` -github-feed/ +gitlab-feed/ ├── main.go # Main application code -├── db.go # Database operations for caching GitHub data +├── db.go # Database operations for caching activity data ├── README.md # This file ├── CLAUDE.md # Instructions for Claude Code AI assistant ├── .goreleaser.yml # GoReleaser configuration for builds @@ -270,9 +274,9 @@ github-feed/ │ └── workflows/ │ └── release.yml # GitHub Actions workflow for releases -~/.github-feed/ # Config directory (auto-created) +~/.gitlab-feed/ # Config directory (auto-created) ├── .env # Configuration file with credentials - └── github.db # BBolt database for caching + └── gitlab.db # BBolt database for caching ``` ### Testing Releases Locally @@ -293,4 +297,3 @@ ls -la dist/ ## License MIT License - Feel free to use and modify as needed. - diff --git a/db.go b/db.go index a407c8b..848d107 100644 --- a/db.go +++ b/db.go @@ -7,31 +7,37 @@ import ( "strings" "time" - "github.com/google/go-github/v57/github" bolt "go.etcd.io/bbolt" ) var ( - pullRequestsBucket = []byte("pull_requests") - issuesBucket = []byte("issues") - commentsBucket = []byte("comments") + gitlabMergeRequestsBkt = []byte("gitlab_merge_requests") + gitlabIssuesBkt = []byte("gitlab_issues") + gitlabNotesBkt = []byte("gitlab_notes") ) type Database struct { db *bolt.DB } -// buildItemKey creates a consistent key format for PRs and issues -func buildItemKey(owner, repo string, number int) string { - return fmt.Sprintf("%s/%s#%d", owner, repo, number) +func buildGitLabMergeRequestKey(pathWithNamespace string, iid int) string { + return fmt.Sprintf("%s#!%d", normalizeProjectPathWithNamespace(pathWithNamespace), iid) } -// buildCommentKey creates a consistent key format for comments -func buildCommentKey(owner, repo string, itemNumber int, commentType string, commentID int64) string { - return fmt.Sprintf("%s/%s#%d/%s/%d", owner, repo, itemNumber, commentType, commentID) +func buildGitLabIssueKey(pathWithNamespace string, iid int) string { + return fmt.Sprintf("%s##%d", normalizeProjectPathWithNamespace(pathWithNamespace), iid) +} + +func buildGitLabNoteKey(pathWithNamespace, itemType string, iid int, noteID int64) string { + return fmt.Sprintf( + "%s|%s|%d|%d", + normalizeProjectPathWithNamespace(pathWithNamespace), + strings.ToLower(strings.TrimSpace(itemType)), + iid, + noteID, + ) } -// save is a generic function to save data to a bucket with consistent error handling and logging func (d *Database) save(bucket []byte, key string, data interface{}, debugMode bool, itemType string) error { jsonData, err := json.Marshal(data) if err != nil { @@ -45,16 +51,17 @@ func (d *Database) save(bucket []byte, key string, data interface{}, debugMode b b := tx.Bucket(bucket) return b.Put([]byte(key), jsonData) }) - if err != nil { if debugMode { fmt.Printf(" [DB] Error saving %s %s: %v\n", itemType, key, err) } - } else if debugMode { - fmt.Printf(" [DB] Saved %s %s\n", itemType, key) + return err } - return err + if debugMode { + fmt.Printf(" [DB] Saved %s %s\n", itemType, key) + } + return nil } func OpenDatabase(path string) (*Database, error) { @@ -64,12 +71,12 @@ func OpenDatabase(path string) (*Database, error) { } if err := os.Chmod(path, 0666); err != nil { - db.Close() + _ = db.Close() return nil, fmt.Errorf("failed to set database permissions: %w", err) } err = db.Update(func(tx *bolt.Tx) error { - buckets := [][]byte{pullRequestsBucket, issuesBucket, commentsBucket} + buckets := [][]byte{gitlabMergeRequestsBkt, gitlabIssuesBkt, gitlabNotesBkt} for _, bucket := range buckets { _, err := tx.CreateBucketIfNotExists(bucket) if err != nil { @@ -78,9 +85,8 @@ func OpenDatabase(path string) (*Database, error) { } return nil }) - if err != nil { - db.Close() + _ = db.Close() return nil, err } @@ -91,451 +97,167 @@ func (d *Database) Close() error { return d.db.Close() } -type PRWithLabel struct { - PR *github.PullRequest +type GitLabMRWithLabel struct { + MR MergeRequestModel Label string } -func (d *Database) SavePullRequest(owner, repo string, pr *github.PullRequest, debugMode bool) error { - key := buildItemKey(owner, repo, pr.GetNumber()) - return d.save(pullRequestsBucket, key, pr, debugMode, "PR") -} - -func (d *Database) SavePullRequestWithLabel(owner, repo string, pr *github.PullRequest, label string, debugMode bool) error { - key := buildItemKey(owner, repo, pr.GetNumber()) - prWithLabel := PRWithLabel{ - PR: pr, - Label: label, - } - return d.save(pullRequestsBucket, key, prWithLabel, debugMode, fmt.Sprintf("PR with label %s", label)) -} - -func (d *Database) GetPullRequest(owner, repo string, number int) (*github.PullRequest, error) { - key := buildItemKey(owner, repo, number) - - var pr github.PullRequest - err := d.db.View(func(tx *bolt.Tx) error { - b := tx.Bucket(pullRequestsBucket) - data := b.Get([]byte(key)) - if data == nil { - return fmt.Errorf("PR not found") - } - - var prWithLabel PRWithLabel - if err := json.Unmarshal(data, &prWithLabel); err == nil && prWithLabel.PR != nil { - pr = *prWithLabel.PR - return nil - } - - return json.Unmarshal(data, &pr) - }) - - if err != nil { - return nil, err - } - return &pr, nil -} - -func (d *Database) GetPullRequestWithLabel(owner, repo string, number int) (*github.PullRequest, string, error) { - key := buildItemKey(owner, repo, number) - - var pr *github.PullRequest - var label string - - err := d.db.View(func(tx *bolt.Tx) error { - b := tx.Bucket(pullRequestsBucket) - data := b.Get([]byte(key)) - if data == nil { - return fmt.Errorf("PR not found") - } - - var prWithLabel PRWithLabel - if err := json.Unmarshal(data, &prWithLabel); err == nil && prWithLabel.PR != nil { - pr = prWithLabel.PR - label = prWithLabel.Label - return nil - } - - var oldPR github.PullRequest - if err := json.Unmarshal(data, &oldPR); err != nil { - return err - } - pr = &oldPR - label = "" - return nil - }) - - if err != nil { - return nil, "", err - } - return pr, label, nil -} - -type IssueWithLabel struct { - Issue *github.Issue +type GitLabIssueWithLabel struct { + Issue IssueModel Label string } -func (d *Database) SaveIssue(owner, repo string, issue *github.Issue, debugMode bool) error { - key := buildItemKey(owner, repo, issue.GetNumber()) - return d.save(issuesBucket, key, issue, debugMode, "issue") +type GitLabNoteRecord struct { + ProjectPath string + ItemType string + ItemIID int + NoteID int64 + Body string + AuthorUsername string + AuthorID int64 } -func (d *Database) SaveIssueWithLabel(owner, repo string, issue *github.Issue, label string, debugMode bool) error { - key := buildItemKey(owner, repo, issue.GetNumber()) - issueWithLabel := IssueWithLabel{ - Issue: issue, - Label: label, - } - return d.save(issuesBucket, key, issueWithLabel, debugMode, fmt.Sprintf("issue with label %s", label)) +func (d *Database) SaveGitLabMergeRequestWithLabel(pathWithNamespace string, mr MergeRequestModel, label string, debugMode bool) error { + key := buildGitLabMergeRequestKey(pathWithNamespace, mr.Number) + item := GitLabMRWithLabel{MR: mr, Label: label} + return d.save(gitlabMergeRequestsBkt, key, item, debugMode, fmt.Sprintf("gitlab merge request with label %s", label)) } -func (d *Database) GetIssue(owner, repo string, number int) (*github.Issue, error) { - key := buildItemKey(owner, repo, number) - - var issue github.Issue - err := d.db.View(func(tx *bolt.Tx) error { - b := tx.Bucket(issuesBucket) - data := b.Get([]byte(key)) - if data == nil { - return fmt.Errorf("issue not found") - } - - var issueWithLabel IssueWithLabel - if err := json.Unmarshal(data, &issueWithLabel); err == nil && issueWithLabel.Issue != nil { - issue = *issueWithLabel.Issue - return nil - } - - return json.Unmarshal(data, &issue) - }) - - if err != nil { - return nil, err - } - return &issue, nil +func (d *Database) SaveGitLabIssueWithLabel(pathWithNamespace string, issue IssueModel, label string, debugMode bool) error { + key := buildGitLabIssueKey(pathWithNamespace, issue.Number) + item := GitLabIssueWithLabel{Issue: issue, Label: label} + return d.save(gitlabIssuesBkt, key, item, debugMode, fmt.Sprintf("gitlab issue with label %s", label)) } -func (d *Database) GetIssueWithLabel(owner, repo string, number int) (*github.Issue, string, error) { - key := buildItemKey(owner, repo, number) - - var issue *github.Issue - var label string - - err := d.db.View(func(tx *bolt.Tx) error { - b := tx.Bucket(issuesBucket) - data := b.Get([]byte(key)) - if data == nil { - return fmt.Errorf("issue not found") - } - - var issueWithLabel IssueWithLabel - if err := json.Unmarshal(data, &issueWithLabel); err == nil && issueWithLabel.Issue != nil { - issue = issueWithLabel.Issue - label = issueWithLabel.Label - return nil - } - - var oldIssue github.Issue - if err := json.Unmarshal(data, &oldIssue); err != nil { - return err - } - issue = &oldIssue - label = "" - return nil - }) - - if err != nil { - return nil, "", err - } - return issue, label, nil +func (d *Database) SaveGitLabNote(note GitLabNoteRecord, debugMode bool) error { + key := buildGitLabNoteKey(note.ProjectPath, note.ItemType, note.ItemIID, note.NoteID) + return d.save(gitlabNotesBkt, key, note, debugMode, "gitlab note") } -func (d *Database) SaveComment(owner, repo string, itemNumber int, comment *github.IssueComment, commentType string) error { - key := buildCommentKey(owner, repo, itemNumber, commentType, comment.GetID()) - - data, err := json.Marshal(comment) - if err != nil { - return fmt.Errorf("failed to marshal comment: %w", err) - } - - return d.db.Update(func(tx *bolt.Tx) error { - b := tx.Bucket(commentsBucket) - return b.Put([]byte(key), data) - }) -} - -func (d *Database) SavePRComment(owner, repo string, prNumber int, comment *github.PullRequestComment, debugMode bool) error { - key := fmt.Sprintf("%s/%s#%d/pr_review_comment/%d", owner, repo, prNumber, comment.GetID()) - - data, err := json.Marshal(comment) - if err != nil { - if debugMode { - fmt.Printf(" [DB] Error marshaling PR comment %s: %v\n", key, err) - } - return fmt.Errorf("failed to marshal PR comment: %w", err) - } - - err = d.db.Update(func(tx *bolt.Tx) error { - b := tx.Bucket(commentsBucket) - return b.Put([]byte(key), data) - }) - - if err != nil { - if debugMode { - fmt.Printf(" [DB] Error saving PR comment %s: %v\n", key, err) - } - } else if debugMode { - fmt.Printf(" [DB] Saved PR comment %s\n", key) - } - - return err -} - -func (d *Database) GetComment(owner, repo string, itemNumber int, commentType string, commentID int64) (*github.IssueComment, error) { - key := buildCommentKey(owner, repo, itemNumber, commentType, commentID) - - var comment github.IssueComment - err := d.db.View(func(tx *bolt.Tx) error { - b := tx.Bucket(commentsBucket) - data := b.Get([]byte(key)) - if data == nil { - return fmt.Errorf("comment not found") - } - return json.Unmarshal(data, &comment) - }) - - if err != nil { - return nil, err - } - return &comment, nil -} - -func (d *Database) Stats() (prCount, issueCount, commentCount int, err error) { - err = d.db.View(func(tx *bolt.Tx) error { - prCount = tx.Bucket(pullRequestsBucket).Stats().KeyN - issueCount = tx.Bucket(issuesBucket).Stats().KeyN - commentCount = tx.Bucket(commentsBucket).Stats().KeyN - return nil - }) - return -} - -func (d *Database) GetAllPullRequests(debugMode bool) (map[string]*github.PullRequest, error) { - prs := make(map[string]*github.PullRequest) +func (d *Database) GetAllGitLabMergeRequestsWithLabels(debugMode bool) (map[string]MergeRequestModel, map[string]string, error) { + items := make(map[string]MergeRequestModel) + labels := make(map[string]string) if debugMode { - fmt.Printf(" [DB] Reading all PRs from database...\n") + fmt.Printf(" [DB] Reading all GitLab merge requests with labels from database...\n") } err := d.db.View(func(tx *bolt.Tx) error { - b := tx.Bucket(pullRequestsBucket) + b := tx.Bucket(gitlabMergeRequestsBkt) return b.ForEach(func(k, v []byte) error { - var prWithLabel PRWithLabel - if err := json.Unmarshal(v, &prWithLabel); err == nil && prWithLabel.PR != nil { - prs[string(k)] = prWithLabel.PR - return nil - } - - var pr github.PullRequest - if err := json.Unmarshal(v, &pr); err != nil { + key := string(k) + var item GitLabMRWithLabel + if err := json.Unmarshal(v, &item); err != nil { if debugMode { - fmt.Printf(" [DB] Error unmarshaling PR %s: %v\n", string(k), err) + fmt.Printf(" [DB] Error unmarshaling gitlab merge request %s: %v\n", key, err) } return err } - prs[string(k)] = &pr + items[key] = item.MR + labels[key] = item.Label return nil }) }) - if err != nil { if debugMode { - fmt.Printf(" [DB] Error reading PRs: %v\n", err) + fmt.Printf(" [DB] Error reading GitLab merge requests: %v\n", err) } - return nil, err + return nil, nil, err } if debugMode { - fmt.Printf(" [DB] Loaded %d PRs from database\n", len(prs)) + fmt.Printf(" [DB] Loaded %d GitLab merge requests from database\n", len(items)) } - return prs, nil + return items, labels, nil } -func (d *Database) GetAllPullRequestsWithLabels(debugMode bool) (map[string]*github.PullRequest, map[string]string, error) { - prs := make(map[string]*github.PullRequest) +func (d *Database) GetAllGitLabIssuesWithLabels(debugMode bool) (map[string]IssueModel, map[string]string, error) { + items := make(map[string]IssueModel) labels := make(map[string]string) if debugMode { - fmt.Printf(" [DB] Reading all PRs with labels from database...\n") + fmt.Printf(" [DB] Reading all GitLab issues with labels from database...\n") } err := d.db.View(func(tx *bolt.Tx) error { - b := tx.Bucket(pullRequestsBucket) + b := tx.Bucket(gitlabIssuesBkt) return b.ForEach(func(k, v []byte) error { key := string(k) - - var prWithLabel PRWithLabel - if err := json.Unmarshal(v, &prWithLabel); err == nil && prWithLabel.PR != nil { - prs[key] = prWithLabel.PR - labels[key] = prWithLabel.Label - return nil - } - - var pr github.PullRequest - if err := json.Unmarshal(v, &pr); err != nil { + var item GitLabIssueWithLabel + if err := json.Unmarshal(v, &item); err != nil { if debugMode { - fmt.Printf(" [DB] Error unmarshaling PR %s: %v\n", key, err) + fmt.Printf(" [DB] Error unmarshaling gitlab issue %s: %v\n", key, err) } return err } - prs[key] = &pr - labels[key] = "" // No label in old format + items[key] = item.Issue + labels[key] = item.Label return nil }) }) - if err != nil { if debugMode { - fmt.Printf(" [DB] Error reading PRs: %v\n", err) + fmt.Printf(" [DB] Error reading GitLab issues: %v\n", err) } return nil, nil, err } if debugMode { - fmt.Printf(" [DB] Loaded %d PRs from database\n", len(prs)) + fmt.Printf(" [DB] Loaded %d GitLab issues from database\n", len(items)) } - return prs, labels, nil + return items, labels, nil } -func (d *Database) GetAllIssues(debugMode bool) (map[string]*github.Issue, error) { - issues := make(map[string]*github.Issue) - - if debugMode { - fmt.Printf(" [DB] Reading all issues from database...\n") - } - +func (d *Database) HasGitLabData() (bool, error) { + hasData := false err := d.db.View(func(tx *bolt.Tx) error { - b := tx.Bucket(issuesBucket) - return b.ForEach(func(k, v []byte) error { - var issueWithLabel IssueWithLabel - if err := json.Unmarshal(v, &issueWithLabel); err == nil && issueWithLabel.Issue != nil { - issues[string(k)] = issueWithLabel.Issue - return nil - } - - var issue github.Issue - if err := json.Unmarshal(v, &issue); err != nil { - if debugMode { - fmt.Printf(" [DB] Error unmarshaling issue %s: %v\n", string(k), err) - } - return err - } - issues[string(k)] = &issue + b := tx.Bucket(gitlabMergeRequestsBkt) + if b != nil && b.Stats().KeyN > 0 { + hasData = true return nil - }) - }) - - if err != nil { - if debugMode { - fmt.Printf(" [DB] Error reading issues: %v\n", err) } - return nil, err - } - if debugMode { - fmt.Printf(" [DB] Loaded %d issues from database\n", len(issues)) - } - - return issues, nil -} - -func (d *Database) GetAllIssuesWithLabels(debugMode bool) (map[string]*github.Issue, map[string]string, error) { - issues := make(map[string]*github.Issue) - labels := make(map[string]string) - - if debugMode { - fmt.Printf(" [DB] Reading all issues with labels from database...\n") - } - - err := d.db.View(func(tx *bolt.Tx) error { - b := tx.Bucket(issuesBucket) - return b.ForEach(func(k, v []byte) error { - key := string(k) - - var issueWithLabel IssueWithLabel - if err := json.Unmarshal(v, &issueWithLabel); err == nil && issueWithLabel.Issue != nil { - issues[key] = issueWithLabel.Issue - labels[key] = issueWithLabel.Label - return nil - } - - var issue github.Issue - if err := json.Unmarshal(v, &issue); err != nil { - if debugMode { - fmt.Printf(" [DB] Error unmarshaling issue %s: %v\n", key, err) - } - return err - } - issues[key] = &issue - labels[key] = "" // No label in old format - return nil - }) + b = tx.Bucket(gitlabIssuesBkt) + if b != nil && b.Stats().KeyN > 0 { + hasData = true + } + return nil }) - if err != nil { - if debugMode { - fmt.Printf(" [DB] Error reading issues: %v\n", err) - } - return nil, nil, err + return false, err } - - if debugMode { - fmt.Printf(" [DB] Loaded %d issues from database\n", len(issues)) - } - - return issues, labels, nil + return hasData, nil } -func (d *Database) GetAllComments() ([]string, error) { - var comments []string +func (d *Database) GetGitLabNotes(pathWithNamespace, itemType string, iid int) ([]GitLabNoteRecord, error) { + notes := make([]GitLabNoteRecord, 0) + prefix := fmt.Sprintf( + "%s|%s|%d|", + normalizeProjectPathWithNamespace(pathWithNamespace), + strings.ToLower(strings.TrimSpace(itemType)), + iid, + ) err := d.db.View(func(tx *bolt.Tx) error { - b := tx.Bucket(commentsBucket) - return b.ForEach(func(k, v []byte) error { - comments = append(comments, string(v)) + b := tx.Bucket(gitlabNotesBkt) + if b == nil { return nil - }) - }) - - if err != nil { - return nil, err - } - return comments, nil -} - -func (d *Database) GetPRComments(owner, repo string, prNumber int) ([]*github.PullRequestComment, error) { - var comments []*github.PullRequestComment - prefix := fmt.Sprintf("%s/%s#%d/pr_review_comment/", owner, repo, prNumber) + } - err := d.db.View(func(tx *bolt.Tx) error { - b := tx.Bucket(commentsBucket) c := b.Cursor() - for k, v := c.Seek([]byte(prefix)); k != nil && strings.HasPrefix(string(k), prefix); k, v = c.Next() { - var comment github.PullRequestComment - if err := json.Unmarshal(v, &comment); err != nil { + var record GitLabNoteRecord + if err := json.Unmarshal(v, &record); err != nil { return err } - comments = append(comments, &comment) + notes = append(notes, record) } return nil }) - if err != nil { return nil, err } - return comments, nil + return notes, nil } diff --git a/go.mod b/go.mod index 32e0bd7..f29d970 100644 --- a/go.mod +++ b/go.mod @@ -4,13 +4,17 @@ go 1.25 require ( github.com/fatih/color v1.18.0 - github.com/google/go-github/v57 v57.0.0 + gitlab.com/gitlab-org/api/client-go v1.30.0 + go.etcd.io/bbolt v1.4.3 ) require ( - github.com/google/go-querystring v1.1.0 // indirect + github.com/google/go-querystring v1.2.0 // indirect + github.com/hashicorp/go-cleanhttp v0.5.2 // indirect + github.com/hashicorp/go-retryablehttp v0.7.8 // indirect github.com/mattn/go-colorable v0.1.13 // indirect github.com/mattn/go-isatty v0.0.20 // indirect - go.etcd.io/bbolt v1.4.3 // indirect - golang.org/x/sys v0.29.0 // indirect + golang.org/x/oauth2 v0.34.0 // indirect + golang.org/x/sys v0.39.0 // indirect + golang.org/x/time v0.14.0 // indirect ) diff --git a/go.sum b/go.sum index b370a01..ebc515d 100644 --- a/go.sum +++ b/go.sum @@ -1,23 +1,39 @@ +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM= github.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU= -github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= -github.com/google/go-github/v57 v57.0.0 h1:L+Y3UPTY8ALM8x+TV0lg+IEBI+upibemtBD8Q9u7zHs= -github.com/google/go-github/v57 v57.0.0/go.mod h1:s0omdnye0hvK/ecLvpsGfJMiRt85PimQh4oygmLIxHw= -github.com/google/go-querystring v1.1.0 h1:AnCroh3fv4ZBgVIf1Iwtovgjaw/GiKJo8M8yD/fhyJ8= -github.com/google/go-querystring v1.1.0/go.mod h1:Kcdr2DB4koayq7X8pmAG4sNG59So17icRSOU623lUBU= +github.com/google/go-querystring v1.2.0 h1:yhqkPbu2/OH+V9BfpCVPZkNmUXhb2gBxJArfhIxNtP0= +github.com/google/go-querystring v1.2.0/go.mod h1:8IFJqpSRITyJ8QhQ13bmbeMBDfmeEJZD5A0egEOmkqU= +github.com/hashicorp/go-cleanhttp v0.5.2 h1:035FKYIWjmULyFRBKPs8TBQoi0x6d9G4xc9neXJWAZQ= +github.com/hashicorp/go-cleanhttp v0.5.2/go.mod h1:kO/YDlP8L1346E6Sodw+PrpBSV4/SoxCXGY6BqNFT48= +github.com/hashicorp/go-hclog v1.6.3 h1:Qr2kF+eVWjTiYmU7Y31tYlP1h0q/X3Nl3tPGdaB11/k= +github.com/hashicorp/go-hclog v1.6.3/go.mod h1:W4Qnvbt70Wk/zYJryRzDRU/4r0kIg0PVHBcfoyhpF5M= +github.com/hashicorp/go-retryablehttp v0.7.8 h1:ylXZWnqa7Lhqpk0L1P1LzDtGcCR0rPVUrx/c8Unxc48= +github.com/hashicorp/go-retryablehttp v0.7.8/go.mod h1:rjiScheydd+CxvumBsIrFKlx3iS0jrZ7LvzFGFmuKbw= github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA= github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg= github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM= github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= +github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +gitlab.com/gitlab-org/api/client-go v1.30.0 h1:VZV1Dbjr6KKWpZBs2nTgiWB11gw5dWnBweCAK0jUjNU= +gitlab.com/gitlab-org/api/client-go v1.30.0/go.mod h1:1LZ/6Q075HHVa1u9GBQjt8StFwFTRvfjo596slHmDbo= go.etcd.io/bbolt v1.4.3 h1:dEadXpI6G79deX5prL3QRNP6JB8UxVkqo4UPnHaNXJo= go.etcd.io/bbolt v1.4.3/go.mod h1:tKQlpPaYCVFctUIgFKFnAlvbmB3tpy1vkTnDWohtc0E= +golang.org/x/oauth2 v0.34.0 h1:hqK/t4AKgbqWkdkcAeI8XLmbK+4m4G5YeQRrmiotGlw= +golang.org/x/oauth2 v0.34.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA= +golang.org/x/sync v0.10.0 h1:3NQrjDixjgGwUOCaF8w2+VYHv0Ve/vGYSbdkTa98gmQ= +golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.25.0 h1:r+8e+loiHxRqhXVl6ML1nO3l1+oFoWbnlu2Ehimmi34= -golang.org/x/sys v0.25.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= -golang.org/x/sys v0.29.0 h1:TPYlXGxvx1MGTn2GiZDhnjPA9wZzZeGKHHmKhHYvgaU= -golang.org/x/sys v0.29.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= -golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/sys v0.39.0 h1:CvCKL8MeisomCi6qNZ+wbb0DN9E5AATixKsvNtMoMFk= +golang.org/x/sys v0.39.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= +golang.org/x/time v0.14.0 h1:MRx4UaLrDotUKUdCIqzPC48t1Y9hANFKIRpNx+Te8PI= +golang.org/x/time v0.14.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/main.go b/main.go index f6f8f78..cb2fed4 100644 --- a/main.go +++ b/main.go @@ -8,24 +8,26 @@ import ( "fmt" "hash/fnv" "math" + "net/http" + "net/url" "os" "path/filepath" + "regexp" "sort" "strconv" "strings" - "sync" "sync/atomic" "time" "github.com/fatih/color" - "github.com/google/go-github/v57/github" + gitlab "gitlab.com/gitlab-org/api/client-go" ) type PRActivity struct { Label string Owner string Repo string - PR *github.PullRequest + MR MergeRequestModel UpdatedAt time.Time HasUpdates bool Issues []IssueActivity @@ -35,32 +37,104 @@ type IssueActivity struct { Label string Owner string Repo string - Issue *github.Issue + Issue IssueModel UpdatedAt time.Time HasUpdates bool } +type MergeRequestModel struct { + Number int + Title string + Body string + State string + UpdatedAt time.Time + WebURL string + UserLogin string + Merged bool +} + +type IssueModel struct { + Number int + Title string + Body string + State string + UpdatedAt time.Time + WebURL string + UserLogin string +} + +type CommentModel struct { + Body string +} + type Progress struct { current atomic.Int32 total atomic.Int32 } type Config struct { - debugMode bool - localMode bool - showLinks bool - timeRange time.Duration - username string - allowedRepos map[string]bool - client *github.Client - db *Database - progress *Progress - ctx context.Context - dbErrorCount atomic.Int32 + debugMode bool + localMode bool + gitlabUserID int64 + showLinks bool + timeRange time.Duration + gitlabUsername string + allowedRepos map[string]bool + gitlabClient *gitlab.Client + db *Database + progress *Progress + ctx context.Context + dbErrorCount atomic.Int32 } var config Config +var retryAfter = time.After + +const defaultGitLabBaseURL = "https://gitlab.com" + +func normalizeGitLabBaseURL(raw string) (string, error) { + baseURL := strings.TrimSpace(raw) + if baseURL == "" { + baseURL = defaultGitLabBaseURL + } + + parsed, err := url.Parse(baseURL) + if err != nil { + return "", fmt.Errorf("invalid GitLab base URL %q: %w", baseURL, err) + } + + if parsed.Scheme == "" || parsed.Host == "" { + return "", fmt.Errorf("invalid GitLab base URL %q: must include scheme and host", baseURL) + } + + normalizedPath := strings.TrimSuffix(parsed.EscapedPath(), "/") + if normalizedPath == "" { + normalizedPath = "/api/v4" + } else if !strings.HasSuffix(normalizedPath, "/api/v4") { + normalizedPath += "/api/v4" + } + + parsed.Path = normalizedPath + parsed.RawPath = "" + + return parsed.String(), nil +} + +func newGitLabClient(token, rawBaseURL string) (*gitlab.Client, string, error) { + normalizedBaseURL, err := normalizeGitLabBaseURL(rawBaseURL) + if err != nil { + return nil, "", err + } + + client, err := gitlab.NewClient(token, gitlab.WithBaseURL(normalizedBaseURL)) + if err != nil { + return nil, "", fmt.Errorf("failed to create GitLab client: %w", err) + } + + return client, normalizedBaseURL, nil +} + func getPRLabelPriority(label string) int { priorities := map[string]int{ "Authored": 1, @@ -170,6 +244,10 @@ func retryWithBackoff(operation func() error, operationName string) error { backoff := initialBackoff attempt := 1 + retryCtx := config.ctx + if retryCtx == nil { + retryCtx = context.Background() + } for { err := operation() @@ -177,51 +255,42 @@ func retryWithBackoff(operation func() error, operationName string) error { return nil } - // Check if this is a GitHub rate limit error with reset time - var rateLimitErr *github.RateLimitError - var abuseRateLimitErr *github.AbuseRateLimitError + var gitLabErr *gitlab.ErrorResponse var waitTime time.Duration var isRateLimitError bool + var isTransientServerError bool + shouldRetry := true + + if errors.As(err, &gitLabErr) && gitLabErr.Response != nil { + statusCode := gitLabErr.Response.StatusCode + + if statusCode == http.StatusTooManyRequests { + isRateLimitError = true + retryAfterSeconds, parseErr := strconv.Atoi(strings.TrimSpace(gitLabErr.Response.Header.Get("Retry-After"))) + if parseErr == nil && retryAfterSeconds > 0 { + waitTime = time.Duration(retryAfterSeconds) * time.Second + } else if resetWait, ok := gitLabRateLimitResetWait(gitLabErr.Response.Header.Get("Ratelimit-Reset")); ok { + waitTime = resetWait + } else { + waitTime = time.Duration(math.Min(float64(backoff), float64(maxBackoff))) + } - if errors.As(err, &rateLimitErr) { - // Primary rate limit error - use the reset time from the error - isRateLimitError = true - resetTime := rateLimitErr.Rate.Reset.Time - waitTime = time.Until(resetTime) - - // Add a small buffer to ensure the rate limit has definitely reset - waitTime += 2 * time.Second - - // Cap at a reasonable maximum to avoid waiting forever if clock is wrong - if waitTime > 1*time.Hour { - waitTime = 1 * time.Hour - } - - // Ensure we wait at least 1 second - if waitTime < 1*time.Second { - waitTime = 1 * time.Second - } + if config.debugMode { + fmt.Printf(" [%s] GitLab rate limit hit (attempt %d), waiting %v before retry...\n", + operationName, attempt, waitTime.Round(time.Second)) + } + } else if statusCode >= http.StatusInternalServerError && statusCode <= 599 { + isTransientServerError = true + waitTime = time.Duration(math.Min(float64(backoff), float64(maxBackoff))) - if config.debugMode { - fmt.Printf(" [%s] Rate limit hit (attempt %d), reset at %v, waiting %v before retry...\n", - operationName, attempt, resetTime.Format("15:04:05"), waitTime.Round(time.Second)) - } - } else if errors.As(err, &abuseRateLimitErr) { - // Secondary/abuse rate limit error - use RetryAfter if available - isRateLimitError = true - if abuseRateLimitErr.RetryAfter != nil { - waitTime = *abuseRateLimitErr.RetryAfter + if config.debugMode { + fmt.Printf(" [%s] GitLab server error %d (attempt %d), waiting %v before retry...\n", + operationName, statusCode, attempt, waitTime) + } } else { - // If no RetryAfter specified, use a default wait time - waitTime = 60 * time.Second - } - - if config.debugMode { - fmt.Printf(" [%s] Abuse rate limit hit (attempt %d), waiting %v before retry...\n", - operationName, attempt, waitTime.Round(time.Second)) + shouldRetry = false } - } else { - // Check if error message suggests rate limiting (fallback for older errors) + } else { isRateLimitError = strings.Contains(err.Error(), "rate limit") || strings.Contains(err.Error(), "API rate limit exceeded") || strings.Contains(err.Error(), "403") @@ -236,12 +305,16 @@ func retryWithBackoff(operation func() error, operationName string) error { } } + if !shouldRetry { + return err + } + if isRateLimitError { if config.debugMode { select { - case <-config.ctx.Done(): - return config.ctx.Err() - case <-time.After(waitTime): + case <-retryCtx.Done(): + return retryCtx.Err() + case <-retryAfter(waitTime): } } else { ticker := time.NewTicker(1 * time.Second) @@ -254,8 +327,35 @@ func retryWithBackoff(operation func() error, operationName string) error { } select { - case <-config.ctx.Done(): - return config.ctx.Err() + case <-retryCtx.Done(): + return retryCtx.Err() + case <-ticker.C: + remaining-- + } + } + } + + backoff = time.Duration(float64(backoff) * backoffFactor) + } else if isTransientServerError { + if config.debugMode { + select { + case <-retryCtx.Done(): + return retryCtx.Err() + case <-retryAfter(waitTime): + } + } else { + ticker := time.NewTicker(1 * time.Second) + defer ticker.Stop() + + remaining := int(waitTime.Seconds()) + for remaining > 0 { + if config.progress != nil { + config.progress.displayWithWarning(fmt.Sprintf("API error, retrying in %ds", remaining)) + } + + select { + case <-retryCtx.Done(): + return retryCtx.Err() case <-ticker.C: remaining-- } @@ -271,9 +371,9 @@ func retryWithBackoff(operation func() error, operationName string) error { fmt.Printf(" [%s] Error (attempt %d): %v, waiting %v before retry...\n", operationName, attempt, err, waitTime) select { - case <-config.ctx.Done(): - return config.ctx.Err() - case <-time.After(waitTime): + case <-retryCtx.Done(): + return retryCtx.Err() + case <-retryAfter(waitTime): } } else { ticker := time.NewTicker(1 * time.Second) @@ -286,8 +386,8 @@ func retryWithBackoff(operation func() error, operationName string) error { } select { - case <-config.ctx.Done(): - return config.ctx.Err() + case <-retryCtx.Done(): + return retryCtx.Err() case <-ticker.C: remaining-- } @@ -301,6 +401,21 @@ func retryWithBackoff(operation func() error, operationName string) error { } } +func gitLabRateLimitResetWait(rawHeader string) (time.Duration, bool) { + resetAtUnix, err := strconv.ParseInt(strings.TrimSpace(rawHeader), 10, 64) + if err != nil || resetAtUnix <= 0 { + return 0, false + } + + resetTime := time.Unix(resetAtUnix, 0) + waitTime := time.Until(resetTime) + if waitTime <= 0 { + return 1 * time.Second, true + } + + return waitTime, true +} + func getLabelColor(label string) *color.Color { labelColors := map[string]*color.Color{ "Authored": color.New(color.FgCyan), @@ -372,6 +487,9 @@ func loadEnvFile(path string) error { if len(parts) == 2 { key := strings.TrimSpace(parts[0]) value := strings.TrimSpace(parts[1]) + if _, exists := os.LookupEnv(key); exists { + continue + } os.Setenv(key, value) } } @@ -423,24 +541,26 @@ func main() { flag.StringVar(&timeRangeStr, "time", "1m", "Show items from last time range (1h, 2d, 3w, 4m, 1y)") flag.BoolVar(&debugMode, "debug", false, "Show detailed API logging") - flag.BoolVar(&localMode, "local", false, "Use local database instead of GitHub API") + flag.BoolVar(&localMode, "local", false, "Use local database instead of GitLab API") flag.BoolVar(&showLinks, "links", false, "Show hyperlinks underneath each PR/issue") flag.BoolVar(&llMode, "ll", false, "Shortcut for --local --links (offline mode with links)") flag.BoolVar(&cleanCache, "clean", false, "Delete and recreate the database cache") - flag.StringVar(&allowedReposFlag, "allowed-repos", "", "Comma-separated list of allowed repos (e.g., user/repo1,user/repo2)") + flag.StringVar(&allowedReposFlag, "allowed-repos", "", "Comma-separated list of allowed repos (e.g., group/repo,group/subgroup/repo)") // Custom usage message flag.Usage = func() { fmt.Fprintf(os.Stderr, "Usage: %s [options]\n\n", os.Args[0]) - fmt.Fprintln(os.Stderr, "GitHub Feed - Monitor GitHub pull requests and issues across repositories") + fmt.Fprintln(os.Stderr, "GitLab Feed - Monitor pull requests and issues across repositories") fmt.Fprintln(os.Stderr, "\nOptions:") flag.PrintDefaults() fmt.Fprintln(os.Stderr, "\nEnvironment Variables:") - fmt.Fprintln(os.Stderr, " GITHUB_TOKEN or GITHUB_ACTIVITY_TOKEN - GitHub Personal Access Token") - fmt.Fprintln(os.Stderr, " GITHUB_USERNAME or GITHUB_USER - Your GitHub username") - fmt.Fprintln(os.Stderr, " ALLOWED_REPOS - Comma-separated list of allowed repos") + fmt.Fprintln(os.Stderr, " GITLAB_TOKEN or GITLAB_ACTIVITY_TOKEN - GitLab Personal Access Token") + fmt.Fprintln(os.Stderr, " GITLAB_USERNAME or GITLAB_USER - Optional GitLab username") + fmt.Fprintln(os.Stderr, " GITLAB_HOST - Optional GitLab host (overrides GITLAB_BASE_URL when set)") + fmt.Fprintln(os.Stderr, " GITLAB_BASE_URL - Optional GitLab base URL (default: https://gitlab.com)") + fmt.Fprintln(os.Stderr, " ALLOWED_REPOS - Required in online mode (group[/subgroup]/repo)") fmt.Fprintln(os.Stderr, "\nConfiguration File:") - fmt.Fprintln(os.Stderr, " ~/.github-feed/.env - Configuration file (auto-created)") + fmt.Fprintln(os.Stderr, " ~/.gitlab-feed/.env - Configuration file (auto-created)") } flag.Parse() @@ -465,7 +585,7 @@ func main() { os.Exit(1) } - configDir := filepath.Join(homeDir, ".github-feed") + configDir := filepath.Join(homeDir, ".gitlab-feed") if err := os.MkdirAll(configDir, 0o755); err != nil { fmt.Printf("Error: Could not create config directory %s: %v\n", configDir, err) os.Exit(1) @@ -473,19 +593,28 @@ func main() { envPath := filepath.Join(configDir, ".env") if _, err := os.Stat(envPath); os.IsNotExist(err) { - envTemplate := `# GitHub Feed Configuration -# Add your GitHub credentials here + envTemplate := `# Activity Feed Configuration +# Add your API credentials here -# Your GitHub Personal Access Token (required) -# Generate at: https://github.com/settings/tokens -# Required scopes: repo, read:org -GITHUB_TOKEN= +# Your GitLab Personal Access Token (required for online mode) +# Recommended scope: read_api (or api on some self-managed instances) +GITLAB_TOKEN= -# Your GitHub username (required) -GITHUB_USERNAME= +# Optional username (the app also resolves the current user via API) +GITLAB_USERNAME= -# Optional: Comma-separated list of allowed repos (e.g., user/repo1,user/repo2) -# Leave empty to allow all repos + # Optional: GitLab host for self-managed/cloud instances + # Example: http://10.10.1.207/ + # If set, this overrides GITLAB_BASE_URL. + GITLAB_HOST= + + # Optional: full GitLab base URL (supports path prefixes) + # Default: https://gitlab.com + GITLAB_BASE_URL=https://gitlab.com + +# Required in online mode: comma-separated allowed repos +# Format: group/repo or group/subgroup/repo +# Example self-managed repo path: platform/backend/gitlab-feed ALLOWED_REPOS= ` if err := os.WriteFile(envPath, []byte(envTemplate), 0o600); err != nil { @@ -495,11 +624,6 @@ ALLOWED_REPOS= _ = loadEnvFile(envPath) - username := os.Getenv("GITHUB_USERNAME") - if username == "" { - username = os.Getenv("GITHUB_USER") - } - allowedReposStr := allowedReposFlag if allowedReposStr == "" { allowedReposStr = os.Getenv("ALLOWED_REPOS") @@ -520,7 +644,7 @@ ALLOWED_REPOS= } } - dbPath := filepath.Join(configDir, "github.db") + dbPath := filepath.Join(configDir, "gitlab.db") if cleanCache { fmt.Println("Cleaning database cache...") @@ -544,20 +668,62 @@ ALLOWED_REPOS= defer db.Close() } - token := os.Getenv("GITHUB_ACTIVITY_TOKEN") + token := os.Getenv("GITLAB_ACTIVITY_TOKEN") if token == "" { - token = os.Getenv("GITHUB_TOKEN") + token = os.Getenv("GITLAB_TOKEN") + } + + rawGitLabHost := os.Getenv("GITLAB_HOST") + rawGitLabBaseURL := os.Getenv("GITLAB_BASE_URL") + selectedGitLabBaseURL := rawGitLabBaseURL + if strings.TrimSpace(rawGitLabHost) != "" { + selectedGitLabBaseURL = rawGitLabHost + } + + normalizedGitLabBaseURL, err := normalizeGitLabBaseURL(selectedGitLabBaseURL) + if err != nil { + if strings.TrimSpace(selectedGitLabBaseURL) != "" { + fmt.Printf("Configuration Error: %v\n", err) + os.Exit(1) + } + + normalizedGitLabBaseURL, _ = normalizeGitLabBaseURL("") + } + + var gitlabClient *gitlab.Client + gitlabUsername := "" + var gitlabUserID int64 + if !localMode && token != "" { + client, _, err := newGitLabClient(token, selectedGitLabBaseURL) + if err != nil { + fmt.Printf("Configuration Error: %v\n", err) + os.Exit(1) + } + gitlabClient = client + + currentUser, _, err := gitlabClient.Users.CurrentUser(gitlab.WithContext(context.Background())) + if err != nil { + fmt.Printf("Configuration Error: failed to fetch GitLab current user: %v\n", err) + os.Exit(1) + } + gitlabUsername = strings.TrimSpace(currentUser.Username) + gitlabUserID = currentUser.ID + if gitlabUsername == "" { + fmt.Println("Configuration Error: GitLab current user has empty username") + os.Exit(1) + } } // Validate configuration - if err := validateConfig(username, token, localMode, envPath); err != nil { + if err := validateConfig(token, localMode, envPath, allowedRepos); err != nil { fmt.Printf("Configuration Error: %v\n\n", err) os.Exit(1) } if debugMode { - fmt.Printf("Monitoring GitHub PR activity for user: %s\n", username) + fmt.Println("Monitoring GitLab merge request and issue activity") fmt.Printf("Showing items from the last %v\n", timeRange) + fmt.Printf("GitLab API base URL: %s\n", normalizedGitLabBaseURL) } if debugMode { fmt.Println("Debug mode enabled") @@ -565,297 +731,175 @@ ALLOWED_REPOS= config.debugMode = debugMode config.localMode = localMode + config.gitlabUserID = gitlabUserID config.showLinks = showLinks config.timeRange = timeRange - config.username = username + config.gitlabUsername = gitlabUsername config.allowedRepos = allowedRepos config.db = db config.ctx = context.Background() - config.client = github.NewClient(nil).WithAuthToken(token) + config.gitlabClient = gitlabClient fetchAndDisplayActivity() } -func validateConfig(username, token string, localMode bool, envPath string) error { +func validateConfig(token string, localMode bool, envPath string, allowedRepos map[string]bool) error { if localMode { return nil // No validation needed for offline mode } - if username == "" { - return fmt.Errorf("GitHub username is required.\n\nTo fix this:\n - Set GITHUB_USERNAME environment variable\n - Or add it to %s", envPath) - } - if token == "" { - return fmt.Errorf("GitHub token is required.\n\nTo fix this:\n 1. Generate a token at https://github.com/settings/tokens\n 2. Click 'Generate new token' -> 'Generate new token (classic)'\n 3. Give it a name and select scopes: 'repo', 'read:org'\n 4. Generate and copy the token\n 5. Set GITHUB_TOKEN environment variable\n 6. Or add it to %s", envPath) + return fmt.Errorf("token is required for GitLab API mode.\n\nTo fix this:\n - Set GITLAB_TOKEN or GITLAB_ACTIVITY_TOKEN\n - Or add it to %s", envPath) } - - // Validate token format (GitHub PAT tokens start with ghp_, gho_, or github_pat_) - if !strings.HasPrefix(token, "ghp_") && - !strings.HasPrefix(token, "gho_") && - !strings.HasPrefix(token, "github_pat_") { - return fmt.Errorf("GitHub token format looks invalid.\n\nGitHub Personal Access Tokens should start with:\n - 'ghp_' (classic PAT)\n - 'gho_' (OAuth token)\n - 'github_pat_' (fine-grained PAT)\n\nYour token starts with: '%s'\n\nPlease check your token at https://github.com/settings/tokens", token[:min(10, len(token))]) + if len(allowedRepos) == 0 { + return fmt.Errorf("ALLOWED_REPOS is required for GitLab API mode to keep API usage bounded.\n\nTo fix this:\n - Set ALLOWED_REPOS with group[/subgroup]/repo paths\n - Example: ALLOWED_REPOS=team/service,platform/backend/gitlab-feed\n - Or add it to %s", envPath) } - return nil } -func isRepoAllowed(owner, repo string) bool { - if config.allowedRepos == nil || len(config.allowedRepos) == 0 { - return true - } - repoKey := fmt.Sprintf("%s/%s", owner, repo) - return config.allowedRepos[repoKey] +func fetchAndDisplayActivity() { + fetchAndDisplayGitLabActivity() } -func checkRateLimit() error { - var rateLimits *github.RateLimits - var err error - - retryErr := retryWithBackoff(func() error { - rateLimits, _, err = config.client.RateLimit.Get(config.ctx) - return err - }, "RateLimitCheck") - - if retryErr != nil { - return fmt.Errorf("failed to fetch rate limit: %w", retryErr) - } - - core := rateLimits.Core - search := rateLimits.Search - - if config.debugMode { - fmt.Printf("Rate Limits - Core: %d/%d, Search: %d/%d\n", - core.Remaining, core.Limit, - search.Remaining, search.Limit) - } - - if core.Remaining == 0 { - resetTime := core.Reset.Time.Sub(time.Now()) - fmt.Printf("WARNING: Core API rate limit exceeded! Resets in %v\n", resetTime.Round(time.Second)) - return fmt.Errorf("rate limit exceeded, resets at %v", core.Reset.Time.Format("15:04:05")) - } - - if search.Remaining == 0 { - resetTime := search.Reset.Time.Sub(time.Now()) - fmt.Printf("WARNING: Search API rate limit exceeded! Resets in %v\n", resetTime.Round(time.Second)) - return fmt.Errorf("search rate limit exceeded, resets at %v", search.Reset.Time.Format("15:04:05")) - } - - coreThreshold := core.Limit / 5 - if core.Remaining < coreThreshold && core.Remaining > 0 { - fmt.Printf("WARNING: Core API rate limit running low (%d remaining)\n", core.Remaining) - } - - if search.Remaining < 5 && search.Remaining > 0 { - fmt.Printf("WARNING: Search API rate limit running low (%d remaining)\n", search.Remaining) - } - - return nil +type DisplayConfig struct { + Owner string + Repo string + Number int + Title string + User string + UpdatedAt time.Time + WebURL string + Label string + HasUpdates bool + IsIndented bool + State string } -func fetchAndDisplayActivity() { - startTime := time.Now() - - if !config.localMode { - if err := checkRateLimit(); err != nil { - fmt.Printf("Skipping this cycle due to rate limit: %v\n", err) - return - } - if config.debugMode { - fmt.Println() - } - } - - var seenPRs sync.Map // Maps prKey -> label - activitiesMap := sync.Map{} // Maps prKey -> *PRActivity - - // 6 PR queries + 4 issue queries = 10 total - initialTotal := 10 - if !config.localMode { - initialTotal += 3 // Add 3 for event pages - } - config.progress = &Progress{} - config.progress.current.Store(0) - config.progress.total.Store(int32(initialTotal)) - - if config.debugMode { - fmt.Println("Running optimized search queries...") - } else { - fmt.Print("Fetching data from GitHub... ") - config.progress.display() +func displayItem(cfg DisplayConfig) { + dateStr := " " + if !cfg.UpdatedAt.IsZero() { + dateStr = cfg.UpdatedAt.Format("2006/01/02") } - dateAgo := time.Now().Add(-config.timeRange).Format("2006-01-02") - dateFilter := fmt.Sprintf("updated:>=%s", dateAgo) - - buildQuery := func(base string) string { - return fmt.Sprintf("%s %s", base, dateFilter) + indent := "" + linkIndent := " " + if cfg.IsIndented && cfg.State != "" { + state := strings.ToUpper(cfg.State) + stateColor := getStateColor(cfg.State) + indent = fmt.Sprintf("-- %s ", stateColor.Sprint(state)) + linkIndent = " " } - var prWg sync.WaitGroup - - prQueries := []struct { - query string - label string - }{ - {buildQuery(fmt.Sprintf("is:pr reviewed-by:%s", config.username)), "Reviewed"}, - {buildQuery(fmt.Sprintf("is:pr review-requested:%s", config.username)), "Review Requested"}, - {buildQuery(fmt.Sprintf("is:pr author:%s", config.username)), "Authored"}, - {buildQuery(fmt.Sprintf("is:pr assignee:%s", config.username)), "Assigned"}, - {buildQuery(fmt.Sprintf("is:pr commenter:%s", config.username)), "Commented"}, - {buildQuery(fmt.Sprintf("is:pr mentions:%s", config.username)), "Mentioned"}, - } + labelColor := getLabelColor(cfg.Label) + userColor := getUserColor(cfg.User) - for _, pq := range prQueries { - query := pq.query - label := pq.label - prWg.Go(func() { - collectSearchResults(query, label, &seenPRs, &activitiesMap) - }) + updateIcon := "" + if cfg.HasUpdates { + updateIcon = color.New(color.FgYellow, color.Bold).Sprint("● ") } - prWg.Wait() - - if config.debugMode { - fmt.Println() - fmt.Println("Running issue search queries...") + repoDisplay := "" + if cfg.Repo == "" { + repoDisplay = fmt.Sprintf("%s#%d", cfg.Owner, cfg.Number) + } else { + repoDisplay = fmt.Sprintf("%s/%s#%d", cfg.Owner, cfg.Repo, cfg.Number) } - var seenIssues sync.Map // Maps issueKey -> label - issueActivitiesMap := sync.Map{} // Maps issueKey -> *IssueActivity - var issueWg sync.WaitGroup - - issueQueries := []struct { - query string - label string - }{ - {buildQuery(fmt.Sprintf("is:issue author:%s", config.username)), "Authored"}, - {buildQuery(fmt.Sprintf("is:issue mentions:%s", config.username)), "Mentioned"}, - {buildQuery(fmt.Sprintf("is:issue assignee:%s", config.username)), "Assigned"}, - {buildQuery(fmt.Sprintf("is:issue commenter:%s", config.username)), "Commented"}, - } + fmt.Printf("%s%s%s %s %s %s - %s\n", + updateIcon, + indent, + dateStr, + labelColor.Sprint(strings.ToUpper(cfg.Label)), + userColor.Sprint(cfg.User), + repoDisplay, + cfg.Title, + ) - for _, iq := range issueQueries { - query := iq.query - label := iq.label - issueWg.Go(func() { - collectIssueSearchResults(query, label, &seenIssues, &issueActivitiesMap) - }) + if config.showLinks && cfg.WebURL != "" { + fmt.Printf("%s🔗 %s\n", linkIndent, cfg.WebURL) } +} - issueWg.Wait() - - // Convert activitiesMap to slice - activities := []PRActivity{} - activitiesMap.Range(func(key, value interface{}) bool { - if activity, ok := value.(*PRActivity); ok { - activities = append(activities, *activity) - } - return true +func displayMergeRequest(label, owner, repo string, mr MergeRequestModel, hasUpdates bool) { + displayItem(DisplayConfig{ + Owner: owner, + Repo: repo, + Number: mr.Number, + Title: mr.Title, + User: mr.UserLogin, + UpdatedAt: mr.UpdatedAt, + WebURL: mr.WebURL, + Label: label, + HasUpdates: hasUpdates, + IsIndented: false, }) +} - // Convert issueActivitiesMap to slice - issueActivities := []IssueActivity{} - issueActivitiesMap.Range(func(key, value interface{}) bool { - if activity, ok := value.(*IssueActivity); ok { - issueActivities = append(issueActivities, *activity) - } - return true +func displayIssue(label, owner, repo string, issue IssueModel, indented bool, hasUpdates bool) { + displayItem(DisplayConfig{ + Owner: owner, + Repo: repo, + Number: issue.Number, + Title: issue.Title, + User: issue.UserLogin, + UpdatedAt: issue.UpdatedAt, + WebURL: issue.WebURL, + Label: label, + HasUpdates: hasUpdates, + IsIndented: indented, + State: issue.State, }) +} - if config.debugMode { - fmt.Println("Checking cross-references between PRs and issues...") - } +type gitLabProject struct { + PathWithNamespace string + ID int64 +} - linkedIssues := make(map[string]bool) +func fetchAndDisplayGitLabActivity() { + startTime := time.Now() - // Channel-based approach to avoid race conditions - type crossRefResult struct { - prIndex int - issue IssueActivity - issueKey string - debugInfo string + if config.debugMode { + fmt.Println("Fetching data from GitLab...") + } else { + fmt.Print("Fetching data from GitLab... ") } - resultsChan := make(chan crossRefResult, 100) - var wg sync.WaitGroup + cutoffTime := time.Now().Add(-config.timeRange) + var ( + activities []PRActivity + issueActivities []IssueActivity + err error + ) - // Launch collector goroutine to handle all appends and map writes - collectorDone := make(chan struct{}) - go func() { - for result := range resultsChan { - // Safe to modify since only this goroutine accesses these - activities[result.prIndex].Issues = append(activities[result.prIndex].Issues, result.issue) - linkedIssues[result.issueKey] = true - if config.debugMode { - fmt.Println(result.debugInfo) - } - } - close(collectorDone) - }() - - for j := range issueActivities { - issue := &issueActivities[j] - issueKey := buildItemKey(issue.Owner, issue.Repo, issue.Issue.GetNumber()) - - for i := range activities { - pr := &activities[i] - if pr.Owner == issue.Owner && pr.Repo == issue.Repo { - // Capture loop variables - prIndex := i - issueCopy := *issue - issueKeyCopy := issueKey - prCopy := pr - wg.Go(func() { - if areCrossReferenced(prCopy, &issueCopy) { - debugInfo := "" - if config.debugMode { - debugInfo = fmt.Sprintf(" Linked %s/%s#%d <-> %s/%s#%d", - prCopy.Owner, prCopy.Repo, prCopy.PR.GetNumber(), - issueCopy.Owner, issueCopy.Repo, issueCopy.Issue.GetNumber()) - } - resultsChan <- crossRefResult{ - prIndex: prIndex, - issue: issueCopy, - issueKey: issueKeyCopy, - debugInfo: debugInfo, - } - } - }) - } - } + if config.localMode { + activities, issueActivities, err = loadGitLabCachedActivities(cutoffTime) + } else { + activities, issueActivities, err = fetchGitLabProjectActivities( + config.ctx, + config.gitlabClient, + config.allowedRepos, + cutoffTime, + config.gitlabUsername, + config.gitlabUserID, + config.db, + ) } - - wg.Wait() - close(resultsChan) - <-collectorDone - - standaloneIssues := []IssueActivity{} - for _, issue := range issueActivities { - issueKey := buildItemKey(issue.Owner, issue.Repo, issue.Issue.GetNumber()) - if !linkedIssues[issueKey] { - standaloneIssues = append(standaloneIssues, issue) - } + if err != nil { + fmt.Printf("Error fetching GitLab activity: %v\n", err) + return } - duration := time.Since(startTime) if config.debugMode { fmt.Println() - fmt.Printf("Total fetch time: %v\n", duration.Round(time.Millisecond)) - fmt.Printf("Found %d unique PRs and %d unique issues\n", len(activities), len(issueActivities)) - - if config.db != nil { - prCount, issueCount, commentCount, err := config.db.Stats() - if err == nil { - fmt.Printf("Database stats: %d PRs, %d issues, %d comments\n", prCount, issueCount, commentCount) - } - } + fmt.Printf("Total fetch time: %v\n", time.Since(startTime).Round(time.Millisecond)) + fmt.Printf("Found %d unique merge requests and %d unique issues\n", len(activities), len(issueActivities)) fmt.Println() } else { fmt.Print("\r" + strings.Repeat(" ", 80) + "\r") } - if len(activities) == 0 && len(standaloneIssues) == 0 { + if len(activities) == 0 && len(issueActivities) == 0 { fmt.Println("No open activity found") return } @@ -863,14 +907,14 @@ func fetchAndDisplayActivity() { sort.Slice(activities, func(i, j int) bool { return activities[i].UpdatedAt.After(activities[j].UpdatedAt) }) - sort.Slice(standaloneIssues, func(i, j int) bool { - return standaloneIssues[i].UpdatedAt.After(standaloneIssues[j].UpdatedAt) + sort.Slice(issueActivities, func(i, j int) bool { + return issueActivities[i].UpdatedAt.After(issueActivities[j].UpdatedAt) }) var openPRs, closedPRs, mergedPRs []PRActivity for _, activity := range activities { - if activity.PR.State != nil && *activity.PR.State == "closed" { - if activity.PR.Merged != nil && *activity.PR.Merged { + if activity.MR.State == "closed" { + if activity.MR.Merged { mergedPRs = append(mergedPRs, activity) } else { closedPRs = append(closedPRs, activity) @@ -881,8 +925,8 @@ func fetchAndDisplayActivity() { } var openIssues, closedIssues []IssueActivity - for _, issue := range standaloneIssues { - if issue.Issue.State != nil && *issue.Issue.State == "closed" { + for _, issue := range issueActivities { + if issue.Issue.State == "closed" { closedIssues = append(closedIssues, issue) } else { openIssues = append(openIssues, issue) @@ -894,7 +938,7 @@ func fetchAndDisplayActivity() { fmt.Println(titleColor.Sprint("OPEN PULL REQUESTS:")) fmt.Println("------------------------------------------") for _, activity := range openPRs { - displayPR(activity.Label, activity.Owner, activity.Repo, activity.PR, activity.HasUpdates) + displayMergeRequest(activity.Label, activity.Owner, activity.Repo, activity.MR, activity.HasUpdates) if len(activity.Issues) > 0 { for _, issue := range activity.Issues { displayIssue(issue.Label, issue.Owner, issue.Repo, issue.Issue, true, issue.HasUpdates) @@ -903,13 +947,21 @@ func fetchAndDisplayActivity() { } } - if len(closedPRs) > 0 { + if len(closedPRs) > 0 || len(mergedPRs) > 0 { fmt.Println() titleColor := color.New(color.FgHiRed, color.Bold) fmt.Println(titleColor.Sprint("CLOSED/MERGED PULL REQUESTS:")) fmt.Println("------------------------------------------") + for _, activity := range mergedPRs { + displayMergeRequest(activity.Label, activity.Owner, activity.Repo, activity.MR, activity.HasUpdates) + if len(activity.Issues) > 0 { + for _, issue := range activity.Issues { + displayIssue(issue.Label, issue.Owner, issue.Repo, issue.Issue, true, issue.HasUpdates) + } + } + } for _, activity := range closedPRs { - displayPR(activity.Label, activity.Owner, activity.Repo, activity.PR, activity.HasUpdates) + displayMergeRequest(activity.Label, activity.Owner, activity.Repo, activity.MR, activity.HasUpdates) if len(activity.Issues) > 0 { for _, issue := range activity.Issues { displayIssue(issue.Label, issue.Owner, issue.Repo, issue.Issue, true, issue.HasUpdates) @@ -937,720 +989,1006 @@ func fetchAndDisplayActivity() { displayIssue(issue.Label, issue.Owner, issue.Repo, issue.Issue, false, issue.HasUpdates) } } - - // Warn about database errors if any occurred - if dbErrors := config.dbErrorCount.Load(); dbErrors > 0 { - fmt.Printf("\n") - warningColor := color.New(color.FgYellow, color.Bold) - fmt.Printf("%s %d database write error(s) occurred. Offline mode may be incomplete.\n", - warningColor.Sprint("Warning:"), dbErrors) - if !config.debugMode { - fmt.Println("Run with --debug to see detailed error messages.") - } - } } -func areCrossReferenced(pr *PRActivity, issue *IssueActivity) bool { - prNumber := pr.PR.GetNumber() - issueNumber := issue.Issue.GetNumber() - - if config.debugMode { - fmt.Printf(" Checking cross-reference: PR %s/%s#%d <-> Issue %s/%s#%d\n", - pr.Owner, pr.Repo, prNumber, - issue.Owner, issue.Repo, issueNumber) +func fetchGitLabProjectActivities( + ctx context.Context, + client *gitlab.Client, + allowedRepos map[string]bool, + cutoff time.Time, + currentUsername string, + currentUserID int64, + db *Database, +) ([]PRActivity, []IssueActivity, error) { + projects, err := resolveAllowedGitLabProjects(ctx, client, allowedRepos) + if err != nil { + return nil, nil, err } - prBody := pr.PR.GetBody() - if mentionsNumber(prBody, issueNumber, pr.Owner, pr.Repo) { - return true + currentUsername = strings.TrimSpace(currentUsername) + if currentUsername == "" { + return nil, nil, fmt.Errorf("gitlab current username is required") } - issueBody := issue.Issue.GetBody() - if mentionsNumber(issueBody, prNumber, issue.Owner, issue.Repo) { - return true + if len(projects) == 0 { + return []PRActivity{}, []IssueActivity{}, nil } - var prComments []*github.PullRequestComment - var err error + activities := make([]PRActivity, 0) + issueActivities := make([]IssueActivity, 0) + seenMergeRequests := make(map[string]struct{}) + seenIssues := make(map[string]struct{}) + projectIDByPath := make(map[string]int64, len(projects)) + mrNotesByKey := make(map[string][]*gitlab.Note) - if config.localMode { - if config.db != nil { - prComments, err = config.db.GetPRComments(pr.Owner, pr.Repo, prNumber) - if err != nil && config.debugMode { - fmt.Printf(" Warning: Could not fetch comments from database for %s/%s#%d: %v\n", - pr.Owner, pr.Repo, prNumber, err) - } - } - } else { - config.progress.addToTotal(1) - if !config.debugMode { - config.progress.display() + for _, project := range projects { + projectIDByPath[normalizeProjectPathWithNamespace(project.PathWithNamespace)] = project.ID + } + + for _, project := range projects { + projectMergeRequests, err := listGitLabProjectMergeRequests(ctx, client, project.ID, cutoff) + if err != nil { + return nil, nil, fmt.Errorf("list merge requests for %s: %w", project.PathWithNamespace, err) } - retryErr := retryWithBackoff(func() error { - prComments, _, err = config.client.PullRequests.ListComments(config.ctx, pr.Owner, pr.Repo, prNumber, &github.PullRequestListCommentsOptions{ - ListOptions: github.ListOptions{PerPage: 100}, - }) - return err - }, fmt.Sprintf("Comments-PR#%d", prNumber)) + for _, item := range projectMergeRequests { + key := buildGitLabDedupKey(project.PathWithNamespace, "mr", item.IID) + if _, exists := seenMergeRequests[key]; exists { + continue + } + seenMergeRequests[key] = struct{}{} - config.progress.increment() - if !config.debugMode { - config.progress.display() - } + model := toMergeRequestModelFromGitLab(item) + if model.UpdatedAt.IsZero() || model.UpdatedAt.Before(cutoff) { + continue + } - if retryErr != nil { - err = retryErr - } + label, notes, err := deriveGitLabMergeRequestLabel(ctx, client, project.ID, item, currentUsername, currentUserID) + if err != nil { + return nil, nil, fmt.Errorf("derive merge request label for %s!%d: %w", project.PathWithNamespace, item.IID, err) + } - if err == nil && config.db != nil { - for _, comment := range prComments { - if err := config.db.SavePRComment(pr.Owner, pr.Repo, prNumber, comment, config.debugMode); err != nil { + if db != nil { + if err := db.SaveGitLabMergeRequestWithLabel(project.PathWithNamespace, model, label, config.debugMode); err != nil { config.dbErrorCount.Add(1) if config.debugMode { - fmt.Printf(" [DB] Warning: Failed to save PR comment for %s/%s#%d: %v\n", pr.Owner, pr.Repo, prNumber, err) + fmt.Printf(" [DB] Warning: Failed to save GitLab MR %s!%d: %v\n", project.PathWithNamespace, item.IID, err) } } - } + if err := persistGitLabNotes(db, project.PathWithNamespace, "mr", int(item.IID), notes); err != nil { + config.dbErrorCount.Add(1) + if config.debugMode { + fmt.Printf(" [DB] Warning: Failed to save GitLab MR notes %s!%d: %v\n", project.PathWithNamespace, item.IID, err) + } + } + } + + mrNotesByKey[buildGitLabMergeRequestKey(project.PathWithNamespace, model.Number)] = notes + + activities = append(activities, PRActivity{ + Label: label, + Owner: project.PathWithNamespace, + Repo: "", + MR: model, + UpdatedAt: model.UpdatedAt, + }) } - } - if err == nil { - for _, comment := range prComments { - if mentionsNumber(comment.GetBody(), issueNumber, pr.Owner, pr.Repo) { - return true + projectIssues, err := listGitLabProjectIssues(ctx, client, project.ID, cutoff) + if err != nil { + return nil, nil, fmt.Errorf("list issues for %s: %w", project.PathWithNamespace, err) + } + + for _, item := range projectIssues { + key := buildGitLabDedupKey(project.PathWithNamespace, "issue", item.IID) + if _, exists := seenIssues[key]; exists { + continue + } + seenIssues[key] = struct{}{} + + model := toIssueModelFromGitLab(item) + if model.UpdatedAt.IsZero() || model.UpdatedAt.Before(cutoff) { + continue + } + + label, notes, err := deriveGitLabIssueLabel(ctx, client, project.ID, item, currentUsername, currentUserID) + if err != nil { + return nil, nil, fmt.Errorf("derive issue label for %s#%d: %w", project.PathWithNamespace, item.IID, err) + } + + if db != nil { + if err := db.SaveGitLabIssueWithLabel(project.PathWithNamespace, model, label, config.debugMode); err != nil { + config.dbErrorCount.Add(1) + if config.debugMode { + fmt.Printf(" [DB] Warning: Failed to save GitLab issue %s#%d: %v\n", project.PathWithNamespace, item.IID, err) + } + } + if err := persistGitLabNotes(db, project.PathWithNamespace, "issue", int(item.IID), notes); err != nil { + config.dbErrorCount.Add(1) + if config.debugMode { + fmt.Printf(" [DB] Warning: Failed to save GitLab issue notes %s#%d: %v\n", project.PathWithNamespace, item.IID, err) + } + } } + + issueActivities = append(issueActivities, IssueActivity{ + Label: label, + Owner: project.PathWithNamespace, + Repo: "", + Issue: model, + UpdatedAt: model.UpdatedAt, + }) } } - return false + activities, issueActivities, err = linkGitLabCrossReferencesOnline(ctx, client, activities, issueActivities, projectIDByPath, mrNotesByKey, db) + if err != nil { + return nil, nil, err + } + + return activities, issueActivities, nil } -func mentionsNumber(text string, number int, owner string, repo string) bool { - if text == "" { - return false +func deriveGitLabMergeRequestLabel( + ctx context.Context, + client *gitlab.Client, + projectID int64, + item *gitlab.BasicMergeRequest, + currentUsername string, + currentUserID int64, +) (string, []*gitlab.Note, error) { + if item == nil { + return "Involved", nil, nil } - lowerText := strings.ToLower(text) - - urlPatterns := []string{ - fmt.Sprintf("github.com/%s/%s/issues/%d", strings.ToLower(owner), strings.ToLower(repo), number), - fmt.Sprintf("github.com/%s/%s/pull/%d", strings.ToLower(owner), strings.ToLower(repo), number), + currentLabel := "" + if matchesGitLabBasicUser(item.Author, currentUsername, currentUserID) { + currentLabel = mergeLabelWithPriority(currentLabel, "Authored", true) } - for _, pattern := range urlPatterns { - if strings.Contains(lowerText, pattern) { - return true - } + if gitLabBasicUserListContains(item.Assignees, currentUsername, currentUserID) || matchesGitLabBasicUser(item.Assignee, currentUsername, currentUserID) { + currentLabel = mergeLabelWithPriority(currentLabel, "Assigned", true) } - patterns := []string{ - fmt.Sprintf("#%d", number), - fmt.Sprintf("fixes #%d", number), - fmt.Sprintf("closes #%d", number), - fmt.Sprintf("resolves #%d", number), - fmt.Sprintf("fixed #%d", number), - fmt.Sprintf("closed #%d", number), - fmt.Sprintf("resolved #%d", number), - fmt.Sprintf("fix #%d", number), - fmt.Sprintf("close #%d", number), - fmt.Sprintf("resolve #%d", number), + if currentLabel == "Authored" || currentLabel == "Assigned" { + return currentLabel, nil, nil } - for _, pattern := range patterns { - if strings.Contains(lowerText, pattern) { - return true - } + var approvalState *gitlab.MergeRequestApprovalState + err := retryWithBackoff(func() error { + var apiErr error + approvalState, _, apiErr = client.MergeRequestApprovals.GetApprovalState(projectID, item.IID, gitlab.WithContext(ctx)) + return apiErr + }, fmt.Sprintf("GitLabGetApprovalState %d!%d", projectID, item.IID)) + if err != nil { + return "", nil, err + } + if gitLabApprovalStateReviewedByCurrentUser(approvalState, currentUsername, currentUserID) { + currentLabel = mergeLabelWithPriority(currentLabel, "Reviewed", true) } - return false -} + if gitLabBasicUserListContains(item.Reviewers, currentUsername, currentUserID) { + currentLabel = mergeLabelWithPriority(currentLabel, "Review Requested", true) + } -func collectSearchResults(query, label string, seenPRs *sync.Map, activitiesMap *sync.Map) { - if config.localMode { - if config.db == nil { - return + if !needsLowerPriorityPRChecks(currentLabel) { + if currentLabel == "" { + return "Involved", nil, nil } + return currentLabel, nil, nil + } - allPRs, prLabels, err := config.db.GetAllPullRequestsWithLabels(config.debugMode) - if err != nil { - if config.debugMode { - fmt.Printf(" [%s] Error loading from database: %v\n", label, err) - } - return - } + notes, err := listAllGitLabMergeRequestNotes(ctx, client, projectID, item.IID) + if err != nil { + return "", nil, err + } - if config.debugMode { - fmt.Printf(" [%s] Loading from database...\n", label) - } + commented, mentioned := gitLabNotesInvolvement(notes, item.Description, currentUsername, currentUserID) + if commented { + currentLabel = mergeLabelWithPriority(currentLabel, "Commented", true) + } + if mentioned { + currentLabel = mergeLabelWithPriority(currentLabel, "Mentioned", true) + } - totalFound := 0 - cutoffTime := time.Now().Add(-config.timeRange) - for key, pr := range allPRs { - storedLabel := prLabels[key] + if currentLabel == "" { + return "Involved", notes, nil + } + return currentLabel, notes, nil +} - if storedLabel != label { - continue - } +func deriveGitLabIssueLabel( + ctx context.Context, + client *gitlab.Client, + projectID int64, + item *gitlab.Issue, + currentUsername string, + currentUserID int64, +) (string, []*gitlab.Note, error) { + if item == nil { + return "Involved", nil, nil + } - if pr.GetUpdatedAt().Time.Before(cutoffTime) { - continue - } + currentLabel := "" + if matchesGitLabIssueAuthor(item.Author, currentUsername, currentUserID) { + currentLabel = mergeLabelWithPriority(currentLabel, "Authored", false) + } + if gitLabIssueAssigneeListContains(item.Assignees, currentUsername, currentUserID) || matchesGitLabIssueAssignee(item.Assignee, currentUsername, currentUserID) { + currentLabel = mergeLabelWithPriority(currentLabel, "Assigned", false) + } - parts := strings.Split(key, "/") - if len(parts) < 2 { - continue - } - owner := parts[0] - repoAndNum := parts[1] - repoParts := strings.Split(repoAndNum, "#") - if len(repoParts) < 2 { - continue - } - repo := repoParts[0] + if currentLabel == "Authored" || currentLabel == "Assigned" { + return currentLabel, nil, nil + } - if !isRepoAllowed(owner, repo) { - continue - } + notes, err := listAllGitLabIssueNotes(ctx, client, projectID, item.IID) + if err != nil { + return "", nil, err + } - prKey := key + commented, mentioned := gitLabNotesInvolvement(notes, item.Description, currentUsername, currentUserID) + if commented { + currentLabel = mergeLabelWithPriority(currentLabel, "Commented", false) + } + if mentioned { + currentLabel = mergeLabelWithPriority(currentLabel, "Mentioned", false) + } - // Check if we've already processed this PR in activitiesMap - existingActivity, alreadyProcessed := activitiesMap.Load(prKey) - shouldProcess := true + if currentLabel == "" { + return "Involved", notes, nil + } + return currentLabel, notes, nil +} - if alreadyProcessed { - existingPR := existingActivity.(*PRActivity) - if shouldUpdateLabel(existingPR.Label, label, true) { - // New label has higher priority, we'll update it - if config.debugMode { - fmt.Printf(" [%s] Updating label for %s from %s to %s (higher priority)\n", label, prKey, existingPR.Label, label) - } - } else { - // Existing label has higher or equal priority, skip - shouldProcess = false - } - } +func persistGitLabNotes(db *Database, projectPath, itemType string, itemIID int, notes []*gitlab.Note) error { + if db == nil || len(notes) == 0 { + return nil + } - if shouldProcess { - activity := PRActivity{ - Label: label, - Owner: owner, - Repo: repo, - PR: pr, - UpdatedAt: pr.GetUpdatedAt().Time, - } - activitiesMap.Store(prKey, &activity) - totalFound++ - } + for _, note := range notes { + if note == nil { + continue } - if config.debugMode && totalFound > 0 { - fmt.Printf(" [%s] Complete: %d PRs found\n", label, totalFound) + authorUsername := "" + authorID := int64(0) + author := note.Author + authorUsername = strings.TrimSpace(author.Username) + authorID = author.ID + + record := GitLabNoteRecord{ + ProjectPath: projectPath, + ItemType: itemType, + ItemIID: itemIID, + NoteID: int64(note.ID), + Body: note.Body, + AuthorUsername: authorUsername, + AuthorID: authorID, } - return + if err := db.SaveGitLabNote(record, config.debugMode); err != nil { + return err + } } - opts := &github.SearchOptions{ - ListOptions: github.ListOptions{PerPage: 100}, + return nil +} + +func loadGitLabCachedActivities(cutoff time.Time) ([]PRActivity, []IssueActivity, error) { + if config.db == nil { + return []PRActivity{}, []IssueActivity{}, nil } - totalFound := 0 + allMRs, mrLabels, err := config.db.GetAllGitLabMergeRequestsWithLabels(config.debugMode) + if err != nil { + return nil, nil, err + } - page := 1 - for { - if config.debugMode { - fmt.Printf(" [%s] Searching page %d with query: %s\n", label, page, query) + activities := make([]PRActivity, 0, len(allMRs)) + for key, mr := range allMRs { + if mr.UpdatedAt.IsZero() || mr.UpdatedAt.Before(cutoff) { + continue } - var result *github.IssuesSearchResult - var resp *github.Response - var err error + projectPath, ok := parseGitLabMRProjectPath(key) + if !ok || !isGitLabProjectAllowed(projectPath) { + continue + } - retryErr := retryWithBackoff(func() error { - result, resp, err = config.client.Search.Issues(config.ctx, query, opts) - return err - }, fmt.Sprintf("%s-page%d", label, page)) + activities = append(activities, PRActivity{ + Label: mrLabels[key], + Owner: projectPath, + Repo: "", + MR: mr, + UpdatedAt: mr.UpdatedAt, + }) + } - config.progress.increment() - if !config.debugMode { - config.progress.display() - } + allIssues, issueLabels, err := config.db.GetAllGitLabIssuesWithLabels(config.debugMode) + if err != nil { + return nil, nil, err + } - if page == 1 && resp != nil && resp.NextPage != 0 { - lastPage := resp.LastPage - if lastPage > 1 { - additionalPages := lastPage - 1 - config.progress.addToTotal(additionalPages) - if !config.debugMode { - config.progress.display() - } - } + issueActivities := make([]IssueActivity, 0, len(allIssues)) + for key, issue := range allIssues { + if issue.UpdatedAt.IsZero() || issue.UpdatedAt.Before(cutoff) { + continue } - if retryErr != nil { - fmt.Printf(" [%s] Error searching after retries: %v\n", label, retryErr) - if resp != nil { - fmt.Printf(" [%s] Rate limit remaining: %d/%d\n", label, resp.Rate.Remaining, resp.Rate.Limit) - } - return + projectPath, ok := parseGitLabIssueProjectPath(key) + if !ok || !isGitLabProjectAllowed(projectPath) { + continue } - if config.debugMode && resp != nil { - fmt.Printf(" [%s] API Response: %d results, Rate: %d/%d\n", label, len(result.Issues), resp.Rate.Remaining, resp.Rate.Limit) - } + issueActivities = append(issueActivities, IssueActivity{ + Label: issueLabels[key], + Owner: projectPath, + Repo: "", + Issue: issue, + UpdatedAt: issue.UpdatedAt, + }) + } - pageResults := 0 - for _, issue := range result.Issues { - if issue.PullRequestLinks == nil { - continue - } + activities, issueActivities, err = linkGitLabCrossReferencesOffline(config.db, activities, issueActivities) + if err != nil { + return nil, nil, err + } - repoURL := *issue.RepositoryURL - parts := strings.Split(repoURL, "/") - if len(parts) < 2 { - fmt.Printf(" [%s] Error: Invalid repository URL format: %s\n", label, repoURL) - continue - } - owner := parts[len(parts)-2] - repo := parts[len(parts)-1] + return activities, issueActivities, nil +} - if !isRepoAllowed(owner, repo) { - continue - } +var ( + gitLabIssueSameProjectRefPattern = regexp.MustCompile(`(?i)(?:^|[^a-z0-9_])#([0-9]+)\b`) + gitLabIssueQualifiedRefPattern = regexp.MustCompile(`(?i)([a-z0-9_.-]+(?:/[a-z0-9_.-]+)+)#([0-9]+)\b`) + gitLabIssueURLRefPattern = regexp.MustCompile(`(?i)https?://[^\s]+/([a-z0-9_.-]+(?:/[a-z0-9_.-]+)+)/-/issues/([0-9]+)\b`) + gitLabIssueRelativeURLRefPattern = regexp.MustCompile(`(?i)/-/issues/([0-9]+)\b`) +) - prKey := buildItemKey(owner, repo, *issue.Number) +func linkGitLabCrossReferencesOnline( + ctx context.Context, + client *gitlab.Client, + activities []PRActivity, + issueActivities []IssueActivity, + projectIDByPath map[string]int64, + mrNotesByKey map[string][]*gitlab.Note, + db *Database, +) ([]PRActivity, []IssueActivity, error) { + mrToIssueKeys := make(map[string]map[string]struct{}, len(activities)) - // Check if we've already processed this PR in activitiesMap - existingActivity, alreadyProcessed := activitiesMap.Load(prKey) - shouldProcess := true + for _, activity := range activities { + projectPath := normalizeProjectPathWithNamespace(activity.Owner) + projectID, ok := projectIDByPath[projectPath] + if !ok { + continue + } - if alreadyProcessed { - // PR is already in activitiesMap, check if we need to update the label - existingPR := existingActivity.(*PRActivity) - if shouldUpdateLabel(existingPR.Label, label, true) { - // New label has higher priority, we'll update it - if config.debugMode { - fmt.Printf(" [%s] Updating label for %s from %s to %s (higher priority)\n", label, prKey, existingPR.Label, label) - } - } else { - // Existing label has higher or equal priority, skip fetching again - shouldProcess = false + mrKey := buildGitLabMergeRequestKey(projectPath, activity.MR.Number) + closedIssues, err := listGitLabIssuesClosedOnMergeRequest(ctx, client, projectID, int64(activity.MR.Number)) + if err == nil { + resolvedKeys := make(map[string]struct{}) + for _, item := range closedIssues { + issueKey, ok := gitLabIssueKeyFromIssue(item, projectPath) + if !ok { + continue } + resolvedKeys[issueKey] = struct{}{} } + if len(resolvedKeys) > 0 { + mrToIssueKeys[mrKey] = resolvedKeys + } + continue + } - if shouldProcess { - // Store in seenPRs to prevent other goroutines from fetching the same PR - seenPRs.Store(prKey, label) - config.progress.addToTotal(1) - if !config.debugMode { - config.progress.display() - } - - var pr *github.PullRequest - // var prErr error - - config.progress.increment() - if !config.debugMode { - config.progress.display() - } - - pr = &github.PullRequest{ - Number: issue.Number, - Title: issue.Title, - Body: issue.Body, - State: issue.State, - UpdatedAt: issue.UpdatedAt, - User: issue.User, - HTMLURL: issue.HTMLURL, - } - // } - - hasUpdates := false - - if config.db != nil { - cachedPR, err := config.db.GetPullRequest(owner, repo, *issue.Number) - if err == nil { - if pr.GetUpdatedAt().After(cachedPR.GetUpdatedAt().Time) { - hasUpdates = true + fallbackKeys := gitLabIssueReferenceKeysFromText(activity.MR.Body, projectPath) + if len(fallbackKeys) == 0 { + notes := mrNotesByKey[mrKey] + if len(notes) == 0 { + notes, err = listAllGitLabMergeRequestNotes(ctx, client, projectID, int64(activity.MR.Number)) + if err == nil { + mrNotesByKey[mrKey] = notes + if db != nil { + if persistErr := persistGitLabNotes(db, projectPath, "mr", activity.MR.Number, notes); persistErr != nil { + config.dbErrorCount.Add(1) if config.debugMode { - fmt.Printf(" [%s] Update detected: %s/%s#%d (API: %s > DB: %s)\n", - label, owner, repo, *issue.Number, - pr.GetUpdatedAt().Format("2006-01-02 15:04:05"), - cachedPR.GetUpdatedAt().Time.Format("2006-01-02 15:04:05")) + fmt.Printf(" [DB] Warning: Failed to save GitLab MR notes %s!%d: %v\n", projectPath, activity.MR.Number, persistErr) } - } else if config.debugMode { - fmt.Printf(" [%s] No update: %s/%s#%d (API: %s == DB: %s)\n", - label, owner, repo, *issue.Number, - pr.GetUpdatedAt().Format("2006-01-02 15:04:05"), - cachedPR.GetUpdatedAt().Time.Format("2006-01-02 15:04:05")) - } - } else { - // If there's no cached version, this is a new PR, so it has "updates" - hasUpdates = true - if config.debugMode { - fmt.Printf(" [%s] New PR (not in DB): %s/%s#%d\n", - label, owner, repo, *issue.Number) } } } + } - // Determine the final label to use - finalLabel := label - if alreadyProcessed { - existingPR := existingActivity.(*PRActivity) - if !shouldUpdateLabel(existingPR.Label, label, true) { - // Keep the existing higher-priority label - finalLabel = existingPR.Label - if config.debugMode { - fmt.Printf(" [%s] Keeping existing label %s for %s (higher priority)\n", label, finalLabel, prKey) - } - } + for _, note := range notes { + if note == nil { + continue } - - if config.db != nil { - if err := config.db.SavePullRequestWithLabel(owner, repo, pr, finalLabel, config.debugMode); err != nil { - config.dbErrorCount.Add(1) - if config.debugMode { - fmt.Printf(" [DB] Warning: Failed to save PR %s/%s#%d: %v\n", owner, repo, pr.GetNumber(), err) - } - } + for issueKey := range gitLabIssueReferenceKeysFromText(note.Body, projectPath) { + fallbackKeys[issueKey] = struct{}{} } + } + } - activity := PRActivity{ - Label: finalLabel, - Owner: owner, - Repo: repo, - PR: pr, - UpdatedAt: pr.GetUpdatedAt().Time, - HasUpdates: hasUpdates, + if len(fallbackKeys) > 0 { + mrToIssueKeys[mrKey] = fallbackKeys + } + } + + nestedActivities := nestGitLabIssues(activities, issueActivities, mrToIssueKeys) + return nestedActivities, filterStandaloneGitLabIssues(nestedActivities, issueActivities), nil +} + +func linkGitLabCrossReferencesOffline(db *Database, activities []PRActivity, issueActivities []IssueActivity) ([]PRActivity, []IssueActivity, error) { + mrToIssueKeys := make(map[string]map[string]struct{}, len(activities)) + + for _, activity := range activities { + projectPath := normalizeProjectPathWithNamespace(activity.Owner) + mrKey := buildGitLabMergeRequestKey(projectPath, activity.MR.Number) + linked := gitLabIssueReferenceKeysFromText(activity.MR.Body, projectPath) + if len(linked) == 0 && db != nil { + notes, err := db.GetGitLabNotes(projectPath, "mr", activity.MR.Number) + if err != nil { + return nil, nil, err + } + for _, note := range notes { + for issueKey := range gitLabIssueReferenceKeysFromText(note.Body, projectPath) { + linked[issueKey] = struct{}{} } - activitiesMap.Store(prKey, &activity) - pageResults++ - totalFound++ } } - if config.debugMode { - fmt.Printf(" [%s] Page %d: found %d new PRs (total: %d)\n", label, page, pageResults, totalFound) + if len(linked) > 0 { + mrToIssueKeys[mrKey] = linked } + } - if resp.NextPage == 0 { + nestedActivities := nestGitLabIssues(activities, issueActivities, mrToIssueKeys) + return nestedActivities, filterStandaloneGitLabIssues(nestedActivities, issueActivities), nil +} + +func listGitLabIssuesClosedOnMergeRequest(ctx context.Context, client *gitlab.Client, projectID int64, mergeRequestIID int64) ([]*gitlab.Issue, error) { + allIssues := make([]*gitlab.Issue, 0) + opts := &gitlab.GetIssuesClosedOnMergeOptions{ListOptions: gitlab.ListOptions{PerPage: 100, Page: 1}} + + for { + issues, resp, err := client.MergeRequests.GetIssuesClosedOnMerge(projectID, mergeRequestIID, opts, gitlab.WithContext(ctx)) + if err != nil { + return nil, err + } + allIssues = append(allIssues, issues...) + if resp == nil || resp.NextPage == 0 { break } opts.Page = resp.NextPage - page++ } - if config.debugMode && totalFound > 0 { - fmt.Printf(" [%s] Complete: %d PRs found\n", label, totalFound) - } + return allIssues, nil } -// DisplayConfig holds all the information needed to display a PR or issue -type DisplayConfig struct { - Owner string - Repo string - Number int - Title string - User string - UpdatedAt *github.Timestamp - HTMLURL *string - Label string - HasUpdates bool - IsIndented bool // for nested display under PRs - State *string // for issues nested under PRs (OPEN/CLOSED) -} - -// displayItem is the unified display function for both PRs and issues -func displayItem(cfg DisplayConfig) { - dateStr := " " - if cfg.UpdatedAt != nil { - dateStr = cfg.UpdatedAt.Format("2006/01/02") +func nestGitLabIssues(activities []PRActivity, issueActivities []IssueActivity, mrToIssueKeys map[string]map[string]struct{}) []PRActivity { + issueByKey := make(map[string]IssueActivity, len(issueActivities)) + for _, issue := range issueActivities { + issueByKey[buildGitLabIssueKey(issue.Owner, issue.Issue.Number)] = issue } - indent := "" - linkIndent := " " - if cfg.IsIndented && cfg.State != nil { - state := strings.ToUpper(*cfg.State) - stateColor := getStateColor(*cfg.State) - indent = fmt.Sprintf("-- %s ", stateColor.Sprint(state)) - linkIndent = " " + for i := range activities { + activities[i].Issues = nil + mrKey := buildGitLabMergeRequestKey(activities[i].Owner, activities[i].MR.Number) + linkedKeys := mrToIssueKeys[mrKey] + if len(linkedKeys) == 0 { + continue + } + for issueKey := range linkedKeys { + issue, ok := issueByKey[issueKey] + if !ok { + continue + } + activities[i].Issues = append(activities[i].Issues, issue) + } + sort.Slice(activities[i].Issues, func(a, b int) bool { + return activities[i].Issues[a].UpdatedAt.After(activities[i].Issues[b].UpdatedAt) + }) } - labelColor := getLabelColor(cfg.Label) - userColor := getUserColor(cfg.User) + return activities +} - updateIcon := "" - if cfg.HasUpdates { - updateIcon = color.New(color.FgYellow, color.Bold).Sprint("● ") +func filterStandaloneGitLabIssues(activities []PRActivity, issueActivities []IssueActivity) []IssueActivity { + linkedIssueKeys := make(map[string]struct{}) + for _, activity := range activities { + for _, issue := range activity.Issues { + linkedIssueKeys[buildGitLabIssueKey(issue.Owner, issue.Issue.Number)] = struct{}{} + } } - fmt.Printf("%s%s%s %s %s %s/%s#%d - %s\n", - updateIcon, - indent, - dateStr, - labelColor.Sprint(strings.ToUpper(cfg.Label)), - userColor.Sprint(cfg.User), - cfg.Owner, cfg.Repo, cfg.Number, - cfg.Title, - ) - - if config.showLinks && cfg.HTMLURL != nil { - fmt.Printf("%s🔗 %s\n", linkIndent, *cfg.HTMLURL) + standalone := make([]IssueActivity, 0, len(issueActivities)) + for _, issue := range issueActivities { + issueKey := buildGitLabIssueKey(issue.Owner, issue.Issue.Number) + if _, linked := linkedIssueKeys[issueKey]; linked { + continue + } + standalone = append(standalone, issue) } -} -func displayPR(label, owner, repo string, pr *github.PullRequest, hasUpdates bool) { - displayItem(DisplayConfig{ - Owner: owner, - Repo: repo, - Number: pr.GetNumber(), - Title: pr.GetTitle(), - User: pr.User.GetLogin(), - UpdatedAt: pr.UpdatedAt, - HTMLURL: pr.HTMLURL, - Label: label, - HasUpdates: hasUpdates, - IsIndented: false, - }) + return standalone } -func displayIssue(label, owner, repo string, issue *github.Issue, indented bool, hasUpdates bool) { - displayItem(DisplayConfig{ - Owner: owner, - Repo: repo, - Number: issue.GetNumber(), - Title: issue.GetTitle(), - User: issue.User.GetLogin(), - UpdatedAt: issue.UpdatedAt, - HTMLURL: issue.HTMLURL, - Label: label, - HasUpdates: hasUpdates, - IsIndented: indented, - State: issue.State, - }) -} +func gitLabIssueReferenceKeysFromText(text, defaultProjectPath string) map[string]struct{} { + results := make(map[string]struct{}) + if strings.TrimSpace(text) == "" { + return results + } -func collectIssueSearchResults(query, label string, seenIssues *sync.Map, issueActivitiesMap *sync.Map) { - if config.localMode { - if config.db == nil { - return + for _, match := range gitLabIssueURLRefPattern.FindAllStringSubmatch(text, -1) { + if len(match) < 3 { + continue } - - allIssues, issueLabels, err := config.db.GetAllIssuesWithLabels(config.debugMode) - if err != nil { - if config.debugMode { - fmt.Printf(" [%s] Error loading from database: %v\n", label, err) - } - return + iid, ok := parsePositiveInt(match[2]) + if !ok { + continue } + results[buildGitLabIssueKey(match[1], iid)] = struct{}{} + } - if config.debugMode { - fmt.Printf(" [%s] Loading from database...\n", label) + for _, match := range gitLabIssueQualifiedRefPattern.FindAllStringSubmatch(text, -1) { + if len(match) < 3 { + continue } + iid, ok := parsePositiveInt(match[2]) + if !ok { + continue + } + results[buildGitLabIssueKey(match[1], iid)] = struct{}{} + } - totalFound := 0 - cutoffTime := time.Now().Add(-config.timeRange) - for key, issue := range allIssues { - storedLabel := issueLabels[key] - - if storedLabel != label { + defaultProjectPath = normalizeProjectPathWithNamespace(defaultProjectPath) + if defaultProjectPath != "" { + for _, match := range gitLabIssueRelativeURLRefPattern.FindAllStringSubmatch(text, -1) { + if len(match) < 2 { continue } - - if issue.GetUpdatedAt().Time.Before(cutoffTime) { + iid, ok := parsePositiveInt(match[1]) + if !ok { continue } + results[buildGitLabIssueKey(defaultProjectPath, iid)] = struct{}{} + } - parts := strings.Split(key, "/") - if len(parts) < 2 { + for _, match := range gitLabIssueSameProjectRefPattern.FindAllStringSubmatch(text, -1) { + if len(match) < 2 { continue } - owner := parts[0] - repoAndNum := parts[1] - repoParts := strings.Split(repoAndNum, "#") - if len(repoParts) < 2 { + iid, ok := parsePositiveInt(match[1]) + if !ok { continue } - repo := repoParts[0] + results[buildGitLabIssueKey(defaultProjectPath, iid)] = struct{}{} + } + } - if !isRepoAllowed(owner, repo) { - continue - } + return results +} - issueKey := key +func gitLabIssueKeyFromIssue(item *gitlab.Issue, defaultProjectPath string) (string, bool) { + if item == nil || item.IID <= 0 { + return "", false + } - // Check if we've already processed this issue in issueActivitiesMap - existingActivity, alreadyProcessed := issueActivitiesMap.Load(issueKey) - shouldProcess := true + if item.References != nil { + if projectPath, iid, ok := parseGitLabQualifiedReference(item.References.Full); ok { + return buildGitLabIssueKey(projectPath, iid), true + } + } - if alreadyProcessed { - // Issue is already in issueActivitiesMap, check if we need to update the label - existingIssue := existingActivity.(*IssueActivity) - if shouldUpdateLabel(existingIssue.Label, label, false) { - // New label has higher priority, we'll update it - if config.debugMode { - fmt.Printf(" [%s] Updating label for %s from %s to %s (higher priority)\n", label, issueKey, existingIssue.Label, label) - } - } else { - // Existing label has higher or equal priority, skip - shouldProcess = false - } - } + defaultProjectPath = normalizeProjectPathWithNamespace(defaultProjectPath) + if defaultProjectPath == "" { + return "", false + } + return buildGitLabIssueKey(defaultProjectPath, int(item.IID)), true +} - if shouldProcess { - activity := IssueActivity{ - Label: label, - Owner: owner, - Repo: repo, - Issue: issue, - UpdatedAt: issue.GetUpdatedAt().Time, - } - issueActivitiesMap.Store(issueKey, &activity) - totalFound++ - } +func parseGitLabQualifiedReference(reference string) (string, int, bool) { + for _, match := range gitLabIssueQualifiedRefPattern.FindAllStringSubmatch(reference, -1) { + if len(match) < 3 { + continue } + iid, ok := parsePositiveInt(match[2]) + if !ok { + continue + } + return normalizeProjectPathWithNamespace(match[1]), iid, true + } + return "", 0, false +} - if config.debugMode && totalFound > 0 { - fmt.Printf(" [%s] Complete: %d issues found\n", label, totalFound) +func parsePositiveInt(raw string) (int, bool) { + value, err := strconv.Atoi(strings.TrimSpace(raw)) + if err != nil || value <= 0 { + return 0, false + } + return value, true +} + +func parseGitLabMRProjectPath(key string) (string, bool) { + idx := strings.LastIndex(key, "#!") + if idx <= 0 { + return "", false + } + return key[:idx], true +} + +func parseGitLabIssueProjectPath(key string) (string, bool) { + idx := strings.LastIndex(key, "##") + if idx <= 0 { + return "", false + } + return key[:idx], true +} + +func isGitLabProjectAllowed(projectPath string) bool { + if config.allowedRepos == nil || len(config.allowedRepos) == 0 { + return true + } + + normalized := normalizeProjectPathWithNamespace(projectPath) + for repo := range config.allowedRepos { + if strings.EqualFold(normalizeProjectPathWithNamespace(repo), normalized) { + return true } + } - return + return false +} + +func needsLowerPriorityPRChecks(currentLabel string) bool { + return shouldUpdateLabel(currentLabel, "Commented", true) || shouldUpdateLabel(currentLabel, "Mentioned", true) +} + +func mergeLabelWithPriority(currentLabel, candidateLabel string, isPR bool) string { + if shouldUpdateLabel(currentLabel, candidateLabel, isPR) { + return candidateLabel } + return currentLabel +} - opts := &github.SearchOptions{ - ListOptions: github.ListOptions{PerPage: 100}, +func listAllGitLabMergeRequestNotes(ctx context.Context, client *gitlab.Client, projectID int64, mrIID int64) ([]*gitlab.Note, error) { + allNotes := make([]*gitlab.Note, 0) + options := &gitlab.ListMergeRequestNotesOptions{ + ListOptions: gitlab.ListOptions{PerPage: 100, Page: 1}, } - totalFound := 0 + for { + var ( + notes []*gitlab.Note + response *gitlab.Response + ) + err := retryWithBackoff(func() error { + var apiErr error + notes, response, apiErr = client.Notes.ListMergeRequestNotes(projectID, mrIID, options, gitlab.WithContext(ctx)) + return apiErr + }, fmt.Sprintf("GitLabListMergeRequestNotes %d!%d page %d", projectID, mrIID, options.Page)) + if err != nil { + return nil, err + } + allNotes = append(allNotes, notes...) + + if response == nil || response.NextPage == 0 { + break + } + options.Page = response.NextPage + } + + return allNotes, nil +} + +func listAllGitLabIssueNotes(ctx context.Context, client *gitlab.Client, projectID int64, issueIID int64) ([]*gitlab.Note, error) { + allNotes := make([]*gitlab.Note, 0) + options := &gitlab.ListIssueNotesOptions{ + ListOptions: gitlab.ListOptions{PerPage: 100, Page: 1}, + } - page := 1 for { - if config.debugMode { - fmt.Printf(" [%s] Searching page %d with query: %s\n", label, page, query) + var ( + notes []*gitlab.Note + response *gitlab.Response + ) + err := retryWithBackoff(func() error { + var apiErr error + notes, response, apiErr = client.Notes.ListIssueNotes(projectID, issueIID, options, gitlab.WithContext(ctx)) + return apiErr + }, fmt.Sprintf("GitLabListIssueNotes %d#%d page %d", projectID, issueIID, options.Page)) + if err != nil { + return nil, err } + allNotes = append(allNotes, notes...) - var result *github.IssuesSearchResult - var resp *github.Response - var err error + if response == nil || response.NextPage == 0 { + break + } + options.Page = response.NextPage + } - retryErr := retryWithBackoff(func() error { - result, resp, err = config.client.Search.Issues(config.ctx, query, opts) - return err - }, fmt.Sprintf("%s-issues-page%d", label, page)) + return allNotes, nil +} - config.progress.increment() - if !config.debugMode { - config.progress.display() +func gitLabNotesInvolvement(notes []*gitlab.Note, description, currentUsername string, currentUserID int64) (bool, bool) { + commented := false + mentioned := containsGitLabUserMention(description, currentUsername) + + for _, note := range notes { + if note == nil { + continue + } + if matchesGitLabNoteAuthor(note.Author, currentUsername, currentUserID) { + commented = true } + if !mentioned && containsGitLabUserMention(note.Body, currentUsername) { + mentioned = true + } + if commented && mentioned { + break + } + } - if page == 1 && resp != nil && resp.NextPage != 0 { - lastPage := resp.LastPage - if lastPage > 1 { - additionalPages := lastPage - 1 - config.progress.addToTotal(additionalPages) - if !config.debugMode { - config.progress.display() - } - } + return commented, mentioned +} + +func containsGitLabUserMention(text, username string) bool { + if text == "" || username == "" { + return false + } + needle := "@" + strings.ToLower(strings.TrimSpace(username)) + if needle == "@" { + return false + } + return strings.Contains(strings.ToLower(text), needle) +} + +func matchesGitLabNoteAuthor(author gitlab.NoteAuthor, username string, userID int64) bool { + if userID > 0 && author.ID == userID { + return true + } + return strings.EqualFold(strings.TrimSpace(author.Username), strings.TrimSpace(username)) +} + +func matchesGitLabBasicUser(user *gitlab.BasicUser, username string, userID int64) bool { + if user == nil { + return false + } + if userID > 0 && user.ID == userID { + return true + } + return strings.EqualFold(strings.TrimSpace(user.Username), strings.TrimSpace(username)) +} + +func matchesGitLabIssueAuthor(author *gitlab.IssueAuthor, username string, userID int64) bool { + if author == nil { + return false + } + if userID > 0 && author.ID == userID { + return true + } + return strings.EqualFold(strings.TrimSpace(author.Username), strings.TrimSpace(username)) +} + +func matchesGitLabIssueAssignee(assignee *gitlab.IssueAssignee, username string, userID int64) bool { + if assignee == nil { + return false + } + if userID > 0 && assignee.ID == userID { + return true + } + return strings.EqualFold(strings.TrimSpace(assignee.Username), strings.TrimSpace(username)) +} + +func gitLabIssueAssigneeListContains(assignees []*gitlab.IssueAssignee, username string, userID int64) bool { + for _, assignee := range assignees { + if matchesGitLabIssueAssignee(assignee, username, userID) { + return true } + } + return false +} - if retryErr != nil { - fmt.Printf(" [%s] Error searching after retries: %v\n", label, retryErr) - if resp != nil { - fmt.Printf(" [%s] Rate limit remaining: %d/%d\n", label, resp.Rate.Remaining, resp.Rate.Limit) - } - return +func gitLabBasicUserListContains(users []*gitlab.BasicUser, username string, userID int64) bool { + for _, user := range users { + if matchesGitLabBasicUser(user, username, userID) { + return true } + } + return false +} - if config.debugMode && resp != nil { - fmt.Printf(" [%s] API Response: %d results, Rate: %d/%d\n", label, len(result.Issues), resp.Rate.Remaining, resp.Rate.Limit) +func gitLabApprovalStateReviewedByCurrentUser(state *gitlab.MergeRequestApprovalState, username string, userID int64) bool { + if state == nil { + return false + } + for _, rule := range state.Rules { + if rule == nil { + continue + } + if gitLabBasicUserListContains(rule.ApprovedBy, username, userID) { + return true } + } + return false +} - pageResults := 0 - for _, issue := range result.Issues { - if issue.PullRequestLinks != nil { - continue - } +func resolveAllowedGitLabProjects(ctx context.Context, client *gitlab.Client, allowedRepos map[string]bool) ([]gitLabProject, error) { + if client == nil { + return nil, fmt.Errorf("gitlab client is not configured") + } - repoURL := *issue.RepositoryURL - parts := strings.Split(repoURL, "/") - if len(parts) < 2 { - fmt.Printf(" [%s] Error: Invalid repository URL format: %s\n", label, repoURL) - continue - } - owner := parts[len(parts)-2] - repo := parts[len(parts)-1] + if len(allowedRepos) == 0 { + return []gitLabProject{}, nil + } - if !isRepoAllowed(owner, repo) { - continue - } + repoPaths := make([]string, 0, len(allowedRepos)) + for repo := range allowedRepos { + normalized := normalizeProjectPathWithNamespace(repo) + if normalized != "" { + repoPaths = append(repoPaths, normalized) + } + } + sort.Strings(repoPaths) - issueKey := buildItemKey(owner, repo, *issue.Number) + projectIDCache := make(map[string]int64, len(repoPaths)) + projects := make([]gitLabProject, 0, len(repoPaths)) + for _, pathWithNamespace := range repoPaths { + if id, ok := projectIDCache[pathWithNamespace]; ok { + projects = append(projects, gitLabProject{PathWithNamespace: pathWithNamespace, ID: id}) + continue + } - // Check if we've already processed this issue in issueActivitiesMap - existingActivity, alreadyProcessed := issueActivitiesMap.Load(issueKey) - shouldProcess := true + var project *gitlab.Project + err := retryWithBackoff(func() error { + var apiErr error + project, _, apiErr = client.Projects.GetProject(pathWithNamespace, nil, gitlab.WithContext(ctx)) + return apiErr + }, fmt.Sprintf("GitLabGetProject %s", pathWithNamespace)) + if err != nil { + return nil, fmt.Errorf("resolve project %s: %w", pathWithNamespace, err) + } - if alreadyProcessed { - // Issue is already in issueActivitiesMap, check if we need to update the label - existingIssue := existingActivity.(*IssueActivity) - if shouldUpdateLabel(existingIssue.Label, label, false) { - // New label has higher priority, we'll update it - if config.debugMode { - fmt.Printf(" [%s] Updating label for %s from %s to %s (higher priority)\n", label, issueKey, existingIssue.Label, label) - } - } else { - // Existing label has higher or equal priority, skip - shouldProcess = false - } - } + projectIDCache[pathWithNamespace] = project.ID + projects = append(projects, gitLabProject{PathWithNamespace: pathWithNamespace, ID: project.ID}) + } - if shouldProcess { - hasUpdates := false + return projects, nil +} - if config.db != nil { - cachedIssue, err := config.db.GetIssue(owner, repo, *issue.Number) - if err == nil { - if issue.GetUpdatedAt().After(cachedIssue.GetUpdatedAt().Time) { - hasUpdates = true - } - } - if err := config.db.SaveIssueWithLabel(owner, repo, issue, label, config.debugMode); err != nil { - config.dbErrorCount.Add(1) - if config.debugMode { - fmt.Printf(" [DB] Warning: Failed to save issue %s/%s#%d: %v\n", owner, repo, *issue.Number, err) - } - } - } +func listGitLabProjectMergeRequests(ctx context.Context, client *gitlab.Client, projectID int64, cutoff time.Time) ([]*gitlab.BasicMergeRequest, error) { + allItems := make([]*gitlab.BasicMergeRequest, 0) + options := &gitlab.ListProjectMergeRequestsOptions{ + ListOptions: gitlab.ListOptions{PerPage: 100, Page: 1}, + State: gitlab.Ptr("all"), + UpdatedAfter: &cutoff, + } - activity := IssueActivity{ - Label: label, - Owner: owner, - Repo: repo, - Issue: issue, - UpdatedAt: issue.GetUpdatedAt().Time, - HasUpdates: hasUpdates, - } - issueActivitiesMap.Store(issueKey, &activity) - pageResults++ - totalFound++ - } + for { + var ( + items []*gitlab.BasicMergeRequest + response *gitlab.Response + ) + err := retryWithBackoff(func() error { + var apiErr error + items, response, apiErr = client.MergeRequests.ListProjectMergeRequests(projectID, options, gitlab.WithContext(ctx)) + return apiErr + }, fmt.Sprintf("GitLabListProjectMergeRequests %d page %d", projectID, options.Page)) + if err != nil { + return nil, err } + allItems = append(allItems, items...) + + if response == nil || response.NextPage == 0 { + break + } + options.Page = response.NextPage + } - if config.debugMode { - fmt.Printf(" [%s] Page %d: found %d new issues (total: %d)\n", label, page, pageResults, totalFound) + return allItems, nil +} + +func listGitLabProjectIssues(ctx context.Context, client *gitlab.Client, projectID int64, cutoff time.Time) ([]*gitlab.Issue, error) { + allItems := make([]*gitlab.Issue, 0) + options := &gitlab.ListProjectIssuesOptions{ + ListOptions: gitlab.ListOptions{PerPage: 100, Page: 1}, + State: gitlab.Ptr("all"), + UpdatedAfter: &cutoff, + } + + for { + var ( + items []*gitlab.Issue + response *gitlab.Response + ) + err := retryWithBackoff(func() error { + var apiErr error + items, response, apiErr = client.Issues.ListProjectIssues(projectID, options, gitlab.WithContext(ctx)) + return apiErr + }, fmt.Sprintf("GitLabListProjectIssues %d page %d", projectID, options.Page)) + if err != nil { + return nil, err } + allItems = append(allItems, items...) - if resp.NextPage == 0 { + if response == nil || response.NextPage == 0 { break } - opts.Page = resp.NextPage - page++ + options.Page = response.NextPage + } + + return allItems, nil +} + +func normalizeProjectPathWithNamespace(repo string) string { + trimmed := strings.TrimSpace(repo) + return strings.Trim(trimmed, "/") +} + +func buildGitLabDedupKey(projectPath, itemType string, iid int64) string { + return fmt.Sprintf("%s|%s|%d", strings.ToLower(projectPath), itemType, iid) +} + +func toMergeRequestModelFromGitLab(item *gitlab.BasicMergeRequest) MergeRequestModel { + if item == nil { + return MergeRequestModel{} + } + + state := strings.ToLower(item.State) + merged := state == "merged" || item.MergedAt != nil + normalizedState := "open" + if merged || state == "closed" { + normalizedState = "closed" + } + + updatedAt := time.Time{} + if item.UpdatedAt != nil { + updatedAt = *item.UpdatedAt + } + + userLogin := "" + if item.Author != nil { + userLogin = item.Author.Username + } + + return MergeRequestModel{ + Number: int(item.IID), + Title: item.Title, + Body: item.Description, + State: normalizedState, + UpdatedAt: updatedAt, + WebURL: item.WebURL, + UserLogin: userLogin, + Merged: merged, + } +} + +func toIssueModelFromGitLab(item *gitlab.Issue) IssueModel { + if item == nil { + return IssueModel{} + } + + state := strings.ToLower(item.State) + normalizedState := "open" + if state == "closed" { + normalizedState = "closed" + } + + updatedAt := time.Time{} + if item.UpdatedAt != nil { + updatedAt = *item.UpdatedAt + } + + userLogin := "" + if item.Author != nil { + userLogin = item.Author.Username } - if config.debugMode && totalFound > 0 { - fmt.Printf(" [%s] Complete: %d issues found\n", label, totalFound) + return IssueModel{ + Number: int(item.IID), + Title: item.Title, + Body: item.Description, + State: normalizedState, + UpdatedAt: updatedAt, + WebURL: item.WebURL, + UserLogin: userLogin, } } diff --git a/priority_test.go b/priority_test.go index 099f720..72b4200 100644 --- a/priority_test.go +++ b/priority_test.go @@ -1,7 +1,24 @@ package main import ( + "bytes" + "context" + "encoding/json" + "fmt" + "net/http" + "net/http/httptest" + "os" + "os/exec" + "path/filepath" + "sort" + "strconv" + "strings" + "sync/atomic" "testing" + "time" + + bolt "go.etcd.io/bbolt" + gitlab "gitlab.com/gitlab-org/api/client-go" ) func TestPRLabelPriority(t *testing.T) { @@ -103,3 +120,1180 @@ func TestShouldUpdateLabel_Issue(t *testing.T) { }) } } + +func TestNormalizeGitLabBaseURL(t *testing.T) { + tests := []struct { + name string + raw string + want string + wantErr bool + }{ + { + name: "defaults to gitlab.com when empty", + raw: "", + want: "https://gitlab.com/api/v4", + }, + { + name: "normalizes host with trailing slash", + raw: "http://10.10.1.207/", + want: "http://10.10.1.207/api/v4", + }, + { + name: "normalizes host without trailing slash", + raw: "http://10.10.1.207", + want: "http://10.10.1.207/api/v4", + }, + { + name: "normalizes gitlab.com", + raw: "https://gitlab.com", + want: "https://gitlab.com/api/v4", + }, + { + name: "normalizes subpath base", + raw: "https://gitlab.example.com/gitlab", + want: "https://gitlab.example.com/gitlab/api/v4", + }, + { + name: "does not double append api v4", + raw: "https://host/api/v4", + want: "https://host/api/v4", + }, + { + name: "trims trailing slash on existing api v4 path", + raw: "https://host/api/v4/", + want: "https://host/api/v4", + }, + { + name: "rejects missing scheme", + raw: "gitlab.example.com", + wantErr: true, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, err := normalizeGitLabBaseURL(tt.raw) + if tt.wantErr { + if err == nil { + t.Fatalf("normalizeGitLabBaseURL(%q) expected error, got nil", tt.raw) + } + return + } + + if err != nil { + t.Fatalf("normalizeGitLabBaseURL(%q) unexpected error: %v", tt.raw, err) + } + + if got != tt.want { + t.Errorf("normalizeGitLabBaseURL(%q) = %q, want %q", tt.raw, got, tt.want) + } + }) + } +} + +func TestRetryWithBackoff_GitLab429UsesRetryAfterHeader(t *testing.T) { + var calls atomic.Int32 + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path != "/retry" { + t.Fatalf("unexpected path: %s", r.URL.Path) + } + + if calls.Add(1) == 1 { + w.Header().Set("Retry-After", "7") + w.WriteHeader(http.StatusTooManyRequests) + fmt.Fprint(w, `{"message":"rate limited"}`) + return + } + + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusOK) + fmt.Fprint(w, `{"id":42,"username":"tester"}`) + })) + defer server.Close() + + oldDebugMode := config.debugMode + oldCtx := config.ctx + oldProgress := config.progress + oldRetryAfter := retryAfter + t.Cleanup(func() { + config.debugMode = oldDebugMode + config.ctx = oldCtx + config.progress = oldProgress + retryAfter = oldRetryAfter + }) + + config.debugMode = true + config.ctx = context.Background() + config.progress = nil + + waits := make([]time.Duration, 0, 2) + retryAfter = func(d time.Duration) <-chan time.Time { + waits = append(waits, d) + ch := make(chan time.Time, 1) + ch <- time.Now() + return ch + } + + err := retryWithBackoff(func() error { + request, reqErr := http.NewRequestWithContext(config.ctx, http.MethodGet, server.URL+"/retry", nil) + if reqErr != nil { + return reqErr + } + + response, reqErr := http.DefaultClient.Do(request) + if reqErr != nil { + return reqErr + } + defer response.Body.Close() + + if response.StatusCode >= http.StatusBadRequest { + return gitlab.CheckResponse(response) + } + + return nil + }, "GitLabCurrentUser") + if err != nil { + t.Fatalf("retryWithBackoff failed: %v", err) + } + + if calls.Load() != 2 { + t.Fatalf("expected 2 API calls, got %d", calls.Load()) + } + if len(waits) != 1 { + t.Fatalf("expected one retry wait, got %d", len(waits)) + } + if waits[0] != 7*time.Second { + t.Fatalf("expected Retry-After wait 7s, got %v", waits[0]) + } +} + +func TestRetryWithBackoff_GitLab429FallsBackWhenRetryAfterMissing(t *testing.T) { + var calls atomic.Int32 + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path != "/retry" { + t.Fatalf("unexpected path: %s", r.URL.Path) + } + + if calls.Add(1) == 1 { + w.WriteHeader(http.StatusTooManyRequests) + fmt.Fprint(w, `{"message":"rate limited"}`) + return + } + + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusOK) + fmt.Fprint(w, `{"id":42,"username":"tester"}`) + })) + defer server.Close() + + oldDebugMode := config.debugMode + oldCtx := config.ctx + oldProgress := config.progress + oldRetryAfter := retryAfter + t.Cleanup(func() { + config.debugMode = oldDebugMode + config.ctx = oldCtx + config.progress = oldProgress + retryAfter = oldRetryAfter + }) + + config.debugMode = true + config.ctx = context.Background() + config.progress = nil + + waits := make([]time.Duration, 0, 2) + retryAfter = func(d time.Duration) <-chan time.Time { + waits = append(waits, d) + ch := make(chan time.Time, 1) + ch <- time.Now() + return ch + } + + err := retryWithBackoff(func() error { + request, reqErr := http.NewRequestWithContext(config.ctx, http.MethodGet, server.URL+"/retry", nil) + if reqErr != nil { + return reqErr + } + + response, reqErr := http.DefaultClient.Do(request) + if reqErr != nil { + return reqErr + } + defer response.Body.Close() + + if response.StatusCode >= http.StatusBadRequest { + return gitlab.CheckResponse(response) + } + + return nil + }, "GitLabCurrentUser") + if err != nil { + t.Fatalf("retryWithBackoff failed: %v", err) + } + + if calls.Load() != 2 { + t.Fatalf("expected 2 API calls, got %d", calls.Load()) + } + if len(waits) != 1 { + t.Fatalf("expected one retry wait, got %d", len(waits)) + } + if waits[0] != 1*time.Second { + t.Fatalf("expected fallback wait 1s, got %v", waits[0]) + } +} + +func TestRetryWithBackoff_GitLab429UsesRateLimitResetWhenRetryAfterMissing(t *testing.T) { + var calls atomic.Int32 + resetUnix := time.Now().Add(10 * time.Second).Unix() + + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path != "/retry" { + t.Fatalf("unexpected path: %s", r.URL.Path) + } + + if calls.Add(1) == 1 { + w.Header().Set("Ratelimit-Reset", strconv.FormatInt(resetUnix, 10)) + w.WriteHeader(http.StatusTooManyRequests) + fmt.Fprint(w, `{"message":"rate limited"}`) + return + } + + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusOK) + fmt.Fprint(w, `{"id":42,"username":"tester"}`) + })) + defer server.Close() + + oldDebugMode := config.debugMode + oldCtx := config.ctx + oldProgress := config.progress + oldRetryAfter := retryAfter + t.Cleanup(func() { + config.debugMode = oldDebugMode + config.ctx = oldCtx + config.progress = oldProgress + retryAfter = oldRetryAfter + }) + + config.debugMode = true + config.ctx = context.Background() + config.progress = nil + + waits := make([]time.Duration, 0, 2) + retryAfter = func(d time.Duration) <-chan time.Time { + waits = append(waits, d) + ch := make(chan time.Time, 1) + ch <- time.Now() + return ch + } + + err := retryWithBackoff(func() error { + request, reqErr := http.NewRequestWithContext(config.ctx, http.MethodGet, server.URL+"/retry", nil) + if reqErr != nil { + return reqErr + } + + response, reqErr := http.DefaultClient.Do(request) + if reqErr != nil { + return reqErr + } + defer response.Body.Close() + + if response.StatusCode >= http.StatusBadRequest { + return gitlab.CheckResponse(response) + } + + return nil + }, "GitLabCurrentUser") + if err != nil { + t.Fatalf("retryWithBackoff failed: %v", err) + } + + if calls.Load() != 2 { + t.Fatalf("expected 2 API calls, got %d", calls.Load()) + } + if len(waits) != 1 { + t.Fatalf("expected one retry wait, got %d", len(waits)) + } + if waits[0] < 8*time.Second || waits[0] > 10*time.Second { + t.Fatalf("expected Ratelimit-Reset wait between 8s and 10s, got %v", waits[0]) + } +} + +func TestRetryWithBackoff_GitLab5xxRetriesWithExponentialBackoff(t *testing.T) { + var calls atomic.Int32 + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path != "/retry" { + t.Fatalf("unexpected path: %s", r.URL.Path) + } + + if calls.Add(1) == 1 { + w.WriteHeader(http.StatusBadGateway) + fmt.Fprint(w, `{"message":"temporary outage"}`) + return + } + + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(http.StatusOK) + fmt.Fprint(w, `{"id":42,"username":"tester"}`) + })) + defer server.Close() + + oldDebugMode := config.debugMode + oldCtx := config.ctx + oldProgress := config.progress + oldRetryAfter := retryAfter + t.Cleanup(func() { + config.debugMode = oldDebugMode + config.ctx = oldCtx + config.progress = oldProgress + retryAfter = oldRetryAfter + }) + + config.debugMode = true + config.ctx = context.Background() + config.progress = nil + + waits := make([]time.Duration, 0, 2) + retryAfter = func(d time.Duration) <-chan time.Time { + waits = append(waits, d) + ch := make(chan time.Time, 1) + ch <- time.Now() + return ch + } + + err := retryWithBackoff(func() error { + request, reqErr := http.NewRequestWithContext(config.ctx, http.MethodGet, server.URL+"/retry", nil) + if reqErr != nil { + return reqErr + } + + response, reqErr := http.DefaultClient.Do(request) + if reqErr != nil { + return reqErr + } + defer response.Body.Close() + + if response.StatusCode >= http.StatusBadRequest { + return gitlab.CheckResponse(response) + } + + return nil + }, "GitLabCurrentUser") + if err != nil { + t.Fatalf("retryWithBackoff failed: %v", err) + } + + if calls.Load() != 2 { + t.Fatalf("expected 2 API calls, got %d", calls.Load()) + } + if len(waits) != 1 { + t.Fatalf("expected one retry wait, got %d", len(waits)) + } + if waits[0] != 1*time.Second { + t.Fatalf("expected 5xx wait 1s, got %v", waits[0]) + } +} + +func TestDatabaseGitLabRoundTripWithLabels(t *testing.T) { + dbPath := filepath.Join(t.TempDir(), "gitlab.db") + db, err := OpenDatabase(dbPath) + if err != nil { + t.Fatalf("OpenDatabase failed: %v", err) + } + defer db.Close() + + updated := time.Date(2026, 1, 15, 12, 0, 0, 0, time.UTC) + mr := MergeRequestModel{ + Number: 7, + Title: "cached mr", + Body: "mr body", + State: "open", + UpdatedAt: updated, + WebURL: "https://gitlab.example/group/repo/-/merge_requests/7", + UserLogin: "alice", + } + issue := IssueModel{ + Number: 11, + Title: "cached issue", + Body: "issue body", + State: "closed", + UpdatedAt: updated.Add(-time.Hour), + WebURL: "https://gitlab.example/group/repo/-/issues/11", + UserLogin: "bob", + } + + if err := db.SaveGitLabMergeRequestWithLabel("group/repo", mr, "Reviewed", false); err != nil { + t.Fatalf("SaveGitLabMergeRequestWithLabel failed: %v", err) + } + if err := db.SaveGitLabIssueWithLabel("group/repo", issue, "Commented", false); err != nil { + t.Fatalf("SaveGitLabIssueWithLabel failed: %v", err) + } + if err := db.SaveGitLabNote(GitLabNoteRecord{ + ProjectPath: "group/repo", + ItemType: "mr", + ItemIID: 7, + NoteID: 301, + Body: "note body", + AuthorUsername: "alice", + AuthorID: 42, + }, false); err != nil { + t.Fatalf("SaveGitLabNote failed: %v", err) + } + + allMRs, mrLabels, err := db.GetAllGitLabMergeRequestsWithLabels(false) + if err != nil { + t.Fatalf("GetAllGitLabMergeRequestsWithLabels failed: %v", err) + } + mrKey := "group/repo#!7" + if len(allMRs) != 1 { + t.Fatalf("MR count = %d, want 1", len(allMRs)) + } + if allMRs[mrKey].Title != "cached mr" { + t.Fatalf("MR title = %q, want cached mr", allMRs[mrKey].Title) + } + if mrLabels[mrKey] != "Reviewed" { + t.Fatalf("MR label = %q, want Reviewed", mrLabels[mrKey]) + } + + allIssues, issueLabels, err := db.GetAllGitLabIssuesWithLabels(false) + if err != nil { + t.Fatalf("GetAllGitLabIssuesWithLabels failed: %v", err) + } + issueKey := "group/repo##11" + if len(allIssues) != 1 { + t.Fatalf("Issue count = %d, want 1", len(allIssues)) + } + if allIssues[issueKey].Title != "cached issue" { + t.Fatalf("Issue title = %q, want cached issue", allIssues[issueKey].Title) + } + if issueLabels[issueKey] != "Commented" { + t.Fatalf("Issue label = %q, want Commented", issueLabels[issueKey]) + } + + noteCount := 0 + err = db.db.View(func(tx *bolt.Tx) error { + return tx.Bucket(gitlabNotesBkt).ForEach(func(_, _ []byte) error { + noteCount++ + return nil + }) + }) + if err != nil { + t.Fatalf("reading gitlab notes bucket failed: %v", err) + } + if noteCount != 1 { + t.Fatalf("GitLab note count = %d, want 1", noteCount) + } + + hasData, err := db.HasGitLabData() + if err != nil { + t.Fatalf("HasGitLabData failed: %v", err) + } + if !hasData { + t.Fatalf("HasGitLabData = false, want true") + } +} + +func TestLoadGitLabCachedActivities_OfflineParityFiltersAndOrder(t *testing.T) { + originalConfig := config + defer func() { config = originalConfig }() + + dbPath := filepath.Join(t.TempDir(), "gitlab.db") + db, err := OpenDatabase(dbPath) + if err != nil { + t.Fatalf("OpenDatabase failed: %v", err) + } + defer db.Close() + + now := time.Date(2026, 2, 1, 12, 0, 0, 0, time.UTC) + newMR := MergeRequestModel{Number: 2, Title: "new mr", State: "open", UpdatedAt: now.Add(-2 * time.Hour), UserLogin: "alice"} + oldMR := MergeRequestModel{Number: 1, Title: "old mr", State: "closed", UpdatedAt: now.Add(-48 * time.Hour), UserLogin: "alice"} + newIssue := IssueModel{Number: 4, Title: "new issue", State: "open", UpdatedAt: now.Add(-90 * time.Minute), UserLogin: "bob"} + + if err := db.SaveGitLabMergeRequestWithLabel("group/repo", newMR, "Authored", false); err != nil { + t.Fatalf("save new MR failed: %v", err) + } + if err := db.SaveGitLabMergeRequestWithLabel("group/repo", oldMR, "Reviewed", false); err != nil { + t.Fatalf("save old MR failed: %v", err) + } + if err := db.SaveGitLabIssueWithLabel("group/repo", newIssue, "Commented", false); err != nil { + t.Fatalf("save issue failed: %v", err) + } + if err := db.SaveGitLabMergeRequestWithLabel("other/repo", MergeRequestModel{Number: 8, Title: "other", UpdatedAt: now.Add(-time.Hour)}, "Authored", false); err != nil { + t.Fatalf("save other repo MR failed: %v", err) + } + + config = Config{ + db: db, + allowedRepos: map[string]bool{"group/repo": true}, + debugMode: false, + } + + activities, issueActivities, err := loadGitLabCachedActivities(now.Add(-24 * time.Hour)) + if err != nil { + t.Fatalf("loadGitLabCachedActivities failed: %v", err) + } + + if len(activities) != 1 { + t.Fatalf("MR activities count = %d, want 1", len(activities)) + } + if activities[0].MR.Title != "new mr" || activities[0].Label != "Authored" || activities[0].Owner != "group/repo" { + t.Fatalf("unexpected MR activity %+v", activities[0]) + } + + if len(issueActivities) != 1 { + t.Fatalf("Issue activities count = %d, want 1", len(issueActivities)) + } + if issueActivities[0].Issue.Title != "new issue" || issueActivities[0].Label != "Commented" || issueActivities[0].Owner != "group/repo" { + t.Fatalf("unexpected issue activity %+v", issueActivities[0]) + } + + sort.Slice(activities, func(i, j int) bool { return activities[i].UpdatedAt.After(activities[j].UpdatedAt) }) + sort.Slice(issueActivities, func(i, j int) bool { return issueActivities[i].UpdatedAt.After(issueActivities[j].UpdatedAt) }) +} + +func TestLoadGitLabCachedActivities_NestsLinkedIssuesAndExcludesStandalone(t *testing.T) { + originalConfig := config + defer func() { config = originalConfig }() + + dbPath := filepath.Join(t.TempDir(), "gitlab.db") + db, err := OpenDatabase(dbPath) + if err != nil { + t.Fatalf("OpenDatabase failed: %v", err) + } + defer db.Close() + + now := time.Date(2026, 2, 1, 12, 0, 0, 0, time.UTC) + mr := MergeRequestModel{Number: 9, Title: "mr", Body: "no direct refs", State: "open", UpdatedAt: now.Add(-2 * time.Hour), UserLogin: "alice"} + linkedIssue := IssueModel{Number: 41, Title: "linked", State: "open", UpdatedAt: now.Add(-time.Hour), UserLogin: "bob"} + standaloneIssue := IssueModel{Number: 42, Title: "standalone", State: "open", UpdatedAt: now.Add(-30 * time.Minute), UserLogin: "carol"} + + if err := db.SaveGitLabMergeRequestWithLabel("group/repo", mr, "Authored", false); err != nil { + t.Fatalf("save MR failed: %v", err) + } + if err := db.SaveGitLabIssueWithLabel("group/repo", linkedIssue, "Commented", false); err != nil { + t.Fatalf("save linked issue failed: %v", err) + } + if err := db.SaveGitLabIssueWithLabel("group/repo", standaloneIssue, "Mentioned", false); err != nil { + t.Fatalf("save standalone issue failed: %v", err) + } + if err := db.SaveGitLabNote(GitLabNoteRecord{ + ProjectPath: "group/repo", + ItemType: "mr", + ItemIID: 9, + NoteID: 9001, + Body: "Tracking issue group/repo#41", + AuthorUsername: "alice", + AuthorID: 1, + }, false); err != nil { + t.Fatalf("save MR note failed: %v", err) + } + + config = Config{ + db: db, + allowedRepos: map[string]bool{"group/repo": true}, + debugMode: false, + } + + activities, issueActivities, err := loadGitLabCachedActivities(now.Add(-24 * time.Hour)) + if err != nil { + t.Fatalf("loadGitLabCachedActivities failed: %v", err) + } + + if len(activities) != 1 { + t.Fatalf("MR activities count = %d, want 1", len(activities)) + } + if len(activities[0].Issues) != 1 || activities[0].Issues[0].Issue.Number != 41 { + t.Fatalf("nested issues = %+v, want only issue 41", activities[0].Issues) + } + + if len(issueActivities) != 1 || issueActivities[0].Issue.Number != 42 { + t.Fatalf("standalone issues = %+v, want only issue 42", issueActivities) + } +} + +func TestFetchGitLabProjectActivities_PaginatesAndFiltersByCutoff(t *testing.T) { + cutoff := time.Date(2026, 1, 10, 12, 0, 0, 0, time.UTC) + + var projectRequestPath string + mrPageCalls := map[int]int{} + issuePageCalls := map[int]int{} + + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + + switch { + case strings.HasPrefix(r.URL.Path, "/api/v4/projects/") && strings.Contains(r.URL.Path, "/closes_issues"): + _, _ = w.Write([]byte(`[]`)) + + case strings.HasPrefix(r.URL.Path, "/api/v4/projects/") && strings.Contains(r.URL.Path, "/approval_state"): + _, _ = w.Write([]byte(`{"approval_rules_overwritten": false, "rules": []}`)) + + case strings.HasPrefix(r.URL.Path, "/api/v4/projects/") && strings.Contains(r.URL.Path, "/notes"): + _, _ = w.Write([]byte(`[]`)) + + case strings.HasPrefix(r.URL.Path, "/api/v4/projects/") && strings.Contains(r.URL.Path, "/merge_requests"): + if r.URL.Query().Get("state") != "all" { + t.Fatalf("merge request state query = %q, want all", r.URL.Query().Get("state")) + } + if r.URL.Query().Get("updated_after") == "" { + t.Fatalf("merge request updated_after query must be set") + } + + page := parsePageQuery(r) + mrPageCalls[page]++ + writePageHeaders(w, page) + + if page == 1 { + _, _ = w.Write([]byte(`[ + {"iid": 7, "title": "MR page 1", "description": "desc", "state": "opened", "updated_at": "2026-01-11T12:00:00Z", "web_url": "https://gitlab.example/mr/7", "author": {"username": "alice"}} + ]`)) + return + } + + _, _ = w.Write([]byte(`[ + {"iid": 8, "title": "MR page 2", "description": "desc", "state": "merged", "updated_at": "2026-01-12T12:00:00Z", "web_url": "https://gitlab.example/mr/8", "author": {"username": "bob"}} + ]`)) + + case strings.HasPrefix(r.URL.Path, "/api/v4/projects/") && strings.Contains(r.URL.Path, "/issues"): + if r.URL.Query().Get("state") != "all" { + t.Fatalf("issue state query = %q, want all", r.URL.Query().Get("state")) + } + if r.URL.Query().Get("updated_after") == "" { + t.Fatalf("issue updated_after query must be set") + } + + page := parsePageQuery(r) + issuePageCalls[page]++ + writePageHeaders(w, page) + + if page == 1 { + _, _ = w.Write([]byte(`[ + {"id": 201, "iid": 11, "title": "Issue page 1", "description": "desc", "state": "opened", "updated_at": "2026-01-11T08:00:00Z", "web_url": "https://gitlab.example/issues/11", "author": {"username": "carol"}} + ]`)) + return + } + + _, _ = w.Write([]byte(`[ + {"id": 202, "iid": 12, "title": "Issue old", "description": "desc", "state": "closed", "updated_at": "2026-01-09T08:00:00Z", "web_url": "https://gitlab.example/issues/12", "author": {"username": "dave"}} + ]`)) + + case strings.HasPrefix(r.URL.Path, "/api/v4/projects/"): + projectRequestPath = r.URL.Path + _ = json.NewEncoder(w).Encode(map[string]any{ + "id": 101, + "path_with_namespace": "group/subgroup/repo", + }) + + default: + t.Fatalf("unexpected request path: %s", r.URL.Path) + } + })) + defer server.Close() + + client, _, err := newGitLabClient("token", server.URL) + if err != nil { + t.Fatalf("newGitLabClient failed: %v", err) + } + + activities, issues, err := fetchGitLabProjectActivities( + context.Background(), + client, + map[string]bool{"group/subgroup/repo": true}, + cutoff, + "alice", + 0, + nil, + ) + if err != nil { + t.Fatalf("fetchGitLabProjectActivities failed: %v", err) + } + + if projectRequestPath != "/api/v4/projects/group%2Fsubgroup%2Frepo" && projectRequestPath != "/api/v4/projects/group/subgroup/repo" { + t.Fatalf("project path = %q, want full path-with-namespace to be preserved", projectRequestPath) + } + + if mrPageCalls[1] != 1 || mrPageCalls[2] != 1 { + t.Fatalf("merge request pagination calls = %+v, want page 1 and 2 exactly once", mrPageCalls) + } + if issuePageCalls[1] != 1 || issuePageCalls[2] != 1 { + t.Fatalf("issue pagination calls = %+v, want page 1 and 2 exactly once", issuePageCalls) + } + + if len(activities) != 2 { + t.Fatalf("got %d merge request activities, want 2", len(activities)) + } + if activities[0].Owner != "group/subgroup/repo" || activities[0].Repo != "" { + t.Fatalf("unexpected project mapping for merge request activity: owner=%q repo=%q", activities[0].Owner, activities[0].Repo) + } + + mergedFound := false + for _, activity := range activities { + if activity.MR.Number == 8 { + if !activity.MR.Merged || activity.MR.State != "closed" { + t.Fatalf("merged MR mapping invalid: merged=%v state=%q", activity.MR.Merged, activity.MR.State) + } + mergedFound = true + } + } + if !mergedFound { + t.Fatalf("expected merged MR iid 8 in results") + } + + if len(issues) != 1 { + t.Fatalf("got %d issue activities, want 1 after cutoff filtering", len(issues)) + } + if issues[0].Issue.Number != 11 { + t.Fatalf("issue number = %d, want 11", issues[0].Issue.Number) + } +} + +func parsePageQuery(r *http.Request) int { + pageParam := r.URL.Query().Get("page") + if pageParam == "" { + return 1 + } + page, err := strconv.Atoi(pageParam) + if err != nil { + return 1 + } + return page +} + +func writePageHeaders(w http.ResponseWriter, page int) { + w.Header().Set("X-Page", strconv.Itoa(page)) + w.Header().Set("X-Per-Page", "100") + w.Header().Set("X-Total", "2") + w.Header().Set("X-Total-Pages", "2") + if page == 1 { + w.Header().Set("X-Next-Page", "2") + } else { + w.Header().Set("X-Next-Page", "") + } +} + +func TestFetchGitLabProjectActivities_DerivesLabelsFromSources(t *testing.T) { + cutoff := time.Date(2026, 1, 10, 0, 0, 0, 0, time.UTC) + + approvalCalls := map[int64]int{} + mrNoteCalls := map[int64]int{} + issueNoteCalls := map[int64]int{} + + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + + switch { + case strings.HasPrefix(r.URL.Path, "/api/v4/projects/") && strings.Contains(r.URL.Path, "/closes_issues"): + _, _ = w.Write([]byte(`[]`)) + + case strings.HasPrefix(r.URL.Path, "/api/v4/projects/") && strings.Contains(r.URL.Path, "/merge_requests/") && strings.HasSuffix(r.URL.Path, "/approval_state"): + iid := parseResourceIID(t, r.URL.Path, "merge_requests", "approval_state") + approvalCalls[iid]++ + if iid == 2 { + _, _ = w.Write([]byte(`{"approval_rules_overwritten": false, "rules": [{"id": 1, "approved_by": [{"id": 42, "username": "me"}]}]}`)) + return + } + _, _ = w.Write([]byte(`{"approval_rules_overwritten": false, "rules": []}`)) + + case strings.HasPrefix(r.URL.Path, "/api/v4/projects/") && strings.Contains(r.URL.Path, "/merge_requests/") && strings.HasSuffix(r.URL.Path, "/notes"): + iid := parseResourceIID(t, r.URL.Path, "merge_requests", "notes") + mrNoteCalls[iid]++ + if iid == 3 { + _, _ = w.Write([]byte(`[ + {"id": 301, "body": "left a review comment", "author": {"id": 42, "username": "me"}} + ]`)) + return + } + _, _ = w.Write([]byte(`[]`)) + + case strings.HasPrefix(r.URL.Path, "/api/v4/projects/") && strings.Contains(r.URL.Path, "/issues/") && strings.HasSuffix(r.URL.Path, "/notes"): + iid := parseResourceIID(t, r.URL.Path, "issues", "notes") + issueNoteCalls[iid]++ + if iid == 21 { + _, _ = w.Write([]byte(`[ + {"id": 401, "body": "I commented", "author": {"id": 42, "username": "me"}} + ]`)) + return + } + if iid == 22 { + _, _ = w.Write([]byte(`[ + {"id": 402, "body": "pinging @Me for visibility", "author": {"id": 7, "username": "alice"}} + ]`)) + return + } + _, _ = w.Write([]byte(`[]`)) + + case strings.HasPrefix(r.URL.Path, "/api/v4/projects/") && strings.Contains(r.URL.Path, "/merge_requests"): + _, _ = w.Write([]byte(`[ + {"iid": 1, "title": "Authored and assigned", "description": "desc", "state": "opened", "updated_at": "2026-01-11T12:00:00Z", "web_url": "https://gitlab.example/mr/1", "author": {"id": 42, "username": "me"}, "assignees": [{"id": 42, "username": "me"}]}, + {"iid": 2, "title": "Reviewed via approvals", "description": "desc", "state": "opened", "updated_at": "2026-01-11T13:00:00Z", "web_url": "https://gitlab.example/mr/2", "author": {"id": 7, "username": "alice"}}, + {"iid": 3, "title": "Commented via notes", "description": "desc", "state": "opened", "updated_at": "2026-01-11T14:00:00Z", "web_url": "https://gitlab.example/mr/3", "author": {"id": 8, "username": "bob"}} + ]`)) + + case strings.HasPrefix(r.URL.Path, "/api/v4/projects/") && strings.Contains(r.URL.Path, "/issues"): + _, _ = w.Write([]byte(`[ + {"id": 521, "iid": 21, "title": "Commented issue", "description": "desc", "state": "opened", "updated_at": "2026-01-11T08:00:00Z", "web_url": "https://gitlab.example/issues/21", "author": {"id": 7, "username": "alice"}}, + {"id": 522, "iid": 22, "title": "Mentioned issue", "description": "desc", "state": "opened", "updated_at": "2026-01-11T09:00:00Z", "web_url": "https://gitlab.example/issues/22", "author": {"id": 9, "username": "carol"}} + ]`)) + + case strings.HasPrefix(r.URL.Path, "/api/v4/projects/"): + _ = json.NewEncoder(w).Encode(map[string]any{ + "id": 101, + "path_with_namespace": "group/subgroup/repo", + }) + + default: + t.Fatalf("unexpected request path: %s", r.URL.Path) + } + })) + defer server.Close() + + client, _, err := newGitLabClient("token", server.URL) + if err != nil { + t.Fatalf("newGitLabClient failed: %v", err) + } + + activities, issues, err := fetchGitLabProjectActivities( + context.Background(), + client, + map[string]bool{"group/subgroup/repo": true}, + cutoff, + "me", + 42, + nil, + ) + if err != nil { + t.Fatalf("fetchGitLabProjectActivities failed: %v", err) + } + + mrLabels := map[int]string{} + for _, activity := range activities { + mrLabels[activity.MR.Number] = activity.Label + } + + if mrLabels[1] != "Authored" { + t.Fatalf("MR 1 label = %q, want Authored", mrLabels[1]) + } + if mrLabels[2] != "Reviewed" { + t.Fatalf("MR 2 label = %q, want Reviewed", mrLabels[2]) + } + if mrLabels[3] != "Commented" { + t.Fatalf("MR 3 label = %q, want Commented", mrLabels[3]) + } + + if approvalCalls[1] != 0 { + t.Fatalf("MR 1 approval calls = %d, want 0 due to authored/assigned short-circuit", approvalCalls[1]) + } + if approvalCalls[2] != 1 || approvalCalls[3] != 1 { + t.Fatalf("approval calls = %+v, want MR 2 and 3 exactly once", approvalCalls) + } + if mrNoteCalls[2] != 0 { + t.Fatalf("MR 2 notes calls = %d, want 0 because Reviewed outranks note-based labels", mrNoteCalls[2]) + } + if mrNoteCalls[3] != 1 { + t.Fatalf("MR 3 notes calls = %d, want 1", mrNoteCalls[3]) + } + + issueLabels := map[int]string{} + for _, issue := range issues { + issueLabels[issue.Issue.Number] = issue.Label + } + if issueLabels[21] != "Commented" { + t.Fatalf("Issue 21 label = %q, want Commented", issueLabels[21]) + } + if issueLabels[22] != "Mentioned" { + t.Fatalf("Issue 22 label = %q, want Mentioned", issueLabels[22]) + } + if issueNoteCalls[21] != 1 || issueNoteCalls[22] != 1 { + t.Fatalf("issue note calls = %+v, want issue 21 and 22 exactly once", issueNoteCalls) + } +} + +func TestMergeLabelWithPriority_TableDriven(t *testing.T) { + tests := []struct { + name string + labels []string + isPR bool + expected string + }{ + { + name: "PR chooses authored over lower-priority labels", + labels: []string{"Mentioned", "Commented", "Authored", "Assigned"}, + isPR: true, + expected: "Authored", + }, + { + name: "PR chooses reviewed over review requested", + labels: []string{"Review Requested", "Reviewed"}, + isPR: true, + expected: "Reviewed", + }, + { + name: "Issue chooses assigned over commented", + labels: []string{"Commented", "Assigned"}, + isPR: false, + expected: "Assigned", + }, + { + name: "Issue keeps commented over mentioned", + labels: []string{"Mentioned", "Commented"}, + isPR: false, + expected: "Commented", + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + current := "" + for _, label := range tt.labels { + current = mergeLabelWithPriority(current, label, tt.isPR) + } + if current != tt.expected { + t.Fatalf("final label = %q, want %q", current, tt.expected) + } + }) + } +} + +func TestFetchGitLabProjectActivities_LinksIssuesUsingEndpointAndFallback(t *testing.T) { + cutoff := time.Date(2026, 1, 10, 0, 0, 0, 0, time.UTC) + + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + + switch { + case strings.HasPrefix(r.URL.Path, "/api/v4/projects/") && strings.Contains(r.URL.Path, "/merge_requests/") && strings.HasSuffix(r.URL.Path, "/closes_issues"): + iid := parseResourceIID(t, r.URL.Path, "merge_requests", "closes_issues") + if iid == 1 { + w.WriteHeader(http.StatusInternalServerError) + _, _ = w.Write([]byte(`{"message":"endpoint unavailable"}`)) + return + } + if iid == 2 { + _, _ = w.Write([]byte(`[ + {"id": 602, "iid": 22, "title": "Issue via endpoint", "state": "opened", "updated_at": "2026-01-11T10:00:00Z", "references": {"full": "group/subgroup/repo#22"}} + ]`)) + return + } + _, _ = w.Write([]byte(`[]`)) + + case strings.HasPrefix(r.URL.Path, "/api/v4/projects/") && strings.Contains(r.URL.Path, "/merge_requests/") && strings.HasSuffix(r.URL.Path, "/notes"): + iid := parseResourceIID(t, r.URL.Path, "merge_requests", "notes") + if iid == 1 { + _, _ = w.Write([]byte(`[ + {"id": 701, "body": "Follow-up in #21", "author": {"id": 7, "username": "alice"}} + ]`)) + return + } + _, _ = w.Write([]byte(`[]`)) + + case strings.HasPrefix(r.URL.Path, "/api/v4/projects/") && strings.Contains(r.URL.Path, "/merge_requests"): + _, _ = w.Write([]byte(`[ + {"iid": 1, "title": "MR fallback", "description": "no issue refs", "state": "opened", "updated_at": "2026-01-11T12:00:00Z", "web_url": "https://gitlab.example/mr/1", "author": {"id": 42, "username": "me"}}, + {"iid": 2, "title": "MR endpoint", "description": "no refs", "state": "opened", "updated_at": "2026-01-11T13:00:00Z", "web_url": "https://gitlab.example/mr/2", "author": {"id": 42, "username": "me"}} + ]`)) + + case strings.HasPrefix(r.URL.Path, "/api/v4/projects/") && strings.Contains(r.URL.Path, "/issues"): + _, _ = w.Write([]byte(`[ + {"id": 521, "iid": 21, "title": "Issue from fallback", "description": "desc", "state": "opened", "updated_at": "2026-01-11T08:00:00Z", "web_url": "https://gitlab.example/issues/21", "author": {"id": 7, "username": "alice"}}, + {"id": 522, "iid": 22, "title": "Issue from endpoint", "description": "desc", "state": "opened", "updated_at": "2026-01-11T09:00:00Z", "web_url": "https://gitlab.example/issues/22", "author": {"id": 8, "username": "bob"}}, + {"id": 523, "iid": 23, "title": "Standalone issue", "description": "desc", "state": "opened", "updated_at": "2026-01-11T07:00:00Z", "web_url": "https://gitlab.example/issues/23", "author": {"id": 9, "username": "carol"}} + ]`)) + + case strings.HasPrefix(r.URL.Path, "/api/v4/projects/"): + _ = json.NewEncoder(w).Encode(map[string]any{ + "id": 101, + "path_with_namespace": "group/subgroup/repo", + }) + + default: + t.Fatalf("unexpected request path: %s", r.URL.Path) + } + })) + defer server.Close() + + client, _, err := newGitLabClient("token", server.URL) + if err != nil { + t.Fatalf("newGitLabClient failed: %v", err) + } + + activities, issues, err := fetchGitLabProjectActivities( + context.Background(), + client, + map[string]bool{"group/subgroup/repo": true}, + cutoff, + "me", + 42, + nil, + ) + if err != nil { + t.Fatalf("fetchGitLabProjectActivities failed: %v", err) + } + + if len(activities) != 2 { + t.Fatalf("got %d merge request activities, want 2", len(activities)) + } + + mrIssues := map[int]map[int]bool{} + for _, activity := range activities { + linked := map[int]bool{} + for _, issue := range activity.Issues { + linked[issue.Issue.Number] = true + } + mrIssues[activity.MR.Number] = linked + } + + if !mrIssues[1][21] { + t.Fatalf("MR 1 should link fallback issue 21") + } + if !mrIssues[2][22] { + t.Fatalf("MR 2 should link endpoint issue 22") + } + + if len(issues) != 1 || issues[0].Issue.Number != 23 { + t.Fatalf("standalone issues = %+v, want only issue 23", issues) + } +} + +func TestGitLabIssueReferenceKeysFromText_ParsesLocalQualifiedAndURLRefs(t *testing.T) { + refs := gitLabIssueReferenceKeysFromText( + "Fixes #12 and group/subgroup/repo#34 and https://gitlab.example/group/other/-/issues/56 and /-/issues/78", + "group/subgroup/repo", + ) + + expected := []string{ + buildGitLabIssueKey("group/subgroup/repo", 12), + buildGitLabIssueKey("group/subgroup/repo", 34), + buildGitLabIssueKey("group/other", 56), + buildGitLabIssueKey("group/subgroup/repo", 78), + } + + for _, key := range expected { + if _, ok := refs[key]; !ok { + t.Fatalf("missing parsed reference key %q in %+v", key, refs) + } + } +} + +func TestGitLabCLIWithMockServer_ShowsMergeRequestsAndIssues(t *testing.T) { + const ( + mrTitle = "MR E2E Unique Title" + issueTitle = "Issue E2E Unique Title" + ) + updatedAt := time.Now().UTC().Format(time.RFC3339) + + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + + switch { + case r.Method == http.MethodGet && r.URL.Path == "/api/v4/user": + _, _ = w.Write([]byte(`{"id":42,"username":"me"}`)) + + case r.Method == http.MethodGet && r.URL.Path == "/api/v4/projects/101/merge_requests/1/closes_issues": + _, _ = w.Write([]byte(`[]`)) + + case r.Method == http.MethodGet && r.URL.Path == "/api/v4/projects/101/merge_requests": + _, _ = w.Write([]byte(`[ + {"iid":1,"title":"` + mrTitle + `","description":"desc","state":"opened","updated_at":"` + updatedAt + `","web_url":"https://gitlab.example/mr/1","author":{"id":42,"username":"me"}} + ]`)) + + case r.Method == http.MethodGet && r.URL.Path == "/api/v4/projects/101/issues": + _, _ = w.Write([]byte(`[ + {"id":301,"iid":2,"title":"` + issueTitle + `","description":"desc","state":"opened","updated_at":"` + updatedAt + `","web_url":"https://gitlab.example/issues/2","author":{"id":42,"username":"me"}} + ]`)) + + case r.Method == http.MethodGet && strings.HasPrefix(r.URL.Path, "/api/v4/projects/") && !strings.Contains(r.URL.Path, "/merge_requests") && !strings.Contains(r.URL.Path, "/issues"): + _, _ = w.Write([]byte(`{"id":101,"path_with_namespace":"group/subgroup/repo"}`)) + + default: + t.Fatalf("unexpected request path: %s", r.URL.Path) + } + })) + defer server.Close() + + ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second) + defer cancel() + + homeDir := t.TempDir() + configDir := filepath.Join(homeDir, ".gitlab-feed") + if err := os.MkdirAll(configDir, 0o755); err != nil { + t.Fatalf("failed to create config directory: %v", err) + } + envFile := filepath.Join(configDir, ".env") + envContent := strings.Join([]string{ + "GITLAB_BASE_URL=" + server.URL, + "GITLAB_TOKEN=token", + "ALLOWED_REPOS=group/subgroup/repo", + "", + }, "\n") + if err := os.WriteFile(envFile, []byte(envContent), 0o600); err != nil { + t.Fatalf("failed to write test env file: %v", err) + } + + modCache := filepath.Join(homeDir, "gomodcache") + goCache := filepath.Join(homeDir, "gocache") + if err := os.MkdirAll(modCache, 0o755); err != nil { + t.Fatalf("failed to create GOMODCACHE: %v", err) + } + if err := os.MkdirAll(goCache, 0o755); err != nil { + t.Fatalf("failed to create GOCACHE: %v", err) + } + + cmd := exec.CommandContext(ctx, "go", "run", ".", "--debug", "--time", "1d") + cmd.Dir = "/home/wintermute/repos/github-feed" + var stdoutBuf bytes.Buffer + var stderrBuf bytes.Buffer + cmd.Stdout = &stdoutBuf + cmd.Stderr = &stderrBuf + cmd.Env = append(os.Environ(), + "HOME="+homeDir, + "GITLAB_BASE_URL="+server.URL, + "GITLAB_TOKEN=token", + "ALLOWED_REPOS=group/subgroup/repo", + "GOMODCACHE="+modCache, + "GOCACHE="+goCache, + "GOFLAGS=-modcacherw", + ) + + err := cmd.Run() + if ctx.Err() == context.DeadlineExceeded { + t.Fatalf("go run timed out") + } + if err != nil { + t.Fatalf("go run failed: %v\nstdout:\n%s\nstderr:\n%s", err, stdoutBuf.String(), stderrBuf.String()) + } + + output := stdoutBuf.String() + if !strings.Contains(output, mrTitle) { + t.Fatalf("stdout missing MR title %q\nstdout:\n%s", mrTitle, output) + } + if !strings.Contains(output, issueTitle) { + t.Fatalf("stdout missing issue title %q\nstdout:\n%s", issueTitle, output) + } + if !strings.Contains(output, "OPEN PULL REQUESTS:") { + t.Fatalf("stdout missing section header OPEN PULL REQUESTS:\nstdout:\n%s", output) + } +} + +func parseResourceIID(t *testing.T, path string, resource string, suffix string) int64 { + t.Helper() + parts := strings.Split(path, "/") + resourceIndex := -1 + for i := range parts { + if parts[i] == resource { + resourceIndex = i + break + } + } + if resourceIndex == -1 || resourceIndex+1 >= len(parts) { + t.Fatalf("could not parse resource iid from path %q", path) + } + if !strings.HasSuffix(path, "/"+suffix) { + t.Fatalf("path %q missing expected suffix %q", path, suffix) + } + iid, err := strconv.ParseInt(parts[resourceIndex+1], 10, 64) + if err != nil { + t.Fatalf("could not parse iid from path %q: %v", path, err) + } + return iid +} From 937a1896e9d97604171ce01c636c3cd4c13a85fc Mon Sep 17 00:00:00 2001 From: Aidar Siraev Date: Wed, 11 Feb 2026 17:30:28 +0300 Subject: [PATCH 02/11] fix: test cleaned up, readme.md and claude.md updated --- CLAUDE.md | 15 ++- README.md | 319 ++++++++++++++--------------------------------- priority_test.go | 31 ++--- 3 files changed, 118 insertions(+), 247 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index ef682d4..9845190 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -25,13 +25,24 @@ The app loads configuration from: 1) Environment variables 2) `~/.gitlab-feed/.env` (auto-created on first run) -Environment variables take precedence over values in the `.env` file. +Precedence order: +1) CLI flags +2) Environment variables +3) `~/.gitlab-feed/.env` +4) Built-in defaults Environment variables: -- `GITLAB_TOKEN` or `GITLAB_ACTIVITY_TOKEN` +- `GITLAB_TOKEN` or `GITLAB_ACTIVITY_TOKEN` (required online) - `GITLAB_HOST` (optional host override; takes precedence over `GITLAB_BASE_URL`) - `GITLAB_BASE_URL` (optional; default: `https://gitlab.com`) - `ALLOWED_REPOS` (required online; comma-separated `group[/subgroup]/repo`) +- `GITLAB_USERNAME` or `GITLAB_USER` (optional legacy override; user is normally auto-resolved via API) + +Token scopes: +- `read_api` (recommended) +- `api` only if your self-managed instance requires broader scope + +Reference: https://docs.gitlab.com/user/profile/personal_access_tokens/ Database cache: - `~/.gitlab-feed/gitlab.db` (BBolt) diff --git a/README.md b/README.md index bfabdd8..c822547 100644 --- a/README.md +++ b/README.md @@ -1,299 +1,166 @@ -# GitAI - Activity Monitor +# GitLab Feed (gitlab-feed) -A fast, colorful CLI tool for monitoring pull requests and issues across repositories. Track your contributions, reviews, and assignments with real-time progress visualization. +GitLab Feed is a Go CLI for monitoring GitLab merge requests and issues across a bounded set of projects. ## Features -- **Parallel API Calls** - Fetches data concurrently for maximum speed -- **Colorized Output** - Easy-to-read color-coded labels, states, and progress -- **Smart Cross-Referencing** - Automatically links related PRs and issues -- **Real-Time Progress Bar** - Visual feedback with color-coded completion status -- **Comprehensive Search** - Tracks authored, mentioned, assigned, commented, and reviewed items -- **Time Filtering** - View items from the last month by default (configurable with `--time`) -- **Organized Display** - Separates open, merged, and closed items into clear sections +- Parallel API fetching for faster scans +- Colorized, grouped output (open/closed/merged) +- Smart MR/issue cross-reference nesting +- Online mode with local BBolt cache +- Offline mode from cache (`--local`) +- Time-window filtering (`--time 1h|2d|3w|4m|1y`) +- Retry/backoff for GitLab rate-limit and transient API errors ## Installation -### Pre-built Binaries (Recommended) - -Download the latest release for your platform from the [releases page](https://github.com/zveinn/github-feed/releases): - -**macOS** -```bash -# Intel Mac -curl -L https://github.com/zveinn/github-feed/releases/latest/download/gitlab-feed__Darwin_x86_64.tar.gz | tar xz -chmod +x gitlab-feed -sudo mv gitlab-feed /usr/local/bin/ - -# Apple Silicon Mac -curl -L https://github.com/zveinn/github-feed/releases/latest/download/gitlab-feed__Darwin_arm64.tar.gz | tar xz -chmod +x gitlab-feed -sudo mv gitlab-feed /usr/local/bin/ -``` - -**Linux** -```bash -# x86_64 -curl -L https://github.com/zveinn/github-feed/releases/latest/download/gitlab-feed__Linux_x86_64.tar.gz | tar xz -chmod +x gitlab-feed -sudo mv gitlab-feed /usr/local/bin/ - -# ARM64 -curl -L https://github.com/zveinn/github-feed/releases/latest/download/gitlab-feed__Linux_arm64.tar.gz | tar xz -chmod +x gitlab-feed -sudo mv gitlab-feed /usr/local/bin/ - - -``` - -**Windows** - -Download the appropriate `.zip` file from the releases page, extract it, and add `gitlab-feed.exe` to your PATH. - -### Build from Source +### Build from source ```bash go build -o gitlab-feed . ``` -### Release Management +### Pre-built binaries -Releases are automatically built and published via GitHub Actions using GoReleaser: +Download from GitHub Releases: -```bash -# Create a new release -git tag -a v1.0.0 -m "Release v1.0.0" -git push origin v1.0.0 -``` - -This will automatically: -- Build binaries for Linux (amd64, arm64), macOS (Intel, Apple Silicon), and Windows (amd64) -- Generate checksums for all releases -- Create a GitHub release with installation instructions -- Publish all artifacts to the releases page +- https://github.com/zveinn/github-feed/releases ## Configuration -### First Run Setup +Online mode requires: -On first run, GitAI automatically creates a configuration directory at `~/.gitlab-feed/` with: -- `.env` - Configuration file (with helpful template) -- `gitlab.db` - Local database for caching activity data +- `GITLAB_TOKEN` (or `GITLAB_ACTIVITY_TOKEN`) +- `ALLOWED_REPOS` -### API Token Setup +The app loads configuration in this order: -Create a GitLab Personal Access Token with: -- `read_api` (recommended) -- `api` (use this only if your self-managed instance requires it) +1. Environment variables +2. `~/.gitlab-feed/.env` (auto-created on first run) -GitLab.com token page: https://gitlab.com/-/user_settings/personal_access_tokens +Environment variables take precedence over `.env` values. -### Environment Setup +### Environment variables -You can provide your token and optional username in two ways: +- `GITLAB_TOKEN` or `GITLAB_ACTIVITY_TOKEN` (required online) +- `GITLAB_HOST` (optional host override; takes precedence over `GITLAB_BASE_URL`) +- `GITLAB_BASE_URL` (optional base URL, default: `https://gitlab.com`) +- `ALLOWED_REPOS` (required online; comma-separated `group[/subgroup]/repo`) -**Option 1: Configuration File (Recommended)** +### Example `.env` -Edit `~/.gitlab-feed/.env` and add your credentials: ```bash -# Your GitLab Personal Access Token (required for online mode) GITLAB_TOKEN=your_token_here -# Optional fallback token variable -# GITLAB_ACTIVITY_TOKEN=your_token_here - -# Optional username (the tool can also resolve current user via API) -GITLAB_USERNAME=your_username +# Optional host override, e.g. self-managed GitLab +# If set, this overrides GITLAB_BASE_URL. +GITLAB_HOST=http://1.1.1.1 -# Optional for self-managed GitLab (defaults to https://gitlab.com) -# Example: http://10.10.1.207/ +# Optional explicit base URL (defaults to https://gitlab.com) GITLAB_BASE_URL=https://gitlab.com -# Required in online mode: comma-separated project paths -# Format: group[/subgroup]/repo +# Required in online mode ALLOWED_REPOS=team/repo1,platform/backend/repo2 ``` -**Option 2: Environment Variables** -```bash -export GITLAB_TOKEN="your_token_here" -export GITLAB_USERNAME="your_username" # Optional -export GITLAB_BASE_URL="https://gitlab.com" # Optional -export ALLOWED_REPOS="team/repo1,platform/backend/repo2" # Required in online mode -``` - -**Note:** Environment variables take precedence over the `.env` file. - -If both are set, `GITLAB_HOST` takes precedence over `GITLAB_BASE_URL`. - -## Usage - -### Basic Usage +### Token scopes -```bash -# Monitor merge requests and issues from the last month (default, fetches from API) -gitlab-feed +Create a GitLab Personal Access Token with: -# Show items from the last 3 hours -gitlab-feed --time 3h +- `read_api` (recommended) +- `api` only if your self-managed setup requires broader scope -# Show items from the last 2 days -gitlab-feed --time 2d +Reference: -# Show items from the last 3 weeks -gitlab-feed --time 3w +- https://docs.gitlab.com/user/profile/personal_access_tokens/ -# Show items from the last 6 months -gitlab-feed --time 6m +## Usage -# Show items from the last year -gitlab-feed --time 1y +```bash +# Default: last month (1m), online mode +./gitlab-feed -# Show detailed logging output -gitlab-feed --debug +# Time window examples +./gitlab-feed --time 3h +./gitlab-feed --time 2d +./gitlab-feed --time 3w +./gitlab-feed --time 6m +./gitlab-feed --time 1y -# Use local database instead of API (offline mode) -gitlab-feed --local +# Debug output +./gitlab-feed --debug -# Show hyperlinks underneath each merge request/issue -gitlab-feed --links +# Offline from cache +./gitlab-feed --local -# Delete and recreate the database cache (start fresh) -gitlab-feed --clean +# Show links +./gitlab-feed --links -# Filter to specific repositories only -gitlab-feed --allowed-repos="team/repo1,platform/backend/repo2" +# Shortcut: --local --links +./gitlab-feed --ll -# Quick offline mode with links (combines --local and --links) -gitlab-feed --ll +# Recreate cache DB +./gitlab-feed --clean -# Combine flags -gitlab-feed --local --time 2w --debug --links --allowed-repos="team/repo1,platform/backend/repo2" +# Override allowed repos from CLI +./gitlab-feed --allowed-repos "group/repo,group/subgroup/repo" ``` -### Command Line Options +### Flags | Flag | Description | |------|-------------| -| `--time RANGE` | Show items from the last time range (default: `1m`)
Examples: `1h` (hour), `2d` (days), `3w` (weeks), `4m` (months), `1y` (year) | -| `--debug` | Show detailed API call progress instead of progress bar | -| `--local` | Use local database instead of GitLab API (offline mode, no token required) | -| `--links` | Show hyperlinks underneath each PR and issue | -| `--ll` | Shortcut for `--local --links` (offline mode with links) | -| `--clean` | Delete and recreate the database cache (useful for starting fresh or fixing corrupted cache) | -| `--allowed-repos REPOS` | Filter to specific repositories (comma-separated: `group/repo,group/subgroup/repo`) | - -### Color Coding - -**Labels:** -- `AUTHORED` - Cyan -- `MENTIONED` - Yellow -- `ASSIGNED` - Magenta -- `COMMENTED` - Blue -- `REVIEWED` - Green -- `REVIEW REQUESTED` - Red -- `INVOLVED` - Gray - -**States:** -- `OPEN` - Green -- `CLOSED` - Red -- `MERGED` - Magenta - -**Usernames:** Each user gets a consistent color based on hash - -## How It Works - -### Online Mode (Default) - -1. **Scoped Fetching** - Loads merge requests and issues from `ALLOWED_REPOS` project paths - - Uses GitLab identity to label authored/assigned/reviewed/commented/mentioned activity - - Applies your `--time` window to keep output focused +| `--time RANGE` | Show items from last time range (default: `1m`). Examples: `1h`, `2d`, `3w`, `4m`, `1y` | +| `--debug` | Show detailed API logging | +| `--local` | Use local database instead of GitLab API | +| `--links` | Show hyperlinks under each MR/issue | +| `--ll` | Shortcut for `--local --links` | +| `--clean` | Delete and recreate the database cache | +| `--allowed-repos REPOS` | Comma-separated project paths (`group/repo,group/subgroup/repo`) | -2. **Local Caching** - All fetched data is automatically saved to a local BBolt database (`~/.gitlab-feed/gitlab.db`) - - Merge requests, issues, and notes are cached for offline access - - Each item is stored/updated with a unique key - - Database grows as you fetch more data +## Data and cache -3. **Cross-Reference Detection** - Automatically finds connections between merge requests and issues - - Displays linked issues directly under their related merge requests +On first run, the tool creates: -4. **Smart Filtering**: - - Shows both open and closed items from the specified time period - - **Default**: Items updated in last month (`1m`) - - **Custom**: Use `--time` with values like `1h`, `2d`, `3w`, `6m`, `1y` +- `~/.gitlab-feed/.env` +- `~/.gitlab-feed/gitlab.db` -### Offline Mode (`--local`) - -- Reads all data from the local database instead of GitLab API -- No internet connection or token required -- Displays all cached merge requests and issues -- Useful for: - - Working offline - - Faster lookups when you don't need fresh data - - Reviewing previously fetched data - -## API Rate Limits - -GitAI monitors API limits and retries automatically when responses indicate throttling. -Rate limit and retry status is displayed in debug mode. - -### Automatic Retry & Backoff - -When rate limits are hit, GitAI automatically retries with exponential backoff: -- Detects rate limit errors (429, 403 responses) -- Waits progressively longer between retries (1s → 2s → 4s → ... up to 30s max) -- Continues indefinitely until the request succeeds -- Shows clear warnings: `Rate limit hit, waiting [duration] before retry...` -- No manual intervention required - the tool handles rate limits gracefully +Online mode fetches GitLab activity and updates cache. +Offline mode (`--local`) reads from cache only. ## Troubleshooting -### "token is required for GitLab API mode" -Set `GITLAB_TOKEN` (or `GITLAB_ACTIVITY_TOKEN`) as described in [Configuration](#configuration). +### `token is required for GitLab API mode` -### "ALLOWED_REPOS is required for GitLab API mode" -Set `ALLOWED_REPOS` with one or more GitLab project paths in `group[/subgroup]/repo` format. +Set `GITLAB_TOKEN` (or `GITLAB_ACTIVITY_TOKEN`) in env or `~/.gitlab-feed/.env`. -### "Rate limit exceeded" -Wait for the rate limit to reset. Use `--debug` to see current rate limits. +### `ALLOWED_REPOS is required for GitLab API mode` -### Progress bar looks garbled -Your terminal may not support ANSI colors properly. Use `--debug` mode for plain text output. +Set `ALLOWED_REPOS` with valid project paths (`group[/subgroup]/repo`). -## Development +### No open activity found -### Project Structure -``` -gitlab-feed/ -├── main.go # Main application code -├── db.go # Database operations for caching activity data -├── README.md # This file -├── CLAUDE.md # Instructions for Claude Code AI assistant -├── .goreleaser.yml # GoReleaser configuration for builds -├── .github/ -│ └── workflows/ -│ └── release.yml # GitHub Actions workflow for releases - -~/.gitlab-feed/ # Config directory (auto-created) - ├── .env # Configuration file with credentials - └── gitlab.db # BBolt database for caching -``` +Try: -### Testing Releases Locally +- `--debug` to inspect resolved repos and API base URL +- a wider window (for example `--time 24h`) +- verifying `ALLOWED_REPOS` matches exact project paths -You can test the GoReleaser build locally before pushing a tag: +## Development ```bash -# Install goreleaser -go install github.com/goreleaser/goreleaser/v2@latest +go test ./... -count=1 +go build -o gitlab-feed . +``` -# Test the build (creates snapshot without publishing) -goreleaser release --snapshot --clean +Current core files: -# Check the dist/ folder for built binaries -ls -la dist/ -``` +- `main.go` +- `db.go` +- `priority_test.go` +- `CLAUDE.md` +- `README.md` ## License -MIT License - Feel free to use and modify as needed. +MIT diff --git a/priority_test.go b/priority_test.go index 72b4200..47fcae0 100644 --- a/priority_test.go +++ b/priority_test.go @@ -10,7 +10,6 @@ import ( "os" "os/exec" "path/filepath" - "sort" "strconv" "strings" "sync/atomic" @@ -648,8 +647,6 @@ func TestLoadGitLabCachedActivities_OfflineParityFiltersAndOrder(t *testing.T) { t.Fatalf("unexpected issue activity %+v", issueActivities[0]) } - sort.Slice(activities, func(i, j int) bool { return activities[i].UpdatedAt.After(activities[j].UpdatedAt) }) - sort.Slice(issueActivities, func(i, j int) bool { return issueActivities[i].UpdatedAt.After(issueActivities[j].UpdatedAt) }) } func TestLoadGitLabCachedActivities_NestsLinkedIssuesAndExcludesStandalone(t *testing.T) { @@ -1017,26 +1014,14 @@ func TestMergeLabelWithPriority_TableDriven(t *testing.T) { expected string }{ { - name: "PR chooses authored over lower-priority labels", - labels: []string{"Mentioned", "Commented", "Authored", "Assigned"}, + name: "PR fold keeps highest-priority label despite later lower-priority candidates", + labels: []string{"Mentioned", "Authored", "Review Requested", "Commented", "Assigned"}, isPR: true, expected: "Authored", }, { - name: "PR chooses reviewed over review requested", - labels: []string{"Review Requested", "Reviewed"}, - isPR: true, - expected: "Reviewed", - }, - { - name: "Issue chooses assigned over commented", - labels: []string{"Commented", "Assigned"}, - isPR: false, - expected: "Assigned", - }, - { - name: "Issue keeps commented over mentioned", - labels: []string{"Mentioned", "Commented"}, + name: "Issue fold ignores unknown labels and preserves best known label", + labels: []string{"Mentioned", "Commented", "Unknown", "Mentioned"}, isPR: false, expected: "Commented", }, @@ -1173,6 +1158,14 @@ func TestGitLabIssueReferenceKeysFromText_ParsesLocalQualifiedAndURLRefs(t *test t.Fatalf("missing parsed reference key %q in %+v", key, refs) } } + + noiseRefs := gitLabIssueReferenceKeysFromText( + "ignore #0 #x project/repo#-5 /-/issues/0 https://gitlab.example/group/repo/-/issues/not-a-number and text#42", + "group/subgroup/repo", + ) + if len(noiseRefs) != 0 { + t.Fatalf("unexpected refs parsed from noise: %+v", noiseRefs) + } } func TestGitLabCLIWithMockServer_ShowsMergeRequestsAndIssues(t *testing.T) { From dd9eaf8106140a17b51691c818c8633405ba268d Mon Sep 17 00:00:00 2001 From: L0thlorien Date: Wed, 11 Feb 2026 18:18:20 +0300 Subject: [PATCH 03/11] Rename project to GitAI and add fork details Updated project name and added fork information. --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index c822547..3be4363 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,9 @@ -# GitLab Feed (gitlab-feed) +# GitAI - GitLab Activity Monitor GitLab Feed is a Go CLI for monitoring GitLab merge requests and issues across a bounded set of projects. +fork from [GitAI GitHub feed](https://github.com/zveinn/github-feed) + ## Features - Parallel API fetching for faster scans From 39de1c8d9ee698d9f4011662f791b33fb363ac7d Mon Sep 17 00:00:00 2001 From: L0thlorien Date: Wed, 11 Feb 2026 18:35:15 +0300 Subject: [PATCH 04/11] Revise README description for GitLab CLI tool Updated project description for clarity and detail. --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 3be4363..11c7686 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # GitAI - GitLab Activity Monitor -GitLab Feed is a Go CLI for monitoring GitLab merge requests and issues across a bounded set of projects. +A fast, colorful CLI tool for monitoring GitLab pull requests and issues across repositories. Track your contributions, reviews, and assignments with real-time progress visualization. fork from [GitAI GitHub feed](https://github.com/zveinn/github-feed) From 7e48ca0413f7ca9f5eb26069593161c3baa828ab Mon Sep 17 00:00:00 2001 From: Aidar Siraev Date: Wed, 11 Feb 2026 20:31:04 +0300 Subject: [PATCH 05/11] fix: project name fixed in .goreleaser --- .goreleaser.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.goreleaser.yml b/.goreleaser.yml index f528279..88e5f59 100644 --- a/.goreleaser.yml +++ b/.goreleaser.yml @@ -1,5 +1,7 @@ version: 2 +project_name: gitlab-feed + before: hooks: # You may remove this if you don't use go modules. From 1197b06d6877192b4b21fad936792c38effbb98e Mon Sep 17 00:00:00 2001 From: Aidar Siraev Date: Wed, 11 Feb 2026 20:53:40 +0300 Subject: [PATCH 06/11] fix: fixed repo name and link --- .goreleaser.yml | 4 ++-- README.md | 2 +- go.mod | 2 +- priority_test.go | 3 +-- 4 files changed, 5 insertions(+), 6 deletions(-) diff --git a/.goreleaser.yml b/.goreleaser.yml index 88e5f59..e7a6d91 100644 --- a/.goreleaser.yml +++ b/.goreleaser.yml @@ -64,8 +64,8 @@ changelog: release: github: - owner: zveinn - name: github-feed + owner: L0thlorien + name: gitlab-feed draft: false prerelease: auto mode: replace diff --git a/README.md b/README.md index 11c7686..7e3dd7f 100644 --- a/README.md +++ b/README.md @@ -26,7 +26,7 @@ go build -o gitlab-feed . Download from GitHub Releases: -- https://github.com/zveinn/github-feed/releases +- https://github.com/L0thlorien/gitlab-feed/releases ## Configuration diff --git a/go.mod b/go.mod index f29d970..0de1a0b 100644 --- a/go.mod +++ b/go.mod @@ -1,4 +1,4 @@ -module github.com/sveinn/github-feed +module github.com/L0thlorien/gitlab-feed go 1.25 diff --git a/priority_test.go b/priority_test.go index 47fcae0..2569f89 100644 --- a/priority_test.go +++ b/priority_test.go @@ -16,8 +16,8 @@ import ( "testing" "time" - bolt "go.etcd.io/bbolt" gitlab "gitlab.com/gitlab-org/api/client-go" + bolt "go.etcd.io/bbolt" ) func TestPRLabelPriority(t *testing.T) { @@ -1233,7 +1233,6 @@ func TestGitLabCLIWithMockServer_ShowsMergeRequestsAndIssues(t *testing.T) { } cmd := exec.CommandContext(ctx, "go", "run", ".", "--debug", "--time", "1d") - cmd.Dir = "/home/wintermute/repos/github-feed" var stdoutBuf bytes.Buffer var stderrBuf bytes.Buffer cmd.Stdout = &stdoutBuf From 1a89aef695cc4259947fe2450349c075b5b2a327 Mon Sep 17 00:00:00 2001 From: Aidar Siraev Date: Thu, 12 Feb 2026 19:06:18 +0300 Subject: [PATCH 07/11] feat: added github/gitlab platform flag to switch between them --- CLAUDE.md | 39 +- README.md | 74 +- db.go | 190 +++++- go.mod | 3 +- go.sum | 2 + main.go | 1595 ++++---------------------------------------- platform_github.go | 756 +++++++++++++++++++++ platform_gitlab.go | 1479 ++++++++++++++++++++++++++++++++++++++++ priority_test.go | 54 +- 9 files changed, 2685 insertions(+), 1507 deletions(-) create mode 100644 platform_github.go create mode 100644 platform_gitlab.go diff --git a/CLAUDE.md b/CLAUDE.md index 9845190..b1c3ebd 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,6 +1,6 @@ # GitLab Feed (gitlab-feed) -GitLab Feed is a Go CLI for monitoring GitLab merge requests and issues across a bounded set of projects. +GitLab Feed is a Go CLI for monitoring GitHub pull requests and GitLab merge requests and issues across a bounded set of projects. ## Build & Run @@ -8,35 +8,51 @@ GitLab Feed is a Go CLI for monitoring GitLab merge requests and issues across a go build -o gitlab-feed . ./gitlab-feed +./gitlab-feed --platform github +./gitlab-feed --platform gitlab ./gitlab-feed --time 3h ./gitlab-feed --debug ./gitlab-feed --local ./gitlab-feed --links ./gitlab-feed --ll ./gitlab-feed --clean -./gitlab-feed --allowed-repos "group/repo,group/subgroup/repo" +./gitlab-feed --allowed-repos "owner/repo,owner/other" +./gitlab-feed --platform gitlab --allowed-repos "group/repo,group/subgroup/repo" ``` ## Configuration -Online mode requires a GitLab Personal Access Token and `ALLOWED_REPOS`. +Select the platform via `--platform github|gitlab` (default: `github`). + +Online mode requirements depend on platform: + +- GitHub: `GITHUB_TOKEN`, `GITHUB_USERNAME` (and optionally `ALLOWED_REPOS`) +- GitLab: `GITLAB_TOKEN` (or `GITLAB_ACTIVITY_TOKEN`) and `ALLOWED_REPOS` The app loads configuration from: 1) Environment variables -2) `~/.gitlab-feed/.env` (auto-created on first run) +2) Platform `.env` file (auto-created on first run) + - GitHub: `~/.github-feed/.env` + - GitLab: `~/.gitlab-feed/.env` Precedence order: 1) CLI flags 2) Environment variables -3) `~/.gitlab-feed/.env` +3) Platform `.env` file 4) Built-in defaults Environment variables: -- `GITLAB_TOKEN` or `GITLAB_ACTIVITY_TOKEN` (required online) -- `GITLAB_HOST` (optional host override; takes precedence over `GITLAB_BASE_URL`) -- `GITLAB_BASE_URL` (optional; default: `https://gitlab.com`) -- `ALLOWED_REPOS` (required online; comma-separated `group[/subgroup]/repo`) -- `GITLAB_USERNAME` or `GITLAB_USER` (optional legacy override; user is normally auto-resolved via API) +- GitHub + - `GITHUB_TOKEN` (required online) + - `GITHUB_USERNAME` (required online) + - `ALLOWED_REPOS` (optional; comma-separated `owner/repo`) + +- GitLab + - `GITLAB_TOKEN` or `GITLAB_ACTIVITY_TOKEN` (required online) + - `GITLAB_HOST` (optional host override; takes precedence over `GITLAB_BASE_URL`) + - `GITLAB_BASE_URL` (optional; default: `https://gitlab.com`) + - `ALLOWED_REPOS` (required online; comma-separated `group[/subgroup]/repo`) + - `GITLAB_USERNAME` or `GITLAB_USER` (optional legacy override; user is normally auto-resolved via API) Token scopes: - `read_api` (recommended) @@ -45,7 +61,8 @@ Token scopes: Reference: https://docs.gitlab.com/user/profile/personal_access_tokens/ Database cache: -- `~/.gitlab-feed/gitlab.db` (BBolt) +- GitHub: `~/.github-feed/github.db` (BBolt) +- GitLab: `~/.gitlab-feed/gitlab.db` (BBolt) ## Testing diff --git a/README.md b/README.md index 7e3dd7f..ffc81dd 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ -# GitAI - GitLab Activity Monitor +# GitAI - Activity Monitor -A fast, colorful CLI tool for monitoring GitLab pull requests and issues across repositories. Track your contributions, reviews, and assignments with real-time progress visualization. +A fast, colorful CLI tool for monitoring GitHub pull requests and GitLab merge requests and issues across repositories. Track your contributions, reviews, and assignments with real-time progress visualization. fork from [GitAI GitHub feed](https://github.com/zveinn/github-feed) @@ -12,7 +12,7 @@ fork from [GitAI GitHub feed](https://github.com/zveinn/github-feed) - Online mode with local BBolt cache - Offline mode from cache (`--local`) - Time-window filtering (`--time 1h|2d|3w|4m|1y`) -- Retry/backoff for GitLab rate-limit and transient API errors +- Retry/backoff for API rate-limit and transient API errors ## Installation @@ -30,20 +30,32 @@ Download from GitHub Releases: ## Configuration -Online mode requires: +Select the platform via `--platform github|gitlab` (default: `github`). -- `GITLAB_TOKEN` (or `GITLAB_ACTIVITY_TOKEN`) -- `ALLOWED_REPOS` +Online mode requirements depend on platform: -The app loads configuration in this order: +- GitHub: `GITHUB_TOKEN`, `GITHUB_USERNAME` (and optionally `ALLOWED_REPOS`) +- GitLab: `GITLAB_TOKEN` (or `GITLAB_ACTIVITY_TOKEN`) and `ALLOWED_REPOS` -1. Environment variables -2. `~/.gitlab-feed/.env` (auto-created on first run) +The app loads configuration in this order: -Environment variables take precedence over `.env` values. +1. CLI flags +2. Environment variables +3. Platform `.env` file (auto-created on first run) + - GitHub: `~/.github-feed/.env` + - GitLab: `~/.gitlab-feed/.env` +4. Built-in defaults ### Environment variables +GitHub: + +- `GITHUB_TOKEN` (required online) +- `GITHUB_USERNAME` (required online) +- `ALLOWED_REPOS` (optional; comma-separated `owner/repo`) + +GitLab: + - `GITLAB_TOKEN` or `GITLAB_ACTIVITY_TOKEN` (required online) - `GITLAB_HOST` (optional host override; takes precedence over `GITLAB_BASE_URL`) - `GITLAB_BASE_URL` (optional base URL, default: `https://gitlab.com`) @@ -52,6 +64,14 @@ Environment variables take precedence over `.env` values. ### Example `.env` ```bash +# GitHub (`--platform github`) +GITHUB_TOKEN=your_token_here +GITHUB_USERNAME=your_username + +# Optional in online mode; if omitted, the tool relies on platform defaults/behavior. +ALLOWED_REPOS=owner/repo1,owner/repo2 + +# GitLab (`--platform gitlab`) GITLAB_TOKEN=your_token_here # Optional host override, e.g. self-managed GitLab @@ -79,9 +99,13 @@ Reference: ## Usage ```bash -# Default: last month (1m), online mode +# Default: last month (1m), online mode, platform=github ./gitlab-feed +# Explicit platform +./gitlab-feed --platform github +./gitlab-feed --platform gitlab + # Time window examples ./gitlab-feed --time 3h ./gitlab-feed --time 2d @@ -105,7 +129,8 @@ Reference: ./gitlab-feed --clean # Override allowed repos from CLI -./gitlab-feed --allowed-repos "group/repo,group/subgroup/repo" +./gitlab-feed --allowed-repos "owner/repo,owner/other" +./gitlab-feed --platform gitlab --allowed-repos "group/repo,group/subgroup/repo" ``` ### Flags @@ -113,30 +138,39 @@ Reference: | Flag | Description | |------|-------------| | `--time RANGE` | Show items from last time range (default: `1m`). Examples: `1h`, `2d`, `3w`, `4m`, `1y` | +| `--platform PLATFORM` | Activity source platform: `github` or `gitlab` (default: `github`) | | `--debug` | Show detailed API logging | -| `--local` | Use local database instead of GitLab API | +| `--local` | Use local database instead of API | | `--links` | Show hyperlinks under each MR/issue | | `--ll` | Shortcut for `--local --links` | | `--clean` | Delete and recreate the database cache | -| `--allowed-repos REPOS` | Comma-separated project paths (`group/repo,group/subgroup/repo`) | +| `--allowed-repos REPOS` | Comma-separated repo paths (GitHub: `owner/repo`; GitLab: `group[/subgroup]/repo`) | ## Data and cache -On first run, the tool creates: +On first run, the tool creates a platform-specific config dir and cache DB: -- `~/.gitlab-feed/.env` -- `~/.gitlab-feed/gitlab.db` +- GitHub: + - `~/.github-feed/.env` + - `~/.github-feed/github.db` +- GitLab: + - `~/.gitlab-feed/.env` + - `~/.gitlab-feed/gitlab.db` -Online mode fetches GitLab activity and updates cache. +Online mode fetches platform activity and updates cache. Offline mode (`--local`) reads from cache only. ## Troubleshooting -### `token is required for GitLab API mode` +### GitHub online mode missing token/user + +Set `GITHUB_TOKEN` and `GITHUB_USERNAME` in env or `~/.github-feed/.env`. + +### GitLab online mode missing token Set `GITLAB_TOKEN` (or `GITLAB_ACTIVITY_TOKEN`) in env or `~/.gitlab-feed/.env`. -### `ALLOWED_REPOS is required for GitLab API mode` +### GitLab online mode missing `ALLOWED_REPOS` Set `ALLOWED_REPOS` with valid project paths (`group[/subgroup]/repo`). diff --git a/db.go b/db.go index 848d107..22ad8ad 100644 --- a/db.go +++ b/db.go @@ -14,6 +14,9 @@ var ( gitlabMergeRequestsBkt = []byte("gitlab_merge_requests") gitlabIssuesBkt = []byte("gitlab_issues") gitlabNotesBkt = []byte("gitlab_notes") + githubPullRequestsBkt = []byte("pull_requests") + githubIssuesBkt = []byte("issues") + githubCommentsBkt = []byte("comments") ) type Database struct { @@ -38,6 +41,14 @@ func buildGitLabNoteKey(pathWithNamespace, itemType string, iid int, noteID int6 ) } +func buildGitHubItemKey(owner, repo string, number int) string { + return fmt.Sprintf("%s/%s#%d", strings.TrimSpace(owner), strings.TrimSpace(repo), number) +} + +func buildGitHubPRReviewCommentKey(owner, repo string, prNumber int, commentID int64) string { + return fmt.Sprintf("%s/%s#%d/pr_review_comment/%d", strings.TrimSpace(owner), strings.TrimSpace(repo), prNumber, commentID) +} + func (d *Database) save(bucket []byte, key string, data interface{}, debugMode bool, itemType string) error { jsonData, err := json.Marshal(data) if err != nil { @@ -76,7 +87,14 @@ func OpenDatabase(path string) (*Database, error) { } err = db.Update(func(tx *bolt.Tx) error { - buckets := [][]byte{gitlabMergeRequestsBkt, gitlabIssuesBkt, gitlabNotesBkt} + buckets := [][]byte{ + gitlabMergeRequestsBkt, + gitlabIssuesBkt, + gitlabNotesBkt, + githubPullRequestsBkt, + githubIssuesBkt, + githubCommentsBkt, + } for _, bucket := range buckets { _, err := tx.CreateBucketIfNotExists(bucket) if err != nil { @@ -117,6 +135,26 @@ type GitLabNoteRecord struct { AuthorID int64 } +type GitHubPRWithLabel struct { + PR MergeRequestModel + Label string +} + +type GitHubIssueWithLabel struct { + Issue IssueModel + Label string +} + +type GitHubPRReviewCommentRecord struct { + Owner string + Repo string + PRNumber int + CommentID int64 + Body string + AuthorUsername string + AuthorID int64 +} + func (d *Database) SaveGitLabMergeRequestWithLabel(pathWithNamespace string, mr MergeRequestModel, label string, debugMode bool) error { key := buildGitLabMergeRequestKey(pathWithNamespace, mr.Number) item := GitLabMRWithLabel{MR: mr, Label: label} @@ -134,6 +172,23 @@ func (d *Database) SaveGitLabNote(note GitLabNoteRecord, debugMode bool) error { return d.save(gitlabNotesBkt, key, note, debugMode, "gitlab note") } +func (d *Database) SaveGitHubPullRequestWithLabel(owner, repo string, pr MergeRequestModel, label string, debugMode bool) error { + key := buildGitHubItemKey(owner, repo, pr.Number) + item := GitHubPRWithLabel{PR: pr, Label: label} + return d.save(githubPullRequestsBkt, key, item, debugMode, fmt.Sprintf("github pull request with label %s", label)) +} + +func (d *Database) SaveGitHubIssueWithLabel(owner, repo string, issue IssueModel, label string, debugMode bool) error { + key := buildGitHubItemKey(owner, repo, issue.Number) + item := GitHubIssueWithLabel{Issue: issue, Label: label} + return d.save(githubIssuesBkt, key, item, debugMode, fmt.Sprintf("github issue with label %s", label)) +} + +func (d *Database) SaveGitHubPRReviewComment(comment GitHubPRReviewCommentRecord, debugMode bool) error { + key := buildGitHubPRReviewCommentKey(comment.Owner, comment.Repo, comment.PRNumber, comment.CommentID) + return d.save(githubCommentsBkt, key, comment, debugMode, "github pr review comment") +} + func (d *Database) GetAllGitLabMergeRequestsWithLabels(debugMode bool) (map[string]MergeRequestModel, map[string]string, error) { items := make(map[string]MergeRequestModel) labels := make(map[string]string) @@ -210,6 +265,112 @@ func (d *Database) GetAllGitLabIssuesWithLabels(debugMode bool) (map[string]Issu return items, labels, nil } +func (d *Database) GetAllGitHubPullRequestsWithLabels(debugMode bool) (map[string]MergeRequestModel, map[string]string, error) { + items := make(map[string]MergeRequestModel) + labels := make(map[string]string) + + if debugMode { + fmt.Printf(" [DB] Reading all GitHub pull requests with labels from database...\n") + } + + err := d.db.View(func(tx *bolt.Tx) error { + b := tx.Bucket(githubPullRequestsBkt) + if b == nil { + return nil + } + + return b.ForEach(func(k, v []byte) error { + key := string(k) + + var item GitHubPRWithLabel + if err := json.Unmarshal(v, &item); err == nil { + if item.PR.Number != 0 || item.Label != "" { + items[key] = item.PR + labels[key] = item.Label + return nil + } + } + + var pr MergeRequestModel + if err := json.Unmarshal(v, &pr); err != nil { + if debugMode { + fmt.Printf(" [DB] Error unmarshaling github pull request %s: %v\n", key, err) + } + return err + } + + items[key] = pr + labels[key] = "" + return nil + }) + }) + if err != nil { + if debugMode { + fmt.Printf(" [DB] Error reading GitHub pull requests: %v\n", err) + } + return nil, nil, err + } + + if debugMode { + fmt.Printf(" [DB] Loaded %d GitHub pull requests from database\n", len(items)) + } + + return items, labels, nil +} + +func (d *Database) GetAllGitHubIssuesWithLabels(debugMode bool) (map[string]IssueModel, map[string]string, error) { + items := make(map[string]IssueModel) + labels := make(map[string]string) + + if debugMode { + fmt.Printf(" [DB] Reading all GitHub issues with labels from database...\n") + } + + err := d.db.View(func(tx *bolt.Tx) error { + b := tx.Bucket(githubIssuesBkt) + if b == nil { + return nil + } + + return b.ForEach(func(k, v []byte) error { + key := string(k) + + var item GitHubIssueWithLabel + if err := json.Unmarshal(v, &item); err == nil { + if item.Issue.Number != 0 || item.Label != "" { + items[key] = item.Issue + labels[key] = item.Label + return nil + } + } + + var issue IssueModel + if err := json.Unmarshal(v, &issue); err != nil { + if debugMode { + fmt.Printf(" [DB] Error unmarshaling github issue %s: %v\n", key, err) + } + return err + } + + items[key] = issue + labels[key] = "" + return nil + }) + }) + if err != nil { + if debugMode { + fmt.Printf(" [DB] Error reading GitHub issues: %v\n", err) + } + return nil, nil, err + } + + if debugMode { + fmt.Printf(" [DB] Loaded %d GitHub issues from database\n", len(items)) + } + + return items, labels, nil +} + func (d *Database) HasGitLabData() (bool, error) { hasData := false err := d.db.View(func(tx *bolt.Tx) error { @@ -261,3 +422,30 @@ func (d *Database) GetGitLabNotes(pathWithNamespace, itemType string, iid int) ( } return notes, nil } + +func (d *Database) GetGitHubPRReviewComments(owner, repo string, prNumber int) ([]GitHubPRReviewCommentRecord, error) { + comments := make([]GitHubPRReviewCommentRecord, 0) + prefix := fmt.Sprintf("%s/%s#%d/pr_review_comment/", strings.TrimSpace(owner), strings.TrimSpace(repo), prNumber) + + err := d.db.View(func(tx *bolt.Tx) error { + b := tx.Bucket(githubCommentsBkt) + if b == nil { + return nil + } + + c := b.Cursor() + for k, v := c.Seek([]byte(prefix)); k != nil && strings.HasPrefix(string(k), prefix); k, v = c.Next() { + var record GitHubPRReviewCommentRecord + if err := json.Unmarshal(v, &record); err != nil { + return err + } + comments = append(comments, record) + } + return nil + }) + if err != nil { + return nil, err + } + + return comments, nil +} diff --git a/go.mod b/go.mod index 0de1a0b..26038f2 100644 --- a/go.mod +++ b/go.mod @@ -4,8 +4,10 @@ go 1.25 require ( github.com/fatih/color v1.18.0 + github.com/google/go-github/v57 v57.0.0 gitlab.com/gitlab-org/api/client-go v1.30.0 go.etcd.io/bbolt v1.4.3 + golang.org/x/oauth2 v0.34.0 ) require ( @@ -14,7 +16,6 @@ require ( github.com/hashicorp/go-retryablehttp v0.7.8 // indirect github.com/mattn/go-colorable v0.1.13 // indirect github.com/mattn/go-isatty v0.0.20 // indirect - golang.org/x/oauth2 v0.34.0 // indirect golang.org/x/sys v0.39.0 // indirect golang.org/x/time v0.14.0 // indirect ) diff --git a/go.sum b/go.sum index ebc515d..8d76d87 100644 --- a/go.sum +++ b/go.sum @@ -4,6 +4,8 @@ github.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM= github.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU= github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= +github.com/google/go-github/v57 v57.0.0 h1:L+Y3UPTY8ALM8x+TV0lg+IEBI+upibemtBD8Q9u7zHs= +github.com/google/go-github/v57 v57.0.0/go.mod h1:s0omdnye0hvK/ecLvpsGfJMiRt85PimQh4oygmLIxHw= github.com/google/go-querystring v1.2.0 h1:yhqkPbu2/OH+V9BfpCVPZkNmUXhb2gBxJArfhIxNtP0= github.com/google/go-querystring v1.2.0/go.mod h1:8IFJqpSRITyJ8QhQ13bmbeMBDfmeEJZD5A0egEOmkqU= github.com/hashicorp/go-cleanhttp v0.5.2 h1:035FKYIWjmULyFRBKPs8TBQoi0x6d9G4xc9neXJWAZQ= diff --git a/main.go b/main.go index cb2fed4..a8839f4 100644 --- a/main.go +++ b/main.go @@ -3,17 +3,11 @@ package main import ( "bufio" "context" - "errors" "flag" "fmt" "hash/fnv" - "math" - "net/http" - "net/url" "os" "path/filepath" - "regexp" - "sort" "strconv" "strings" "sync/atomic" @@ -73,113 +67,24 @@ type Progress struct { } type Config struct { - debugMode bool - localMode bool - gitlabUserID int64 - showLinks bool - timeRange time.Duration - gitlabUsername string - allowedRepos map[string]bool - gitlabClient *gitlab.Client - db *Database - progress *Progress - ctx context.Context - dbErrorCount atomic.Int32 + debugMode bool + localMode bool + gitlabUserID int64 + githubToken string + githubUsername string + showLinks bool + timeRange time.Duration + gitlabUsername string + allowedRepos map[string]bool + gitlabClient *gitlab.Client + db *Database + progress *Progress + ctx context.Context + dbErrorCount atomic.Int32 } var config Config -var retryAfter = time.After - -const defaultGitLabBaseURL = "https://gitlab.com" - -func normalizeGitLabBaseURL(raw string) (string, error) { - baseURL := strings.TrimSpace(raw) - if baseURL == "" { - baseURL = defaultGitLabBaseURL - } - - parsed, err := url.Parse(baseURL) - if err != nil { - return "", fmt.Errorf("invalid GitLab base URL %q: %w", baseURL, err) - } - - if parsed.Scheme == "" || parsed.Host == "" { - return "", fmt.Errorf("invalid GitLab base URL %q: must include scheme and host", baseURL) - } - - normalizedPath := strings.TrimSuffix(parsed.EscapedPath(), "/") - if normalizedPath == "" { - normalizedPath = "/api/v4" - } else if !strings.HasSuffix(normalizedPath, "/api/v4") { - normalizedPath += "/api/v4" - } - - parsed.Path = normalizedPath - parsed.RawPath = "" - - return parsed.String(), nil -} - -func newGitLabClient(token, rawBaseURL string) (*gitlab.Client, string, error) { - normalizedBaseURL, err := normalizeGitLabBaseURL(rawBaseURL) - if err != nil { - return nil, "", err - } - - client, err := gitlab.NewClient(token, gitlab.WithBaseURL(normalizedBaseURL)) - if err != nil { - return nil, "", fmt.Errorf("failed to create GitLab client: %w", err) - } - - return client, normalizedBaseURL, nil -} - -func getPRLabelPriority(label string) int { - priorities := map[string]int{ - "Authored": 1, - "Assigned": 2, - "Reviewed": 3, - "Review Requested": 4, - "Commented": 5, - "Mentioned": 6, - } - if priority, ok := priorities[label]; ok { - return priority - } - return 999 // Unknown labels get lowest priority -} - -func getIssueLabelPriority(label string) int { - priorities := map[string]int{ - "Authored": 1, - "Assigned": 2, - "Commented": 3, - "Mentioned": 4, - } - if priority, ok := priorities[label]; ok { - return priority - } - return 999 // Unknown labels get lowest priority -} - -func shouldUpdateLabel(currentLabel, newLabel string, isPR bool) bool { - if currentLabel == "" { - return true - } - - var currentPriority, newPriority int - if isPR { - currentPriority = getPRLabelPriority(currentLabel) - newPriority = getPRLabelPriority(newLabel) - } else { - currentPriority = getIssueLabelPriority(currentLabel) - newPriority = getIssueLabelPriority(newLabel) - } - - return newPriority < currentPriority -} - func (p *Progress) increment() { p.current.Add(1) } @@ -235,187 +140,6 @@ func (p *Progress) displayWithWarning(message string) { color.New(color.FgYellow).Sprint("! "+message)) } -func retryWithBackoff(operation func() error, operationName string) error { - const ( - initialBackoff = 1 * time.Second - maxBackoff = 30 * time.Second - backoffFactor = 1.5 - ) - - backoff := initialBackoff - attempt := 1 - retryCtx := config.ctx - if retryCtx == nil { - retryCtx = context.Background() - } - - for { - err := operation() - if err == nil { - return nil - } - - var gitLabErr *gitlab.ErrorResponse - var waitTime time.Duration - var isRateLimitError bool - var isTransientServerError bool - shouldRetry := true - - if errors.As(err, &gitLabErr) && gitLabErr.Response != nil { - statusCode := gitLabErr.Response.StatusCode - - if statusCode == http.StatusTooManyRequests { - isRateLimitError = true - retryAfterSeconds, parseErr := strconv.Atoi(strings.TrimSpace(gitLabErr.Response.Header.Get("Retry-After"))) - if parseErr == nil && retryAfterSeconds > 0 { - waitTime = time.Duration(retryAfterSeconds) * time.Second - } else if resetWait, ok := gitLabRateLimitResetWait(gitLabErr.Response.Header.Get("Ratelimit-Reset")); ok { - waitTime = resetWait - } else { - waitTime = time.Duration(math.Min(float64(backoff), float64(maxBackoff))) - } - - if config.debugMode { - fmt.Printf(" [%s] GitLab rate limit hit (attempt %d), waiting %v before retry...\n", - operationName, attempt, waitTime.Round(time.Second)) - } - } else if statusCode >= http.StatusInternalServerError && statusCode <= 599 { - isTransientServerError = true - waitTime = time.Duration(math.Min(float64(backoff), float64(maxBackoff))) - - if config.debugMode { - fmt.Printf(" [%s] GitLab server error %d (attempt %d), waiting %v before retry...\n", - operationName, statusCode, attempt, waitTime) - } - } else { - shouldRetry = false - } - } else { - isRateLimitError = strings.Contains(err.Error(), "rate limit") || - strings.Contains(err.Error(), "API rate limit exceeded") || - strings.Contains(err.Error(), "403") - - if isRateLimitError { - // Fallback to exponential backoff if we can't extract reset time - waitTime = time.Duration(math.Min(float64(backoff), float64(maxBackoff))) - if config.debugMode { - fmt.Printf(" [%s] Rate limit hit (attempt %d), waiting %v before retry...\n", - operationName, attempt, waitTime) - } - } - } - - if !shouldRetry { - return err - } - - if isRateLimitError { - if config.debugMode { - select { - case <-retryCtx.Done(): - return retryCtx.Err() - case <-retryAfter(waitTime): - } - } else { - ticker := time.NewTicker(1 * time.Second) - defer ticker.Stop() - - remaining := int(waitTime.Seconds()) - for remaining > 0 { - if config.progress != nil { - config.progress.displayWithWarning(fmt.Sprintf("Rate limit hit, retrying in %ds", remaining)) - } - - select { - case <-retryCtx.Done(): - return retryCtx.Err() - case <-ticker.C: - remaining-- - } - } - } - - backoff = time.Duration(float64(backoff) * backoffFactor) - } else if isTransientServerError { - if config.debugMode { - select { - case <-retryCtx.Done(): - return retryCtx.Err() - case <-retryAfter(waitTime): - } - } else { - ticker := time.NewTicker(1 * time.Second) - defer ticker.Stop() - - remaining := int(waitTime.Seconds()) - for remaining > 0 { - if config.progress != nil { - config.progress.displayWithWarning(fmt.Sprintf("API error, retrying in %ds", remaining)) - } - - select { - case <-retryCtx.Done(): - return retryCtx.Err() - case <-ticker.C: - remaining-- - } - } - } - - backoff = time.Duration(float64(backoff) * backoffFactor) - } else { - // Non-rate-limit error - use exponential backoff - waitTime := time.Duration(math.Min(float64(backoff)/2, float64(5*time.Second))) - - if config.debugMode { - fmt.Printf(" [%s] Error (attempt %d): %v, waiting %v before retry...\n", - operationName, attempt, err, waitTime) - select { - case <-retryCtx.Done(): - return retryCtx.Err() - case <-retryAfter(waitTime): - } - } else { - ticker := time.NewTicker(1 * time.Second) - defer ticker.Stop() - - remaining := int(waitTime.Seconds()) - for remaining > 0 { - if config.progress != nil { - config.progress.displayWithWarning(fmt.Sprintf("API error, retrying in %ds", remaining)) - } - - select { - case <-retryCtx.Done(): - return retryCtx.Err() - case <-ticker.C: - remaining-- - } - } - } - - backoff = time.Duration(float64(backoff) * backoffFactor) - } - - attempt++ - } -} - -func gitLabRateLimitResetWait(rawHeader string) (time.Duration, bool) { - resetAtUnix, err := strconv.ParseInt(strings.TrimSpace(rawHeader), 10, 64) - if err != nil || resetAtUnix <= 0 { - return 0, false - } - - resetTime := time.Unix(resetAtUnix, 0) - waitTime := time.Until(resetTime) - if waitTime <= 0 { - return 1 * time.Second, true - } - - return waitTime, true -} - func getLabelColor(label string) *color.Color { labelColors := map[string]*color.Color{ "Authored": color.New(color.FgCyan), @@ -532,6 +256,7 @@ func parseTimeRange(timeStr string) (time.Duration, error) { func main() { // Define flags var timeRangeStr string + var platform string var debugMode bool var localMode bool var showLinks bool @@ -540,6 +265,7 @@ func main() { var cleanCache bool flag.StringVar(&timeRangeStr, "time", "1m", "Show items from last time range (1h, 2d, 3w, 4m, 1y)") + flag.StringVar(&platform, "platform", "github", "Platform to use (gitlab|github)") flag.BoolVar(&debugMode, "debug", false, "Show detailed API logging") flag.BoolVar(&localMode, "local", false, "Use local database instead of GitLab API") flag.BoolVar(&showLinks, "links", false, "Show hyperlinks underneath each PR/issue") @@ -558,9 +284,12 @@ func main() { fmt.Fprintln(os.Stderr, " GITLAB_USERNAME or GITLAB_USER - Optional GitLab username") fmt.Fprintln(os.Stderr, " GITLAB_HOST - Optional GitLab host (overrides GITLAB_BASE_URL when set)") fmt.Fprintln(os.Stderr, " GITLAB_BASE_URL - Optional GitLab base URL (default: https://gitlab.com)") + fmt.Fprintln(os.Stderr, " GITHUB_TOKEN - GitHub Personal Access Token") + fmt.Fprintln(os.Stderr, " GITHUB_USERNAME - Required in GitHub online mode") fmt.Fprintln(os.Stderr, " ALLOWED_REPOS - Required in online mode (group[/subgroup]/repo)") fmt.Fprintln(os.Stderr, "\nConfiguration File:") - fmt.Fprintln(os.Stderr, " ~/.gitlab-feed/.env - Configuration file (auto-created)") + fmt.Fprintln(os.Stderr, " ~/.gitlab-feed/.env - GitLab configuration file (auto-created)") + fmt.Fprintln(os.Stderr, " ~/.github-feed/.env - GitHub configuration file (auto-created)") } flag.Parse() @@ -571,6 +300,12 @@ func main() { showLinks = true } + platform = strings.ToLower(strings.TrimSpace(platform)) + if platform != "gitlab" && platform != "github" { + fmt.Printf("Error: invalid --platform value %q (allowed: gitlab|github)\n", platform) + os.Exit(1) + } + // Parse time range timeRange, err := parseTimeRange(timeRangeStr) if err != nil { @@ -585,15 +320,26 @@ func main() { os.Exit(1) } - configDir := filepath.Join(homeDir, ".gitlab-feed") - if err := os.MkdirAll(configDir, 0o755); err != nil { - fmt.Printf("Error: Could not create config directory %s: %v\n", configDir, err) - os.Exit(1) - } + configDir := filepath.Join(homeDir, ".github-feed") + dbFileName := "github.db" + envTemplate := `# Activity Feed Configuration +# Add your API credentials here - envPath := filepath.Join(configDir, ".env") - if _, err := os.Stat(envPath); os.IsNotExist(err) { - envTemplate := `# Activity Feed Configuration +# Your GitHub Personal Access Token (required for online mode) +GITHUB_TOKEN= + +# Required in online mode +GITHUB_USERNAME= + +# Optional: comma-separated allowed repos +# Example: owner/repo,owner/another-repo +ALLOWED_REPOS= +` + + if platform == "gitlab" { + configDir = filepath.Join(homeDir, ".gitlab-feed") + dbFileName = "gitlab.db" + envTemplate = `# Activity Feed Configuration # Add your API credentials here # Your GitLab Personal Access Token (required for online mode) @@ -617,6 +363,15 @@ GITLAB_USERNAME= # Example self-managed repo path: platform/backend/gitlab-feed ALLOWED_REPOS= ` + } + + if err := os.MkdirAll(configDir, 0o755); err != nil { + fmt.Printf("Error: Could not create config directory %s: %v\n", configDir, err) + os.Exit(1) + } + + envPath := filepath.Join(configDir, ".env") + if _, err := os.Stat(envPath); os.IsNotExist(err) { if err := os.WriteFile(envPath, []byte(envTemplate), 0o600); err != nil { fmt.Printf("Warning: Could not create .env file at %s: %v\n", envPath, err) } @@ -644,7 +399,7 @@ ALLOWED_REPOS= } } - dbPath := filepath.Join(configDir, "gitlab.db") + dbPath := filepath.Join(configDir, dbFileName) if cleanCache { fmt.Println("Cleaning database cache...") @@ -668,32 +423,49 @@ ALLOWED_REPOS= defer db.Close() } - token := os.Getenv("GITLAB_ACTIVITY_TOKEN") - if token == "" { - token = os.Getenv("GITLAB_TOKEN") + var token string + if platform == "gitlab" { + token = os.Getenv("GITLAB_ACTIVITY_TOKEN") + if token == "" { + token = os.Getenv("GITLAB_TOKEN") + } + } else { + token = os.Getenv("GITHUB_TOKEN") } - rawGitLabHost := os.Getenv("GITLAB_HOST") - rawGitLabBaseURL := os.Getenv("GITLAB_BASE_URL") - selectedGitLabBaseURL := rawGitLabBaseURL - if strings.TrimSpace(rawGitLabHost) != "" { - selectedGitLabBaseURL = rawGitLabHost - } + githubUsername := strings.TrimSpace(os.Getenv("GITHUB_USERNAME")) - normalizedGitLabBaseURL, err := normalizeGitLabBaseURL(selectedGitLabBaseURL) - if err != nil { - if strings.TrimSpace(selectedGitLabBaseURL) != "" { - fmt.Printf("Configuration Error: %v\n", err) - os.Exit(1) + normalizedGitLabBaseURL := "" + if platform == "gitlab" { + rawGitLabHost := os.Getenv("GITLAB_HOST") + rawGitLabBaseURL := os.Getenv("GITLAB_BASE_URL") + selectedGitLabBaseURL := rawGitLabBaseURL + if strings.TrimSpace(rawGitLabHost) != "" { + selectedGitLabBaseURL = rawGitLabHost } - normalizedGitLabBaseURL, _ = normalizeGitLabBaseURL("") + normalizedGitLabBaseURL, err = normalizeGitLabBaseURL(selectedGitLabBaseURL) + if err != nil { + if strings.TrimSpace(selectedGitLabBaseURL) != "" { + fmt.Printf("Configuration Error: %v\n", err) + os.Exit(1) + } + + normalizedGitLabBaseURL, _ = normalizeGitLabBaseURL("") + } } var gitlabClient *gitlab.Client gitlabUsername := "" var gitlabUserID int64 - if !localMode && token != "" { + if platform == "gitlab" && !localMode && token != "" { + rawGitLabHost := os.Getenv("GITLAB_HOST") + rawGitLabBaseURL := os.Getenv("GITLAB_BASE_URL") + selectedGitLabBaseURL := rawGitLabBaseURL + if strings.TrimSpace(rawGitLabHost) != "" { + selectedGitLabBaseURL = rawGitLabHost + } + client, _, err := newGitLabClient(token, selectedGitLabBaseURL) if err != nil { fmt.Printf("Configuration Error: %v\n", err) @@ -715,15 +487,19 @@ ALLOWED_REPOS= } // Validate configuration - if err := validateConfig(token, localMode, envPath, allowedRepos); err != nil { + if err := validateConfig(platform, token, githubUsername, localMode, envPath, allowedRepos); err != nil { fmt.Printf("Configuration Error: %v\n\n", err) os.Exit(1) } if debugMode { - fmt.Println("Monitoring GitLab merge request and issue activity") + if platform == "gitlab" { + fmt.Println("Monitoring GitLab merge request and issue activity") + fmt.Printf("GitLab API base URL: %s\n", normalizedGitLabBaseURL) + } else { + fmt.Println("Monitoring GitHub pull request and issue activity") + } fmt.Printf("Showing items from the last %v\n", timeRange) - fmt.Printf("GitLab API base URL: %s\n", normalizedGitLabBaseURL) } if debugMode { fmt.Println("Debug mode enabled") @@ -732,6 +508,8 @@ ALLOWED_REPOS= config.debugMode = debugMode config.localMode = localMode config.gitlabUserID = gitlabUserID + config.githubToken = token + config.githubUsername = githubUsername config.showLinks = showLinks config.timeRange = timeRange config.gitlabUsername = gitlabUsername @@ -740,25 +518,44 @@ ALLOWED_REPOS= config.ctx = context.Background() config.gitlabClient = gitlabClient - fetchAndDisplayActivity() + fetchAndDisplayActivity(platform) } -func validateConfig(token string, localMode bool, envPath string, allowedRepos map[string]bool) error { +func validateConfig(platform, token, githubUsername string, localMode bool, envPath string, allowedRepos map[string]bool) error { if localMode { return nil // No validation needed for offline mode } - if token == "" { - return fmt.Errorf("token is required for GitLab API mode.\n\nTo fix this:\n - Set GITLAB_TOKEN or GITLAB_ACTIVITY_TOKEN\n - Or add it to %s", envPath) - } - if len(allowedRepos) == 0 { - return fmt.Errorf("ALLOWED_REPOS is required for GitLab API mode to keep API usage bounded.\n\nTo fix this:\n - Set ALLOWED_REPOS with group[/subgroup]/repo paths\n - Example: ALLOWED_REPOS=team/service,platform/backend/gitlab-feed\n - Or add it to %s", envPath) + switch platform { + case "gitlab": + if token == "" { + return fmt.Errorf("token is required for GitLab API mode.\n\nTo fix this:\n - Set GITLAB_TOKEN or GITLAB_ACTIVITY_TOKEN\n - Or add it to %s", envPath) + } + if len(allowedRepos) == 0 { + return fmt.Errorf("ALLOWED_REPOS is required for GitLab API mode to keep API usage bounded.\n\nTo fix this:\n - Set ALLOWED_REPOS with group[/subgroup]/repo paths\n - Example: ALLOWED_REPOS=team/service,platform/backend/gitlab-feed\n - Or add it to %s", envPath) + } + case "github": + if token == "" { + return fmt.Errorf("token is required for GitHub API mode.\n\nTo fix this:\n - Set GITHUB_TOKEN\n - Or add it to %s", envPath) + } + if githubUsername == "" { + return fmt.Errorf("username is required for GitHub API mode.\n\nTo fix this:\n - Set GITHUB_USERNAME\n - Or add it to %s", envPath) + } + default: + return fmt.Errorf("unsupported platform %q", platform) } return nil } -func fetchAndDisplayActivity() { - fetchAndDisplayGitLabActivity() +func fetchAndDisplayActivity(platform string) { + switch platform { + case "gitlab": + fetchAndDisplayGitLabActivity() + case "github": + fetchAndDisplayGitHubActivity() + default: + fmt.Printf("Unsupported platform: %s\n", platform) + } } type DisplayConfig struct { @@ -850,1145 +647,3 @@ func displayIssue(label, owner, repo string, issue IssueModel, indented bool, ha State: issue.State, }) } - -type gitLabProject struct { - PathWithNamespace string - ID int64 -} - -func fetchAndDisplayGitLabActivity() { - startTime := time.Now() - - if config.debugMode { - fmt.Println("Fetching data from GitLab...") - } else { - fmt.Print("Fetching data from GitLab... ") - } - - cutoffTime := time.Now().Add(-config.timeRange) - var ( - activities []PRActivity - issueActivities []IssueActivity - err error - ) - - if config.localMode { - activities, issueActivities, err = loadGitLabCachedActivities(cutoffTime) - } else { - activities, issueActivities, err = fetchGitLabProjectActivities( - config.ctx, - config.gitlabClient, - config.allowedRepos, - cutoffTime, - config.gitlabUsername, - config.gitlabUserID, - config.db, - ) - } - if err != nil { - fmt.Printf("Error fetching GitLab activity: %v\n", err) - return - } - - if config.debugMode { - fmt.Println() - fmt.Printf("Total fetch time: %v\n", time.Since(startTime).Round(time.Millisecond)) - fmt.Printf("Found %d unique merge requests and %d unique issues\n", len(activities), len(issueActivities)) - fmt.Println() - } else { - fmt.Print("\r" + strings.Repeat(" ", 80) + "\r") - } - - if len(activities) == 0 && len(issueActivities) == 0 { - fmt.Println("No open activity found") - return - } - - sort.Slice(activities, func(i, j int) bool { - return activities[i].UpdatedAt.After(activities[j].UpdatedAt) - }) - sort.Slice(issueActivities, func(i, j int) bool { - return issueActivities[i].UpdatedAt.After(issueActivities[j].UpdatedAt) - }) - - var openPRs, closedPRs, mergedPRs []PRActivity - for _, activity := range activities { - if activity.MR.State == "closed" { - if activity.MR.Merged { - mergedPRs = append(mergedPRs, activity) - } else { - closedPRs = append(closedPRs, activity) - } - } else { - openPRs = append(openPRs, activity) - } - } - - var openIssues, closedIssues []IssueActivity - for _, issue := range issueActivities { - if issue.Issue.State == "closed" { - closedIssues = append(closedIssues, issue) - } else { - openIssues = append(openIssues, issue) - } - } - - if len(openPRs) > 0 { - titleColor := color.New(color.FgHiGreen, color.Bold) - fmt.Println(titleColor.Sprint("OPEN PULL REQUESTS:")) - fmt.Println("------------------------------------------") - for _, activity := range openPRs { - displayMergeRequest(activity.Label, activity.Owner, activity.Repo, activity.MR, activity.HasUpdates) - if len(activity.Issues) > 0 { - for _, issue := range activity.Issues { - displayIssue(issue.Label, issue.Owner, issue.Repo, issue.Issue, true, issue.HasUpdates) - } - } - } - } - - if len(closedPRs) > 0 || len(mergedPRs) > 0 { - fmt.Println() - titleColor := color.New(color.FgHiRed, color.Bold) - fmt.Println(titleColor.Sprint("CLOSED/MERGED PULL REQUESTS:")) - fmt.Println("------------------------------------------") - for _, activity := range mergedPRs { - displayMergeRequest(activity.Label, activity.Owner, activity.Repo, activity.MR, activity.HasUpdates) - if len(activity.Issues) > 0 { - for _, issue := range activity.Issues { - displayIssue(issue.Label, issue.Owner, issue.Repo, issue.Issue, true, issue.HasUpdates) - } - } - } - for _, activity := range closedPRs { - displayMergeRequest(activity.Label, activity.Owner, activity.Repo, activity.MR, activity.HasUpdates) - if len(activity.Issues) > 0 { - for _, issue := range activity.Issues { - displayIssue(issue.Label, issue.Owner, issue.Repo, issue.Issue, true, issue.HasUpdates) - } - } - } - } - - if len(openIssues) > 0 { - fmt.Println() - titleColor := color.New(color.FgHiGreen, color.Bold) - fmt.Println(titleColor.Sprint("OPEN ISSUES:")) - fmt.Println("------------------------------------------") - for _, issue := range openIssues { - displayIssue(issue.Label, issue.Owner, issue.Repo, issue.Issue, false, issue.HasUpdates) - } - } - - if len(closedIssues) > 0 { - fmt.Println() - titleColor := color.New(color.FgHiRed, color.Bold) - fmt.Println(titleColor.Sprint("CLOSED ISSUES:")) - fmt.Println("------------------------------------------") - for _, issue := range closedIssues { - displayIssue(issue.Label, issue.Owner, issue.Repo, issue.Issue, false, issue.HasUpdates) - } - } -} - -func fetchGitLabProjectActivities( - ctx context.Context, - client *gitlab.Client, - allowedRepos map[string]bool, - cutoff time.Time, - currentUsername string, - currentUserID int64, - db *Database, -) ([]PRActivity, []IssueActivity, error) { - projects, err := resolveAllowedGitLabProjects(ctx, client, allowedRepos) - if err != nil { - return nil, nil, err - } - - currentUsername = strings.TrimSpace(currentUsername) - if currentUsername == "" { - return nil, nil, fmt.Errorf("gitlab current username is required") - } - - if len(projects) == 0 { - return []PRActivity{}, []IssueActivity{}, nil - } - - activities := make([]PRActivity, 0) - issueActivities := make([]IssueActivity, 0) - seenMergeRequests := make(map[string]struct{}) - seenIssues := make(map[string]struct{}) - projectIDByPath := make(map[string]int64, len(projects)) - mrNotesByKey := make(map[string][]*gitlab.Note) - - for _, project := range projects { - projectIDByPath[normalizeProjectPathWithNamespace(project.PathWithNamespace)] = project.ID - } - - for _, project := range projects { - projectMergeRequests, err := listGitLabProjectMergeRequests(ctx, client, project.ID, cutoff) - if err != nil { - return nil, nil, fmt.Errorf("list merge requests for %s: %w", project.PathWithNamespace, err) - } - - for _, item := range projectMergeRequests { - key := buildGitLabDedupKey(project.PathWithNamespace, "mr", item.IID) - if _, exists := seenMergeRequests[key]; exists { - continue - } - seenMergeRequests[key] = struct{}{} - - model := toMergeRequestModelFromGitLab(item) - if model.UpdatedAt.IsZero() || model.UpdatedAt.Before(cutoff) { - continue - } - - label, notes, err := deriveGitLabMergeRequestLabel(ctx, client, project.ID, item, currentUsername, currentUserID) - if err != nil { - return nil, nil, fmt.Errorf("derive merge request label for %s!%d: %w", project.PathWithNamespace, item.IID, err) - } - - if db != nil { - if err := db.SaveGitLabMergeRequestWithLabel(project.PathWithNamespace, model, label, config.debugMode); err != nil { - config.dbErrorCount.Add(1) - if config.debugMode { - fmt.Printf(" [DB] Warning: Failed to save GitLab MR %s!%d: %v\n", project.PathWithNamespace, item.IID, err) - } - } - if err := persistGitLabNotes(db, project.PathWithNamespace, "mr", int(item.IID), notes); err != nil { - config.dbErrorCount.Add(1) - if config.debugMode { - fmt.Printf(" [DB] Warning: Failed to save GitLab MR notes %s!%d: %v\n", project.PathWithNamespace, item.IID, err) - } - } - } - - mrNotesByKey[buildGitLabMergeRequestKey(project.PathWithNamespace, model.Number)] = notes - - activities = append(activities, PRActivity{ - Label: label, - Owner: project.PathWithNamespace, - Repo: "", - MR: model, - UpdatedAt: model.UpdatedAt, - }) - } - - projectIssues, err := listGitLabProjectIssues(ctx, client, project.ID, cutoff) - if err != nil { - return nil, nil, fmt.Errorf("list issues for %s: %w", project.PathWithNamespace, err) - } - - for _, item := range projectIssues { - key := buildGitLabDedupKey(project.PathWithNamespace, "issue", item.IID) - if _, exists := seenIssues[key]; exists { - continue - } - seenIssues[key] = struct{}{} - - model := toIssueModelFromGitLab(item) - if model.UpdatedAt.IsZero() || model.UpdatedAt.Before(cutoff) { - continue - } - - label, notes, err := deriveGitLabIssueLabel(ctx, client, project.ID, item, currentUsername, currentUserID) - if err != nil { - return nil, nil, fmt.Errorf("derive issue label for %s#%d: %w", project.PathWithNamespace, item.IID, err) - } - - if db != nil { - if err := db.SaveGitLabIssueWithLabel(project.PathWithNamespace, model, label, config.debugMode); err != nil { - config.dbErrorCount.Add(1) - if config.debugMode { - fmt.Printf(" [DB] Warning: Failed to save GitLab issue %s#%d: %v\n", project.PathWithNamespace, item.IID, err) - } - } - if err := persistGitLabNotes(db, project.PathWithNamespace, "issue", int(item.IID), notes); err != nil { - config.dbErrorCount.Add(1) - if config.debugMode { - fmt.Printf(" [DB] Warning: Failed to save GitLab issue notes %s#%d: %v\n", project.PathWithNamespace, item.IID, err) - } - } - } - - issueActivities = append(issueActivities, IssueActivity{ - Label: label, - Owner: project.PathWithNamespace, - Repo: "", - Issue: model, - UpdatedAt: model.UpdatedAt, - }) - } - } - - activities, issueActivities, err = linkGitLabCrossReferencesOnline(ctx, client, activities, issueActivities, projectIDByPath, mrNotesByKey, db) - if err != nil { - return nil, nil, err - } - - return activities, issueActivities, nil -} - -func deriveGitLabMergeRequestLabel( - ctx context.Context, - client *gitlab.Client, - projectID int64, - item *gitlab.BasicMergeRequest, - currentUsername string, - currentUserID int64, -) (string, []*gitlab.Note, error) { - if item == nil { - return "Involved", nil, nil - } - - currentLabel := "" - if matchesGitLabBasicUser(item.Author, currentUsername, currentUserID) { - currentLabel = mergeLabelWithPriority(currentLabel, "Authored", true) - } - if gitLabBasicUserListContains(item.Assignees, currentUsername, currentUserID) || matchesGitLabBasicUser(item.Assignee, currentUsername, currentUserID) { - currentLabel = mergeLabelWithPriority(currentLabel, "Assigned", true) - } - - if currentLabel == "Authored" || currentLabel == "Assigned" { - return currentLabel, nil, nil - } - - var approvalState *gitlab.MergeRequestApprovalState - err := retryWithBackoff(func() error { - var apiErr error - approvalState, _, apiErr = client.MergeRequestApprovals.GetApprovalState(projectID, item.IID, gitlab.WithContext(ctx)) - return apiErr - }, fmt.Sprintf("GitLabGetApprovalState %d!%d", projectID, item.IID)) - if err != nil { - return "", nil, err - } - if gitLabApprovalStateReviewedByCurrentUser(approvalState, currentUsername, currentUserID) { - currentLabel = mergeLabelWithPriority(currentLabel, "Reviewed", true) - } - - if gitLabBasicUserListContains(item.Reviewers, currentUsername, currentUserID) { - currentLabel = mergeLabelWithPriority(currentLabel, "Review Requested", true) - } - - if !needsLowerPriorityPRChecks(currentLabel) { - if currentLabel == "" { - return "Involved", nil, nil - } - return currentLabel, nil, nil - } - - notes, err := listAllGitLabMergeRequestNotes(ctx, client, projectID, item.IID) - if err != nil { - return "", nil, err - } - - commented, mentioned := gitLabNotesInvolvement(notes, item.Description, currentUsername, currentUserID) - if commented { - currentLabel = mergeLabelWithPriority(currentLabel, "Commented", true) - } - if mentioned { - currentLabel = mergeLabelWithPriority(currentLabel, "Mentioned", true) - } - - if currentLabel == "" { - return "Involved", notes, nil - } - return currentLabel, notes, nil -} - -func deriveGitLabIssueLabel( - ctx context.Context, - client *gitlab.Client, - projectID int64, - item *gitlab.Issue, - currentUsername string, - currentUserID int64, -) (string, []*gitlab.Note, error) { - if item == nil { - return "Involved", nil, nil - } - - currentLabel := "" - if matchesGitLabIssueAuthor(item.Author, currentUsername, currentUserID) { - currentLabel = mergeLabelWithPriority(currentLabel, "Authored", false) - } - if gitLabIssueAssigneeListContains(item.Assignees, currentUsername, currentUserID) || matchesGitLabIssueAssignee(item.Assignee, currentUsername, currentUserID) { - currentLabel = mergeLabelWithPriority(currentLabel, "Assigned", false) - } - - if currentLabel == "Authored" || currentLabel == "Assigned" { - return currentLabel, nil, nil - } - - notes, err := listAllGitLabIssueNotes(ctx, client, projectID, item.IID) - if err != nil { - return "", nil, err - } - - commented, mentioned := gitLabNotesInvolvement(notes, item.Description, currentUsername, currentUserID) - if commented { - currentLabel = mergeLabelWithPriority(currentLabel, "Commented", false) - } - if mentioned { - currentLabel = mergeLabelWithPriority(currentLabel, "Mentioned", false) - } - - if currentLabel == "" { - return "Involved", notes, nil - } - return currentLabel, notes, nil -} - -func persistGitLabNotes(db *Database, projectPath, itemType string, itemIID int, notes []*gitlab.Note) error { - if db == nil || len(notes) == 0 { - return nil - } - - for _, note := range notes { - if note == nil { - continue - } - - authorUsername := "" - authorID := int64(0) - author := note.Author - authorUsername = strings.TrimSpace(author.Username) - authorID = author.ID - - record := GitLabNoteRecord{ - ProjectPath: projectPath, - ItemType: itemType, - ItemIID: itemIID, - NoteID: int64(note.ID), - Body: note.Body, - AuthorUsername: authorUsername, - AuthorID: authorID, - } - - if err := db.SaveGitLabNote(record, config.debugMode); err != nil { - return err - } - } - - return nil -} - -func loadGitLabCachedActivities(cutoff time.Time) ([]PRActivity, []IssueActivity, error) { - if config.db == nil { - return []PRActivity{}, []IssueActivity{}, nil - } - - allMRs, mrLabels, err := config.db.GetAllGitLabMergeRequestsWithLabels(config.debugMode) - if err != nil { - return nil, nil, err - } - - activities := make([]PRActivity, 0, len(allMRs)) - for key, mr := range allMRs { - if mr.UpdatedAt.IsZero() || mr.UpdatedAt.Before(cutoff) { - continue - } - - projectPath, ok := parseGitLabMRProjectPath(key) - if !ok || !isGitLabProjectAllowed(projectPath) { - continue - } - - activities = append(activities, PRActivity{ - Label: mrLabels[key], - Owner: projectPath, - Repo: "", - MR: mr, - UpdatedAt: mr.UpdatedAt, - }) - } - - allIssues, issueLabels, err := config.db.GetAllGitLabIssuesWithLabels(config.debugMode) - if err != nil { - return nil, nil, err - } - - issueActivities := make([]IssueActivity, 0, len(allIssues)) - for key, issue := range allIssues { - if issue.UpdatedAt.IsZero() || issue.UpdatedAt.Before(cutoff) { - continue - } - - projectPath, ok := parseGitLabIssueProjectPath(key) - if !ok || !isGitLabProjectAllowed(projectPath) { - continue - } - - issueActivities = append(issueActivities, IssueActivity{ - Label: issueLabels[key], - Owner: projectPath, - Repo: "", - Issue: issue, - UpdatedAt: issue.UpdatedAt, - }) - } - - activities, issueActivities, err = linkGitLabCrossReferencesOffline(config.db, activities, issueActivities) - if err != nil { - return nil, nil, err - } - - return activities, issueActivities, nil -} - -var ( - gitLabIssueSameProjectRefPattern = regexp.MustCompile(`(?i)(?:^|[^a-z0-9_])#([0-9]+)\b`) - gitLabIssueQualifiedRefPattern = regexp.MustCompile(`(?i)([a-z0-9_.-]+(?:/[a-z0-9_.-]+)+)#([0-9]+)\b`) - gitLabIssueURLRefPattern = regexp.MustCompile(`(?i)https?://[^\s]+/([a-z0-9_.-]+(?:/[a-z0-9_.-]+)+)/-/issues/([0-9]+)\b`) - gitLabIssueRelativeURLRefPattern = regexp.MustCompile(`(?i)/-/issues/([0-9]+)\b`) -) - -func linkGitLabCrossReferencesOnline( - ctx context.Context, - client *gitlab.Client, - activities []PRActivity, - issueActivities []IssueActivity, - projectIDByPath map[string]int64, - mrNotesByKey map[string][]*gitlab.Note, - db *Database, -) ([]PRActivity, []IssueActivity, error) { - mrToIssueKeys := make(map[string]map[string]struct{}, len(activities)) - - for _, activity := range activities { - projectPath := normalizeProjectPathWithNamespace(activity.Owner) - projectID, ok := projectIDByPath[projectPath] - if !ok { - continue - } - - mrKey := buildGitLabMergeRequestKey(projectPath, activity.MR.Number) - closedIssues, err := listGitLabIssuesClosedOnMergeRequest(ctx, client, projectID, int64(activity.MR.Number)) - if err == nil { - resolvedKeys := make(map[string]struct{}) - for _, item := range closedIssues { - issueKey, ok := gitLabIssueKeyFromIssue(item, projectPath) - if !ok { - continue - } - resolvedKeys[issueKey] = struct{}{} - } - if len(resolvedKeys) > 0 { - mrToIssueKeys[mrKey] = resolvedKeys - } - continue - } - - fallbackKeys := gitLabIssueReferenceKeysFromText(activity.MR.Body, projectPath) - if len(fallbackKeys) == 0 { - notes := mrNotesByKey[mrKey] - if len(notes) == 0 { - notes, err = listAllGitLabMergeRequestNotes(ctx, client, projectID, int64(activity.MR.Number)) - if err == nil { - mrNotesByKey[mrKey] = notes - if db != nil { - if persistErr := persistGitLabNotes(db, projectPath, "mr", activity.MR.Number, notes); persistErr != nil { - config.dbErrorCount.Add(1) - if config.debugMode { - fmt.Printf(" [DB] Warning: Failed to save GitLab MR notes %s!%d: %v\n", projectPath, activity.MR.Number, persistErr) - } - } - } - } - } - - for _, note := range notes { - if note == nil { - continue - } - for issueKey := range gitLabIssueReferenceKeysFromText(note.Body, projectPath) { - fallbackKeys[issueKey] = struct{}{} - } - } - } - - if len(fallbackKeys) > 0 { - mrToIssueKeys[mrKey] = fallbackKeys - } - } - - nestedActivities := nestGitLabIssues(activities, issueActivities, mrToIssueKeys) - return nestedActivities, filterStandaloneGitLabIssues(nestedActivities, issueActivities), nil -} - -func linkGitLabCrossReferencesOffline(db *Database, activities []PRActivity, issueActivities []IssueActivity) ([]PRActivity, []IssueActivity, error) { - mrToIssueKeys := make(map[string]map[string]struct{}, len(activities)) - - for _, activity := range activities { - projectPath := normalizeProjectPathWithNamespace(activity.Owner) - mrKey := buildGitLabMergeRequestKey(projectPath, activity.MR.Number) - linked := gitLabIssueReferenceKeysFromText(activity.MR.Body, projectPath) - if len(linked) == 0 && db != nil { - notes, err := db.GetGitLabNotes(projectPath, "mr", activity.MR.Number) - if err != nil { - return nil, nil, err - } - for _, note := range notes { - for issueKey := range gitLabIssueReferenceKeysFromText(note.Body, projectPath) { - linked[issueKey] = struct{}{} - } - } - } - - if len(linked) > 0 { - mrToIssueKeys[mrKey] = linked - } - } - - nestedActivities := nestGitLabIssues(activities, issueActivities, mrToIssueKeys) - return nestedActivities, filterStandaloneGitLabIssues(nestedActivities, issueActivities), nil -} - -func listGitLabIssuesClosedOnMergeRequest(ctx context.Context, client *gitlab.Client, projectID int64, mergeRequestIID int64) ([]*gitlab.Issue, error) { - allIssues := make([]*gitlab.Issue, 0) - opts := &gitlab.GetIssuesClosedOnMergeOptions{ListOptions: gitlab.ListOptions{PerPage: 100, Page: 1}} - - for { - issues, resp, err := client.MergeRequests.GetIssuesClosedOnMerge(projectID, mergeRequestIID, opts, gitlab.WithContext(ctx)) - if err != nil { - return nil, err - } - allIssues = append(allIssues, issues...) - if resp == nil || resp.NextPage == 0 { - break - } - opts.Page = resp.NextPage - } - - return allIssues, nil -} - -func nestGitLabIssues(activities []PRActivity, issueActivities []IssueActivity, mrToIssueKeys map[string]map[string]struct{}) []PRActivity { - issueByKey := make(map[string]IssueActivity, len(issueActivities)) - for _, issue := range issueActivities { - issueByKey[buildGitLabIssueKey(issue.Owner, issue.Issue.Number)] = issue - } - - for i := range activities { - activities[i].Issues = nil - mrKey := buildGitLabMergeRequestKey(activities[i].Owner, activities[i].MR.Number) - linkedKeys := mrToIssueKeys[mrKey] - if len(linkedKeys) == 0 { - continue - } - for issueKey := range linkedKeys { - issue, ok := issueByKey[issueKey] - if !ok { - continue - } - activities[i].Issues = append(activities[i].Issues, issue) - } - sort.Slice(activities[i].Issues, func(a, b int) bool { - return activities[i].Issues[a].UpdatedAt.After(activities[i].Issues[b].UpdatedAt) - }) - } - - return activities -} - -func filterStandaloneGitLabIssues(activities []PRActivity, issueActivities []IssueActivity) []IssueActivity { - linkedIssueKeys := make(map[string]struct{}) - for _, activity := range activities { - for _, issue := range activity.Issues { - linkedIssueKeys[buildGitLabIssueKey(issue.Owner, issue.Issue.Number)] = struct{}{} - } - } - - standalone := make([]IssueActivity, 0, len(issueActivities)) - for _, issue := range issueActivities { - issueKey := buildGitLabIssueKey(issue.Owner, issue.Issue.Number) - if _, linked := linkedIssueKeys[issueKey]; linked { - continue - } - standalone = append(standalone, issue) - } - - return standalone -} - -func gitLabIssueReferenceKeysFromText(text, defaultProjectPath string) map[string]struct{} { - results := make(map[string]struct{}) - if strings.TrimSpace(text) == "" { - return results - } - - for _, match := range gitLabIssueURLRefPattern.FindAllStringSubmatch(text, -1) { - if len(match) < 3 { - continue - } - iid, ok := parsePositiveInt(match[2]) - if !ok { - continue - } - results[buildGitLabIssueKey(match[1], iid)] = struct{}{} - } - - for _, match := range gitLabIssueQualifiedRefPattern.FindAllStringSubmatch(text, -1) { - if len(match) < 3 { - continue - } - iid, ok := parsePositiveInt(match[2]) - if !ok { - continue - } - results[buildGitLabIssueKey(match[1], iid)] = struct{}{} - } - - defaultProjectPath = normalizeProjectPathWithNamespace(defaultProjectPath) - if defaultProjectPath != "" { - for _, match := range gitLabIssueRelativeURLRefPattern.FindAllStringSubmatch(text, -1) { - if len(match) < 2 { - continue - } - iid, ok := parsePositiveInt(match[1]) - if !ok { - continue - } - results[buildGitLabIssueKey(defaultProjectPath, iid)] = struct{}{} - } - - for _, match := range gitLabIssueSameProjectRefPattern.FindAllStringSubmatch(text, -1) { - if len(match) < 2 { - continue - } - iid, ok := parsePositiveInt(match[1]) - if !ok { - continue - } - results[buildGitLabIssueKey(defaultProjectPath, iid)] = struct{}{} - } - } - - return results -} - -func gitLabIssueKeyFromIssue(item *gitlab.Issue, defaultProjectPath string) (string, bool) { - if item == nil || item.IID <= 0 { - return "", false - } - - if item.References != nil { - if projectPath, iid, ok := parseGitLabQualifiedReference(item.References.Full); ok { - return buildGitLabIssueKey(projectPath, iid), true - } - } - - defaultProjectPath = normalizeProjectPathWithNamespace(defaultProjectPath) - if defaultProjectPath == "" { - return "", false - } - return buildGitLabIssueKey(defaultProjectPath, int(item.IID)), true -} - -func parseGitLabQualifiedReference(reference string) (string, int, bool) { - for _, match := range gitLabIssueQualifiedRefPattern.FindAllStringSubmatch(reference, -1) { - if len(match) < 3 { - continue - } - iid, ok := parsePositiveInt(match[2]) - if !ok { - continue - } - return normalizeProjectPathWithNamespace(match[1]), iid, true - } - return "", 0, false -} - -func parsePositiveInt(raw string) (int, bool) { - value, err := strconv.Atoi(strings.TrimSpace(raw)) - if err != nil || value <= 0 { - return 0, false - } - return value, true -} - -func parseGitLabMRProjectPath(key string) (string, bool) { - idx := strings.LastIndex(key, "#!") - if idx <= 0 { - return "", false - } - return key[:idx], true -} - -func parseGitLabIssueProjectPath(key string) (string, bool) { - idx := strings.LastIndex(key, "##") - if idx <= 0 { - return "", false - } - return key[:idx], true -} - -func isGitLabProjectAllowed(projectPath string) bool { - if config.allowedRepos == nil || len(config.allowedRepos) == 0 { - return true - } - - normalized := normalizeProjectPathWithNamespace(projectPath) - for repo := range config.allowedRepos { - if strings.EqualFold(normalizeProjectPathWithNamespace(repo), normalized) { - return true - } - } - - return false -} - -func needsLowerPriorityPRChecks(currentLabel string) bool { - return shouldUpdateLabel(currentLabel, "Commented", true) || shouldUpdateLabel(currentLabel, "Mentioned", true) -} - -func mergeLabelWithPriority(currentLabel, candidateLabel string, isPR bool) string { - if shouldUpdateLabel(currentLabel, candidateLabel, isPR) { - return candidateLabel - } - return currentLabel -} - -func listAllGitLabMergeRequestNotes(ctx context.Context, client *gitlab.Client, projectID int64, mrIID int64) ([]*gitlab.Note, error) { - allNotes := make([]*gitlab.Note, 0) - options := &gitlab.ListMergeRequestNotesOptions{ - ListOptions: gitlab.ListOptions{PerPage: 100, Page: 1}, - } - - for { - var ( - notes []*gitlab.Note - response *gitlab.Response - ) - err := retryWithBackoff(func() error { - var apiErr error - notes, response, apiErr = client.Notes.ListMergeRequestNotes(projectID, mrIID, options, gitlab.WithContext(ctx)) - return apiErr - }, fmt.Sprintf("GitLabListMergeRequestNotes %d!%d page %d", projectID, mrIID, options.Page)) - if err != nil { - return nil, err - } - allNotes = append(allNotes, notes...) - - if response == nil || response.NextPage == 0 { - break - } - options.Page = response.NextPage - } - - return allNotes, nil -} - -func listAllGitLabIssueNotes(ctx context.Context, client *gitlab.Client, projectID int64, issueIID int64) ([]*gitlab.Note, error) { - allNotes := make([]*gitlab.Note, 0) - options := &gitlab.ListIssueNotesOptions{ - ListOptions: gitlab.ListOptions{PerPage: 100, Page: 1}, - } - - for { - var ( - notes []*gitlab.Note - response *gitlab.Response - ) - err := retryWithBackoff(func() error { - var apiErr error - notes, response, apiErr = client.Notes.ListIssueNotes(projectID, issueIID, options, gitlab.WithContext(ctx)) - return apiErr - }, fmt.Sprintf("GitLabListIssueNotes %d#%d page %d", projectID, issueIID, options.Page)) - if err != nil { - return nil, err - } - allNotes = append(allNotes, notes...) - - if response == nil || response.NextPage == 0 { - break - } - options.Page = response.NextPage - } - - return allNotes, nil -} - -func gitLabNotesInvolvement(notes []*gitlab.Note, description, currentUsername string, currentUserID int64) (bool, bool) { - commented := false - mentioned := containsGitLabUserMention(description, currentUsername) - - for _, note := range notes { - if note == nil { - continue - } - if matchesGitLabNoteAuthor(note.Author, currentUsername, currentUserID) { - commented = true - } - if !mentioned && containsGitLabUserMention(note.Body, currentUsername) { - mentioned = true - } - if commented && mentioned { - break - } - } - - return commented, mentioned -} - -func containsGitLabUserMention(text, username string) bool { - if text == "" || username == "" { - return false - } - needle := "@" + strings.ToLower(strings.TrimSpace(username)) - if needle == "@" { - return false - } - return strings.Contains(strings.ToLower(text), needle) -} - -func matchesGitLabNoteAuthor(author gitlab.NoteAuthor, username string, userID int64) bool { - if userID > 0 && author.ID == userID { - return true - } - return strings.EqualFold(strings.TrimSpace(author.Username), strings.TrimSpace(username)) -} - -func matchesGitLabBasicUser(user *gitlab.BasicUser, username string, userID int64) bool { - if user == nil { - return false - } - if userID > 0 && user.ID == userID { - return true - } - return strings.EqualFold(strings.TrimSpace(user.Username), strings.TrimSpace(username)) -} - -func matchesGitLabIssueAuthor(author *gitlab.IssueAuthor, username string, userID int64) bool { - if author == nil { - return false - } - if userID > 0 && author.ID == userID { - return true - } - return strings.EqualFold(strings.TrimSpace(author.Username), strings.TrimSpace(username)) -} - -func matchesGitLabIssueAssignee(assignee *gitlab.IssueAssignee, username string, userID int64) bool { - if assignee == nil { - return false - } - if userID > 0 && assignee.ID == userID { - return true - } - return strings.EqualFold(strings.TrimSpace(assignee.Username), strings.TrimSpace(username)) -} - -func gitLabIssueAssigneeListContains(assignees []*gitlab.IssueAssignee, username string, userID int64) bool { - for _, assignee := range assignees { - if matchesGitLabIssueAssignee(assignee, username, userID) { - return true - } - } - return false -} - -func gitLabBasicUserListContains(users []*gitlab.BasicUser, username string, userID int64) bool { - for _, user := range users { - if matchesGitLabBasicUser(user, username, userID) { - return true - } - } - return false -} - -func gitLabApprovalStateReviewedByCurrentUser(state *gitlab.MergeRequestApprovalState, username string, userID int64) bool { - if state == nil { - return false - } - for _, rule := range state.Rules { - if rule == nil { - continue - } - if gitLabBasicUserListContains(rule.ApprovedBy, username, userID) { - return true - } - } - return false -} - -func resolveAllowedGitLabProjects(ctx context.Context, client *gitlab.Client, allowedRepos map[string]bool) ([]gitLabProject, error) { - if client == nil { - return nil, fmt.Errorf("gitlab client is not configured") - } - - if len(allowedRepos) == 0 { - return []gitLabProject{}, nil - } - - repoPaths := make([]string, 0, len(allowedRepos)) - for repo := range allowedRepos { - normalized := normalizeProjectPathWithNamespace(repo) - if normalized != "" { - repoPaths = append(repoPaths, normalized) - } - } - sort.Strings(repoPaths) - - projectIDCache := make(map[string]int64, len(repoPaths)) - projects := make([]gitLabProject, 0, len(repoPaths)) - for _, pathWithNamespace := range repoPaths { - if id, ok := projectIDCache[pathWithNamespace]; ok { - projects = append(projects, gitLabProject{PathWithNamespace: pathWithNamespace, ID: id}) - continue - } - - var project *gitlab.Project - err := retryWithBackoff(func() error { - var apiErr error - project, _, apiErr = client.Projects.GetProject(pathWithNamespace, nil, gitlab.WithContext(ctx)) - return apiErr - }, fmt.Sprintf("GitLabGetProject %s", pathWithNamespace)) - if err != nil { - return nil, fmt.Errorf("resolve project %s: %w", pathWithNamespace, err) - } - - projectIDCache[pathWithNamespace] = project.ID - projects = append(projects, gitLabProject{PathWithNamespace: pathWithNamespace, ID: project.ID}) - } - - return projects, nil -} - -func listGitLabProjectMergeRequests(ctx context.Context, client *gitlab.Client, projectID int64, cutoff time.Time) ([]*gitlab.BasicMergeRequest, error) { - allItems := make([]*gitlab.BasicMergeRequest, 0) - options := &gitlab.ListProjectMergeRequestsOptions{ - ListOptions: gitlab.ListOptions{PerPage: 100, Page: 1}, - State: gitlab.Ptr("all"), - UpdatedAfter: &cutoff, - } - - for { - var ( - items []*gitlab.BasicMergeRequest - response *gitlab.Response - ) - err := retryWithBackoff(func() error { - var apiErr error - items, response, apiErr = client.MergeRequests.ListProjectMergeRequests(projectID, options, gitlab.WithContext(ctx)) - return apiErr - }, fmt.Sprintf("GitLabListProjectMergeRequests %d page %d", projectID, options.Page)) - if err != nil { - return nil, err - } - allItems = append(allItems, items...) - - if response == nil || response.NextPage == 0 { - break - } - options.Page = response.NextPage - } - - return allItems, nil -} - -func listGitLabProjectIssues(ctx context.Context, client *gitlab.Client, projectID int64, cutoff time.Time) ([]*gitlab.Issue, error) { - allItems := make([]*gitlab.Issue, 0) - options := &gitlab.ListProjectIssuesOptions{ - ListOptions: gitlab.ListOptions{PerPage: 100, Page: 1}, - State: gitlab.Ptr("all"), - UpdatedAfter: &cutoff, - } - - for { - var ( - items []*gitlab.Issue - response *gitlab.Response - ) - err := retryWithBackoff(func() error { - var apiErr error - items, response, apiErr = client.Issues.ListProjectIssues(projectID, options, gitlab.WithContext(ctx)) - return apiErr - }, fmt.Sprintf("GitLabListProjectIssues %d page %d", projectID, options.Page)) - if err != nil { - return nil, err - } - allItems = append(allItems, items...) - - if response == nil || response.NextPage == 0 { - break - } - options.Page = response.NextPage - } - - return allItems, nil -} - -func normalizeProjectPathWithNamespace(repo string) string { - trimmed := strings.TrimSpace(repo) - return strings.Trim(trimmed, "/") -} - -func buildGitLabDedupKey(projectPath, itemType string, iid int64) string { - return fmt.Sprintf("%s|%s|%d", strings.ToLower(projectPath), itemType, iid) -} - -func toMergeRequestModelFromGitLab(item *gitlab.BasicMergeRequest) MergeRequestModel { - if item == nil { - return MergeRequestModel{} - } - - state := strings.ToLower(item.State) - merged := state == "merged" || item.MergedAt != nil - normalizedState := "open" - if merged || state == "closed" { - normalizedState = "closed" - } - - updatedAt := time.Time{} - if item.UpdatedAt != nil { - updatedAt = *item.UpdatedAt - } - - userLogin := "" - if item.Author != nil { - userLogin = item.Author.Username - } - - return MergeRequestModel{ - Number: int(item.IID), - Title: item.Title, - Body: item.Description, - State: normalizedState, - UpdatedAt: updatedAt, - WebURL: item.WebURL, - UserLogin: userLogin, - Merged: merged, - } -} - -func toIssueModelFromGitLab(item *gitlab.Issue) IssueModel { - if item == nil { - return IssueModel{} - } - - state := strings.ToLower(item.State) - normalizedState := "open" - if state == "closed" { - normalizedState = "closed" - } - - updatedAt := time.Time{} - if item.UpdatedAt != nil { - updatedAt = *item.UpdatedAt - } - - userLogin := "" - if item.Author != nil { - userLogin = item.Author.Username - } - - return IssueModel{ - Number: int(item.IID), - Title: item.Title, - Body: item.Description, - State: normalizedState, - UpdatedAt: updatedAt, - WebURL: item.WebURL, - UserLogin: userLogin, - } -} diff --git a/platform_github.go b/platform_github.go new file mode 100644 index 0000000..2ec11a6 --- /dev/null +++ b/platform_github.go @@ -0,0 +1,756 @@ +package main + +import ( + "context" + "fmt" + "net/url" + "regexp" + "sort" + "strconv" + "strings" + "time" + + "github.com/fatih/color" + "github.com/google/go-github/v57/github" + "golang.org/x/oauth2" +) + +var ( + githubCrossRefKeywordPattern = regexp.MustCompile(`(?i)\b(?:fix(?:e[sd])?|close[sd]?|resolve[sd]?)\b`) + githubCrossRefSameRefPattern = regexp.MustCompile(`(?i)(?:^|[^a-z0-9_])#([0-9]+)\b`) + githubCrossRefQualifiedRef = regexp.MustCompile(`(?i)\b([a-z0-9_.-]+/[a-z0-9_.-]+)#([0-9]+)\b`) + githubCrossRefURLPattern = regexp.MustCompile(`(?i)https?://github\.com/([a-z0-9_.-]+)/([a-z0-9_.-]+)/(?:issues|pull)/([0-9]+)\b`) +) + +func fetchAndDisplayGitHubActivity() { + startTime := time.Now() + + if config.debugMode { + fmt.Println("Fetching data from GitHub...") + } else { + fmt.Print("Fetching data from GitHub... ") + } + + cutoffTime := time.Now().Add(-config.timeRange) + var ( + activities []PRActivity + issueActivities []IssueActivity + err error + ) + + if config.localMode { + activities, issueActivities, err = loadGitHubCachedActivities(cutoffTime) + } else { + ctx := config.ctx + if ctx == nil { + ctx = context.Background() + } + activities, issueActivities, err = fetchGitHubActivitiesOnline(ctx, cutoffTime) + } + if err != nil { + fmt.Printf("Error fetching GitHub activity: %v\n", err) + return + } + + if config.debugMode { + fmt.Println() + fmt.Printf("Total fetch time: %v\n", time.Since(startTime).Round(time.Millisecond)) + fmt.Printf("Found %d unique pull requests and %d unique issues\n", len(activities), len(issueActivities)) + fmt.Println() + } else { + fmt.Print("\r" + strings.Repeat(" ", 80) + "\r") + } + + if len(activities) == 0 && len(issueActivities) == 0 { + fmt.Println("No open activity found") + return + } + + sort.Slice(activities, func(i, j int) bool { + return activities[i].UpdatedAt.After(activities[j].UpdatedAt) + }) + sort.Slice(issueActivities, func(i, j int) bool { + return issueActivities[i].UpdatedAt.After(issueActivities[j].UpdatedAt) + }) + + var openPRs, closedPRs, mergedPRs []PRActivity + for _, activity := range activities { + if activity.MR.State == "closed" { + if activity.MR.Merged { + mergedPRs = append(mergedPRs, activity) + } else { + closedPRs = append(closedPRs, activity) + } + } else { + openPRs = append(openPRs, activity) + } + } + + var openIssues, closedIssues []IssueActivity + for _, issue := range issueActivities { + if issue.Issue.State == "closed" { + closedIssues = append(closedIssues, issue) + } else { + openIssues = append(openIssues, issue) + } + } + + if len(openPRs) > 0 { + titleColor := color.New(color.FgHiGreen, color.Bold) + fmt.Println(titleColor.Sprint("OPEN PULL REQUESTS:")) + fmt.Println("------------------------------------------") + for _, activity := range openPRs { + displayMergeRequest(activity.Label, activity.Owner, activity.Repo, activity.MR, activity.HasUpdates) + for _, issue := range activity.Issues { + displayIssue(issue.Label, issue.Owner, issue.Repo, issue.Issue, true, issue.HasUpdates) + } + } + } + + if len(closedPRs) > 0 || len(mergedPRs) > 0 { + fmt.Println() + titleColor := color.New(color.FgHiRed, color.Bold) + fmt.Println(titleColor.Sprint("CLOSED/MERGED PULL REQUESTS:")) + fmt.Println("------------------------------------------") + for _, activity := range mergedPRs { + displayMergeRequest(activity.Label, activity.Owner, activity.Repo, activity.MR, activity.HasUpdates) + for _, issue := range activity.Issues { + displayIssue(issue.Label, issue.Owner, issue.Repo, issue.Issue, true, issue.HasUpdates) + } + } + for _, activity := range closedPRs { + displayMergeRequest(activity.Label, activity.Owner, activity.Repo, activity.MR, activity.HasUpdates) + for _, issue := range activity.Issues { + displayIssue(issue.Label, issue.Owner, issue.Repo, issue.Issue, true, issue.HasUpdates) + } + } + } + + if len(openIssues) > 0 { + fmt.Println() + titleColor := color.New(color.FgHiGreen, color.Bold) + fmt.Println(titleColor.Sprint("OPEN ISSUES:")) + fmt.Println("------------------------------------------") + for _, issue := range openIssues { + displayIssue(issue.Label, issue.Owner, issue.Repo, issue.Issue, false, issue.HasUpdates) + } + } + + if len(closedIssues) > 0 { + fmt.Println() + titleColor := color.New(color.FgHiRed, color.Bold) + fmt.Println(titleColor.Sprint("CLOSED ISSUES:")) + fmt.Println("------------------------------------------") + for _, issue := range closedIssues { + displayIssue(issue.Label, issue.Owner, issue.Repo, issue.Issue, false, issue.HasUpdates) + } + } +} + +func fetchGitHubActivitiesOnline(ctx context.Context, cutoff time.Time) ([]PRActivity, []IssueActivity, error) { + client := newGitHubClient(config.githubToken) + dateFilter := cutoff.Format("2006-01-02") + + prActivities, prReviewComments, err := collectGitHubPRSearchResults(ctx, client, config.githubUsername, dateFilter, cutoff) + if err != nil { + return nil, nil, err + } + + issueActivities, err := collectGitHubIssueSearchResults(ctx, client, config.githubUsername, dateFilter, cutoff) + if err != nil { + return nil, nil, err + } + + nestedPRs := nestGitHubIssues(prActivities, issueActivities, prReviewComments) + standaloneIssues := filterStandaloneGitHubIssues(nestedPRs, issueActivities) + return nestedPRs, standaloneIssues, nil +} + +func collectGitHubPRSearchResults( + ctx context.Context, + client *github.Client, + username, dateFilter string, + cutoff time.Time, +) ([]PRActivity, map[string][]GitHubPRReviewCommentRecord, error) { + queries := []struct { + Label string + Query string + }{ + {Label: "Reviewed", Query: fmt.Sprintf("is:pr reviewed-by:%s updated:>=%s", username, dateFilter)}, + {Label: "Review Requested", Query: fmt.Sprintf("is:pr review-requested:%s updated:>=%s", username, dateFilter)}, + {Label: "Authored", Query: fmt.Sprintf("is:pr author:%s updated:>=%s", username, dateFilter)}, + {Label: "Assigned", Query: fmt.Sprintf("is:pr assignee:%s updated:>=%s", username, dateFilter)}, + {Label: "Commented", Query: fmt.Sprintf("is:pr commenter:%s updated:>=%s", username, dateFilter)}, + {Label: "Mentioned", Query: fmt.Sprintf("is:pr mentions:%s updated:>=%s", username, dateFilter)}, + } + + byKey := make(map[string]PRActivity) + prReviewComments := make(map[string][]GitHubPRReviewCommentRecord) + + for _, q := range queries { + items, err := searchGitHubIssues(ctx, client, q.Query) + if err != nil { + return nil, nil, fmt.Errorf("search pull requests for %s: %w", q.Label, err) + } + + for _, item := range items { + if item == nil || item.GetPullRequestLinks() == nil { + continue + } + owner, repo, ok := parseGitHubRepoFromSearchItem(item) + if !ok || !isGitHubRepoAllowed(owner, repo) { + continue + } + + pr, err := getGitHubPullRequest(ctx, client, owner, repo, item.GetNumber()) + if err != nil { + return nil, nil, err + } + model := toMergeRequestModelFromGitHubPR(pr) + if model.UpdatedAt.IsZero() || model.UpdatedAt.Before(cutoff) { + continue + } + + key := buildGitHubItemKey(owner, repo, model.Number) + activity, exists := byKey[key] + if !exists { + activity = PRActivity{Owner: owner, Repo: repo, MR: model, UpdatedAt: model.UpdatedAt} + } else { + if model.UpdatedAt.After(activity.UpdatedAt) { + activity.UpdatedAt = model.UpdatedAt + } + activity.MR = model + } + if shouldUpdateLabel(activity.Label, q.Label, true) { + activity.Label = q.Label + } + + if config.db != nil { + if err := config.db.SaveGitHubPullRequestWithLabel(owner, repo, model, activity.Label, config.debugMode); err != nil { + config.dbErrorCount.Add(1) + if config.debugMode { + fmt.Printf(" [DB] Warning: Failed to save GitHub PR %s/%s#%d: %v\n", owner, repo, model.Number, err) + } + } + } + + reviewComments, err := listGitHubPRReviewComments(ctx, client, owner, repo, model.Number) + if err != nil { + return nil, nil, err + } + records := make([]GitHubPRReviewCommentRecord, 0, len(reviewComments)) + for _, comment := range reviewComments { + record := toGitHubPRReviewCommentRecord(owner, repo, model.Number, comment) + records = append(records, record) + if config.db != nil { + if err := config.db.SaveGitHubPRReviewComment(record, config.debugMode); err != nil { + config.dbErrorCount.Add(1) + if config.debugMode { + fmt.Printf(" [DB] Warning: Failed to save GitHub PR review comment %s/%s#%d/%d: %v\n", owner, repo, model.Number, record.CommentID, err) + } + } + } + } + prReviewComments[key] = records + + byKey[key] = activity + } + } + + activities := make([]PRActivity, 0, len(byKey)) + for _, activity := range byKey { + activities = append(activities, activity) + } + return activities, prReviewComments, nil +} + +func collectGitHubIssueSearchResults( + ctx context.Context, + client *github.Client, + username, dateFilter string, + cutoff time.Time, +) ([]IssueActivity, error) { + queries := []struct { + Label string + Query string + }{ + {Label: "Authored", Query: fmt.Sprintf("is:issue author:%s updated:>=%s", username, dateFilter)}, + {Label: "Mentioned", Query: fmt.Sprintf("is:issue mentions:%s updated:>=%s", username, dateFilter)}, + {Label: "Assigned", Query: fmt.Sprintf("is:issue assignee:%s updated:>=%s", username, dateFilter)}, + {Label: "Commented", Query: fmt.Sprintf("is:issue commenter:%s updated:>=%s", username, dateFilter)}, + } + + byKey := make(map[string]IssueActivity) + + for _, q := range queries { + items, err := searchGitHubIssues(ctx, client, q.Query) + if err != nil { + return nil, fmt.Errorf("search issues for %s: %w", q.Label, err) + } + + for _, item := range items { + if item == nil || item.GetPullRequestLinks() != nil { + continue + } + owner, repo, ok := parseGitHubRepoFromSearchItem(item) + if !ok || !isGitHubRepoAllowed(owner, repo) { + continue + } + + issue, err := getGitHubIssue(ctx, client, owner, repo, item.GetNumber()) + if err != nil { + return nil, err + } + model := toIssueModelFromGitHubIssue(issue) + if model.UpdatedAt.IsZero() || model.UpdatedAt.Before(cutoff) { + continue + } + + key := buildGitHubItemKey(owner, repo, model.Number) + activity, exists := byKey[key] + if !exists { + activity = IssueActivity{Owner: owner, Repo: repo, Issue: model, UpdatedAt: model.UpdatedAt} + } else { + if model.UpdatedAt.After(activity.UpdatedAt) { + activity.UpdatedAt = model.UpdatedAt + } + activity.Issue = model + } + if shouldUpdateLabel(activity.Label, q.Label, false) { + activity.Label = q.Label + } + + if config.db != nil { + if err := config.db.SaveGitHubIssueWithLabel(owner, repo, model, activity.Label, config.debugMode); err != nil { + config.dbErrorCount.Add(1) + if config.debugMode { + fmt.Printf(" [DB] Warning: Failed to save GitHub issue %s/%s#%d: %v\n", owner, repo, model.Number, err) + } + } + } + + byKey[key] = activity + } + } + + activities := make([]IssueActivity, 0, len(byKey)) + for _, activity := range byKey { + activities = append(activities, activity) + } + return activities, nil +} + +func loadGitHubCachedActivities(cutoff time.Time) ([]PRActivity, []IssueActivity, error) { + if config.db == nil { + return []PRActivity{}, []IssueActivity{}, nil + } + + allPRs, prLabels, err := config.db.GetAllGitHubPullRequestsWithLabels(config.debugMode) + if err != nil { + return nil, nil, err + } + + activities := make([]PRActivity, 0, len(allPRs)) + prReviewComments := make(map[string][]GitHubPRReviewCommentRecord) + for key, pr := range allPRs { + if pr.UpdatedAt.IsZero() || pr.UpdatedAt.Before(cutoff) { + continue + } + + owner, repo, _, ok := parseGitHubItemKey(key) + if !ok || !isGitHubRepoAllowed(owner, repo) { + continue + } + + activities = append(activities, PRActivity{ + Label: prLabels[key], + Owner: owner, + Repo: repo, + MR: pr, + UpdatedAt: pr.UpdatedAt, + }) + + comments, err := config.db.GetGitHubPRReviewComments(owner, repo, pr.Number) + if err != nil { + return nil, nil, err + } + prReviewComments[key] = comments + } + + allIssues, issueLabels, err := config.db.GetAllGitHubIssuesWithLabels(config.debugMode) + if err != nil { + return nil, nil, err + } + + issueActivities := make([]IssueActivity, 0, len(allIssues)) + for key, issue := range allIssues { + if issue.UpdatedAt.IsZero() || issue.UpdatedAt.Before(cutoff) { + continue + } + + owner, repo, _, ok := parseGitHubItemKey(key) + if !ok || !isGitHubRepoAllowed(owner, repo) { + continue + } + + issueActivities = append(issueActivities, IssueActivity{ + Label: issueLabels[key], + Owner: owner, + Repo: repo, + Issue: issue, + UpdatedAt: issue.UpdatedAt, + }) + } + + nestedPRs := nestGitHubIssues(activities, issueActivities, prReviewComments) + standaloneIssues := filterStandaloneGitHubIssues(nestedPRs, issueActivities) + return nestedPRs, standaloneIssues, nil +} + +func searchGitHubIssues(ctx context.Context, client *github.Client, query string) ([]*github.Issue, error) { + allIssues := make([]*github.Issue, 0) + options := &github.SearchOptions{ListOptions: github.ListOptions{PerPage: 100, Page: 1}} + + for { + result, resp, err := client.Search.Issues(ctx, query, options) + if err != nil { + return nil, err + } + allIssues = append(allIssues, result.Issues...) + if resp == nil || resp.NextPage == 0 { + break + } + options.Page = resp.NextPage + } + + return allIssues, nil +} + +func newGitHubClient(token string) *github.Client { + tokenSource := oauth2.StaticTokenSource(&oauth2.Token{AccessToken: strings.TrimSpace(token)}) + httpClient := oauth2.NewClient(context.Background(), tokenSource) + return github.NewClient(httpClient) +} + +func getGitHubPullRequest(ctx context.Context, client *github.Client, owner, repo string, number int) (*github.PullRequest, error) { + pr, _, err := client.PullRequests.Get(ctx, owner, repo, number) + if err != nil { + return nil, fmt.Errorf("get pull request %s/%s#%d: %w", owner, repo, number, err) + } + return pr, nil +} + +func getGitHubIssue(ctx context.Context, client *github.Client, owner, repo string, number int) (*github.Issue, error) { + issue, _, err := client.Issues.Get(ctx, owner, repo, number) + if err != nil { + return nil, fmt.Errorf("get issue %s/%s#%d: %w", owner, repo, number, err) + } + return issue, nil +} + +func listGitHubPRReviewComments(ctx context.Context, client *github.Client, owner, repo string, number int) ([]*github.PullRequestComment, error) { + allComments := make([]*github.PullRequestComment, 0) + options := &github.PullRequestListCommentsOptions{ListOptions: github.ListOptions{PerPage: 100, Page: 1}} + + for { + comments, resp, err := client.PullRequests.ListComments(ctx, owner, repo, number, options) + if err != nil { + return nil, fmt.Errorf("list PR review comments for %s/%s#%d: %w", owner, repo, number, err) + } + allComments = append(allComments, comments...) + if resp == nil || resp.NextPage == 0 { + break + } + options.Page = resp.NextPage + } + + return allComments, nil +} + +func parseGitHubRepoFromSearchItem(item *github.Issue) (string, string, bool) { + if item == nil { + return "", "", false + } + + repoURL := strings.TrimSpace(item.GetRepositoryURL()) + if repoURL == "" { + htmlURL := strings.TrimSpace(item.GetHTMLURL()) + if htmlURL == "" { + return "", "", false + } + parsed, err := url.Parse(htmlURL) + if err != nil { + return "", "", false + } + parts := splitPathParts(parsed.Path) + if len(parts) < 2 { + return "", "", false + } + return parts[0], parts[1], true + } + + parsed, err := url.Parse(repoURL) + if err != nil { + return "", "", false + } + parts := splitPathParts(parsed.Path) + if len(parts) < 3 { + return "", "", false + } + if !strings.EqualFold(parts[0], "repos") { + return "", "", false + } + return parts[1], parts[2], true +} + +func splitPathParts(path string) []string { + trimmed := strings.Trim(path, "/") + if trimmed == "" { + return nil + } + parts := strings.Split(trimmed, "/") + out := make([]string, 0, len(parts)) + for _, part := range parts { + value := strings.TrimSpace(part) + if value == "" { + continue + } + out = append(out, value) + } + return out +} + +func toMergeRequestModelFromGitHubPR(pr *github.PullRequest) MergeRequestModel { + if pr == nil { + return MergeRequestModel{} + } + + updatedAt := time.Time{} + if pr.UpdatedAt != nil { + updatedAt = pr.UpdatedAt.Time + } + + state := strings.ToLower(pr.GetState()) + if state == "" { + state = "open" + } + + userLogin := "" + if pr.User != nil { + userLogin = pr.User.GetLogin() + } + + return MergeRequestModel{ + Number: pr.GetNumber(), + Title: pr.GetTitle(), + Body: pr.GetBody(), + State: state, + UpdatedAt: updatedAt, + WebURL: pr.GetHTMLURL(), + UserLogin: userLogin, + Merged: pr.GetMerged(), + } +} + +func toIssueModelFromGitHubIssue(issue *github.Issue) IssueModel { + if issue == nil { + return IssueModel{} + } + + updatedAt := time.Time{} + if issue.UpdatedAt != nil { + updatedAt = issue.UpdatedAt.Time + } + + state := strings.ToLower(issue.GetState()) + if state == "" { + state = "open" + } + + userLogin := "" + if issue.User != nil { + userLogin = issue.User.GetLogin() + } + + return IssueModel{ + Number: issue.GetNumber(), + Title: issue.GetTitle(), + Body: issue.GetBody(), + State: state, + UpdatedAt: updatedAt, + WebURL: issue.GetHTMLURL(), + UserLogin: userLogin, + } +} + +func toGitHubPRReviewCommentRecord(owner, repo string, prNumber int, comment *github.PullRequestComment) GitHubPRReviewCommentRecord { + record := GitHubPRReviewCommentRecord{Owner: owner, Repo: repo, PRNumber: prNumber} + if comment == nil { + return record + } + + record.CommentID = comment.GetID() + record.Body = comment.GetBody() + if comment.User != nil { + record.AuthorUsername = comment.User.GetLogin() + record.AuthorID = comment.User.GetID() + } + + return record +} + +func parseGitHubItemKey(key string) (string, string, int, bool) { + parts := strings.SplitN(key, "#", 2) + if len(parts) != 2 { + return "", "", 0, false + } + + repoParts := strings.SplitN(parts[0], "/", 2) + if len(repoParts) != 2 { + return "", "", 0, false + } + + number, err := strconv.Atoi(strings.TrimSpace(parts[1])) + if err != nil || number <= 0 { + return "", "", 0, false + } + + owner := strings.TrimSpace(repoParts[0]) + repo := strings.TrimSpace(repoParts[1]) + if owner == "" || repo == "" { + return "", "", 0, false + } + + return owner, repo, number, true +} + +func isGitHubRepoAllowed(owner, repo string) bool { + if len(config.allowedRepos) == 0 { + return true + } + + target := strings.ToLower(strings.TrimSpace(owner + "/" + repo)) + for allowed := range config.allowedRepos { + if strings.ToLower(strings.TrimSpace(allowed)) == target { + return true + } + } + return false +} + +func nestGitHubIssues( + activities []PRActivity, + issueActivities []IssueActivity, + prReviewComments map[string][]GitHubPRReviewCommentRecord, +) []PRActivity { + issueByKey := make(map[string]IssueActivity, len(issueActivities)) + for _, issue := range issueActivities { + issueByKey[buildGitHubItemKey(issue.Owner, issue.Repo, issue.Issue.Number)] = issue + } + + for i := range activities { + activities[i].Issues = nil + for _, issue := range issueActivities { + key := buildGitHubItemKey(activities[i].Owner, activities[i].Repo, activities[i].MR.Number) + if areGitHubCrossReferenced(activities[i], issue, prReviewComments[key]) { + issueKey := buildGitHubItemKey(issue.Owner, issue.Repo, issue.Issue.Number) + if nestedIssue, ok := issueByKey[issueKey]; ok { + activities[i].Issues = append(activities[i].Issues, nestedIssue) + } + } + } + sort.Slice(activities[i].Issues, func(a, b int) bool { + return activities[i].Issues[a].UpdatedAt.After(activities[i].Issues[b].UpdatedAt) + }) + } + + return activities +} + +func filterStandaloneGitHubIssues(activities []PRActivity, issueActivities []IssueActivity) []IssueActivity { + nestedIssueKeys := make(map[string]struct{}) + for _, activity := range activities { + for _, issue := range activity.Issues { + nestedIssueKeys[buildGitHubItemKey(issue.Owner, issue.Repo, issue.Issue.Number)] = struct{}{} + } + } + + standalone := make([]IssueActivity, 0, len(issueActivities)) + for _, issue := range issueActivities { + key := buildGitHubItemKey(issue.Owner, issue.Repo, issue.Issue.Number) + if _, nested := nestedIssueKeys[key]; nested { + continue + } + standalone = append(standalone, issue) + } + + return standalone +} + +func areGitHubCrossReferenced(prActivity PRActivity, issueActivity IssueActivity, reviewComments []GitHubPRReviewCommentRecord) bool { + if !strings.EqualFold(strings.TrimSpace(prActivity.Owner), strings.TrimSpace(issueActivity.Owner)) { + return false + } + if !strings.EqualFold(strings.TrimSpace(prActivity.Repo), strings.TrimSpace(issueActivity.Repo)) { + return false + } + + if mentionsNumber(prActivity.MR.Body, issueActivity.Issue.Number, prActivity.Owner, prActivity.Repo) { + return true + } + if mentionsNumber(issueActivity.Issue.Body, prActivity.MR.Number, prActivity.Owner, prActivity.Repo) { + return true + } + for _, comment := range reviewComments { + if mentionsNumber(comment.Body, issueActivity.Issue.Number, prActivity.Owner, prActivity.Repo) { + return true + } + } + + return false +} + +func mentionsNumber(text string, number int, owner, repo string) bool { + if strings.TrimSpace(text) == "" || number <= 0 { + return false + } + + targetRepo := strings.ToLower(strings.TrimSpace(owner + "/" + repo)) + targetNumber := strconv.Itoa(number) + keywordOnly := githubCrossRefKeywordPattern.MatchString(text) + + for _, match := range githubCrossRefURLPattern.FindAllStringSubmatch(text, -1) { + if len(match) < 4 { + continue + } + if strings.EqualFold(strings.TrimSpace(match[1]), strings.TrimSpace(owner)) && + strings.EqualFold(strings.TrimSpace(match[2]), strings.TrimSpace(repo)) && + strings.TrimSpace(match[3]) == targetNumber { + return true + } + } + + for _, match := range githubCrossRefQualifiedRef.FindAllStringSubmatch(text, -1) { + if len(match) < 3 { + continue + } + if strings.EqualFold(strings.TrimSpace(match[1]), targetRepo) && strings.TrimSpace(match[2]) == targetNumber { + return true + } + } + + for _, match := range githubCrossRefSameRefPattern.FindAllStringSubmatch(text, -1) { + if len(match) < 2 { + continue + } + if strings.TrimSpace(match[1]) == targetNumber { + return true + } + } + + if keywordOnly { + return false + } + + return false +} diff --git a/platform_gitlab.go b/platform_gitlab.go new file mode 100644 index 0000000..cf80aa8 --- /dev/null +++ b/platform_gitlab.go @@ -0,0 +1,1479 @@ +package main + +import ( + "context" + "errors" + "fmt" + "math" + "net/http" + "net/url" + "regexp" + "sort" + "strconv" + "strings" + "time" + + "github.com/fatih/color" + gitlab "gitlab.com/gitlab-org/api/client-go" +) + +var retryAfter = time.After + +const defaultGitLabBaseURL = "https://gitlab.com" + +func normalizeGitLabBaseURL(raw string) (string, error) { + baseURL := strings.TrimSpace(raw) + if baseURL == "" { + baseURL = defaultGitLabBaseURL + } + + parsed, err := url.Parse(baseURL) + if err != nil { + return "", fmt.Errorf("invalid GitLab base URL %q: %w", baseURL, err) + } + + if parsed.Scheme == "" || parsed.Host == "" { + return "", fmt.Errorf("invalid GitLab base URL %q: must include scheme and host", baseURL) + } + + normalizedPath := strings.TrimSuffix(parsed.EscapedPath(), "/") + if normalizedPath == "" { + normalizedPath = "/api/v4" + } else if !strings.HasSuffix(normalizedPath, "/api/v4") { + normalizedPath += "/api/v4" + } + + parsed.Path = normalizedPath + parsed.RawPath = "" + + return parsed.String(), nil +} + +func newGitLabClient(token, rawBaseURL string) (*gitlab.Client, string, error) { + normalizedBaseURL, err := normalizeGitLabBaseURL(rawBaseURL) + if err != nil { + return nil, "", err + } + + client, err := gitlab.NewClient(token, gitlab.WithBaseURL(normalizedBaseURL)) + if err != nil { + return nil, "", fmt.Errorf("failed to create GitLab client: %w", err) + } + + return client, normalizedBaseURL, nil +} + +func getPRLabelPriority(label string) int { + priorities := map[string]int{ + "Authored": 1, + "Assigned": 2, + "Reviewed": 3, + "Review Requested": 4, + "Commented": 5, + "Mentioned": 6, + } + if priority, ok := priorities[label]; ok { + return priority + } + return 999 +} + +func getIssueLabelPriority(label string) int { + priorities := map[string]int{ + "Authored": 1, + "Assigned": 2, + "Commented": 3, + "Mentioned": 4, + } + if priority, ok := priorities[label]; ok { + return priority + } + return 999 +} + +func shouldUpdateLabel(currentLabel, newLabel string, isPR bool) bool { + if currentLabel == "" { + return true + } + + var currentPriority, newPriority int + if isPR { + currentPriority = getPRLabelPriority(currentLabel) + newPriority = getPRLabelPriority(newLabel) + } else { + currentPriority = getIssueLabelPriority(currentLabel) + newPriority = getIssueLabelPriority(newLabel) + } + + return newPriority < currentPriority +} + +func retryWithBackoff(operation func() error, operationName string) error { + const ( + initialBackoff = 1 * time.Second + maxBackoff = 30 * time.Second + backoffFactor = 1.5 + ) + + backoff := initialBackoff + attempt := 1 + retryCtx := config.ctx + if retryCtx == nil { + retryCtx = context.Background() + } + + for { + err := operation() + if err == nil { + return nil + } + + var gitLabErr *gitlab.ErrorResponse + var waitTime time.Duration + var isRateLimitError bool + var isTransientServerError bool + shouldRetry := true + + if errors.As(err, &gitLabErr) && gitLabErr.Response != nil { + statusCode := gitLabErr.Response.StatusCode + + if statusCode == http.StatusTooManyRequests { + isRateLimitError = true + retryAfterSeconds, parseErr := strconv.Atoi(strings.TrimSpace(gitLabErr.Response.Header.Get("Retry-After"))) + if parseErr == nil && retryAfterSeconds > 0 { + waitTime = time.Duration(retryAfterSeconds) * time.Second + } else if resetWait, ok := gitLabRateLimitResetWait(gitLabErr.Response.Header.Get("Ratelimit-Reset")); ok { + waitTime = resetWait + } else { + waitTime = time.Duration(math.Min(float64(backoff), float64(maxBackoff))) + } + + if config.debugMode { + fmt.Printf(" [%s] GitLab rate limit hit (attempt %d), waiting %v before retry...\n", + operationName, attempt, waitTime.Round(time.Second)) + } + } else if statusCode >= http.StatusInternalServerError && statusCode <= 599 { + isTransientServerError = true + waitTime = time.Duration(math.Min(float64(backoff), float64(maxBackoff))) + + if config.debugMode { + fmt.Printf(" [%s] GitLab server error %d (attempt %d), waiting %v before retry...\n", + operationName, statusCode, attempt, waitTime) + } + } else { + shouldRetry = false + } + } else { + isRateLimitError = strings.Contains(err.Error(), "rate limit") || + strings.Contains(err.Error(), "API rate limit exceeded") || + strings.Contains(err.Error(), "403") + + if isRateLimitError { + waitTime = time.Duration(math.Min(float64(backoff), float64(maxBackoff))) + if config.debugMode { + fmt.Printf(" [%s] Rate limit hit (attempt %d), waiting %v before retry...\n", + operationName, attempt, waitTime) + } + } + } + + if !shouldRetry { + return err + } + + if isRateLimitError { + if config.debugMode { + select { + case <-retryCtx.Done(): + return retryCtx.Err() + case <-retryAfter(waitTime): + } + } else { + ticker := time.NewTicker(1 * time.Second) + defer ticker.Stop() + + remaining := int(waitTime.Seconds()) + for remaining > 0 { + if config.progress != nil { + config.progress.displayWithWarning(fmt.Sprintf("Rate limit hit, retrying in %ds", remaining)) + } + + select { + case <-retryCtx.Done(): + return retryCtx.Err() + case <-ticker.C: + remaining-- + } + } + } + + backoff = time.Duration(float64(backoff) * backoffFactor) + } else if isTransientServerError { + if config.debugMode { + select { + case <-retryCtx.Done(): + return retryCtx.Err() + case <-retryAfter(waitTime): + } + } else { + ticker := time.NewTicker(1 * time.Second) + defer ticker.Stop() + + remaining := int(waitTime.Seconds()) + for remaining > 0 { + if config.progress != nil { + config.progress.displayWithWarning(fmt.Sprintf("API error, retrying in %ds", remaining)) + } + + select { + case <-retryCtx.Done(): + return retryCtx.Err() + case <-ticker.C: + remaining-- + } + } + } + + backoff = time.Duration(float64(backoff) * backoffFactor) + } else { + waitTime := time.Duration(math.Min(float64(backoff)/2, float64(5*time.Second))) + + if config.debugMode { + fmt.Printf(" [%s] Error (attempt %d): %v, waiting %v before retry...\n", + operationName, attempt, err, waitTime) + select { + case <-retryCtx.Done(): + return retryCtx.Err() + case <-retryAfter(waitTime): + } + } else { + ticker := time.NewTicker(1 * time.Second) + defer ticker.Stop() + + remaining := int(waitTime.Seconds()) + for remaining > 0 { + if config.progress != nil { + config.progress.displayWithWarning(fmt.Sprintf("API error, retrying in %ds", remaining)) + } + + select { + case <-retryCtx.Done(): + return retryCtx.Err() + case <-ticker.C: + remaining-- + } + } + } + + backoff = time.Duration(float64(backoff) * backoffFactor) + } + + attempt++ + } +} + +func gitLabRateLimitResetWait(rawHeader string) (time.Duration, bool) { + resetAtUnix, err := strconv.ParseInt(strings.TrimSpace(rawHeader), 10, 64) + if err != nil || resetAtUnix <= 0 { + return 0, false + } + + resetTime := time.Unix(resetAtUnix, 0) + waitTime := time.Until(resetTime) + if waitTime <= 0 { + return 1 * time.Second, true + } + + return waitTime, true +} + +type gitLabProject struct { + PathWithNamespace string + ID int64 +} + +func fetchAndDisplayGitLabActivity() { + startTime := time.Now() + + if config.debugMode { + fmt.Println("Fetching data from GitLab...") + } else { + fmt.Print("Fetching data from GitLab... ") + } + + cutoffTime := time.Now().Add(-config.timeRange) + var ( + activities []PRActivity + issueActivities []IssueActivity + err error + ) + + if config.localMode { + activities, issueActivities, err = loadGitLabCachedActivities(cutoffTime) + } else { + activities, issueActivities, err = fetchGitLabProjectActivities( + config.ctx, + config.gitlabClient, + config.allowedRepos, + cutoffTime, + config.gitlabUsername, + config.gitlabUserID, + config.db, + ) + } + if err != nil { + fmt.Printf("Error fetching GitLab activity: %v\n", err) + return + } + + if config.debugMode { + fmt.Println() + fmt.Printf("Total fetch time: %v\n", time.Since(startTime).Round(time.Millisecond)) + fmt.Printf("Found %d unique merge requests and %d unique issues\n", len(activities), len(issueActivities)) + fmt.Println() + } else { + fmt.Print("\r" + strings.Repeat(" ", 80) + "\r") + } + + if len(activities) == 0 && len(issueActivities) == 0 { + fmt.Println("No open activity found") + return + } + + sort.Slice(activities, func(i, j int) bool { + return activities[i].UpdatedAt.After(activities[j].UpdatedAt) + }) + sort.Slice(issueActivities, func(i, j int) bool { + return issueActivities[i].UpdatedAt.After(issueActivities[j].UpdatedAt) + }) + + var openPRs, closedPRs, mergedPRs []PRActivity + for _, activity := range activities { + if activity.MR.State == "closed" { + if activity.MR.Merged { + mergedPRs = append(mergedPRs, activity) + } else { + closedPRs = append(closedPRs, activity) + } + } else { + openPRs = append(openPRs, activity) + } + } + + var openIssues, closedIssues []IssueActivity + for _, issue := range issueActivities { + if issue.Issue.State == "closed" { + closedIssues = append(closedIssues, issue) + } else { + openIssues = append(openIssues, issue) + } + } + + if len(openPRs) > 0 { + titleColor := color.New(color.FgHiGreen, color.Bold) + fmt.Println(titleColor.Sprint("OPEN PULL REQUESTS:")) + fmt.Println("------------------------------------------") + for _, activity := range openPRs { + displayMergeRequest(activity.Label, activity.Owner, activity.Repo, activity.MR, activity.HasUpdates) + if len(activity.Issues) > 0 { + for _, issue := range activity.Issues { + displayIssue(issue.Label, issue.Owner, issue.Repo, issue.Issue, true, issue.HasUpdates) + } + } + } + } + + if len(closedPRs) > 0 || len(mergedPRs) > 0 { + fmt.Println() + titleColor := color.New(color.FgHiRed, color.Bold) + fmt.Println(titleColor.Sprint("CLOSED/MERGED PULL REQUESTS:")) + fmt.Println("------------------------------------------") + for _, activity := range mergedPRs { + displayMergeRequest(activity.Label, activity.Owner, activity.Repo, activity.MR, activity.HasUpdates) + if len(activity.Issues) > 0 { + for _, issue := range activity.Issues { + displayIssue(issue.Label, issue.Owner, issue.Repo, issue.Issue, true, issue.HasUpdates) + } + } + } + for _, activity := range closedPRs { + displayMergeRequest(activity.Label, activity.Owner, activity.Repo, activity.MR, activity.HasUpdates) + if len(activity.Issues) > 0 { + for _, issue := range activity.Issues { + displayIssue(issue.Label, issue.Owner, issue.Repo, issue.Issue, true, issue.HasUpdates) + } + } + } + } + + if len(openIssues) > 0 { + fmt.Println() + titleColor := color.New(color.FgHiGreen, color.Bold) + fmt.Println(titleColor.Sprint("OPEN ISSUES:")) + fmt.Println("------------------------------------------") + for _, issue := range openIssues { + displayIssue(issue.Label, issue.Owner, issue.Repo, issue.Issue, false, issue.HasUpdates) + } + } + + if len(closedIssues) > 0 { + fmt.Println() + titleColor := color.New(color.FgHiRed, color.Bold) + fmt.Println(titleColor.Sprint("CLOSED ISSUES:")) + fmt.Println("------------------------------------------") + for _, issue := range closedIssues { + displayIssue(issue.Label, issue.Owner, issue.Repo, issue.Issue, false, issue.HasUpdates) + } + } +} + +func fetchGitLabProjectActivities( + ctx context.Context, + client *gitlab.Client, + allowedRepos map[string]bool, + cutoff time.Time, + currentUsername string, + currentUserID int64, + db *Database, +) ([]PRActivity, []IssueActivity, error) { + projects, err := resolveAllowedGitLabProjects(ctx, client, allowedRepos) + if err != nil { + return nil, nil, err + } + + currentUsername = strings.TrimSpace(currentUsername) + if currentUsername == "" { + return nil, nil, fmt.Errorf("gitlab current username is required") + } + + if len(projects) == 0 { + return []PRActivity{}, []IssueActivity{}, nil + } + + activities := make([]PRActivity, 0) + issueActivities := make([]IssueActivity, 0) + seenMergeRequests := make(map[string]struct{}) + seenIssues := make(map[string]struct{}) + projectIDByPath := make(map[string]int64, len(projects)) + mrNotesByKey := make(map[string][]*gitlab.Note) + + for _, project := range projects { + projectIDByPath[normalizeProjectPathWithNamespace(project.PathWithNamespace)] = project.ID + } + + for _, project := range projects { + projectMergeRequests, err := listGitLabProjectMergeRequests(ctx, client, project.ID, cutoff) + if err != nil { + return nil, nil, fmt.Errorf("list merge requests for %s: %w", project.PathWithNamespace, err) + } + + for _, item := range projectMergeRequests { + key := buildGitLabDedupKey(project.PathWithNamespace, "mr", item.IID) + if _, exists := seenMergeRequests[key]; exists { + continue + } + seenMergeRequests[key] = struct{}{} + + model := toMergeRequestModelFromGitLab(item) + if model.UpdatedAt.IsZero() || model.UpdatedAt.Before(cutoff) { + continue + } + + label, notes, err := deriveGitLabMergeRequestLabel(ctx, client, project.ID, item, currentUsername, currentUserID) + if err != nil { + return nil, nil, fmt.Errorf("derive merge request label for %s!%d: %w", project.PathWithNamespace, item.IID, err) + } + + if db != nil { + if err := db.SaveGitLabMergeRequestWithLabel(project.PathWithNamespace, model, label, config.debugMode); err != nil { + config.dbErrorCount.Add(1) + if config.debugMode { + fmt.Printf(" [DB] Warning: Failed to save GitLab MR %s!%d: %v\n", project.PathWithNamespace, item.IID, err) + } + } + if err := persistGitLabNotes(db, project.PathWithNamespace, "mr", int(item.IID), notes); err != nil { + config.dbErrorCount.Add(1) + if config.debugMode { + fmt.Printf(" [DB] Warning: Failed to save GitLab MR notes %s!%d: %v\n", project.PathWithNamespace, item.IID, err) + } + } + } + + mrNotesByKey[buildGitLabMergeRequestKey(project.PathWithNamespace, model.Number)] = notes + + owner, repo, ok := splitGitLabPathWithNamespace(project.PathWithNamespace) + if !ok { + owner = project.PathWithNamespace + repo = "" + } + + activities = append(activities, PRActivity{ + Label: label, + Owner: owner, + Repo: repo, + MR: model, + UpdatedAt: model.UpdatedAt, + }) + } + + projectIssues, err := listGitLabProjectIssues(ctx, client, project.ID, cutoff) + if err != nil { + return nil, nil, fmt.Errorf("list issues for %s: %w", project.PathWithNamespace, err) + } + + for _, item := range projectIssues { + key := buildGitLabDedupKey(project.PathWithNamespace, "issue", item.IID) + if _, exists := seenIssues[key]; exists { + continue + } + seenIssues[key] = struct{}{} + + model := toIssueModelFromGitLab(item) + if model.UpdatedAt.IsZero() || model.UpdatedAt.Before(cutoff) { + continue + } + + label, notes, err := deriveGitLabIssueLabel(ctx, client, project.ID, item, currentUsername, currentUserID) + if err != nil { + return nil, nil, fmt.Errorf("derive issue label for %s#%d: %w", project.PathWithNamespace, item.IID, err) + } + + if db != nil { + if err := db.SaveGitLabIssueWithLabel(project.PathWithNamespace, model, label, config.debugMode); err != nil { + config.dbErrorCount.Add(1) + if config.debugMode { + fmt.Printf(" [DB] Warning: Failed to save GitLab issue %s#%d: %v\n", project.PathWithNamespace, item.IID, err) + } + } + if err := persistGitLabNotes(db, project.PathWithNamespace, "issue", int(item.IID), notes); err != nil { + config.dbErrorCount.Add(1) + if config.debugMode { + fmt.Printf(" [DB] Warning: Failed to save GitLab issue notes %s#%d: %v\n", project.PathWithNamespace, item.IID, err) + } + } + } + + owner, repo, ok := splitGitLabPathWithNamespace(project.PathWithNamespace) + if !ok { + owner = project.PathWithNamespace + repo = "" + } + + issueActivities = append(issueActivities, IssueActivity{ + Label: label, + Owner: owner, + Repo: repo, + Issue: model, + UpdatedAt: model.UpdatedAt, + }) + } + } + + activities, issueActivities, err = linkGitLabCrossReferencesOnline(ctx, client, activities, issueActivities, projectIDByPath, mrNotesByKey, db) + if err != nil { + return nil, nil, err + } + + return activities, issueActivities, nil +} + +func deriveGitLabMergeRequestLabel( + ctx context.Context, + client *gitlab.Client, + projectID int64, + item *gitlab.BasicMergeRequest, + currentUsername string, + currentUserID int64, +) (string, []*gitlab.Note, error) { + if item == nil { + return "Involved", nil, nil + } + + currentLabel := "" + if matchesGitLabBasicUser(item.Author, currentUsername, currentUserID) { + currentLabel = mergeLabelWithPriority(currentLabel, "Authored", true) + } + if gitLabBasicUserListContains(item.Assignees, currentUsername, currentUserID) || matchesGitLabBasicUser(item.Assignee, currentUsername, currentUserID) { + currentLabel = mergeLabelWithPriority(currentLabel, "Assigned", true) + } + + if currentLabel == "Authored" || currentLabel == "Assigned" { + return currentLabel, nil, nil + } + + var approvalState *gitlab.MergeRequestApprovalState + err := retryWithBackoff(func() error { + var apiErr error + approvalState, _, apiErr = client.MergeRequestApprovals.GetApprovalState(projectID, item.IID, gitlab.WithContext(ctx)) + return apiErr + }, fmt.Sprintf("GitLabGetApprovalState %d!%d", projectID, item.IID)) + if err != nil { + return "", nil, err + } + if gitLabApprovalStateReviewedByCurrentUser(approvalState, currentUsername, currentUserID) { + currentLabel = mergeLabelWithPriority(currentLabel, "Reviewed", true) + } + + if gitLabBasicUserListContains(item.Reviewers, currentUsername, currentUserID) { + currentLabel = mergeLabelWithPriority(currentLabel, "Review Requested", true) + } + + if !needsLowerPriorityPRChecks(currentLabel) { + if currentLabel == "" { + return "Involved", nil, nil + } + return currentLabel, nil, nil + } + + notes, err := listAllGitLabMergeRequestNotes(ctx, client, projectID, item.IID) + if err != nil { + return "", nil, err + } + + commented, mentioned := gitLabNotesInvolvement(notes, item.Description, currentUsername, currentUserID) + if commented { + currentLabel = mergeLabelWithPriority(currentLabel, "Commented", true) + } + if mentioned { + currentLabel = mergeLabelWithPriority(currentLabel, "Mentioned", true) + } + + if currentLabel == "" { + return "Involved", notes, nil + } + return currentLabel, notes, nil +} + +func deriveGitLabIssueLabel( + ctx context.Context, + client *gitlab.Client, + projectID int64, + item *gitlab.Issue, + currentUsername string, + currentUserID int64, +) (string, []*gitlab.Note, error) { + if item == nil { + return "Involved", nil, nil + } + + currentLabel := "" + if matchesGitLabIssueAuthor(item.Author, currentUsername, currentUserID) { + currentLabel = mergeLabelWithPriority(currentLabel, "Authored", false) + } + if gitLabIssueAssigneeListContains(item.Assignees, currentUsername, currentUserID) || matchesGitLabIssueAssignee(item.Assignee, currentUsername, currentUserID) { + currentLabel = mergeLabelWithPriority(currentLabel, "Assigned", false) + } + + if currentLabel == "Authored" || currentLabel == "Assigned" { + return currentLabel, nil, nil + } + + notes, err := listAllGitLabIssueNotes(ctx, client, projectID, item.IID) + if err != nil { + return "", nil, err + } + + commented, mentioned := gitLabNotesInvolvement(notes, item.Description, currentUsername, currentUserID) + if commented { + currentLabel = mergeLabelWithPriority(currentLabel, "Commented", false) + } + if mentioned { + currentLabel = mergeLabelWithPriority(currentLabel, "Mentioned", false) + } + + if currentLabel == "" { + return "Involved", notes, nil + } + return currentLabel, notes, nil +} + +func persistGitLabNotes(db *Database, projectPath, itemType string, itemIID int, notes []*gitlab.Note) error { + if db == nil || len(notes) == 0 { + return nil + } + + for _, note := range notes { + if note == nil { + continue + } + + authorUsername := "" + authorID := int64(0) + author := note.Author + authorUsername = strings.TrimSpace(author.Username) + authorID = author.ID + + record := GitLabNoteRecord{ + ProjectPath: projectPath, + ItemType: itemType, + ItemIID: itemIID, + NoteID: int64(note.ID), + Body: note.Body, + AuthorUsername: authorUsername, + AuthorID: authorID, + } + + if err := db.SaveGitLabNote(record, config.debugMode); err != nil { + return err + } + } + + return nil +} + +func loadGitLabCachedActivities(cutoff time.Time) ([]PRActivity, []IssueActivity, error) { + if config.db == nil { + return []PRActivity{}, []IssueActivity{}, nil + } + + allMRs, mrLabels, err := config.db.GetAllGitLabMergeRequestsWithLabels(config.debugMode) + if err != nil { + return nil, nil, err + } + + activities := make([]PRActivity, 0, len(allMRs)) + for key, mr := range allMRs { + if mr.UpdatedAt.IsZero() || mr.UpdatedAt.Before(cutoff) { + continue + } + + projectPath, ok := parseGitLabMRProjectPath(key) + if !ok || !isGitLabProjectAllowed(projectPath) { + continue + } + + owner, repo, ok := splitGitLabPathWithNamespace(projectPath) + if !ok { + owner = projectPath + repo = "" + } + + activities = append(activities, PRActivity{ + Label: mrLabels[key], + Owner: owner, + Repo: repo, + MR: mr, + UpdatedAt: mr.UpdatedAt, + }) + } + + allIssues, issueLabels, err := config.db.GetAllGitLabIssuesWithLabels(config.debugMode) + if err != nil { + return nil, nil, err + } + + issueActivities := make([]IssueActivity, 0, len(allIssues)) + for key, issue := range allIssues { + if issue.UpdatedAt.IsZero() || issue.UpdatedAt.Before(cutoff) { + continue + } + + projectPath, ok := parseGitLabIssueProjectPath(key) + if !ok || !isGitLabProjectAllowed(projectPath) { + continue + } + + owner, repo, ok := splitGitLabPathWithNamespace(projectPath) + if !ok { + owner = projectPath + repo = "" + } + + issueActivities = append(issueActivities, IssueActivity{ + Label: issueLabels[key], + Owner: owner, + Repo: repo, + Issue: issue, + UpdatedAt: issue.UpdatedAt, + }) + } + + activities, issueActivities, err = linkGitLabCrossReferencesOffline(config.db, activities, issueActivities) + if err != nil { + return nil, nil, err + } + + return activities, issueActivities, nil +} + +var ( + gitLabIssueSameProjectRefPattern = regexp.MustCompile(`(?i)(?:^|[^a-z0-9_])#([0-9]+)\b`) + gitLabIssueQualifiedRefPattern = regexp.MustCompile(`(?i)([a-z0-9_.-]+(?:/[a-z0-9_.-]+)+)#([0-9]+)\b`) + gitLabIssueURLRefPattern = regexp.MustCompile(`(?i)https?://[^\s]+/([a-z0-9_.-]+(?:/[a-z0-9_.-]+)+)/-/issues/([0-9]+)\b`) + gitLabIssueRelativeURLRefPattern = regexp.MustCompile(`(?i)/-/issues/([0-9]+)\b`) +) + +func linkGitLabCrossReferencesOnline( + ctx context.Context, + client *gitlab.Client, + activities []PRActivity, + issueActivities []IssueActivity, + projectIDByPath map[string]int64, + mrNotesByKey map[string][]*gitlab.Note, + db *Database, +) ([]PRActivity, []IssueActivity, error) { + mrToIssueKeys := make(map[string]map[string]struct{}, len(activities)) + + for _, activity := range activities { + projectPath := normalizeProjectPathWithNamespace(gitLabProjectPath(activity.Owner, activity.Repo)) + projectID, ok := projectIDByPath[projectPath] + if !ok { + continue + } + + mrKey := buildGitLabMergeRequestKey(projectPath, activity.MR.Number) + closedIssues, err := listGitLabIssuesClosedOnMergeRequest(ctx, client, projectID, int64(activity.MR.Number)) + if err == nil { + resolvedKeys := make(map[string]struct{}) + for _, item := range closedIssues { + issueKey, ok := gitLabIssueKeyFromIssue(item, projectPath) + if !ok { + continue + } + resolvedKeys[issueKey] = struct{}{} + } + if len(resolvedKeys) > 0 { + mrToIssueKeys[mrKey] = resolvedKeys + } + continue + } + + fallbackKeys := gitLabIssueReferenceKeysFromText(activity.MR.Body, projectPath) + if len(fallbackKeys) == 0 { + notes := mrNotesByKey[mrKey] + if len(notes) == 0 { + notes, err = listAllGitLabMergeRequestNotes(ctx, client, projectID, int64(activity.MR.Number)) + if err == nil { + mrNotesByKey[mrKey] = notes + if db != nil { + if persistErr := persistGitLabNotes(db, projectPath, "mr", activity.MR.Number, notes); persistErr != nil { + config.dbErrorCount.Add(1) + if config.debugMode { + fmt.Printf(" [DB] Warning: Failed to save GitLab MR notes %s!%d: %v\n", projectPath, activity.MR.Number, persistErr) + } + } + } + } + } + + for _, note := range notes { + if note == nil { + continue + } + for issueKey := range gitLabIssueReferenceKeysFromText(note.Body, projectPath) { + fallbackKeys[issueKey] = struct{}{} + } + } + } + + if len(fallbackKeys) > 0 { + mrToIssueKeys[mrKey] = fallbackKeys + } + } + + nestedActivities := nestGitLabIssues(activities, issueActivities, mrToIssueKeys) + return nestedActivities, filterStandaloneGitLabIssues(nestedActivities, issueActivities), nil +} + +func linkGitLabCrossReferencesOffline(db *Database, activities []PRActivity, issueActivities []IssueActivity) ([]PRActivity, []IssueActivity, error) { + mrToIssueKeys := make(map[string]map[string]struct{}, len(activities)) + + for _, activity := range activities { + projectPath := normalizeProjectPathWithNamespace(gitLabProjectPath(activity.Owner, activity.Repo)) + mrKey := buildGitLabMergeRequestKey(projectPath, activity.MR.Number) + linked := gitLabIssueReferenceKeysFromText(activity.MR.Body, projectPath) + if len(linked) == 0 && db != nil { + notes, err := db.GetGitLabNotes(projectPath, "mr", activity.MR.Number) + if err != nil { + return nil, nil, err + } + for _, note := range notes { + for issueKey := range gitLabIssueReferenceKeysFromText(note.Body, projectPath) { + linked[issueKey] = struct{}{} + } + } + } + + if len(linked) > 0 { + mrToIssueKeys[mrKey] = linked + } + } + + nestedActivities := nestGitLabIssues(activities, issueActivities, mrToIssueKeys) + return nestedActivities, filterStandaloneGitLabIssues(nestedActivities, issueActivities), nil +} + +func listGitLabIssuesClosedOnMergeRequest(ctx context.Context, client *gitlab.Client, projectID int64, mergeRequestIID int64) ([]*gitlab.Issue, error) { + allIssues := make([]*gitlab.Issue, 0) + opts := &gitlab.GetIssuesClosedOnMergeOptions{ListOptions: gitlab.ListOptions{PerPage: 100, Page: 1}} + + for { + issues, resp, err := client.MergeRequests.GetIssuesClosedOnMerge(projectID, mergeRequestIID, opts, gitlab.WithContext(ctx)) + if err != nil { + return nil, err + } + allIssues = append(allIssues, issues...) + if resp == nil || resp.NextPage == 0 { + break + } + opts.Page = resp.NextPage + } + + return allIssues, nil +} + +func nestGitLabIssues(activities []PRActivity, issueActivities []IssueActivity, mrToIssueKeys map[string]map[string]struct{}) []PRActivity { + issueByKey := make(map[string]IssueActivity, len(issueActivities)) + for _, issue := range issueActivities { + projectPath := normalizeProjectPathWithNamespace(gitLabProjectPath(issue.Owner, issue.Repo)) + issueByKey[buildGitLabIssueKey(projectPath, issue.Issue.Number)] = issue + } + + for i := range activities { + activities[i].Issues = nil + projectPath := normalizeProjectPathWithNamespace(gitLabProjectPath(activities[i].Owner, activities[i].Repo)) + mrKey := buildGitLabMergeRequestKey(projectPath, activities[i].MR.Number) + linkedKeys := mrToIssueKeys[mrKey] + if len(linkedKeys) == 0 { + continue + } + for issueKey := range linkedKeys { + issue, ok := issueByKey[issueKey] + if !ok { + continue + } + activities[i].Issues = append(activities[i].Issues, issue) + } + sort.Slice(activities[i].Issues, func(a, b int) bool { + return activities[i].Issues[a].UpdatedAt.After(activities[i].Issues[b].UpdatedAt) + }) + } + + return activities +} + +func filterStandaloneGitLabIssues(activities []PRActivity, issueActivities []IssueActivity) []IssueActivity { + linkedIssueKeys := make(map[string]struct{}) + for _, activity := range activities { + for _, issue := range activity.Issues { + projectPath := normalizeProjectPathWithNamespace(gitLabProjectPath(issue.Owner, issue.Repo)) + linkedIssueKeys[buildGitLabIssueKey(projectPath, issue.Issue.Number)] = struct{}{} + } + } + + standalone := make([]IssueActivity, 0, len(issueActivities)) + for _, issue := range issueActivities { + projectPath := normalizeProjectPathWithNamespace(gitLabProjectPath(issue.Owner, issue.Repo)) + issueKey := buildGitLabIssueKey(projectPath, issue.Issue.Number) + if _, linked := linkedIssueKeys[issueKey]; linked { + continue + } + standalone = append(standalone, issue) + } + + return standalone +} + +func gitLabIssueReferenceKeysFromText(text, defaultProjectPath string) map[string]struct{} { + results := make(map[string]struct{}) + if strings.TrimSpace(text) == "" { + return results + } + + for _, match := range gitLabIssueURLRefPattern.FindAllStringSubmatch(text, -1) { + if len(match) < 3 { + continue + } + iid, ok := parsePositiveInt(match[2]) + if !ok { + continue + } + results[buildGitLabIssueKey(match[1], iid)] = struct{}{} + } + + for _, match := range gitLabIssueQualifiedRefPattern.FindAllStringSubmatch(text, -1) { + if len(match) < 3 { + continue + } + iid, ok := parsePositiveInt(match[2]) + if !ok { + continue + } + results[buildGitLabIssueKey(match[1], iid)] = struct{}{} + } + + defaultProjectPath = normalizeProjectPathWithNamespace(defaultProjectPath) + if defaultProjectPath != "" { + for _, match := range gitLabIssueRelativeURLRefPattern.FindAllStringSubmatch(text, -1) { + if len(match) < 2 { + continue + } + iid, ok := parsePositiveInt(match[1]) + if !ok { + continue + } + results[buildGitLabIssueKey(defaultProjectPath, iid)] = struct{}{} + } + + for _, match := range gitLabIssueSameProjectRefPattern.FindAllStringSubmatch(text, -1) { + if len(match) < 2 { + continue + } + iid, ok := parsePositiveInt(match[1]) + if !ok { + continue + } + results[buildGitLabIssueKey(defaultProjectPath, iid)] = struct{}{} + } + } + + return results +} + +func gitLabIssueKeyFromIssue(item *gitlab.Issue, defaultProjectPath string) (string, bool) { + if item == nil || item.IID <= 0 { + return "", false + } + + if item.References != nil { + if projectPath, iid, ok := parseGitLabQualifiedReference(item.References.Full); ok { + return buildGitLabIssueKey(projectPath, iid), true + } + } + + defaultProjectPath = normalizeProjectPathWithNamespace(defaultProjectPath) + if defaultProjectPath == "" { + return "", false + } + return buildGitLabIssueKey(defaultProjectPath, int(item.IID)), true +} + +func parseGitLabQualifiedReference(reference string) (string, int, bool) { + for _, match := range gitLabIssueQualifiedRefPattern.FindAllStringSubmatch(reference, -1) { + if len(match) < 3 { + continue + } + iid, ok := parsePositiveInt(match[2]) + if !ok { + continue + } + return normalizeProjectPathWithNamespace(match[1]), iid, true + } + return "", 0, false +} + +func parsePositiveInt(raw string) (int, bool) { + value, err := strconv.Atoi(strings.TrimSpace(raw)) + if err != nil || value <= 0 { + return 0, false + } + return value, true +} + +func parseGitLabMRProjectPath(key string) (string, bool) { + idx := strings.LastIndex(key, "#!") + if idx <= 0 { + return "", false + } + return key[:idx], true +} + +func parseGitLabIssueProjectPath(key string) (string, bool) { + idx := strings.LastIndex(key, "##") + if idx <= 0 { + return "", false + } + return key[:idx], true +} + +func isGitLabProjectAllowed(projectPath string) bool { + if config.allowedRepos == nil || len(config.allowedRepos) == 0 { + return true + } + + normalized := normalizeProjectPathWithNamespace(projectPath) + for repo := range config.allowedRepos { + if strings.EqualFold(normalizeProjectPathWithNamespace(repo), normalized) { + return true + } + } + + return false +} + +func needsLowerPriorityPRChecks(currentLabel string) bool { + return shouldUpdateLabel(currentLabel, "Commented", true) || shouldUpdateLabel(currentLabel, "Mentioned", true) +} + +func mergeLabelWithPriority(currentLabel, candidateLabel string, isPR bool) string { + if shouldUpdateLabel(currentLabel, candidateLabel, isPR) { + return candidateLabel + } + return currentLabel +} + +func listAllGitLabMergeRequestNotes(ctx context.Context, client *gitlab.Client, projectID int64, mrIID int64) ([]*gitlab.Note, error) { + allNotes := make([]*gitlab.Note, 0) + options := &gitlab.ListMergeRequestNotesOptions{ + ListOptions: gitlab.ListOptions{PerPage: 100, Page: 1}, + } + + for { + var ( + notes []*gitlab.Note + response *gitlab.Response + ) + err := retryWithBackoff(func() error { + var apiErr error + notes, response, apiErr = client.Notes.ListMergeRequestNotes(projectID, mrIID, options, gitlab.WithContext(ctx)) + return apiErr + }, fmt.Sprintf("GitLabListMergeRequestNotes %d!%d page %d", projectID, mrIID, options.Page)) + if err != nil { + return nil, err + } + allNotes = append(allNotes, notes...) + + if response == nil || response.NextPage == 0 { + break + } + options.Page = response.NextPage + } + + return allNotes, nil +} + +func listAllGitLabIssueNotes(ctx context.Context, client *gitlab.Client, projectID int64, issueIID int64) ([]*gitlab.Note, error) { + allNotes := make([]*gitlab.Note, 0) + options := &gitlab.ListIssueNotesOptions{ + ListOptions: gitlab.ListOptions{PerPage: 100, Page: 1}, + } + + for { + var ( + notes []*gitlab.Note + response *gitlab.Response + ) + err := retryWithBackoff(func() error { + var apiErr error + notes, response, apiErr = client.Notes.ListIssueNotes(projectID, issueIID, options, gitlab.WithContext(ctx)) + return apiErr + }, fmt.Sprintf("GitLabListIssueNotes %d#%d page %d", projectID, issueIID, options.Page)) + if err != nil { + return nil, err + } + allNotes = append(allNotes, notes...) + + if response == nil || response.NextPage == 0 { + break + } + options.Page = response.NextPage + } + + return allNotes, nil +} + +func gitLabNotesInvolvement(notes []*gitlab.Note, description, currentUsername string, currentUserID int64) (bool, bool) { + commented := false + mentioned := containsGitLabUserMention(description, currentUsername) + + for _, note := range notes { + if note == nil { + continue + } + if matchesGitLabNoteAuthor(note.Author, currentUsername, currentUserID) { + commented = true + } + if !mentioned && containsGitLabUserMention(note.Body, currentUsername) { + mentioned = true + } + if commented && mentioned { + break + } + } + + return commented, mentioned +} + +func containsGitLabUserMention(text, username string) bool { + if text == "" || username == "" { + return false + } + needle := "@" + strings.ToLower(strings.TrimSpace(username)) + if needle == "@" { + return false + } + return strings.Contains(strings.ToLower(text), needle) +} + +func matchesGitLabNoteAuthor(author gitlab.NoteAuthor, username string, userID int64) bool { + if userID > 0 && author.ID == userID { + return true + } + return strings.EqualFold(strings.TrimSpace(author.Username), strings.TrimSpace(username)) +} + +func matchesGitLabBasicUser(user *gitlab.BasicUser, username string, userID int64) bool { + if user == nil { + return false + } + if userID > 0 && user.ID == userID { + return true + } + return strings.EqualFold(strings.TrimSpace(user.Username), strings.TrimSpace(username)) +} + +func matchesGitLabIssueAuthor(author *gitlab.IssueAuthor, username string, userID int64) bool { + if author == nil { + return false + } + if userID > 0 && author.ID == userID { + return true + } + return strings.EqualFold(strings.TrimSpace(author.Username), strings.TrimSpace(username)) +} + +func matchesGitLabIssueAssignee(assignee *gitlab.IssueAssignee, username string, userID int64) bool { + if assignee == nil { + return false + } + if userID > 0 && assignee.ID == userID { + return true + } + return strings.EqualFold(strings.TrimSpace(assignee.Username), strings.TrimSpace(username)) +} + +func gitLabIssueAssigneeListContains(assignees []*gitlab.IssueAssignee, username string, userID int64) bool { + for _, assignee := range assignees { + if matchesGitLabIssueAssignee(assignee, username, userID) { + return true + } + } + return false +} + +func gitLabBasicUserListContains(users []*gitlab.BasicUser, username string, userID int64) bool { + for _, user := range users { + if matchesGitLabBasicUser(user, username, userID) { + return true + } + } + return false +} + +func gitLabApprovalStateReviewedByCurrentUser(state *gitlab.MergeRequestApprovalState, username string, userID int64) bool { + if state == nil { + return false + } + for _, rule := range state.Rules { + if rule == nil { + continue + } + if gitLabBasicUserListContains(rule.ApprovedBy, username, userID) { + return true + } + } + return false +} + +func resolveAllowedGitLabProjects(ctx context.Context, client *gitlab.Client, allowedRepos map[string]bool) ([]gitLabProject, error) { + if client == nil { + return nil, fmt.Errorf("gitlab client is not configured") + } + + if len(allowedRepos) == 0 { + return []gitLabProject{}, nil + } + + repoPaths := make([]string, 0, len(allowedRepos)) + for repo := range allowedRepos { + normalized := normalizeProjectPathWithNamespace(repo) + if normalized != "" { + repoPaths = append(repoPaths, normalized) + } + } + sort.Strings(repoPaths) + + projectIDCache := make(map[string]int64, len(repoPaths)) + projects := make([]gitLabProject, 0, len(repoPaths)) + for _, pathWithNamespace := range repoPaths { + if id, ok := projectIDCache[pathWithNamespace]; ok { + projects = append(projects, gitLabProject{PathWithNamespace: pathWithNamespace, ID: id}) + continue + } + + var project *gitlab.Project + err := retryWithBackoff(func() error { + var apiErr error + project, _, apiErr = client.Projects.GetProject(pathWithNamespace, nil, gitlab.WithContext(ctx)) + return apiErr + }, fmt.Sprintf("GitLabGetProject %s", pathWithNamespace)) + if err != nil { + return nil, fmt.Errorf("resolve project %s: %w", pathWithNamespace, err) + } + + projectIDCache[pathWithNamespace] = project.ID + projects = append(projects, gitLabProject{PathWithNamespace: pathWithNamespace, ID: project.ID}) + } + + return projects, nil +} + +func listGitLabProjectMergeRequests(ctx context.Context, client *gitlab.Client, projectID int64, cutoff time.Time) ([]*gitlab.BasicMergeRequest, error) { + allItems := make([]*gitlab.BasicMergeRequest, 0) + options := &gitlab.ListProjectMergeRequestsOptions{ + ListOptions: gitlab.ListOptions{PerPage: 100, Page: 1}, + State: gitlab.Ptr("all"), + UpdatedAfter: &cutoff, + } + + for { + var ( + items []*gitlab.BasicMergeRequest + response *gitlab.Response + ) + err := retryWithBackoff(func() error { + var apiErr error + items, response, apiErr = client.MergeRequests.ListProjectMergeRequests(projectID, options, gitlab.WithContext(ctx)) + return apiErr + }, fmt.Sprintf("GitLabListProjectMergeRequests %d page %d", projectID, options.Page)) + if err != nil { + return nil, err + } + allItems = append(allItems, items...) + + if response == nil || response.NextPage == 0 { + break + } + options.Page = response.NextPage + } + + return allItems, nil +} + +func listGitLabProjectIssues(ctx context.Context, client *gitlab.Client, projectID int64, cutoff time.Time) ([]*gitlab.Issue, error) { + allItems := make([]*gitlab.Issue, 0) + options := &gitlab.ListProjectIssuesOptions{ + ListOptions: gitlab.ListOptions{PerPage: 100, Page: 1}, + State: gitlab.Ptr("all"), + UpdatedAfter: &cutoff, + } + + for { + var ( + items []*gitlab.Issue + response *gitlab.Response + ) + err := retryWithBackoff(func() error { + var apiErr error + items, response, apiErr = client.Issues.ListProjectIssues(projectID, options, gitlab.WithContext(ctx)) + return apiErr + }, fmt.Sprintf("GitLabListProjectIssues %d page %d", projectID, options.Page)) + if err != nil { + return nil, err + } + allItems = append(allItems, items...) + + if response == nil || response.NextPage == 0 { + break + } + options.Page = response.NextPage + } + + return allItems, nil +} + +func normalizeProjectPathWithNamespace(repo string) string { + trimmed := strings.TrimSpace(repo) + return strings.Trim(trimmed, "/") +} + +func splitGitLabPathWithNamespace(path string) (owner string, repo string, ok bool) { + normalized := normalizeProjectPathWithNamespace(path) + idx := strings.LastIndex(normalized, "/") + if idx <= 0 || idx >= len(normalized)-1 { + return "", "", false + } + return normalized[:idx], normalized[idx+1:], true +} + +func gitLabProjectPath(owner, repo string) string { + owner = normalizeProjectPathWithNamespace(owner) + repo = strings.Trim(strings.TrimSpace(repo), "/") + if repo != "" { + if owner == "" { + return repo + } + return owner + "/" + repo + } + return owner +} + +func buildGitLabDedupKey(projectPath, itemType string, iid int64) string { + return fmt.Sprintf("%s|%s|%d", strings.ToLower(projectPath), itemType, iid) +} + +func toMergeRequestModelFromGitLab(item *gitlab.BasicMergeRequest) MergeRequestModel { + if item == nil { + return MergeRequestModel{} + } + + state := strings.ToLower(item.State) + merged := state == "merged" || item.MergedAt != nil + normalizedState := "open" + if merged || state == "closed" { + normalizedState = "closed" + } + + updatedAt := time.Time{} + if item.UpdatedAt != nil { + updatedAt = *item.UpdatedAt + } + + userLogin := "" + if item.Author != nil { + userLogin = item.Author.Username + } + + return MergeRequestModel{ + Number: int(item.IID), + Title: item.Title, + Body: item.Description, + State: normalizedState, + UpdatedAt: updatedAt, + WebURL: item.WebURL, + UserLogin: userLogin, + Merged: merged, + } +} + +func toIssueModelFromGitLab(item *gitlab.Issue) IssueModel { + if item == nil { + return IssueModel{} + } + + state := strings.ToLower(item.State) + normalizedState := "open" + if state == "closed" { + normalizedState = "closed" + } + + updatedAt := time.Time{} + if item.UpdatedAt != nil { + updatedAt = *item.UpdatedAt + } + + userLogin := "" + if item.Author != nil { + userLogin = item.Author.Username + } + + return IssueModel{ + Number: int(item.IID), + Title: item.Title, + Body: item.Description, + State: normalizedState, + UpdatedAt: updatedAt, + WebURL: item.WebURL, + UserLogin: userLogin, + } +} diff --git a/priority_test.go b/priority_test.go index 2569f89..ec5439a 100644 --- a/priority_test.go +++ b/priority_test.go @@ -636,14 +636,14 @@ func TestLoadGitLabCachedActivities_OfflineParityFiltersAndOrder(t *testing.T) { if len(activities) != 1 { t.Fatalf("MR activities count = %d, want 1", len(activities)) } - if activities[0].MR.Title != "new mr" || activities[0].Label != "Authored" || activities[0].Owner != "group/repo" { + if activities[0].MR.Title != "new mr" || activities[0].Label != "Authored" || activities[0].Owner != "group" || activities[0].Repo != "repo" { t.Fatalf("unexpected MR activity %+v", activities[0]) } if len(issueActivities) != 1 { t.Fatalf("Issue activities count = %d, want 1", len(issueActivities)) } - if issueActivities[0].Issue.Title != "new issue" || issueActivities[0].Label != "Commented" || issueActivities[0].Owner != "group/repo" { + if issueActivities[0].Issue.Title != "new issue" || issueActivities[0].Label != "Commented" || issueActivities[0].Owner != "group" || issueActivities[0].Repo != "repo" { t.Fatalf("unexpected issue activity %+v", issueActivities[0]) } @@ -820,7 +820,7 @@ func TestFetchGitLabProjectActivities_PaginatesAndFiltersByCutoff(t *testing.T) if len(activities) != 2 { t.Fatalf("got %d merge request activities, want 2", len(activities)) } - if activities[0].Owner != "group/subgroup/repo" || activities[0].Repo != "" { + if activities[0].Owner != "group/subgroup" || activities[0].Repo != "repo" { t.Fatalf("unexpected project mapping for merge request activity: owner=%q repo=%q", activities[0].Owner, activities[0].Repo) } @@ -1006,6 +1006,52 @@ func TestFetchGitLabProjectActivities_DerivesLabelsFromSources(t *testing.T) { } } +func TestLoadEnvFile_DoesNotOverrideExistingEnv(t *testing.T) { + envPath := filepath.Join(t.TempDir(), ".env") + if err := os.WriteFile(envPath, []byte("FOO=fromfile\n"), 0o644); err != nil { + t.Fatalf("WriteFile failed: %v", err) + } + + t.Setenv("FOO", "fromenv") + + if err := loadEnvFile(envPath); err != nil { + t.Fatalf("loadEnvFile failed: %v", err) + } + + if got := os.Getenv("FOO"); got != "fromenv" { + t.Fatalf("FOO = %q, want fromenv", got) + } +} + +func TestValidateConfig_PlatformBranching(t *testing.T) { + if err := validateConfig("gitlab", "", "", false, "/tmp/.env", nil); err == nil { + t.Fatalf("validateConfig(gitlab, empty token) error = nil, want non-nil") + } + if err := validateConfig("gitlab", "token", "", false, "/tmp/.env", map[string]bool{}); err == nil { + t.Fatalf("validateConfig(gitlab, empty allowed repos) error = nil, want non-nil") + } + if err := validateConfig("gitlab", "token", "", false, "/tmp/.env", map[string]bool{"group/subgroup/repo": true}); err != nil { + t.Fatalf("validateConfig(gitlab, valid inputs) error = %v, want nil", err) + } + + if err := validateConfig("github", "", "user", false, "/tmp/.env", nil); err == nil { + t.Fatalf("validateConfig(github, empty token) error = nil, want non-nil") + } + if err := validateConfig("github", "token", "", false, "/tmp/.env", nil); err == nil { + t.Fatalf("validateConfig(github, empty username) error = nil, want non-nil") + } + if err := validateConfig("github", "token", "user", false, "/tmp/.env", nil); err != nil { + t.Fatalf("validateConfig(github, valid inputs) error = %v, want nil", err) + } + + if err := validateConfig("gitlab", "", "", true, "/tmp/.env", nil); err != nil { + t.Fatalf("validateConfig(gitlab, local mode) error = %v, want nil", err) + } + if err := validateConfig("github", "", "", true, "/tmp/.env", nil); err != nil { + t.Fatalf("validateConfig(github, local mode) error = %v, want nil", err) + } +} + func TestMergeLabelWithPriority_TableDriven(t *testing.T) { tests := []struct { name string @@ -1232,7 +1278,7 @@ func TestGitLabCLIWithMockServer_ShowsMergeRequestsAndIssues(t *testing.T) { t.Fatalf("failed to create GOCACHE: %v", err) } - cmd := exec.CommandContext(ctx, "go", "run", ".", "--debug", "--time", "1d") + cmd := exec.CommandContext(ctx, "go", "run", ".", "--platform", "gitlab", "--debug", "--time", "1d") var stdoutBuf bytes.Buffer var stderrBuf bytes.Buffer cmd.Stdout = &stdoutBuf From 961538bf6686308e10def9710374a9b682c38703 Mon Sep 17 00:00:00 2001 From: Aidar Siraev Date: Thu, 12 Feb 2026 19:32:47 +0300 Subject: [PATCH 08/11] fix: fixed owner and repo name --- .goreleaser.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.goreleaser.yml b/.goreleaser.yml index e7a6d91..53d9581 100644 --- a/.goreleaser.yml +++ b/.goreleaser.yml @@ -64,8 +64,8 @@ changelog: release: github: - owner: L0thlorien - name: gitlab-feed + owner: zveinn + name: git-feed draft: false prerelease: auto mode: replace From 3c6e2861b7ed7be3306f22ce8efd0e6ff510208e Mon Sep 17 00:00:00 2001 From: Aidar Siraev Date: Thu, 12 Feb 2026 20:05:51 +0300 Subject: [PATCH 09/11] fix: renamed to git-feed, shared .env file --- .gitignore | 1 + .goreleaser.yml | 32 ++++++++-------- CLAUDE.md | 20 +++++----- README.md | 48 ++++++++++++----------- main.go | 99 +++++++++++++++++++++++++++++------------------- priority_test.go | 62 +++++++++++++++++++++++++++++- 6 files changed, 174 insertions(+), 88 deletions(-) diff --git a/.gitignore b/.gitignore index 8be55d8..c379762 100644 --- a/.gitignore +++ b/.gitignore @@ -3,3 +3,4 @@ gitlab.db gitai gitai.db .env +git-feed diff --git a/.goreleaser.yml b/.goreleaser.yml index 53d9581..8acfa20 100644 --- a/.goreleaser.yml +++ b/.goreleaser.yml @@ -1,6 +1,6 @@ version: 2 -project_name: gitlab-feed +project_name: git-feed before: hooks: @@ -8,8 +8,8 @@ before: - go mod tidy builds: - - id: gitlab-feed - binary: gitlab-feed + - id: git-feed + binary: git-feed env: - CGO_ENABLED=0 goos: @@ -76,28 +76,28 @@ release: ### macOS ```bash # Intel Mac - tar -xzf gitlab-feed_{{.Version}}_Darwin_x86_64.tar.gz - chmod +x gitlab-feed - sudo mv gitlab-feed /usr/local/bin/ + tar -xzf git-feed_{{.Version}}_Darwin_x86_64.tar.gz + chmod +x git-feed + sudo mv git-feed /usr/local/bin/ # Apple Silicon Mac - tar -xzf gitlab-feed_{{.Version}}_Darwin_arm64.tar.gz - chmod +x gitlab-feed - sudo mv gitlab-feed /usr/local/bin/ + tar -xzf git-feed_{{.Version}}_Darwin_arm64.tar.gz + chmod +x git-feed + sudo mv git-feed /usr/local/bin/ ``` ### Linux ```bash # x86_64 - tar -xzf gitlab-feed_{{.Version}}_Linux_x86_64.tar.gz - chmod +x gitlab-feed - sudo mv gitlab-feed /usr/local/bin/ + tar -xzf git-feed_{{.Version}}_Linux_x86_64.tar.gz + chmod +x git-feed + sudo mv git-feed /usr/local/bin/ # ARM64 - tar -xzf gitlab-feed_{{.Version}}_Linux_arm64.tar.gz - chmod +x gitlab-feed - sudo mv gitlab-feed /usr/local/bin/ + tar -xzf git-feed_{{.Version}}_Linux_arm64.tar.gz + chmod +x git-feed + sudo mv git-feed /usr/local/bin/ ``` ### Windows - Download the appropriate `.zip` file, extract it, and add the `gitlab-feed.exe` to your PATH. + Download the appropriate `.zip` file, extract it, and add the `git-feed.exe` to your PATH. diff --git a/CLAUDE.md b/CLAUDE.md index b1c3ebd..19a4482 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -26,32 +26,32 @@ Select the platform via `--platform github|gitlab` (default: `github`). Online mode requirements depend on platform: -- GitHub: `GITHUB_TOKEN`, `GITHUB_USERNAME` (and optionally `ALLOWED_REPOS`) -- GitLab: `GITLAB_TOKEN` (or `GITLAB_ACTIVITY_TOKEN`) and `ALLOWED_REPOS` +- GitHub: `GITHUB_TOKEN`, `GITHUB_USERNAME` (and optionally `GITHUB_ALLOWED_REPOS`) +- GitLab: `GITLAB_TOKEN` (or `GITLAB_ACTIVITY_TOKEN`) and `GITLAB_ALLOWED_REPOS` The app loads configuration from: 1) Environment variables -2) Platform `.env` file (auto-created on first run) - - GitHub: `~/.github-feed/.env` - - GitLab: `~/.gitlab-feed/.env` +2) Shared `.env` file (auto-created on first run) + - `~/.git-feed/.env` Precedence order: 1) CLI flags 2) Environment variables -3) Platform `.env` file +3) Shared `.env` file 4) Built-in defaults Environment variables: - GitHub - `GITHUB_TOKEN` (required online) - `GITHUB_USERNAME` (required online) - - `ALLOWED_REPOS` (optional; comma-separated `owner/repo`) + - `GITHUB_ALLOWED_REPOS` (optional; comma-separated `owner/repo`) - GitLab - `GITLAB_TOKEN` or `GITLAB_ACTIVITY_TOKEN` (required online) - `GITLAB_HOST` (optional host override; takes precedence over `GITLAB_BASE_URL`) - `GITLAB_BASE_URL` (optional; default: `https://gitlab.com`) - - `ALLOWED_REPOS` (required online; comma-separated `group[/subgroup]/repo`) + - `GITLAB_ALLOWED_REPOS` (required online; comma-separated `group[/subgroup]/repo`) + - `ALLOWED_REPOS` (legacy fallback for either platform when platform-specific vars are unset) - `GITLAB_USERNAME` or `GITLAB_USER` (optional legacy override; user is normally auto-resolved via API) Token scopes: @@ -61,8 +61,8 @@ Token scopes: Reference: https://docs.gitlab.com/user/profile/personal_access_tokens/ Database cache: -- GitHub: `~/.github-feed/github.db` (BBolt) -- GitLab: `~/.gitlab-feed/gitlab.db` (BBolt) +- GitHub: `~/.git-feed/github.db` (BBolt) +- GitLab: `~/.git-feed/gitlab.db` (BBolt) ## Testing diff --git a/README.md b/README.md index ffc81dd..7ac291c 100644 --- a/README.md +++ b/README.md @@ -34,16 +34,15 @@ Select the platform via `--platform github|gitlab` (default: `github`). Online mode requirements depend on platform: -- GitHub: `GITHUB_TOKEN`, `GITHUB_USERNAME` (and optionally `ALLOWED_REPOS`) -- GitLab: `GITLAB_TOKEN` (or `GITLAB_ACTIVITY_TOKEN`) and `ALLOWED_REPOS` +- GitHub: `GITHUB_TOKEN`, `GITHUB_USERNAME` (and optionally `GITHUB_ALLOWED_REPOS`) +- GitLab: `GITLAB_TOKEN` (or `GITLAB_ACTIVITY_TOKEN`) and `GITLAB_ALLOWED_REPOS` The app loads configuration in this order: 1. CLI flags 2. Environment variables -3. Platform `.env` file (auto-created on first run) - - GitHub: `~/.github-feed/.env` - - GitLab: `~/.gitlab-feed/.env` +3. Shared `.env` file (auto-created on first run) + - `~/.git-feed/.env` 4. Built-in defaults ### Environment variables @@ -52,14 +51,15 @@ GitHub: - `GITHUB_TOKEN` (required online) - `GITHUB_USERNAME` (required online) -- `ALLOWED_REPOS` (optional; comma-separated `owner/repo`) +- `GITHUB_ALLOWED_REPOS` (optional; comma-separated `owner/repo`) GitLab: - `GITLAB_TOKEN` or `GITLAB_ACTIVITY_TOKEN` (required online) - `GITLAB_HOST` (optional host override; takes precedence over `GITLAB_BASE_URL`) - `GITLAB_BASE_URL` (optional base URL, default: `https://gitlab.com`) -- `ALLOWED_REPOS` (required online; comma-separated `group[/subgroup]/repo`) +- `GITLAB_ALLOWED_REPOS` (required online; comma-separated `group[/subgroup]/repo`) +- `ALLOWED_REPOS` (legacy fallback for either platform when platform-specific vars are unset) ### Example `.env` @@ -68,8 +68,8 @@ GitLab: GITHUB_TOKEN=your_token_here GITHUB_USERNAME=your_username -# Optional in online mode; if omitted, the tool relies on platform defaults/behavior. -ALLOWED_REPOS=owner/repo1,owner/repo2 +# Optional in GitHub online mode +GITHUB_ALLOWED_REPOS=owner/repo1,owner/repo2 # GitLab (`--platform gitlab`) GITLAB_TOKEN=your_token_here @@ -81,8 +81,11 @@ GITLAB_HOST=http://1.1.1.1 # Optional explicit base URL (defaults to https://gitlab.com) GITLAB_BASE_URL=https://gitlab.com -# Required in online mode -ALLOWED_REPOS=team/repo1,platform/backend/repo2 +# Required in GitLab online mode +GITLAB_ALLOWED_REPOS=team/repo1,platform/backend/repo2 + +# Legacy fallback used only when platform-specific vars are unset +ALLOWED_REPOS= ``` ### Token scopes @@ -148,14 +151,13 @@ Reference: ## Data and cache -On first run, the tool creates a platform-specific config dir and cache DB: +On first run, the tool creates a shared config dir with one env file and platform-specific cache DBs: -- GitHub: - - `~/.github-feed/.env` - - `~/.github-feed/github.db` -- GitLab: - - `~/.gitlab-feed/.env` - - `~/.gitlab-feed/gitlab.db` +- Shared env: + - `~/.git-feed/.env` +- Platform DBs: + - `~/.git-feed/github.db` + - `~/.git-feed/gitlab.db` Online mode fetches platform activity and updates cache. Offline mode (`--local`) reads from cache only. @@ -164,15 +166,15 @@ Offline mode (`--local`) reads from cache only. ### GitHub online mode missing token/user -Set `GITHUB_TOKEN` and `GITHUB_USERNAME` in env or `~/.github-feed/.env`. +Set `GITHUB_TOKEN` and `GITHUB_USERNAME` in env or `~/.git-feed/.env`. ### GitLab online mode missing token -Set `GITLAB_TOKEN` (or `GITLAB_ACTIVITY_TOKEN`) in env or `~/.gitlab-feed/.env`. +Set `GITLAB_TOKEN` (or `GITLAB_ACTIVITY_TOKEN`) in env or `~/.git-feed/.env`. -### GitLab online mode missing `ALLOWED_REPOS` +### GitLab online mode missing `GITLAB_ALLOWED_REPOS` -Set `ALLOWED_REPOS` with valid project paths (`group[/subgroup]/repo`). +Set `GITLAB_ALLOWED_REPOS` with valid project paths (`group[/subgroup]/repo`). ### No open activity found @@ -180,7 +182,7 @@ Try: - `--debug` to inspect resolved repos and API base URL - a wider window (for example `--time 24h`) -- verifying `ALLOWED_REPOS` matches exact project paths +- verifying `GITHUB_ALLOWED_REPOS` / `GITLAB_ALLOWED_REPOS` matches exact project paths ## Development diff --git a/main.go b/main.go index a8839f4..448c9c9 100644 --- a/main.go +++ b/main.go @@ -253,6 +253,23 @@ func parseTimeRange(timeStr string) (time.Duration, error) { return duration, nil } +func resolveAllowedRepos(platform, allowedReposFlag string) string { + if value := strings.TrimSpace(allowedReposFlag); value != "" { + return value + } + + platformVar := "GITHUB_ALLOWED_REPOS" + if platform == "gitlab" { + platformVar = "GITLAB_ALLOWED_REPOS" + } + + if value := strings.TrimSpace(os.Getenv(platformVar)); value != "" { + return value + } + + return strings.TrimSpace(os.Getenv("ALLOWED_REPOS")) +} + func main() { // Define flags var timeRangeStr string @@ -271,7 +288,7 @@ func main() { flag.BoolVar(&showLinks, "links", false, "Show hyperlinks underneath each PR/issue") flag.BoolVar(&llMode, "ll", false, "Shortcut for --local --links (offline mode with links)") flag.BoolVar(&cleanCache, "clean", false, "Delete and recreate the database cache") - flag.StringVar(&allowedReposFlag, "allowed-repos", "", "Comma-separated list of allowed repos (e.g., group/repo,group/subgroup/repo)") + flag.StringVar(&allowedReposFlag, "allowed-repos", "", "Comma-separated list of allowed repos (GitHub: owner/repo; GitLab: group[/subgroup]/repo)") // Custom usage message flag.Usage = func() { @@ -286,10 +303,12 @@ func main() { fmt.Fprintln(os.Stderr, " GITLAB_BASE_URL - Optional GitLab base URL (default: https://gitlab.com)") fmt.Fprintln(os.Stderr, " GITHUB_TOKEN - GitHub Personal Access Token") fmt.Fprintln(os.Stderr, " GITHUB_USERNAME - Required in GitHub online mode") - fmt.Fprintln(os.Stderr, " ALLOWED_REPOS - Required in online mode (group[/subgroup]/repo)") + fmt.Fprintln(os.Stderr, " GITHUB_ALLOWED_REPOS - Optional in GitHub online mode (owner/repo)") + fmt.Fprintln(os.Stderr, " GITLAB_ALLOWED_REPOS - Required in GitLab online mode (group[/subgroup]/repo)") + fmt.Fprintln(os.Stderr, " ALLOWED_REPOS - Legacy fallback when platform-specific vars are unset") fmt.Fprintln(os.Stderr, "\nConfiguration File:") - fmt.Fprintln(os.Stderr, " ~/.gitlab-feed/.env - GitLab configuration file (auto-created)") - fmt.Fprintln(os.Stderr, " ~/.github-feed/.env - GitHub configuration file (auto-created)") + fmt.Fprintln(os.Stderr, " ~/.git-feed/.env - Shared configuration file (auto-created)") + fmt.Fprintln(os.Stderr, " ~/.git-feed/github.db|gitlab.db - Platform-specific cache databases") } flag.Parse() @@ -320,50 +339,57 @@ func main() { os.Exit(1) } - configDir := filepath.Join(homeDir, ".github-feed") + configDir := filepath.Join(homeDir, ".git-feed") dbFileName := "github.db" + if platform == "gitlab" { + dbFileName = "gitlab.db" + } + envTemplate := `# Activity Feed Configuration -# Add your API credentials here +# Shared environment file for both platforms -# Your GitHub Personal Access Token (required for online mode) -GITHUB_TOKEN= +# ========================= +# GitHub (--platform github) +# ========================= -# Required in online mode +# Required in GitHub online mode +GITHUB_TOKEN= GITHUB_USERNAME= -# Optional: comma-separated allowed repos -# Example: owner/repo,owner/another-repo -ALLOWED_REPOS= -` + # Optional in GitHub online mode + # Comma-separated owner/repo values + # Example: owner/repo,owner/another-repo + GITHUB_ALLOWED_REPOS= - if platform == "gitlab" { - configDir = filepath.Join(homeDir, ".gitlab-feed") - dbFileName = "gitlab.db" - envTemplate = `# Activity Feed Configuration -# Add your API credentials here +# ========================= +# GitLab (--platform gitlab) +# ========================= -# Your GitLab Personal Access Token (required for online mode) +# Required in GitLab online mode # Recommended scope: read_api (or api on some self-managed instances) GITLAB_TOKEN= +# Optional alternative token variable supported by the app +GITLAB_ACTIVITY_TOKEN= -# Optional username (the app also resolves the current user via API) +# Optional username (the app can also resolve current user via API) GITLAB_USERNAME= - # Optional: GitLab host for self-managed/cloud instances - # Example: http://10.10.1.207/ - # If set, this overrides GITLAB_BASE_URL. - GITLAB_HOST= +# Optional: GitLab host for self-managed/cloud instances +# If set, this overrides GITLAB_BASE_URL. +GITLAB_HOST= - # Optional: full GitLab base URL (supports path prefixes) - # Default: https://gitlab.com - GITLAB_BASE_URL=https://gitlab.com +# Optional: full GitLab base URL (supports path prefixes) +# Default: https://gitlab.com +GITLAB_BASE_URL=https://gitlab.com -# Required in online mode: comma-separated allowed repos -# Format: group/repo or group/subgroup/repo -# Example self-managed repo path: platform/backend/gitlab-feed -ALLOWED_REPOS= -` - } + # Required in GitLab online mode + # Comma-separated group[/subgroup]/repo values + # Example: team/repo,platform/backend/git-feed + GITLAB_ALLOWED_REPOS= + + # Legacy fallback when platform-specific vars are unset + ALLOWED_REPOS= + ` if err := os.MkdirAll(configDir, 0o755); err != nil { fmt.Printf("Error: Could not create config directory %s: %v\n", configDir, err) @@ -379,10 +405,7 @@ ALLOWED_REPOS= _ = loadEnvFile(envPath) - allowedReposStr := allowedReposFlag - if allowedReposStr == "" { - allowedReposStr = os.Getenv("ALLOWED_REPOS") - } + allowedReposStr := resolveAllowedRepos(platform, allowedReposFlag) var allowedRepos map[string]bool if allowedReposStr != "" { @@ -532,7 +555,7 @@ func validateConfig(platform, token, githubUsername string, localMode bool, envP return fmt.Errorf("token is required for GitLab API mode.\n\nTo fix this:\n - Set GITLAB_TOKEN or GITLAB_ACTIVITY_TOKEN\n - Or add it to %s", envPath) } if len(allowedRepos) == 0 { - return fmt.Errorf("ALLOWED_REPOS is required for GitLab API mode to keep API usage bounded.\n\nTo fix this:\n - Set ALLOWED_REPOS with group[/subgroup]/repo paths\n - Example: ALLOWED_REPOS=team/service,platform/backend/gitlab-feed\n - Or add it to %s", envPath) + return fmt.Errorf("GITLAB_ALLOWED_REPOS is required for GitLab API mode to keep API usage bounded.\n\nTo fix this:\n - Set GITLAB_ALLOWED_REPOS with group[/subgroup]/repo paths\n - Example: GITLAB_ALLOWED_REPOS=team/service,platform/backend/git-feed\n - Or use legacy fallback ALLOWED_REPOS\n - Or add it to %s", envPath) } case "github": if token == "" { diff --git a/priority_test.go b/priority_test.go index ec5439a..57cd8c0 100644 --- a/priority_test.go +++ b/priority_test.go @@ -1023,6 +1023,66 @@ func TestLoadEnvFile_DoesNotOverrideExistingEnv(t *testing.T) { } } +func TestResolveAllowedRepos_PerPlatformAndFallback(t *testing.T) { + tests := []struct { + name string + platform string + flagValue string + githubAllowed string + gitlabAllowed string + legacyAllowed string + want string + }{ + { + name: "flag overrides all env vars", + platform: "gitlab", + flagValue: "flag/repo", + githubAllowed: "gh/repo", + gitlabAllowed: "gl/repo", + legacyAllowed: "legacy/repo", + want: "flag/repo", + }, + { + name: "github uses platform-specific var", + platform: "github", + githubAllowed: "owner/repo1,owner/repo2", + legacyAllowed: "legacy/repo", + want: "owner/repo1,owner/repo2", + }, + { + name: "gitlab uses platform-specific var", + platform: "gitlab", + gitlabAllowed: "group/repo,group/subgroup/repo", + legacyAllowed: "legacy/repo", + want: "group/repo,group/subgroup/repo", + }, + { + name: "fallback to legacy var when platform var missing", + platform: "gitlab", + legacyAllowed: "legacy/team/repo", + want: "legacy/team/repo", + }, + { + name: "empty when nothing provided", + platform: "github", + want: "", + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Setenv("GITHUB_ALLOWED_REPOS", tt.githubAllowed) + t.Setenv("GITLAB_ALLOWED_REPOS", tt.gitlabAllowed) + t.Setenv("ALLOWED_REPOS", tt.legacyAllowed) + + got := resolveAllowedRepos(tt.platform, tt.flagValue) + if got != tt.want { + t.Fatalf("resolveAllowedRepos(%q, %q) = %q, want %q", tt.platform, tt.flagValue, got, tt.want) + } + }) + } +} + func TestValidateConfig_PlatformBranching(t *testing.T) { if err := validateConfig("gitlab", "", "", false, "/tmp/.env", nil); err == nil { t.Fatalf("validateConfig(gitlab, empty token) error = nil, want non-nil") @@ -1254,7 +1314,7 @@ func TestGitLabCLIWithMockServer_ShowsMergeRequestsAndIssues(t *testing.T) { defer cancel() homeDir := t.TempDir() - configDir := filepath.Join(homeDir, ".gitlab-feed") + configDir := filepath.Join(homeDir, ".git-feed") if err := os.MkdirAll(configDir, 0o755); err != nil { t.Fatalf("failed to create config directory: %v", err) } From 5925a5e08fc9e9ccc8edb2a8d9ec83ba4d4dd0b7 Mon Sep 17 00:00:00 2001 From: Aidar Siraev Date: Thu, 12 Feb 2026 22:46:07 +0300 Subject: [PATCH 10/11] fix: quick fix claude.md, readme.md, and some minor changes --- CLAUDE.md | 311 ++++++++++++++++++++++++++++++++++++++++++++++++++---- README.md | 53 +++++----- go.mod | 2 +- main.go | 4 +- 4 files changed, 318 insertions(+), 52 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 19a4482..725400f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,23 +1,54 @@ -# GitLab Feed (gitlab-feed) +# CLAUDE.md -GitLab Feed is a Go CLI for monitoring GitHub pull requests and GitLab merge requests and issues across a bounded set of projects. +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Project Overview + +Git Feed is a Go CLI for monitoring: +- GitHub pull requests and issues +- GitLab merge requests and issues + +The README uses the branding name "GitAI", but the binary is `git-feed`. + +This repo is the unified (GitHub + GitLab) version. Avoid adding documentation that assumes GitHub-only behavior. ## Build & Run ```bash -go build -o gitlab-feed . - -./gitlab-feed -./gitlab-feed --platform github -./gitlab-feed --platform gitlab -./gitlab-feed --time 3h -./gitlab-feed --debug -./gitlab-feed --local -./gitlab-feed --links -./gitlab-feed --ll -./gitlab-feed --clean -./gitlab-feed --allowed-repos "owner/repo,owner/other" -./gitlab-feed --platform gitlab --allowed-repos "group/repo,group/subgroup/repo" +go build -o git-feed . + +# Default: GitHub platform, online mode +./git-feed + +# Select platform +./git-feed --platform github +./git-feed --platform gitlab + +# Time window (default: 1m) +./git-feed --time 3h +./git-feed --time 2d +./git-feed --time 3w +./git-feed --time 6m +./git-feed --time 1y + +# Debug output (verbose logging) +./git-feed --debug + +# Offline mode: read from local cache DB only +./git-feed --local + +# Show hyperlinks under each MR/PR/issue +./git-feed --links + +# Shortcut: --local --links +./git-feed --ll + +# Delete and recreate the cache DB for the selected platform +./git-feed --clean + +# Restrict to a bounded set of repos/projects +./git-feed --allowed-repos "owner/repo,owner/other" +./git-feed --platform gitlab --allowed-repos "group/repo,group/subgroup/repo" ``` ## Configuration @@ -29,17 +60,17 @@ Online mode requirements depend on platform: - GitHub: `GITHUB_TOKEN`, `GITHUB_USERNAME` (and optionally `GITHUB_ALLOWED_REPOS`) - GitLab: `GITLAB_TOKEN` (or `GITLAB_ACTIVITY_TOKEN`) and `GITLAB_ALLOWED_REPOS` -The app loads configuration from: -1) Environment variables -2) Shared `.env` file (auto-created on first run) - - `~/.git-feed/.env` - Precedence order: 1) CLI flags 2) Environment variables 3) Shared `.env` file 4) Built-in defaults +The `.env` file is auto-created on first run at: +- `~/.git-feed/.env` + +Important: `.env` loading does not override already-set environment variables. + Environment variables: - GitHub - `GITHUB_TOKEN` (required online) @@ -52,7 +83,7 @@ Environment variables: - `GITLAB_BASE_URL` (optional; default: `https://gitlab.com`) - `GITLAB_ALLOWED_REPOS` (required online; comma-separated `group[/subgroup]/repo`) - `ALLOWED_REPOS` (legacy fallback for either platform when platform-specific vars are unset) - - `GITLAB_USERNAME` or `GITLAB_USER` (optional legacy override; user is normally auto-resolved via API) + - `GITLAB_USERNAME` or `GITLAB_USER` (documented in help/template, but the current code resolves the GitLab user via API and does not read these vars) Token scopes: - `read_api` (recommended) @@ -64,6 +95,244 @@ Database cache: - GitHub: `~/.git-feed/github.db` (BBolt) - GitLab: `~/.git-feed/gitlab.db` (BBolt) +Notes: +- The tool uses a platform-specific database file (based on `--platform`). Both files share the same on-disk schema. +- You will only see both `github.db` and `gitlab.db` after running the tool at least once for each platform. + +## First Run Behavior + +On first run the app creates `~/.git-feed/` (permissions: 0755) and ensures: +- `~/.git-feed/.env` exists (permissions: 0600) +- The platform database exists (permissions: 0666) + +The `.env` file contains a template with both GitHub and GitLab variables. + +## Architecture & Key Components + +This repo is organized as a single CLI entrypoint (`main.go`) that dispatches to one of two platform implementations: +- `platform_github.go` (GitHub) +- `platform_gitlab.go` (GitLab) + +Both platforms share: +- common models (`MergeRequestModel`, `IssueModel`) used for display +- label priority logic (`shouldUpdateLabel`) and its tests +- a BBolt cache (`db.go`) used for offline mode and persistence + +### Data Flow + +#### Platform Selection +`main.go` parses flags, sets up `~/.git-feed/.env` and the cache database file, loads environment variables, validates online requirements, then calls `fetchAndDisplayActivity(platform)`. + +#### GitHub Online Mode (Default when `--platform github` and not `--local`) +1. **Search**: runs several GitHub Search API queries to find PRs and issues the user is involved in. +2. **Hydrate details**: fetches full PR/issue objects by number (not just search items). +3. **Review comment collection**: fetches PR review comments for cross-reference detection. +4. **Caching**: stores PRs, issues, and PR review comments to `~/.git-feed/github.db`. +5. **Cross-reference nesting**: nests issues under PRs when references are detected in bodies or review comments. +6. **Rendering**: prints grouped sections (open PRs, closed/merged PRs, open issues, closed issues), optionally with links. + +#### GitHub Offline Mode (`--local`) +1. **Database loading**: reads PRs, issues, and PR review comments from `~/.git-feed/github.db`. +2. **Filtering**: applies the cutoff time (`--time`) and optional allowed repos. +3. **Cross-reference nesting**: uses the same cross-reference logic as online mode. +4. **Rendering**: same output layout as online mode. + +#### GitLab Online Mode (Default when `--platform gitlab` and not `--local`) +GitLab online mode is intentionally bounded: `GITLAB_ALLOWED_REPOS` (or `ALLOWED_REPOS`) must be set. + +1. **Project resolution**: resolves each allowed `group[/subgroup]/repo` path to a project ID via the Projects API. +2. **Per-project scans**: lists merge requests and issues updated after the cutoff using project-scoped list endpoints. +3. **Label derivation**: + - Uses MR/issue author and assignees first. + - Uses approval state for "Reviewed" on merge requests. + - Uses reviewers list for "Review Requested". + - Uses notes (comments) to detect "Commented" and "Mentioned". +4. **Caching**: stores merge requests, issues, and relevant notes to `~/.git-feed/gitlab.db`. +5. **Cross-reference nesting**: + - Preferred: uses GitLab's "issues closed on merge request" endpoint. + - Fallback: parses MR bodies/notes for issue references (same-project refs, qualified refs, and issue URLs). +6. **Rendering**: same section layout as GitHub mode, using the unified models. + +#### GitLab Offline Mode (`--local`) +1. **Database loading**: reads cached MRs, issues, and notes from `~/.git-feed/gitlab.db`. +2. **Filtering**: applies cutoff time and allowed projects. +3. **Cross-reference nesting**: parses MR bodies and cached notes for issue references. +4. **Rendering**: same output layout. + +### Core Data Structures + +**Config** (`main.go`): runtime configuration and shared references. +- Controls mode flags (`debugMode`, `localMode`, `showLinks`), platform credentials, allowed repos, and cache handle. + +**PRActivity** (`main.go`): a unified "merge request" activity record. +- Used for both GitHub pull requests and GitLab merge requests. +- Fields: involvement label, owner/repo, `MergeRequestModel`, update time, nested issues. + +**IssueActivity** (`main.go`): a unified issue activity record. + +**MergeRequestModel** / **IssueModel** (`main.go`): simplified, platform-neutral view models. +- These are the types stored in BBolt for both platforms. + +### Label Priority System + +When the same item matches multiple involvement sources, the display shows the most important label. + +**PR/MR Label Priorities** (highest to lowest): +1. Authored +2. Assigned +3. Reviewed +4. Review Requested +5. Commented +6. Mentioned + +**Issue Label Priorities** (highest to lowest): +1. Authored +2. Assigned +3. Commented +4. Mentioned + +The shared helper `shouldUpdateLabel(current, candidate, isPR)` implements this rule. + +### Cross-Reference Detection + +**GitHub** (`platform_github.go`): +- Only nests issues under PRs within the same repository. +- Detects references in: + - PR body and issue body + - PR review comments +- Supported patterns include: + - `#123` + - `owner/repo#123` + - `https://github.com/owner/repo/issues/123` (and `/pull/`) + +**GitLab** (`platform_gitlab.go`): +- Preferred nesting via API endpoint: issues closed on merge request. +- Fallback parsing from MR bodies and notes: + - same-project `#123` + - qualified `group/subgroup/repo#123` + - issue URLs (including relative `/-/issues/123`) + +## GitHub API Integration + +Uses `google/go-github/v57`. + +Key API patterns in `platform_github.go`: +- Search: `client.Search.Issues()` to discover candidate items. +- Details: `client.PullRequests.Get()` and `client.Issues.Get()` to fetch canonical fields. +- Comments: `client.PullRequests.ListComments()` for review comment bodies (used for cross-reference detection). + +## GitLab API Integration + +Uses `gitlab.com/gitlab-org/api/client-go`. + +Base URL handling: +- `GITLAB_HOST` or `GITLAB_BASE_URL` is normalized to include `/api/v4` (and supports path prefixes). + +Retry strategy: +- GitLab requests are wrapped via `retryWithBackoff()` for 429 rate limits and transient 5xx errors. +- For 429 responses the code respects `Retry-After` when present, otherwise uses `Ratelimit-Reset` when available. + +## Database Module (db.go) + +The cache uses BBolt and stores platform data as JSON. + +Buckets: +- GitLab: `gitlab_merge_requests`, `gitlab_issues`, `gitlab_notes` +- GitHub: `pull_requests`, `issues`, `comments` + +Key formats: +- GitLab MR key: `path_with_namespace#!IID` +- GitLab issue key: `path_with_namespace##IID` +- GitLab note key: `path|itemType|iid|noteID` +- GitHub item key: `owner/repo#number` +- GitHub PR review comment key: `owner/repo#number/pr_review_comment/commentID` + +Data formats: +- GitHub and GitLab store their simplified models (`MergeRequestModel`, `IssueModel`) wrapped with a `Label` field. +- GitHub readers keep backwards compatibility by falling back to unmarshaling legacy (unwrapped) records. + +## Command-Line Flags + +Flags are parsed with the standard library `flag` package (`main.go`). + +- `--platform github|gitlab` (default: `github`) +- `--time RANGE` (default: `1m`; supports `h`, `d`, `w`, `m`, `y`) +- `--debug` (verbose logging) +- `--local` (offline mode from cache) +- `--links` (print item URLs under each entry) +- `--ll` (shortcut for `--local --links`) +- `--clean` (delete and recreate the selected platform DB) +- `--allowed-repos` (comma-separated) + - GitHub: `owner/repo` + - GitLab: `group[/subgroup]/repo` + +Allowed repo resolution order: +1. `--allowed-repos` +2. `GITHUB_ALLOWED_REPOS` or `GITLAB_ALLOWED_REPOS` (depending on `--platform`) +3. `ALLOWED_REPOS` (legacy fallback) + +## Testing Considerations + +When modifying this codebase: +- Validate both platforms (`--platform github` and `--platform gitlab`). +- Validate both modes (`--local` vs online). +- If touching GitLab behavior, ensure rate limit/backoff behavior still passes the table-driven tests. +- If changing reference parsing, add tests for both GitHub and GitLab patterns. +- Cache schema changes should preserve offline compatibility. + +## Testing + +Run the full test suite: + +```bash +go test ./... -count=1 +``` + +Notable tests (in `priority_test.go`): +- label priority and `shouldUpdateLabel()` behavior +- GitLab base URL normalization (`normalizeGitLabBaseURL`) +- retry/backoff behavior for GitLab rate limits and transient errors (`retryWithBackoff`) +- database round-trip and offline parity for GitLab cache +- end-to-end `go run . --platform gitlab --debug` against a mock GitLab server + +## Known Issues & Discrepancies + +These are documentation/behavior mismatches worth keeping in mind while working on the repo: + +1. GitLab username env vars: `GITLAB_USERNAME` / `GITLAB_USER` appear in the usage text and `.env` template, but the current implementation resolves the current user via API and does not read those variables. +2. Progress bar wiring: a `Progress` type exists and is used for retry countdown messaging when set, but `config.progress` is not initialized in the main execution path (so progress rendering is effectively disabled). + +## Refactoring Opportunities + +Potential improvements that would reduce complexity or improve UX (not required for normal changes): +- Parallelize GitHub query passes and/or per-project GitLab scanning while keeping API usage bounded. +- Wire up `Progress` in non-debug mode (or remove it if not desired). +- Consolidate shared display and nesting logic further, while keeping platform-specific API details isolated. + +## File Structure + +``` +git-feed/ +├── main.go # CLI entrypoint, config, shared models, output rendering +├── platform_github.go # GitHub API fetch + caching + nesting +├── platform_gitlab.go # GitLab API fetch + caching + nesting + retry +├── db.go # BBolt schema and persistence helpers +├── priority_test.go # Unit/integration tests +├── go.mod # Module: github.com/zveinn/git-feed +├── go.sum +├── README.md +├── CLAUDE.md +├── .goreleaser.yml +└── .github/ + └── workflows/ + └── release.yml + +~/.git-feed/ # Config directory (auto-created) + ├── .env # Shared configuration file + ├── github.db # GitHub cache DB (created after running with --platform github) + └── gitlab.db # GitLab cache DB (created after running with --platform gitlab) +``` + ## Testing ```bash diff --git a/README.md b/README.md index 7ac291c..fcd47f1 100644 --- a/README.md +++ b/README.md @@ -1,32 +1,29 @@ -# GitAI - Activity Monitor +# GitAI - Git Feed Activity Monitor A fast, colorful CLI tool for monitoring GitHub pull requests and GitLab merge requests and issues across repositories. Track your contributions, reviews, and assignments with real-time progress visualization. -fork from [GitAI GitHub feed](https://github.com/zveinn/github-feed) - ## Features -- Parallel API fetching for faster scans -- Colorized, grouped output (open/closed/merged) -- Smart MR/issue cross-reference nesting -- Online mode with local BBolt cache -- Offline mode from cache (`--local`) -- Time-window filtering (`--time 1h|2d|3w|4m|1y`) -- Retry/backoff for API rate-limit and transient API errors +- 🚀 **Parallel API fetching** - Faster scans across repositories +- 🎨 **Colorized grouped output** - Easy-to-read open/closed/merged sections +- 🔗 **Smart cross-reference nesting** - Links related MRs and issues +- 💾 **Online + local BBolt cache** - Fetch online or use `--local` offline mode +- ⏱ **Time-window filtering** - Configure with `--time 1h|2d|3w|4m|1y` +- ♻️ **Retry/backoff handling** - Better resilience to API rate limits and transient failures ## Installation ### Build from source ```bash -go build -o gitlab-feed . +go build -o git-feed . ``` ### Pre-built binaries Download from GitHub Releases: -- https://github.com/L0thlorien/gitlab-feed/releases +- https://github.com/zveinn/git-feed/releases ## Configuration @@ -103,37 +100,37 @@ Reference: ```bash # Default: last month (1m), online mode, platform=github -./gitlab-feed +./git-feed # Explicit platform -./gitlab-feed --platform github -./gitlab-feed --platform gitlab +./git-feed --platform github +./git-feed --platform gitlab # Time window examples -./gitlab-feed --time 3h -./gitlab-feed --time 2d -./gitlab-feed --time 3w -./gitlab-feed --time 6m -./gitlab-feed --time 1y +./git-feed --time 3h +./git-feed --time 2d +./git-feed --time 3w +./git-feed --time 6m +./git-feed --time 1y # Debug output -./gitlab-feed --debug +./git-feed --debug # Offline from cache -./gitlab-feed --local +./git-feed --local # Show links -./gitlab-feed --links +./git-feed --links # Shortcut: --local --links -./gitlab-feed --ll +./git-feed --ll # Recreate cache DB -./gitlab-feed --clean +./git-feed --clean # Override allowed repos from CLI -./gitlab-feed --allowed-repos "owner/repo,owner/other" -./gitlab-feed --platform gitlab --allowed-repos "group/repo,group/subgroup/repo" +./git-feed --allowed-repos "owner/repo,owner/other" +./git-feed --platform gitlab --allowed-repos "group/repo,group/subgroup/repo" ``` ### Flags @@ -188,7 +185,7 @@ Try: ```bash go test ./... -count=1 -go build -o gitlab-feed . +go build -o git-feed . ``` Current core files: diff --git a/go.mod b/go.mod index 26038f2..f692e94 100644 --- a/go.mod +++ b/go.mod @@ -1,4 +1,4 @@ -module github.com/L0thlorien/gitlab-feed +module github.com/zveinn/git-feed go 1.25 diff --git a/main.go b/main.go index 448c9c9..1f88138 100644 --- a/main.go +++ b/main.go @@ -284,7 +284,7 @@ func main() { flag.StringVar(&timeRangeStr, "time", "1m", "Show items from last time range (1h, 2d, 3w, 4m, 1y)") flag.StringVar(&platform, "platform", "github", "Platform to use (gitlab|github)") flag.BoolVar(&debugMode, "debug", false, "Show detailed API logging") - flag.BoolVar(&localMode, "local", false, "Use local database instead of GitLab API") + flag.BoolVar(&localMode, "local", false, "Use local database instead of platform API") flag.BoolVar(&showLinks, "links", false, "Show hyperlinks underneath each PR/issue") flag.BoolVar(&llMode, "ll", false, "Shortcut for --local --links (offline mode with links)") flag.BoolVar(&cleanCache, "clean", false, "Delete and recreate the database cache") @@ -293,7 +293,7 @@ func main() { // Custom usage message flag.Usage = func() { fmt.Fprintf(os.Stderr, "Usage: %s [options]\n\n", os.Args[0]) - fmt.Fprintln(os.Stderr, "GitLab Feed - Monitor pull requests and issues across repositories") + fmt.Fprintln(os.Stderr, "Git Feed - Monitor pull requests and issues across repositories") fmt.Fprintln(os.Stderr, "\nOptions:") flag.PrintDefaults() fmt.Fprintln(os.Stderr, "\nEnvironment Variables:") From b1897d5abce766cb21f0d4c92c339d0b48b9a222 Mon Sep 17 00:00:00 2001 From: Aidar Siraev Date: Fri, 13 Feb 2026 09:41:02 +0300 Subject: [PATCH 11/11] fix: refining readme.md --- README.md | 361 ++++++++++++++++++++++++++++++++++++------------------ 1 file changed, 241 insertions(+), 120 deletions(-) diff --git a/README.md b/README.md index fcd47f1..0c83741 100644 --- a/README.md +++ b/README.md @@ -1,201 +1,322 @@ -# GitAI - Git Feed Activity Monitor +# GitAI - GitHub Activity Monitor -A fast, colorful CLI tool for monitoring GitHub pull requests and GitLab merge requests and issues across repositories. Track your contributions, reviews, and assignments with real-time progress visualization. +A fast, colorful CLI tool for monitoring GitHub pull requests/issues and GitLab merge requests/issues across repositories. Track your contributions, reviews, and assignments with real-time progress visualization. ## Features -- 🚀 **Parallel API fetching** - Faster scans across repositories -- 🎨 **Colorized grouped output** - Easy-to-read open/closed/merged sections -- 🔗 **Smart cross-reference nesting** - Links related MRs and issues -- 💾 **Online + local BBolt cache** - Fetch online or use `--local` offline mode -- ⏱ **Time-window filtering** - Configure with `--time 1h|2d|3w|4m|1y` -- ♻️ **Retry/backoff handling** - Better resilience to API rate limits and transient failures +- 🚀 **Parallel API Calls** - Fetches data concurrently for maximum speed +- 🎨 **Colorized Output** - Easy-to-read color-coded labels, states, and progress +- 📊 **Smart Cross-Referencing** - Automatically links related PRs and issues +- ⚡ **Real-Time Progress Bar** - Visual feedback with color-coded completion status +- 🔍 **Comprehensive Search** - Tracks authored, mentioned, assigned, commented, and reviewed items +- 📅 **Time Filtering** - View items from the last month by default (configurable with `--time`) +- 🎯 **Organized Display** - Separates open, merged, and closed items into clear sections ## Installation -### Build from source +### Pre-built Binaries (Recommended) +Download the latest release for your platform from the [releases page](https://github.com/zveinn/git-feed/releases): + +**macOS** ```bash -go build -o git-feed . +# Intel Mac +curl -L https://github.com/zveinn/git-feed/releases/latest/download/git-feed__Darwin_x86_64.tar.gz | tar xz +chmod +x git-feed +sudo mv git-feed /usr/local/bin/ + +# Apple Silicon Mac +curl -L https://github.com/zveinn/git-feed/releases/latest/download/git-feed__Darwin_arm64.tar.gz | tar xz +chmod +x git-feed +sudo mv git-feed /usr/local/bin/ ``` -### Pre-built binaries +**Linux** +```bash +# x86_64 +curl -L https://github.com/zveinn/git-feed/releases/latest/download/git-feed__Linux_x86_64.tar.gz | tar xz +chmod +x git-feed +sudo mv git-feed /usr/local/bin/ + +# ARM64 +curl -L https://github.com/zveinn/git-feed/releases/latest/download/git-feed__Linux_arm64.tar.gz | tar xz +chmod +x git-feed +sudo mv git-feed /usr/local/bin/ +``` -Download from GitHub Releases: +**Windows** -- https://github.com/zveinn/git-feed/releases +Download the appropriate `.zip` file from the releases page, extract it, and add `git-feed.exe` to your PATH. -## Configuration +### Build from Source + +```bash +go build -o git-feed . +``` + +### Release Management -Select the platform via `--platform github|gitlab` (default: `github`). +Releases are automatically built and published via GitHub Actions using GoReleaser: -Online mode requirements depend on platform: +```bash +# Create a new release +git tag -a v1.0.0 -m "Release v1.0.0" +git push origin v1.0.0 +``` -- GitHub: `GITHUB_TOKEN`, `GITHUB_USERNAME` (and optionally `GITHUB_ALLOWED_REPOS`) -- GitLab: `GITLAB_TOKEN` (or `GITLAB_ACTIVITY_TOKEN`) and `GITLAB_ALLOWED_REPOS` +This will automatically: +- Build binaries for Linux (amd64, arm64), macOS (Intel, Apple Silicon), and Windows (amd64) +- Generate checksums for all releases +- Create a GitHub release with installation instructions +- Publish all artifacts to the releases page -The app loads configuration in this order: +## Configuration -1. CLI flags -2. Environment variables -3. Shared `.env` file (auto-created on first run) - - `~/.git-feed/.env` -4. Built-in defaults +### First Run Setup -### Environment variables +On first run, GitAI automatically creates a configuration directory at `~/.git-feed/` with: +- `.env` - Shared configuration file for GitHub and GitLab +- `github.db` - Local database for caching GitHub data +- `gitlab.db` - Local database for caching GitLab data -GitHub: +### GitHub Token Setup -- `GITHUB_TOKEN` (required online) -- `GITHUB_USERNAME` (required online) -- `GITHUB_ALLOWED_REPOS` (optional; comma-separated `owner/repo`) +Create a GitHub Personal Access Token with the following scopes: +- `repo` - Access to repositories +- `read:org` - Read organization data -GitLab: +**Generate token:** https://github.com/settings/tokens -- `GITLAB_TOKEN` or `GITLAB_ACTIVITY_TOKEN` (required online) -- `GITLAB_HOST` (optional host override; takes precedence over `GITLAB_BASE_URL`) -- `GITLAB_BASE_URL` (optional base URL, default: `https://gitlab.com`) -- `GITLAB_ALLOWED_REPOS` (required online; comma-separated `group[/subgroup]/repo`) -- `ALLOWED_REPOS` (legacy fallback for either platform when platform-specific vars are unset) +### Environment Setup -### Example `.env` +You can provide credentials via a shared env file or environment variables. Select platform with `--platform github|gitlab` (default: `github`). +**Option 1: Configuration File (Recommended)** + +Edit `~/.git-feed/.env` and add your credentials: ```bash # GitHub (`--platform github`) +# Required in GitHub online mode GITHUB_TOKEN=your_token_here GITHUB_USERNAME=your_username # Optional in GitHub online mode -GITHUB_ALLOWED_REPOS=owner/repo1,owner/repo2 +GITHUB_ALLOWED_REPOS=user/repo1,user/repo2 # GitLab (`--platform gitlab`) +# Required in GitLab online mode GITLAB_TOKEN=your_token_here +# Optional alternative token variable +GITLAB_ACTIVITY_TOKEN= -# Optional host override, e.g. self-managed GitLab -# If set, this overrides GITLAB_BASE_URL. -GITLAB_HOST=http://1.1.1.1 - -# Optional explicit base URL (defaults to https://gitlab.com) +# Optional host/base URL settings +GITLAB_HOST= GITLAB_BASE_URL=https://gitlab.com # Required in GitLab online mode -GITLAB_ALLOWED_REPOS=team/repo1,platform/backend/repo2 +GITLAB_ALLOWED_REPOS=group/repo1,group/subgroup/repo2 # Legacy fallback used only when platform-specific vars are unset ALLOWED_REPOS= ``` -### Token scopes - -Create a GitLab Personal Access Token with: - -- `read_api` (recommended) -- `api` only if your self-managed setup requires broader scope +**Option 2: Environment Variables** +```bash +export GITHUB_TOKEN="your_token_here" +export GITHUB_USERNAME="your_username" +export GITHUB_ALLOWED_REPOS="user/repo1,user/repo2" # Optional in GitHub mode -Reference: +export GITLAB_TOKEN="your_token_here" +export GITLAB_ALLOWED_REPOS="group/repo1,group/subgroup/repo2" # Required in GitLab mode +``` -- https://docs.gitlab.com/user/profile/personal_access_tokens/ +**Note:** Environment variables take precedence over the `.env` file. ## Usage +### Basic Usage + ```bash -# Default: last month (1m), online mode, platform=github -./git-feed +# Monitor items from the last month (default, platform=github) +git-feed # Explicit platform -./git-feed --platform github -./git-feed --platform gitlab - -# Time window examples -./git-feed --time 3h -./git-feed --time 2d -./git-feed --time 3w -./git-feed --time 6m -./git-feed --time 1y +git-feed --platform github +git-feed --platform gitlab -# Debug output -./git-feed --debug +# Show items from the last 3 hours +git-feed --time 3h -# Offline from cache -./git-feed --local +# Show items from the last 2 days +git-feed --time 2d -# Show links -./git-feed --links +# Show items from the last 3 weeks +git-feed --time 3w -# Shortcut: --local --links -./git-feed --ll +# Show items from the last 6 months +git-feed --time 6m -# Recreate cache DB -./git-feed --clean +# Show items from the last year +git-feed --time 1y -# Override allowed repos from CLI -./git-feed --allowed-repos "owner/repo,owner/other" -./git-feed --platform gitlab --allowed-repos "group/repo,group/subgroup/repo" -``` +# Show detailed logging output +git-feed --debug -### Flags - -| Flag | Description | -|------|-------------| -| `--time RANGE` | Show items from last time range (default: `1m`). Examples: `1h`, `2d`, `3w`, `4m`, `1y` | -| `--platform PLATFORM` | Activity source platform: `github` or `gitlab` (default: `github`) | -| `--debug` | Show detailed API logging | -| `--local` | Use local database instead of API | -| `--links` | Show hyperlinks under each MR/issue | -| `--ll` | Shortcut for `--local --links` | -| `--clean` | Delete and recreate the database cache | -| `--allowed-repos REPOS` | Comma-separated repo paths (GitHub: `owner/repo`; GitLab: `group[/subgroup]/repo`) | +# Use local database instead of GitHub API (offline mode) +git-feed --local -## Data and cache +# Show hyperlinks underneath each PR/issue +git-feed --links -On first run, the tool creates a shared config dir with one env file and platform-specific cache DBs: +# Delete and recreate the database cache (start fresh) +git-feed --clean -- Shared env: - - `~/.git-feed/.env` -- Platform DBs: - - `~/.git-feed/github.db` - - `~/.git-feed/gitlab.db` +# Filter to specific repositories only +git-feed --allowed-repos="user/repo1,user/repo2" -Online mode fetches platform activity and updates cache. -Offline mode (`--local`) reads from cache only. +# Quick offline mode with links (combines --local and --links) +git-feed --ll -## Troubleshooting +# Combine flags +git-feed --local --time 2w --debug --links --allowed-repos="miniohq/ec,tunnels-is/tunnels" +``` -### GitHub online mode missing token/user +### Command Line Options -Set `GITHUB_TOKEN` and `GITHUB_USERNAME` in env or `~/.git-feed/.env`. +| Flag | Description | +|------|-------------| +| `--time RANGE` | Show items from the last time range (default: `1m`)
Examples: `1h` (hour), `2d` (days), `3w` (weeks), `4m` (months), `1y` (year) | +| `--platform PLATFORM` | Activity source platform: `github` or `gitlab` (default: `github`) | +| `--debug` | Show detailed API call progress instead of progress bar | +| `--local` | Use local database instead of platform API (offline mode, no token required) | +| `--links` | Show hyperlinks (with 🔗 icon) underneath each PR and issue | +| `--ll` | Shortcut for `--local --links` (offline mode with links) | +| `--clean` | Delete and recreate the database cache (useful for starting fresh or fixing corrupted cache) | +| `--allowed-repos REPOS` | Filter to specific repositories (GitHub: `owner/repo1`; GitLab: `group[/subgroup]/repo`) | + +### Color Coding + +**Labels:** +- `AUTHORED` - Cyan +- `MENTIONED` - Yellow +- `ASSIGNED` - Magenta +- `COMMENTED` - Blue +- `REVIEWED` - Green +- `REVIEW REQUESTED` - Red +- `INVOLVED` - Gray + +**States:** +- `OPEN` - Green +- `CLOSED` - Red +- `MERGED` - Magenta + +**Usernames:** Each user gets a consistent color based on hash + +## How It Works + +### Online Mode (Default) + +1. **Parallel Fetching** - Simultaneously searches for activity on the selected platform: + - PRs you authored + - PRs where you're mentioned + - PRs assigned to you + - PRs you commented on + - PRs you reviewed + - PRs requesting your review + - PRs involving you + - Your recent activity events + - Issues you authored/mentioned/assigned/commented + +2. **Local Caching** - All fetched data is automatically saved to a local BBolt database (`~/.git-feed/github.db` for GitHub or `~/.git-feed/gitlab.db` for GitLab) + - MRs/PRs, issues, and comments/notes are cached for offline access + - Each item is stored/updated with a unique key + - Database grows as you fetch more data + +3. **Cross-Reference Detection** - Automatically finds connections between PRs and issues by: + - Checking PR body and comments for issue references (`#123`, `fixes #123`, full URLs) + - Checking issue body and comments for PR references + - Displaying linked issues directly under their related PRs + +4. **Smart Filtering**: + - Shows both open and closed items from the specified time period + - **Default**: Items updated in last month (`1m`) + - **Custom**: Use `--time` with values like `1h`, `2d`, `3w`, `6m`, `1y` + +### Offline Mode (`--local`) + +- Reads all data from the selected local database instead of platform APIs +- No internet connection or API token required +- Displays all cached PR/MR and issue activity +- Useful for: + - Working offline + - Faster lookups when you don't need fresh data + - Reviewing previously fetched data + +## API Rate Limits + +GitAI monitors GitHub API rate limits and will warn you when running low: +- **Search API**: 30 requests per minute +- **Core API**: 5000 requests per hour + +Rate limit status is displayed in debug mode. + +### Automatic Retry & Backoff + +When rate limits are hit, GitAI automatically retries with exponential backoff: +- Detects rate limit errors (429, 403 responses) +- Waits progressively longer between retries (1s → 2s → 4s → ... up to 30s max) +- Continues indefinitely until the request succeeds +- Shows clear warnings: `⚠ Rate limit hit, waiting [duration] before retry...` +- No manual intervention required - the tool handles rate limits gracefully -### GitLab online mode missing token +## Troubleshooting -Set `GITLAB_TOKEN` (or `GITLAB_ACTIVITY_TOKEN`) in env or `~/.git-feed/.env`. +### "GITHUB_TOKEN environment variable is required" +Set up your GitHub token (`GITHUB_TOKEN`) for `--platform github`, or GitLab token (`GITLAB_TOKEN` / `GITLAB_ACTIVITY_TOKEN`) plus `GITLAB_ALLOWED_REPOS` for `--platform gitlab`. -### GitLab online mode missing `GITLAB_ALLOWED_REPOS` +### "Rate limit exceeded" +Wait for the rate limit to reset. Use `--debug` to see current rate limits. -Set `GITLAB_ALLOWED_REPOS` with valid project paths (`group[/subgroup]/repo`). +### Progress bar looks garbled +Your terminal may not support ANSI colors properly. Use `--debug` mode for plain text output. -### No open activity found +## Development -Try: +### Project Structure +``` +git-feed/ +├── main.go # Main application code +├── platform_github.go # GitHub platform implementation +├── platform_gitlab.go # GitLab platform implementation +├── db.go # Database operations for caching GitHub data +├── README.md # This file +├── CLAUDE.md # Instructions for Claude Code AI assistant +├── .goreleaser.yml # GoReleaser configuration for builds +├── .github/ +│ └── workflows/ +│ └── release.yml # GitHub Actions workflow for releases + +~/.git-feed/ # Config directory (auto-created) + ├── .env # Shared configuration file + ├── github.db # BBolt database for GitHub cache + └── gitlab.db # BBolt database for GitLab cache +``` -- `--debug` to inspect resolved repos and API base URL -- a wider window (for example `--time 24h`) -- verifying `GITHUB_ALLOWED_REPOS` / `GITLAB_ALLOWED_REPOS` matches exact project paths +### Testing Releases Locally -## Development +You can test the GoReleaser build locally before pushing a tag: ```bash -go test ./... -count=1 -go build -o git-feed . -``` +# Install goreleaser +go install github.com/goreleaser/goreleaser/v2@latest -Current core files: +# Test the build (creates snapshot without publishing) +goreleaser release --snapshot --clean -- `main.go` -- `db.go` -- `priority_test.go` -- `CLAUDE.md` -- `README.md` +# Check the dist/ folder for built binaries +ls -la dist/ +``` ## License -MIT +MIT License - Feel free to use and modify as needed.