Adding the points allocated to users in a PR comment#131
Adding the points allocated to users in a PR comment#131LavanyaK235 wants to merge 80 commits intomainfrom
Conversation
* Added CODEOWNERS * Added .gitignore --------- Co-authored-by: Johan Broberg <johanb@microsoft.com>
Co-authored-by: Johan Broberg <johanb@microsoft.com>
Co-authored-by: Johan Broberg <johanb@microsoft.com>
Co-authored-by: Jesus Terrazas <jterrazas@microsoft.com>
Co-authored-by: Johan Broberg <johanb@microsoft.com>
Co-authored-by: Jesus Terrazas <jterrazas@microsoft.com>
* Adding Python Agent Framework Sample * update to latest --------- Co-authored-by: Jesus Terrazas <jterrazas@microsoft.com>
* Revise README for Agent 365 Sample Agent Updated the README to improve clarity and correct typos. Added sections on prerequisites, running the sample, and troubleshooting. * Update README to remove external sample reference Removed reference to the semantic-kernel-multiturn sample.
* update agents sdk version for langchain * Accept more recent versions * remove authority field, this is set by default --------- Co-authored-by: Jesus Terrazas <jterrazas@microsoft.com>
* Adding Notifications on AF * Update readme * PR comments * small fix * removed observability * update manifest * changes * setup fix
Added comprehensive instructions for GitHub Copilot to follow during code reviews, including rules for checking keywords, copyright headers, and Agent 365 sample validation.
* update with api change * update claude to tool service api name change * name change --------- Co-authored-by: Jesus Terrazas <jterrazas@microsoft.com>
* Add Vercel AI SDK sample * fix comments --------- Co-authored-by: Dominik Bezic <dominikbezic@microsoft.com>
* introducing perplexity sample agent * applying changes from code review * updated readme file with features list --------- Co-authored-by: aubreyquinn <aubreyquinn+odspmdb@microsoft.com>
* Add OpenAI Nodejs Sample * fix: model to string * Add copyright headers * Add build script * fix typo * fix typo in doc * take comments * update doc for env name change * update license path in readme * update walkthrough to follow same structure as AF * include agentsplayground in readme --------- Co-authored-by: Jesus Terrazas <jterrazas@microsoft.com>
…ion (#31) * Initial plan * Rename "Microsoft Agents A365" to "Microsoft Agent 365" Co-authored-by: pontemonti <7850950+pontemonti@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: pontemonti <7850950+pontemonti@users.noreply.github.com>
Co-authored-by: Jesus Terrazas <jterrazas@microsoft.com>
* add updated nodejs claude implementation * fix typos * change mcp endpoint to prod * review comments * add auth from agents-hosting * remove lock
* quickstart initial code * rename folder * update docs on quickstart * add links and update docs * include rename change * copilot suggestions --------- Co-authored-by: Jesus Terrazas <jterrazas@microsoft.com>
* Introducing playground notification handling in Perplexity agent * Introducing playground Teams messaging notification handling in Perplexity agent * applying changes from code review * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * applying changes from code review * Add missing copyright header to playgroundActivityTypes.ts (#36) * Initial plan * Add Microsoft copyright header to playgroundActivityTypes.ts Co-authored-by: aubreyquinn <80953505+aubreyquinn@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: aubreyquinn <80953505+aubreyquinn@users.noreply.github.com> * Add copyright header to playgroundActivityTypes.ts (#35) * Initial plan * Add Microsoft copyright header to playgroundActivityTypes.ts Co-authored-by: aubreyquinn <80953505+aubreyquinn@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: aubreyquinn <80953505+aubreyquinn@users.noreply.github.com> --------- Co-authored-by: aubreyquinn <aubreyquinn+odspmdb@microsoft.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
* Cursor IDE prompt usage --------- Co-authored-by: shirahman <shirahman@microsoft.com>
* Update samples to remove environment id from calls and settings * remove env id from newer samples * Update python/agent-framework/sample-agent/.env.template Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Jesus Terrazas <jterrazas@microsoft.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Fix formatting of 'Microsoft Agent365' to 'Microsoft Agent 365' * Update README to reflect Microsoft Agent 365 SDK * Fix typo in Agent365 authentication comment * Fix comment formatting in client.ts Updated comments to improve clarity and consistency. * Fix typo in AGENT-TESTING.md * Fix description formatting in pyproject.toml * Update package description for clarity * Fix formatting of Agent 365 configuration comment * Update README to reference Microsoft Agent 365 SDK * Fix typo in Agent 365 Observability SDK comment * Update package description for clarity * Fix description formatting in pyproject.toml * Fix formatting in .env.template for Agent 365 * Update README.md * Update README.md * Fix naming in README for Agent Framework and SDK Updated README to correct naming conventions for Agent Framework and Microsoft Agent 365 SDK. * Update license link in README.md * Update perplexityClient.ts * Update README to reference Microsoft Agent 365 SDK * Fix typo in LangChain client documentation * Update README to reflect Microsoft Agent 365 SDK
…lity (#81) * Adding points for the users who contributed towards code review & quality * adding leaderboard.json * added permissions to the workflow * resolved comments --------- Co-authored-by: Lavanya Kappagantu <kalavany@microsoft.com>
* Add Agent Framework and Semantic Kernel enhancements Introduced a new .NET Agent Framework Sample Agent with CI/CD setup, OpenTelemetry observability, and tools for weather and date-time retrieval. Added ASP.NET authentication utilities and Teams app manifest. Updated Semantic Kernel Sample Agent with streaming support, enhanced observability, and plugins for terms and conditions handling. Included configuration files, README, and `.gitignore` for better project structure and documentation. * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Refactor async handling in HTTP operations Updated `InvokeObservedHttpOperation` to return `Task` for better async support and added `ConfigureAwait(false)` in `Program.cs` to prevent synchronization context capture. These changes improve performance, reliability, and adherence to async best practices. * Refactor to support async in HTTP operation handling Updated `Program.cs` to use `await` and `ConfigureAwait(false)` for `AgentMetrics.InvokeObservedHttpOperation` to ensure proper asynchronous execution. Modified `AgentMetrics.cs` to change `InvokeObservedHttpOperation` from `void` to `Task` return type, aligning with async programming practices. Updated the method to handle asynchronous operations and return `Task.CompletedTask` for consistency. --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* adding authorization back to add tool servers call * remove auth config for agentsplayground connection * fixing claude sample * fix indentation * Add useJwtMiddleware if agent is hosted publicly * change useJwt to isProd * Namechange isProduction boolean --------- Co-authored-by: Jesus Terrazas <jterrazas@microsoft.com>
* Creating a PR when the leaderboard needs to be updated * addressed comments --------- Co-authored-by: Lavanya Kappagantu <kalavany@microsoft.com>
* Add bug report issue template This file serves as a template for users to report bugs, detailing the necessary sections for a comprehensive bug report. * Add feature request issue template
* Create workflow for Vercel SDK sample agent * Add static authHandlerName to A365Agent class * Update .github/workflows/ci-nodejs-vercelsdk-sampleagent.yml Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Remove npm cache configuration from CI workflow Removed caching configuration for npm in CI workflow. --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Jesus Terrazas <jterrazas@microsoft.com>
Co-authored-by: Dominik Bezic <dominikbezic@microsoft.com>
…le README (#108) * Initial plan * Fix broken SECURITY.md and LICENSE.md links in LangChain Node.js README Co-authored-by: pontemonti <7850950+pontemonti@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: pontemonti <7850950+pontemonti@users.noreply.github.com>
* Add contributing guidelines to CONTRIBUTING.md Added contributing guidelines to the project, including sections on code of conduct, how to contribute, development process, and licensing. * Update CONTRIBUTING.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update CONTRIBUTING.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Johan Broberg <johan@pontemonti.net> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…n for auto instrument for OpenAi Agent sample. (#96) * fix manual instrument * Configure OpenAI Agents instrumentation * fix PR build error * Update nodejs/openai/sample-agent/src/client.ts Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix invoke scope --------- Co-authored-by: jsl517 <pefan@microsoft.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Johan Broberg <johanb@microsoft.com>
* Introduce per application validation for python Samples * Introduce per application validation for python Samples * Introduce per application validation for python Samples * Introduce per application validation for python Samples * Introduce per application validation for python Samples * Introduce per application validation for python Samples * Introduce per application validation for python Samples * Introduce per application validation for python Samples
* Initial plan * Fix nodemon watch pattern for macOS compatibility in Node.js samples Co-authored-by: pontemonti <7850950+pontemonti@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: pontemonti <7850950+pontemonti@users.noreply.github.com>
* Add survey section to README for user feedback Added a survey link for feedback on the SDK and CLI. * Update README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Remove Top Contributor badge from README Removed the Top Contributor badge from the README. --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This pull request introduces an automated contributor points tracking system that monitors PR activity and displays point allocations in PR comments. The system replaces a previous leaderboard-based approach with real-time point tracking directly on PRs, calculating points for reviews, approvals, merged PRs, and various bonus categories like bug fixes, security improvements, and documentation contributions.
Key Changes:
- Implements a comprehensive Python-based points tracking script that monitors multiple GitHub events (reviews, comments, PR lifecycle events)
- Replaces file-based leaderboard updates with dynamic PR comment updates showing live point tracking
- Expands point categories to include performance improvements, security fixes, and first-time contributor bonuses
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 18 comments.
| File | Description |
|---|---|
| scripts/track_points.py | New 520-line Python script implementing the core points calculation logic, GitHub API integration, and PR comment management |
| scripts/config_points.yml | Updated configuration file expanding point categories from 4 to 10+ types, adding security, performance, and documentation bonuses |
| .github/workflows/points.yml | Refactored workflow to trigger on more event types (pull_request, issues, etc.) and execute the new tracking script instead of creating leaderboard PRs |
| @@ -1,29 +1,37 @@ | |||
| name: Points Allocation | |||
| name: Contributor Points Tracker | |||
There was a problem hiding this comment.
This GitHub Actions workflow file is missing the required Microsoft copyright header. According to repository standards, all source files including workflow YAML files should start with a copyright header before any other content.
| except Exception as e: | ||
| print(f"ERROR: GitHub API request failed: {e}", file=sys.stderr) | ||
| return {} |
There was a problem hiding this comment.
The error handling in this function catches all exceptions and returns an empty dictionary, which can mask genuine errors like authentication failures or network issues. This makes debugging difficult. Consider logging the specific error type and re-raising critical errors (like authentication failures) or at least returning an error indicator so the caller knows the request failed versus legitimately returning an empty result.
| # Keywords that link issues: closes, fixes, resolves (case-insensitive) | ||
| import re | ||
| # Pattern matches: "closes #123", "fixes #456", "resolves #789", etc. | ||
| pattern = r'(?:close[sd]?|fix(?:e[sd])?|resolve[sd]?)\s*:?\s*#(\d+)' |
There was a problem hiding this comment.
This regex pattern has a potential issue: it will match partial words. For example, "closed" would match both "close" and "closed", and "fixing" would match "fix". While the current pattern uses word boundaries implicitly through the grouping, it could still match unintended variations. Consider using more precise patterns like r'\b(closes?|fixe[sd]|resolved?)\s*:?\s*#(\d+)' to ensure proper word boundaries.
| pattern = r'(?:close[sd]?|fix(?:e[sd])?|resolve[sd]?)\s*:?\s*#(\d+)' | |
| pattern = r'\b(closes?|fixe[sd]|resolved?)\b\s*:?\s*#(\d+)' |
| def is_first_time_contributor(username: str, current_pr_number: int) -> bool: | ||
| """Check if the current PR is the user's first merged PR.""" | ||
| # Check if user has any other merged PRs (excluding current one) | ||
| # Limit to recent 100 PRs to avoid API rate limits and performance issues | ||
| try: | ||
| search_result = github_api_request('GET', f'pulls?state=all&creator={username}&per_page=100') | ||
| if isinstance(search_result, list): | ||
| # Get all merged PRs excluding the current one | ||
| other_merged_prs = [ | ||
| pr for pr in search_result | ||
| if pr.get('merged_at') and pr.get('number') != current_pr_number | ||
| ] | ||
| # First-time contributor if no other merged PRs exist | ||
| return len(other_merged_prs) == 0 | ||
| except Exception as e: | ||
| print(f"WARNING: Error checking first-time contributor status: {e}", file=sys.stderr) | ||
| return False |
There was a problem hiding this comment.
This API call could cause rate limit issues for repositories with many PRs from a single user. The function limits to 100 PRs but doesn't implement pagination or rate limit handling. Additionally, checking if a user is a first-time contributor by fetching all their PRs is inefficient. Consider using the GitHub API's contributor statistics endpoint or checking the 'author_association' field which indicates 'FIRST_TIME_CONTRIBUTOR' status.
| def is_first_time_contributor(username: str, current_pr_number: int) -> bool: | |
| """Check if the current PR is the user's first merged PR.""" | |
| # Check if user has any other merged PRs (excluding current one) | |
| # Limit to recent 100 PRs to avoid API rate limits and performance issues | |
| try: | |
| search_result = github_api_request('GET', f'pulls?state=all&creator={username}&per_page=100') | |
| if isinstance(search_result, list): | |
| # Get all merged PRs excluding the current one | |
| other_merged_prs = [ | |
| pr for pr in search_result | |
| if pr.get('merged_at') and pr.get('number') != current_pr_number | |
| ] | |
| # First-time contributor if no other merged PRs exist | |
| return len(other_merged_prs) == 0 | |
| except Exception as e: | |
| print(f"WARNING: Error checking first-time contributor status: {e}", file=sys.stderr) | |
| return False | |
| def is_first_time_contributor(pr_details: dict) -> bool: | |
| """ | |
| Check if the PR author is a first-time contributor using the 'author_association' field. | |
| Returns True if the author_association is 'FIRST_TIME_CONTRIBUTOR'. | |
| """ | |
| author_association = pr_details.get('author_association', '') | |
| return author_association.upper() == 'FIRST_TIME_CONTRIBUTOR' |
| if not has_security: | ||
| for issue in linked_issues: | ||
| issue_labels = [label['name'].lower() for label in issue.get('labels', [])] | ||
| if any('security' in label or 'vulnerability' in label for label in issue_labels): | ||
| has_security = True | ||
| breakdown_source = f'closes issue #{issue["number"]}' | ||
| break | ||
| else: | ||
| breakdown_source = 'PR labeled' | ||
|
|
||
| # Award security bonus only once | ||
| if has_security: | ||
| bonus = config['points'].get('security_fix', 15) | ||
| points += bonus | ||
| breakdown.append(f'Security fix ({breakdown_source}): +{bonus} points') |
There was a problem hiding this comment.
The variable name 'breakdown_source' is defined inside a conditional block but may be referenced outside of it. If 'has_security' is True but was set from PR labels (lines 210-221), the variable 'breakdown_source' is assigned on line 221. However, if the logic changes or the flow is unclear, this could lead to an UnboundLocalError. Consider initializing 'breakdown_source' before the conditional blocks to ensure it's always defined when used on line 227.
| pr_number = pr_details.get('number') | ||
|
|
||
| # Keywords that link issues: closes, fixes, resolves (case-insensitive) | ||
| import re |
There was a problem hiding this comment.
The import statement for the 're' module is placed inside a function rather than at the top of the file with other imports. This violates PEP 8 style guidelines which recommend placing all imports at the top of the module. Move this import to line 38 with the other standard library imports.
| def aggregate_contributor_points(reviews: List[dict], comments: List[dict], pr_details: dict, config: dict) -> Dict[str, dict]: | ||
| """ | ||
| Aggregate points for all contributors on a PR. | ||
|
|
||
| Returns: | ||
| Dict mapping username to {'total': int, 'activities': [list of activity dicts]} | ||
| """ | ||
| contributors = {} | ||
|
|
||
| # Process reviews | ||
| for review in reviews: | ||
| username = review['user']['login'] | ||
| if username not in contributors: | ||
| contributors[username] = {'total': 0, 'activities': []} | ||
|
|
||
| points, breakdown = calculate_review_points(review, config) | ||
| contributors[username]['total'] += points | ||
| contributors[username]['activities'].append({ | ||
| 'type': 'review', | ||
| 'points': points, | ||
| 'breakdown': breakdown, | ||
| 'timestamp': review['submitted_at'], | ||
| 'state': review['state'] | ||
| }) | ||
|
|
||
| # Add PR author points (if PR is merged or has special labels) | ||
| pr_author = pr_details.get('user', {}).get('login') | ||
| if pr_author: | ||
| author_points, author_breakdown = calculate_pr_author_points(pr_details, config) | ||
| if author_points > 0: | ||
| if pr_author not in contributors: | ||
| contributors[pr_author] = {'total': 0, 'activities': []} | ||
|
|
||
| contributors[pr_author]['total'] += author_points | ||
| contributors[pr_author]['activities'].append({ | ||
| 'type': 'pr_author', | ||
| 'points': author_points, | ||
| 'breakdown': author_breakdown, | ||
| 'timestamp': pr_details.get('merged_at') or pr_details.get('created_at') | ||
| }) | ||
|
|
||
| return contributors |
There was a problem hiding this comment.
The function 'aggregate_contributor_points' receives a 'comments' parameter but never uses it. The function's docstring says it aggregates points from "reviews, comments, and pr_details", but the implementation only processes reviews and PR author points - not individual comments. Either implement comment processing (as documented in the module docstring on line 23) or remove the unused parameter and update the docstring.
| """Get issues linked to this PR by checking PR body for closing keywords.""" | ||
| linked_issues = [] | ||
| pr_body = pr_details.get('body', '') or '' | ||
| pr_number = pr_details.get('number') |
There was a problem hiding this comment.
Variable pr_number is not used.
| pr_number = pr_details.get('number') |
|
|
||
| if existing_comment_id: | ||
| # Update existing comment | ||
| result = github_api_request('PATCH', f'issues/comments/{existing_comment_id}', {'body': body}) |
There was a problem hiding this comment.
Variable result is not used.
| result = github_api_request('PATCH', f'issues/comments/{existing_comment_id}', {'body': body}) | |
| github_api_request('PATCH', f'issues/comments/{existing_comment_id}', {'body': body}) |
| print(f"✅ Updated tracking comment (ID: {existing_comment_id})") | ||
| else: | ||
| # Create new comment | ||
| result = github_api_request('POST', f'issues/{pr_number}/comments', {'body': body}) | ||
| print(f"✅ Created new tracking comment") | ||
|
|
There was a problem hiding this comment.
Variable result is not used.
| print(f"✅ Updated tracking comment (ID: {existing_comment_id})") | |
| else: | |
| # Create new comment | |
| result = github_api_request('POST', f'issues/{pr_number}/comments', {'body': body}) | |
| print(f"✅ Created new tracking comment") | |
| if isinstance(result, dict) and 'id' in result: | |
| print(f"✅ Updated tracking comment (ID: {result['id']})") | |
| else: | |
| print(f"⚠️ Failed to update tracking comment (ID: {existing_comment_id}) - API response: {result}") | |
| else: | |
| # Create new comment | |
| result = github_api_request('POST', f'issues/{pr_number}/comments', {'body': body}) | |
| if isinstance(result, dict) and 'id' in result: | |
| print(f"✅ Created new tracking comment (ID: {result['id']})") | |
| else: | |
| print(f"⚠️ Failed to create tracking comment - API response: {result}") |
| """Make GitHub API request.""" | ||
| url = f"https://api.github.com/repos/{GITHUB_REPOSITORY}/{endpoint}" | ||
| headers = { | ||
| 'Authorization': f'token {GITHUB_TOKEN}', |
There was a problem hiding this comment.
Tokens are not safe if we developers run this, they can fetch the token.
72bc142 to
19b3450
Compare
Adding the points allocated to users in a PR comment