Add AI PR review to check that issues are addressed

[braughtg]

GitHub provides for AI Pull Request reviews: https://docs.github.com/en/copilot/how-tos/use-copilot-agents/request-a-code-review/use-code-review

These reviews can be customized based on path: https://docs.github.com/en/copilot/how-tos/configure-custom-instructions/add-repository-instructions

Is it possible to give a general instruction that would apply to all issues without having very specific instructions for every issue?

Consider updating GitKit to include:

• Having students request a PR Review from CoPilot.
• Adding issue specific PR instructions for CoPilot to the repo.
• Having student update PR if it does not address the issue.
• Could this also check if the PR itself includes the correct Closes line?
• Can it also check that no other changes have been made.

A spike project has been created for testing: https://github.com/braughtg/AI-PR-Review-Spike

[braughtg]

Using the PR review feature requires CoPilot Pro. Both faculty and students can obtain access to CoPilot Pro for free by submitting an application:

https://docs.github.com/en/copilot/how-tos/manage-your-account/get-free-access-to-copilot-pro#accessing-copilot-pro-for-free

[braughtg]

@StoneyJackson @kwurst @lpostner How would we feel about adding features to GitKIt that require the use of CoPilot Pro, which requires the additional registration?

We could mark those exercises in some way in the text to indicate that they require this access and that they should be skipped if CoPilot Pro is not available..

[StoneyJackson]

Who has to have CoPilot Pro installed? The upstream repo owner (instructor)? Or each contributors (students)?

[braughtg]

Who has to have CoPilot Pro installed? The upstream repo owner (instructor)? Or each contributors (students)?

Looks like each student would need to have the Pro designation. It is free for them, they just need to fill out an application for the education program. Alternatively, they could be added to an organization that has Pro, but then it looks like the organization has to purchase it.

We can investigate whether if a faculty with Pro through the education program creates an organization and adds students, if those students can then request CoPilot reviews.

One question to investigate is what happens if the faculty member turns on automatic CoPilot reviews for the Repo? Do students then get the review for free without having CoPilot.

[braughtg]

There appears to be an issue that non-collaborators or non-members of the organization cannot request a CoPilot code review? Though I've had that happen in the past where students have requested a CoPilot review on the PR that they submitted. Maybe if they have their own CoPilot?

In any case, some additional investigation will be required here before adding any activities to the GitKit.