You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Welcome to the Dagger hackathon hosted by the CNCF and Code to Cloud! By the end of this hack, you will have a good understanding of how to use Dagger and the various use cases it can help support.
3
+
Welcome to the Dagger hackathon hosted by the [CNCF](https://community.cncf.io/cloud-native-calgary/) and [Code to Cloud](https://www.codetocloud.io/)! By the end of this hack, you will have a good understanding of how to use Dagger and the various use cases it can help support.
4
4
5
5
## 🎯 Expected Outcomes
6
6
@@ -10,36 +10,203 @@ Welcome to the Dagger hackathon hosted by the CNCF and Code to Cloud! By the end
10
10
11
11
✅ Dagger pipeline triggers LLM workflow if tests fail
12
12
13
-
✅ LLM provides feedback on why tests could be failing in a PR
13
+
✅ LLM provides code suggestion to fix failing unit tests
14
+
15
+
✅ Code suggestion is added to PR
14
16
15
17
## ✅ Prerequisites
16
18
17
19
There are two options for this hackathon. We highly recommend the first option to avoid dependency issues.
- This will auto-open VS Code in browser. If you don't want to work from here and have the VS Code app installed, you can close this and open it in the app
Now that we have all of our prerequisites setup, **this is where the fun begins!**
86
+
87
+
The below steps will take you through making a breaking change to a file in our application which will set us up in the next step to have our agent try and troubleshoot it.
26
88
27
-
All the code for the below can be found [here](./dagger-hackathon-pipeline/).
89
+
- Open a terminal
90
+
- Create an environment variable for `DAGGER_CLOUD_TOKEN`, e.g.
91
+
```bash
92
+
export DAGGER_CLOUD_TOKEN="XXX"
93
+
```
94
+
- Change directory into `docs/dagger/dagger-hackathon/`
95
+
- Open the file `docs/dagger/dagger-hackathon/src/addition.py` and break the function
96
+
For example, modify to: `return a + b * 4`
97
+
- Run the unittests
98
+
```bash
99
+
python -m unittest discover tests -v
100
+
```
101
+
and confirm it fails
102
+
- Push the modified `addition.py` to your new feature branch
103
+
- In the GitHub UI, open a PR on the feature branch and compare to `main`
28
104
29
-
Make sure you have the repo cloned and you are in the `docs\dagger\dagger-hackathon-pipeline` directory:
- Rename `docs/dagger/dagger-hackathon/.env-example` to `.env`
112
+
- Populate the placeholder keys with real values (we will provide for Hackathon day)
113
+
- Create environment variables for `GITHUB_TOKEN`, `AZURE_API_KEY`, `AZURE_API_ENDPOINT`
114
+
A GitHub Token can be created in GitHub under Settings → Developer Settings → Create a classic token
115
+
Example:
116
+
```bash
117
+
export GITHUB_TOKEN="XXX"
118
+
```
119
+
120
+
- Put in your values and run in CLI:
121
+
```bash
122
+
dagger call \
123
+
--source="." \
124
+
--github_branch="BRANCH-NAME" \
125
+
--github_repo="USERNAME/REPO-NAME" \
126
+
--github_token="GITHUB_TOKEN" \
127
+
--azure_api_key="AZURE_API_KEY" \
128
+
--azure_endpoint="AZURE_API_ENDPOINT" \
129
+
fix-my-tests-agent
130
+
```
131
+
132
+
> **💡 Tip:** This will be used to enable logging on the Dagger pipeline. Logs can be viewed in [Dagger Cloud](https://dagger.io/cloud) after step 5.
133
+
134
+
- If you see an error, it might be that the Agent has hallucinated — try again
135
+
You can also debug via the Dagger Traces in Dagger Cloud
136
+
137
+
> **💡 Tip:** There is a prompt located at `docs\dagger\dagger-hackathon\dagger-hackathon-pipeline\debug_unit_test_prompt.md` that greatly influences the LLMs behavior. Make some changes to the prompt and see how it influences the results!
Being able to navigate and explore logs in Dagger Cloud is a great skill to have to be able to effectively troubleshoot and build pipelines in Dagger. Now that we have ran our first command in Dagger, lets jump into the logs to see what happened.
146
+
147
+
- Navigate to [Dagger Cloud](https://dagger.io/cloud)
148
+
- You should see your trace(s)
149
+
150
+
151
+
152
+
- Select a trace and explore the logs
153
+
- If all has gone well, you should see something like this at the bottom of the trace detailing the problematic file, line number, and fix for the failing unit test.
- If the Agent finished successfully, you should see something like below in the CLI
164
+
```
165
+
AgentResponse(pr_metadata=PrMetadataResult(pr_number='1', commit_id='c4293304b9fdccf2022e7e15e09e5b1604cecf04'), pr_suggestions=GitHubPrSuggestionResult(body='```suggestion\nreturn a + b\n```', comment_url='https://github.com/david-hurley/platform-engineering/pull/1#discussion_r2127910154'))
166
+
```
167
+
- Click the PR URL and see if there is a suggested code change
168
+
169
+
---
170
+
171
+
### ✅ Step 8: Trigger GitHub Action
172
+
173
+
- Delete the comment the Agent left on the PR in Step 6
174
+
-[Create GitHub secrets](https://docs.github.com/en/actions/security-for-github-actions/security-guides/using-secrets-in-github-actions#creating-secrets-for-a-repository) for everything in `.env` and name them `DAGGER_HACKATHON_GITHUB_TOKEN` and `OPENAI_API_KEY` for the GitHub token and Azure OpenAI API key respectively.
175
+
- Make sure to update your GitHub Action to trigger on a pull request instead of a manual trigger.
176
+
177
+
Update the trigger from this:
178
+
179
+
```yaml
180
+
on:
181
+
workflow_dispatch:
182
+
```
183
+
184
+
To this:
185
+
```yaml
186
+
on:
187
+
pull_request:
188
+
branches: [main]
189
+
types: [opened, synchronize, reopened]
37
190
```
38
191
39
-
### Step 1: Create a Feature Branch
192
+
- Push your changes to Git and navigate back to your forked repo to see the GitHub Action running
193
+
194
+
195
+
196
+
---
197
+
198
+
### ✅ Step 9: Extend the Agent
199
+
200
+
The last step is to extend the Dagger agent to actually make the code fix itself and push the changes to the PR for the human to review. That way there is still a human-in-the-loop to validate the AI output.
40
201
41
-
### 🤖 Step 2: Select your LLM Provider
202
+
---
42
203
43
-
### Step 3: Create a PR
204
+
## ⚠️ Gotchas and Future Improvements
44
205
45
-
### Step 4: Run the Dagger Function
206
+
- The agent can sometimes return incorrect paths to the file with breaking changes, incorrect line number of the breaking change, or incorrect fix (e.g. too verbose)
207
+
- Current state of this does not work with multi-line code changes or multiple breaking changes.
208
+
Only supports **1 breaking change on 1 line**
209
+
- The way commit ID is retrieved is not robust — if a breaking change is pushed to a branch with an open PR and it is commit ID A, and then another file is pushed (commit ID B),
210
+
the code is setup to grab the **latest commit ID**, not the one with the breaking change
211
+
This may result in failure (i.e. no diff)
212
+
- The agent leaves a comment on the PR as the idenity of whoever generated to token. In other words, if you leverage your GitHub token in the above steps, then the agent will leave a comment on the PR as you. This ideally should be a generic account in the future.
0 commit comments