You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This template repository contains the boilerplate code needed in order to create an AWS Lambda function that can be written by any tutor to grade a response area in any way they like.
3
+
This function evaluates the similarity between a student's short text response and the correct answer. It uses Word2Vec (w2v) and Bag-of-Words (BOW) vector similarity to determine semantic equivalence, and supports optional keystring matching to check for the presence or absence of specific terms.
4
+
5
+
For more information, look at the docs in `app/docs/`.
4
6
5
7
This version is specifically for python, however the ultimate goal is to make similar boilerplate repositories in any language, allowing tutors the freedom to code in what they feel most comfortable with.
-[Evaluation Function Template Repository](#evaluation-function-template-repository)
12
-
-[Table of Contents](#table-of-contents)
13
-
-[Repository Structure](#repository-structure)
14
-
-[Usage](#usage)
15
-
-[Getting Started](#getting-started)
16
-
-[How it works](#how-it-works)
17
-
-[Docker & Amazon Web Services (AWS)](#docker--amazon-web-services-aws)
18
-
-[Middleware Functions](#middleware-functions)
19
-
-[GitHub Actions](#github-actions)
20
-
-[Pre-requisites](#pre-requisites)
21
-
-[Contact](#contact)
22
-
23
-
## Repository Structure
24
-
25
-
```bash
26
-
app/
27
-
__init__.py
28
-
evaluation.py # Script containing the main evaluation_function
29
-
docs.md # Documentation page for this function (required)
30
-
evaluation_test.py # Unittests for the main evaluation_function
31
-
requirements.txt # list of packages needed for algorithm.py
32
-
Dockerfile # for building whole image to deploy to AWS
33
-
34
-
.github/
35
-
workflows/
36
-
test-and-deploy.yml # Testing and deployment pipeline
37
-
38
-
config.json # Specify the name of the evaluation function in this file
39
-
.gitignore
40
-
```
41
-
42
-
## Usage
43
12
44
13
### Getting Started
45
14
46
15
1. Clone this repository
47
16
2. Change the name of the evaluation function in `config.json`
48
-
3. The name must be unique. To view existing grading functions, go to:
49
-
50
-
-[Staging API Gateway Integrations](https://eu-west-2.console.aws.amazon.com/apigateway/main/develop/integrations/attach?api=c1o0u8se7b®ion=eu-west-2&routes=0xsoy4q)
51
-
-[Production API Gateway Integrations](https://eu-west-2.console.aws.amazon.com/apigateway/main/develop/integrations/attach?api=cttolq2oph&integration=qpbgva8®ion=eu-west-2&routes=0xsoy4q)
52
17
53
-
4. Merge commits into the default branch
54
-
- This will trigger the `test-and-deploy.yml` workflow, which will build the docker image, push it to a shared ECR repository, then call the backend `grading-function/ensure` route to build the necessary infrastructure to make the function available from the client app.
18
+
3. Merge commits into the default branch
19
+
- This will trigger the `staging-deploy.yml` workflow, which will build the docker image, push it to a shared ECR repository, then call the backend `grading-function/ensure` route to build the necessary infrastructure to make the function available from the client app.
55
20
56
-
5. You are now ready to start developing your function:
21
+
4. You are now ready to start developing your function:
57
22
58
23
- Edit the `app/evaluation.py` file, which ultimately gets called when the function is given the `eval` command
59
24
- Edit the `app/evaluation_tests.py` file to add tests which get run:
60
-
- Every time you commit to this repo, before the image is built and deployed
25
+
- Every time you open a pull request (`test-lint.yml`), before the image is built and deployed
61
26
- Whenever the `healthcheck` command is supplied to the deployed function
62
-
- Edit the `app/docs.md` file to reflect your changes. This file is baked into the function's image, and is made available using the `docs` command. This feature is used to display this function's documentation on our [Documentation](https://lambda-feedback.github.io/Documentation/) website once it's been hooked up!
27
+
- Edit the `app/docs/` files to reflect your changes. These files are baked into the function's image, and are made available using the `docs` command. This feature is used to display this function's documentation on our [Documentation](https://lambda-feedback.github.io/Documentation/) website once it's been hooked up!
63
28
64
29
---
65
30
@@ -79,7 +44,12 @@ In order to run the algorithm and schema on AWS Lambda, some middleware function
79
44
The code needed to build the image using all the middleware functions are available in the [BaseEvaluationFunctionLayer](https://github.com/lambda-feedback/BaseEvalutionFunctionLayer) repository.
80
45
81
46
### GitHub Actions
82
-
Whenever a commit is made to the GitHub repository, the new code will go through a pipeline, where it will be tested for syntax errors and code coverage. The pipeline used is called **GitHub Actions** and the scripts for these can be found in `.github/workflows/`.
47
+
Whenever a commit is made to the GitHub repository, the new code will go through a pipeline, where it will be tested for syntax errors and code coverage. The pipeline used is called **GitHub Actions** and the scripts for these can be found in `.github/workflows/`. The key workflows are:
48
+
49
+
-`test-lint.yml` — runs on every pull request; checks syntax and runs unit tests
50
+
-`staging-deploy.yml` — runs on every push to `main`; builds and deploys to staging
51
+
-`production-deploy.yml` — triggered manually; promotes a release to production
52
+
-`pre_production_tests.yml` — triggered manually; runs pre-production validation tests against the database
83
53
84
54
On top of that, when starting a new evaluation function, you will have to complete a set of unit test scripts, which not only make sure your code is reliable, but also helps you to build a _specification_ for how the code should function before you start programming.
85
55
@@ -92,8 +62,4 @@ Although all programming can be done through the GitHub interface, it is recomme
92
62
93
63
- GitHub Desktop or the `git` CLI.
94
64
95
-
- A code editor such as Atom, VS Code, or Sublime.
96
-
97
-
Copy this template over by clicking **Use this template** button found in the repository on GitHub. Save it to the `lambda-feedback` Organisation.
98
-
99
-
## Contact
65
+
- A code editor such as Atom, VS Code, or Sublime.
0 commit comments