Skip to content

Commit f5025f4

Browse files
m-messerm-messer
committed
Updated the readme
1 parent 76aded5 commit f5025f4

1 file changed

Lines changed: 120 additions & 61 deletions

File tree

‎README.md‎

Lines changed: 120 additions & 61 deletions
Original file line numberDiff line numberDiff line change
@@ -1,97 +1,156 @@
1-
# Evaluation Function Template Repository
1+
# EduVision Evaluation Function
22

3-
This template repository contains the boilerplate code needed in order to create an AWS Lambda function that can be written by any tutor to grade a response area in any way they like.
4-
5-
This version is specifically for python, however the ultimate goal is to make similar boilerplate repositories in any language, allowing tutors the freedom to code in what they feel most comfortable with.
3+
An AWS Lambda evaluation function for the [Lambda Feedback](https://github.com/lambda-feedback) platform that grades student responses by querying the EduVision API. Built on the [BaseEvaluationFunctionLayer](https://github.com/lambda-feedback/BaseEvalutionFunctionLayer).
64

75
## Table of Contents
8-
- [Evaluation Function Template Repository](#evaluation-function-template-repository)
9-
- [Table of Contents](#table-of-contents)
10-
- [Repository Structure](#repository-structure)
11-
- [Usage](#usage)
12-
- [Getting Started](#getting-started)
13-
- [How it works](#how-it-works)
14-
- [Docker & Amazon Web Services (AWS)](#docker--amazon-web-services-aws)
15-
- [Middleware Functions](#middleware-functions)
16-
- [GitHub Actions](#github-actions)
17-
- [Pre-requisites](#pre-requisites)
18-
- [Contact](#contact)
6+
- [Overview](#overview)
7+
- [Repository Structure](#repository-structure)
8+
- [Usage](#usage)
9+
- [Getting Started](#getting-started)
10+
- [API Endpoints](#api-endpoints)
11+
- [Request Format](#request-format)
12+
- [How it works](#how-it-works)
13+
- [Docker & AWS Lambda](#docker--aws-lambda)
14+
- [GitHub Actions](#github-actions)
15+
- [Pre-requisites](#pre-requisites)
16+
17+
## Overview
18+
19+
This function evaluates student responses by making API calls to the EduVision server. It compares the API's response against an expected answer to determine correctness, supporting two grading modes:
20+
21+
- **Resistance** — checks the global resistance of a circuit (float comparison)
22+
- **Resistors** — checks the list of recognised resistors and their resistance values (list comparison)
1923

2024
## Repository Structure
2125

22-
```bash
26+
```
2327
app/
2428
__init__.py
25-
evaluation.py # Script containing the main evaluation_function
26-
preview.py # Script containing the preview_function
27-
docs.md # Documentation page for this function (required)
28-
evaluation_tests.py # Unittests for the main evaluation_function
29-
preview_tests.py # Unittests for the preview_function
30-
requirements.txt # list of packages needed for algorithm.py
31-
Dockerfile # for building whole image to deploy to AWS
29+
evaluation.py # Main evaluation function — called on "eval" command
30+
preview.py # Preview function — called on "preview" command
31+
utility.py # Shared helper (initialize_test_connection)
32+
evaluation_tests.py # Unit tests for the evaluation function
33+
preview_tests.py # Unit tests for the preview function
34+
requirements.txt # Python dependencies
35+
Dockerfile # Container image for AWS Lambda deployment
36+
docs/
37+
user.md # End-user documentation (served via "docs" command)
38+
dev.md # Developer documentation
3239
3340
.github/
3441
workflows/
35-
test-and-deploy.yml # Testing and deployment pipeline
42+
test-and-deploy.yml # CI/CD pipeline
3643
37-
config.json # Specify the name of the evaluation function in this file
44+
config.json # Evaluation function name ("eduVision")
3845
.gitignore
3946
```
4047

4148
## Usage
4249

4350
### Getting Started
4451

45-
1. Clone this repository
46-
2. Change the name of the evaluation function in `config.json`
47-
3. The name must be unique. To view existing grading functions, go to:
48-
49-
- [Staging API Gateway Integrations](https://eu-west-2.console.aws.amazon.com/apigateway/main/develop/integrations/attach?api=c1o0u8se7b&region=eu-west-2&routes=0xsoy4q)
50-
- [Production API Gateway Integrations](https://eu-west-2.console.aws.amazon.com/apigateway/main/develop/integrations/attach?api=cttolq2oph&integration=qpbgva8&region=eu-west-2&routes=0xsoy4q)
51-
52-
4. Merge commits into the default branch
53-
- This will trigger the `test-and-deploy.yml` workflow, which will build the docker image, push it to a shared ECR repository, then call the backend `grading-function/ensure` route to build the necessary infrastructure to make the function available from the client app.
54-
55-
5. You are now ready to start developing your function:
56-
57-
- Edit the `app/evaluation.py` file, which ultimately gets called when the function is given the `eval` command
58-
- Edit the `app/preview.py` file, which is called when the function is passed the `preview` command.
59-
- Edit the `app/evaluation_tests.py` and `app/preview_tests.py` files to add tests which get run:
60-
- Every time you commit to this repo, before the image is built and deployed
61-
- Whenever the `healthcheck` command is supplied to the deployed function
62-
- Edit the `app/docs.md` file to reflect your changes. This file is baked into the function's image, and is made available using the `docs` command. This feature is used to display this function's documentation on our [Documentation](https://lambda-feedback.github.io/Documentation/) website once it's been hooked up!
52+
1. Clone this repository.
53+
2. Set the `API_CONNECTION` environment variable to the EduVision API server URL (e.g. in a `.env` file):
54+
```
55+
API_CONNECTION=http://<host>:<port>
56+
```
57+
3. Install dependencies:
58+
```bash
59+
pip install -r app/requirements.txt
60+
```
61+
4. Run the tests:
62+
```bash
63+
cd app && pytest -v evaluation_tests.py preview_tests.py
64+
```
65+
5. Push to `main` to trigger the CI/CD pipeline, which will build, test, and deploy the function to AWS Lambda.
66+
67+
### API Endpoints
68+
69+
The function proxies requests to the EduVision API. The endpoint is controlled by the `api_endpoint` param:
70+
71+
| Endpoint | Description | Expected answer type |
72+
| --- | --- | --- |
73+
| `resistance/` (default) | Global circuit resistance | `float` |
74+
| `resistors/` | List of recognised resistors | `list[dict]` |
75+
76+
The student's 6-character connection ID is appended to the endpoint:
77+
```
78+
{API_CONNECTION}/{api_endpoint}{response}
79+
```
6380

64-
---
81+
### Request Format
82+
83+
Requests follow the Lambda Feedback schema:
84+
85+
```json
86+
{
87+
"headers": { "command": "eval" },
88+
"body": {
89+
"response": "999999",
90+
"answer": 1000.0,
91+
"params": {
92+
"api_endpoint": "resistance/"
93+
}
94+
}
95+
}
96+
```
6597

66-
## How it works
98+
**Resistance example** — `answer` is a `float`:
99+
```json
100+
{
101+
"body": {
102+
"response": "999999",
103+
"answer": 1000.0,
104+
"params": { "api_endpoint": "resistance/" }
105+
}
106+
}
107+
```
67108

68-
The function is built on top of a custom base layer, [BaseEvaluationFunctionLayer](https://github.com/lambda-feedback/BaseEvalutionFunctionLayer), which tools, tests and schema checking relevant to all evaluation functions.
109+
**Resistors example** — `answer` is a list of dicts:
110+
```json
111+
{
112+
"body": {
113+
"response": "999999",
114+
"answer": [
115+
{ "resistance": 1000.0 },
116+
{ "resistance": 1000.0 }
117+
],
118+
"params": { "api_endpoint": "resistors/" }
119+
}
120+
}
121+
```
69122

70-
### Docker & Amazon Web Services (AWS)
123+
**Response** — the function returns:
124+
```json
125+
{ "is_correct": true, "error": 0 }
126+
```
71127

72-
The grading scripts are hosted AWS Lambda, using containers to run a docker image of the app. Docker is a popular tool in software development that allows programs to be hosted on any machine by bundling all its requirements and dependencies into a single file called an **image**.
128+
`error` codes: `0` = success, `1` = invalid connection ID length, `2`–`5` = answer mismatch variants, `6` = API connection error.
73129

74-
Images are run within **containers** on AWS, which give us a lot of flexibility over what programming language and packages/libraries can be used. For more information on Docker, read this [introduction to containerisation](https://www.freecodecamp.org/news/a-beginner-friendly-introduction-to-containers-vms-and-docker-79a9e3e119b/). To learn more about AWS Lambda, click [here](https://geekflare.com/aws-lambda-for-beginners/).
130+
## How it works
75131

76-
### Middleware Functions
77-
In order to run the algorithm and schema on AWS Lambda, some middleware functions have been provided to handle, validate and return the data so all you need to worry about is the evaluation script and testing.
132+
### Docker & AWS Lambda
78133

79-
The code needed to build the image using all the middleware functions are available in the [BaseEvaluationFunctionLayer](https://github.com/lambda-feedback/BaseEvalutionFunctionLayer) repository.
134+
The function runs as a Docker container on AWS Lambda. On every push to `main`, GitHub Actions builds the image, pushes it to a shared ECR repository, then calls the Lambda Feedback backend to provision the necessary infrastructure.
80135

81136
### GitHub Actions
82-
Whenever a commit is made to the GitHub repository, the new code will go through a pipeline, where it will be tested for syntax errors and code coverage. The pipeline used is called **GitHub Actions** and the scripts for these can be found in `.github/workflows/`.
83137

84-
On top of that, when starting a new evaluation function, you will have to complete a set of unit test scripts, which not only make sure your code is reliable, but also helps you to build a _specification_ for how the code should function before you start programming.
138+
The `.github/workflows/test-and-deploy.yml` pipeline:
85139

86-
Once the code passes all these tests, it will then be uploaded to AWS and will be deployed and ready to go in only a few minutes.
140+
1. **Lint** — runs `flake8` for syntax errors.
141+
2. **Test** — runs `pytest` against `evaluation_tests.py` and `preview_tests.py`.
142+
3. **Deploy (staging)** — builds and pushes the Docker image, then registers the function with the staging environment.
143+
4. **Deploy (production)** — same as staging, but targets the production environment.
87144

88-
## Pre-requisites
89-
Although all programming can be done through the GitHub interface, it is recommended you do this locally on your machine. To do this, you must have installed:
145+
Tests also run on every `healthcheck` command sent to the deployed function.
90146

91-
- Python 3.8 or higher.
147+
## Pre-requisites
92148

93-
- GitHub Desktop or the `git` CLI.
149+
- Python 3.8+
150+
- Docker (for local image builds)
151+
- `git` CLI or GitHub Desktop
152+
- Access to the EduVision API server
94153

95-
- A code editor such as Atom, VS Code, or Sublime.
154+
## Contact
96155

97-
Copy this template over by clicking **Use this template** button found in the repository on GitHub. Save it to the `lambda-feedback` Organisation.
156+
For issues with the Lambda Feedback platform, see the [lambda-feedback organisation](https://github.com/lambda-feedback) on GitHub.

0 commit comments

Comments
 (0)