Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
909d7a7
Updated database
qh2244 Oct 21, 2024
51b601c
Merge pull request #3 from qh2244/Qifeng_Task
qh2244 Oct 21, 2024
b069f44
Updated main comment
qh2244 Oct 21, 2024
783b3f1
Merge pull request #7 from qh2244/Qifeng_Task
qh2244 Oct 21, 2024
751c5de
add in update method
olvmky Oct 21, 2024
5a59949
add in update method
olvmky Oct 21, 2024
f20f90e
testing for changes
olvmky Oct 21, 2024
5ce792f
add in delete method
olvmky Oct 21, 2024
cf7359d
Merge pull request #8 from qh2244/olivia
qh2244 Oct 21, 2024
4b88088
Updated router
qh2244 Oct 21, 2024
66c1256
Merge pull request #9 from qh2244/Qifeng_Task
qh2244 Oct 21, 2024
8f81580
add tests
Oct 21, 2024
a4fe6e8
Merge pull request #10 from qh2244/Xinyi
qh2244 Oct 21, 2024
c35724a
Add the README file
Oct 21, 2024
72cf91a
update README
Oct 21, 2024
9851615
Merge pull request #11 from qh2244/Jerry
qh2244 Oct 21, 2024
92c2edf
Add CI/CD pipeline
qh2244 Nov 23, 2024
d2fb64d
Merge pull request #13 from qh2244/CI/CD_Doc
qh2244 Nov 23, 2024
585a567
Modified main branch in CI/CD
qh2244 Nov 23, 2024
43e9cbc
Merge pull request #14 from qh2244/CI/CD_Doc
qh2244 Nov 23, 2024
0b66153
Update some doc string
qh2244 Nov 23, 2024
f73fd66
Merge pull request #15 from qh2244/CI/CD_Doc
qh2244 Nov 23, 2024
b8189eb
updated ci/cd
qh2244 Nov 23, 2024
7a8d81e
Merge pull request #16 from qh2244/CI/CD_Doc
qh2244 Nov 23, 2024
2e9a0e6
Updated ci/cd rules
qh2244 Nov 23, 2024
d0e898f
Update ci/cd
qh2244 Nov 23, 2024
9b10148
Update pytest ci/cd
qh2244 Nov 23, 2024
29f6c7f
Update ci.yml
qh2244 Nov 23, 2024
5a462f2
Update ci.yml
qh2244 Nov 23, 2024
aa8b500
Added Docker File
qh2244 Nov 27, 2024
7fa2463
modified docker and tests
qh2244 Nov 27, 2024
4f0e41b
update docker file
qh2244 Nov 27, 2024
911cf0c
Updated docker install file
qh2244 Nov 27, 2024
28d08de
update docker
qh2244 Nov 27, 2024
f26c66d
updated docker file
qh2244 Nov 27, 2024
674e014
Updated docker
qh2244 Nov 27, 2024
8d5b8c3
Merge pull request #17 from qh2244/Qifeng_Docker
qh2244 Nov 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
name: CI Pipeline

on:
push:
branches:
- main
pull_request:
branches:
- main

jobs:
test-and-docker:
runs-on: ubuntu-latest

steps:
# Checkout the repository
- name: Checkout code
uses: actions/checkout@v3

# Setup Python environment
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.10'

# Install dependencies from both requirements.txt files
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install -r app/requirements.txt

# Run tests with pytest
- name: Run tests
run: |
export PYTHONPATH=.
python3 tests/test.py

# Run linting with pylint
- name: Run pylint
run: |
pylint app/ || exit 0 # Allow linting to fail without failing the pipeline

# Build Docker image
- name: Build Docker image
run: |
docker build -t commonassessmenttool .

# Run Docker container
- name: Run Docker container
run: |
docker run -d -p 8000:8000 --name app-container commonassessmenttool
29 changes: 29 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Use an official Python image as the base image
FROM python:3.10-slim

# Install necessary build dependencies
RUN apt-get update && apt-get install -y \
gcc \
python3-dev \
libffi-dev \
build-essential \
--no-install-recommends && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

# Set the working directory in the container
WORKDIR /app

# Copy the application code to the container
COPY . .

# Upgrade pip and install dependencies
RUN pip install --upgrade pip && \
pip install -r requirements.txt && \
pip install -r app/requirements.txt

# Expose the port FastAPI will use
EXPOSE 8000

# Command to start the FastAPI application
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
83 changes: 83 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,86 @@ This will contain the model used for the project that based on the input informa
The model works off of dummy data of several combinations of clients alongside the interventions chosen for them as well as their success rate at finding a job afterward. The model will be updated by the case workers by inputing new data for clients with their updated outcome information, and it can be updated on a daily, weekly, or monthly basis.

This also has an API file to interact with the front end, and logic in order to process the interventions coming from the front end. This includes functions to clean data, create a matrix of all possible combinations in order to get the ones with the highest increase of success, and output the results in a way the front end can interact with.

## Assessment Tool Documentation

This API provides several endpoints to handle client-related operations, such as predicting outcomes based on input data, adding client information to an SQLite database, retrieving client information, updating client data, and deleting a client from the database.

### Prerequisites
To run this project, ensure that the following tools are installed on your system:

* Python 3.8 or later
* FastAPI
* SQLite

### API Endpoints
1. POST /clients/predictions
This endpoint accepts input data and returns a prediction result by calling the interpret_and_calculate logic.

Request:
* Method: POST
* URL: /clients/predictions

2. POST /clients/add
This endpoint adds the input data to the SQLite database and returns the newly created client_id along with the prediction result.

Request:
* Method: POST
* URL: /clients/add
* Body: Same as /clients/predictions

3. GET /clients/{client_id}
This endpoint retrieves a client's information by client_id from the database.

Request:
* Method: GET
* URL: /clients/{client_id}

4. PUT /clients/update/{client_id}
This endpoint updates the client information for the provided client_id using the input data.

Request:

* Method: PUT
* URL: /clients/update/{client_id}
* Body: Same as /clients/predictions

5. DELETE /clients/delete/{client_id}
This endpoint deletes the client record from the database using the provided client_id.

Request:
* Method: DELETE
* URL: /clients/delete/{client_id}

### Database Configuration
This project uses an SQLite database to store client information. The database schema can be defined as follows:

CREATE TABLE clients (
id INTEGER PRIMARY KEY AUTOINCREMENT,
age INTEGER,
gender TEXT,
work_experience INTEGER,
canada_workex INTEGER,
dep_num INTEGER,
canada_born BOOLEAN,
citizen_status TEXT,
level_of_schooling TEXT,
fluent_english BOOLEAN,
reading_english_scale INTEGER,
speaking_english_scale INTEGER,
writing_english_scale INTEGER,
numeracy_scale INTEGER,
computer_scale INTEGER,
transportation_bool BOOLEAN,
caregiver_bool BOOLEAN,
housing TEXT,
income_source TEXT,
felony_bool BOOLEAN,
attending_school BOOLEAN,
currently_employed BOOLEAN,
substance_use TEXT,
time_unemployed INTEGER,
need_mental_health_support_bool BOOLEAN
);

You can create the database by running this SQL command on your SQLite database engine.
41 changes: 41 additions & 0 deletions app/clients/database.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
import sqlite3

def get_db_connection():
conn = sqlite3.connect('clients.db')
conn.row_factory = sqlite3.Row
return conn


def create_clients_table():
conn = get_db_connection()
conn.execute('''
CREATE TABLE IF NOT EXISTS clients (
id INTEGER PRIMARY KEY AUTOINCREMENT,
age INTEGER,
gender INTEGER,
work_experience INTEGER,
canada_workex INTEGER,
dep_num INTEGER,
canada_born INTEGER,
citizen_status INTEGER,
level_of_schooling INTEGER,
fluent_english INTEGER,
reading_english_scale INTEGER,
speaking_english_scale INTEGER,
writing_english_scale INTEGER,
numeracy_scale INTEGER,
computer_scale INTEGER,
transportation_bool INTEGER,
caregiver_bool INTEGER,
housing INTEGER,
income_source INTEGER,
felony_bool INTEGER,
attending_school INTEGER,
currently_employed INTEGER,
substance_use INTEGER,
time_unemployed INTEGER,
need_mental_health_support_bool INTEGER
)
''')
conn.commit()
conn.close()
88 changes: 85 additions & 3 deletions app/clients/router.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
from fastapi import APIRouter
from fastapi.responses import HTMLResponse

from fastapi import APIRouter, HTTPException
from app.clients.service.logic import interpret_and_calculate
from app.clients.schema import PredictionInput
from app.clients.database import get_db_connection

router = APIRouter(prefix="/clients", tags=["clients"])

Expand All @@ -12,4 +11,87 @@ async def predict(data: PredictionInput):
print(data.model_dump())
return interpret_and_calculate(data.model_dump())

@router.post("/add")
async def add(data: PredictionInput):
# Input data to SQLite
conn = get_db_connection()
conn.execute('''INSERT INTO clients
(age, gender, work_experience, canada_workex, dep_num, canada_born,
citizen_status, level_of_schooling, fluent_english, reading_english_scale,
speaking_english_scale, writing_english_scale, numeracy_scale, computer_scale,
transportation_bool, caregiver_bool, housing, income_source, felony_bool,
attending_school, currently_employed, substance_use, time_unemployed,
need_mental_health_support_bool)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
''',
(data.age, data.gender, data.work_experience, data.canada_workex, data.dep_num, data.canada_born,
data.citizen_status, data.level_of_schooling, data.fluent_english, data.reading_english_scale,
data.speaking_english_scale, data.writing_english_scale, data.numeracy_scale, data.computer_scale,
data.transportation_bool, data.caregiver_bool, data.housing, data.income_source, data.felony_bool,
data.attending_school, data.currently_employed, data.substance_use, data.time_unemployed,
data.need_mental_health_support_bool))
conn.commit()
client_id = conn.execute("SELECT last_insert_rowid()").fetchone()[0]
conn.close()
return { "client_id": client_id,
"prediction_result": interpret_and_calculate(data.model_dump())}

# get client by id
@router.get("/{client_id}")
async def get_client(client_id: int):
conn = get_db_connection()
client = conn.execute('SELECT * FROM clients WHERE id = ?', (client_id,)).fetchone()
conn.close()
if client is None:
raise HTTPException(status_code=404, detail="Client not found")
return dict(client)

# update clients info by id
@router.put("/update/{client_id}")
async def update_client(client_id: int, data: PredictionInput):
conn = get_db_connection()

client = conn.execute('SELECT * FROM clients WHERE id = ?', (client_id,)).fetchone()
if client is None:
conn.close()
raise HTTPException(status_code=404, detail="Client not found")

conn.execute('''
UPDATE clients
SET age = ?, gender = ?, work_experience = ?, canada_workex = ?, dep_num = ?,
canada_born = ?, citizen_status = ?, level_of_schooling = ?, fluent_english = ?,
reading_english_scale = ?, speaking_english_scale = ?, writing_english_scale = ?,
numeracy_scale = ?, computer_scale = ?, transportation_bool = ?, caregiver_bool = ?,
housing = ?, income_source = ?, felony_bool = ?, attending_school = ?,
currently_employed = ?, substance_use = ?, time_unemployed = ?, need_mental_health_support_bool = ?
WHERE id = ?
''', (
data.age, data.gender, data.work_experience, data.canada_workex, data.dep_num,
data.canada_born, data.citizen_status, data.level_of_schooling, data.fluent_english,
data.reading_english_scale, data.speaking_english_scale, data.writing_english_scale,
data.numeracy_scale, data.computer_scale, data.transportation_bool, data.caregiver_bool,
data.housing, data.income_source, data.felony_bool, data.attending_school,
data.currently_employed, data.substance_use, data.time_unemployed,
data.need_mental_health_support_bool, client_id
))

conn.commit()
conn.close()

return {"message": "Client information updated successfully", "client_id": client_id}

# delete clients info by id
@router.delete("/delete/{client_id}")
async def delete_client(client_id: int):
conn = get_db_connection()

client = conn.execute('SELECT * FROM clients WHERE id = ?', (client_id,)).fetchone()
if client is None:
conn.close()
raise HTTPException(status_code=404, detail="Client not found")

conn.execute('DELETE FROM clients WHERE id = ?', (client_id,))
conn.commit()
conn.close()

return {"message": f"Client with id {client_id} deleted successfully."}
1 change: 1 addition & 0 deletions app/clients/schema.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@

from pydantic import BaseModel

class PredictionInput(BaseModel):
Expand Down
8 changes: 4 additions & 4 deletions app/clients/service/logic.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@
def clean_input_data(data):
#translate input into wahtever we trained the model on, numerical data in a specific order
columns = ["age","gender","work_experience","canada_workex","dep_num", "canada_born",
"citizen_status", "level_of_schooling", "fluent_english", "reading_english_scale",
"speaking_english_scale", "writing_english_scale", "numeracy_scale", "computer_scale",
"transportation_bool", "caregiver_bool", "housing", "income_source", "felony_bool", "attending_school",
"currently_employed", "substance_use", "time_unemployed", "need_mental_health_support_bool"]
"citizen_status", "level_of_schooling", "fluent_english", "reading_english_scale",
"speaking_english_scale", "writing_english_scale", "numeracy_scale", "computer_scale",
"transportation_bool", "caregiver_bool", "housing", "income_source", "felony_bool", "attending_school",
"currently_employed", "substance_use", "time_unemployed", "need_mental_health_support_bool"]
demographics = {
'age': data['age'],
'gender': data['gender'],
Expand Down
8 changes: 6 additions & 2 deletions app/main.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware

from app.clients.router import router as clients_router

from app.clients.database import create_clients_table # input from database.py

app = FastAPI()

Expand All @@ -17,4 +16,9 @@
allow_headers=["*"], # Allows all headers
)

@app.on_event("startup")
async def startup_event():
create_clients_table()

if __name__ == "__main__":
create_clients_table()
Binary file added clients.db
Binary file not shown.
Loading