equal-shares

Description

This project is a voting application and algorithm based Equal Shares
For Ariel University

Table of content

equal-shares
Authors

Technologies

python - As programming language for the backend
conda - For managing python environment
poetry - For managing python dependencies
FastAPI - As framework for the backend
uvicorn - For running the backend usinn ASGI
PostgresSQL - As database
docker - For running the services
docker compose - For running the services
Nginx - For handling SSL and routing different domains to thier services
React - As framework for the frontend
TypeScript - As programming language for the frontend
Vite - As build tool for the frontend
Tailwind CSS - For styling the frontend
MUI - As UI library for the frontend
GitHub Actions - For running the CI
eslint, prettier - For linting and formatting the frontend
isort, black, flake8, mypy - For linting and formatting the backend
pytest, doctest - For testing the backend

Versions

python 3.12 (with poetry 1.7.1)
node 21
PostgresSQL 16

Files Structure

.github
- workflows
  - ci.yml - Running the CI
backend - The backend API
- examples - examples of usage of the algorithm
  - algorithm - examples of the algorithm of equal shares running with logs.
    For debugging and understanding the core algorithm
- src
  - algorithm - the algorithm of equal shares
  - routers
    - admin - routes for managmenet
    - form - routes for the frontend
    - report - routes for the reports of the votes and the algorithm
  - __main__ - the entry of the backend application using CLI
  - app.py - the application of the backend that used by uvicorn
  - cli - CLI commands for the backend
  - config.py - contains the configuration of the backend. uses environment variables
  - database.py - database connection
  - exceptions.py - custom exceptions
  - logger.py - logging configuration
  - models.py - models and queries for comunicating with the database
  - schemas.py - schemas of the API
  - security.py - security functions
- tests - tests of the backend, same structure as src
- Dockerfile - for building the images of the backend
- equal-shares-api-private-key.pem - private RSA key for the API for local development, ignored in git
- equal-shares-api-public-key.pem - public RSA key for the API for local development, ignored in git
- Makefile - commands for development
- poetry.lock - lock file of the dependencies, dont change manually
- pyproject.toml - configuration and poetry dependencies
docs - documentation and spesification files
frontend - The frontend application
- public - static files
- src
  - assets - images and icons
  - components
  - api.ts - for using the backend API
  - App.tsx - the main component
  - config.ts - configuration of the frontend at build time
  - main.tsx - the entry of the frontend application
  - schema.ts - schemas of the API
- Dockerfile - for building the images of the frontend
- package-lock.json - lock file of the dependencies, dont change manually
- package.json - dependencies and commands for development
prod - files for production server
- db
  - pg_hba-original.conf - original pg_hba.conf of PostgresSQL
  - pg_hba.conf - configuration of the PostgresSQL
- frontend
  - default-original.conf - original ` default.conf of Nginx
  - default.conf - configuration of the Nginx for the frontend container
- nginx
  - default - Nginx configuration for the server
- backend.env - environment variables for the backend service, will copy to /app/backend.env
- db.env - environment variables for the database service, will copy to /app/db.env
- frontend.env - environment variables for build the frontend, will copy to /app/frontend.env
res - resources
scripts - scripts for production
- build.sh - building the services
- check-database.sh - checking that the backend can connect to the database
- pull.sh - pulling the latest version of the code, build the services and restart the services
- renew-ssl.sh - renewing the SSL Certificate
- rest-env-files.sh - copying the environment files to the /app directory
- restart.sh - restarting the services
wp
- functions.php - A PHP code for adding the vote website inside iframe as Elementor widget. For WordPress
dev.docker-compose.yml - for local development
environment.yml - conda environment
LICENSE - MIT license
Makefile - commands for development
prod.docker-compose.yml - for production
README.md - this file
test.docker-compose.yml - for local testing
windows-serve.bat - script for running the services locally in Windows
windows-test.bat - script for running the tests locally in Windows

Production Files Structure

/app - the root directory of the project
- backend.env - environment variables for the backend service (container)
- db.env - environment variables for the database service (container)
- equal-shares - the project directory
- frontend.env - environment variables for build the frontend image
- keys
  - equal-shares-api-private-key.pem - private RSA key for the API
  - equal-shares-api-public-key.pem - public RSA key for the API
/etc
- letsencrypt
  - live
    - <server-domain>
      - cert.pem
      - chain.pem
      - fullchain.pem
      - privkey.pem
- nginx
  - sites-available
    - default - Nginx configuration
  - sites-enabled
    - default - Nginx configuration
/root - Home directory
- certbot-creds.ini - credentials for the SSL Certificate

Environment Variables

Backend

Table of the required environment variables for the backend:

Variable	Description
PG_DATABASE	PostgresSQL database name
PG_USER	PostgresSQL user
PG_PASSWORD	PostgresSQL password
PG_HOST	PostgresSQL host
ADMIN_KEY	uuid for admin key
API_RSA_PUBLIC_KEY	path to public RSA key for API
API_RSA_PRIVATE_KEY	path to private RSA key for API

Table of the optional environment variables for the backend:

Variable	Description	Default
PG_PORT	PostgresSQL port	5432
WITHOUT_AUTH_MODE	for using without authentication	false

Frontend

Table of the required environment variables for the frontend, they used at build time:

Variable	Description
VITE_API_HOST	API backend host
VITE_WITHOUT_AUTH_MODE	for using without authentication

Database

Table of the required environment variables for the database:

Variable	Description
POSTGRES_PASSWORD	Password for PostgresSQL

Table of the optional environment variables for the database:

Variable	Description	Default
POSTGRES_USER	defualt user	postgres
POSTGRES_DB	defualt database name	postgres

Diagrams

Communication Diagram

flowchart LR
  react(voting website) --> main-domain
  react(voting website) --> api-sub-domain
  admin(admin dashbord) --> api-sub-domain
  subgraph Server
    subgraph docker-containers
      frontend
      backend --> PostgresSQL
    end

    main-domain -->|Nginx| frontend
    api-sub-domain -->|Nginx| backend
  end

Database ERD

Requirements

For Using Locally

Git

Installation:

sudo apt-get update

sudo apt-get install git

Verify installation:
```
git --version
```

Docker

Installation:

# Add Docker's official GPG key:

sudo apt-get update

sudo apt-get install ca-certificates curl

sudo install -m 0755 -d /etc/apt/keyrings

sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc

sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:

echo \

  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \

  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \

  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt-get update

# Install Docker packages:

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Set up Docker:

# Start the Docker service

sudo systemctl start docker

# Verify that Docker is running

sudo systemctl status docker

# Enable Docker to start on boot

sudo systemctl enable docker

# Verify the installation

sudo docker run hello-world

For Development

conda
node 21.5.0

Clone the repository:

git clone https://github.com/equal-shares/equal-shares.git

cd equal-shares

Generate or copy the RSA keys for the API:

To generate new keys:
```
 cd backend
 ssh-keygen -t rsa -b 2048 -f equal-shares-api-private-key.pem
 mv equal-shares-api-private-key.pem.pub equal-shares-api-public-key.pem
 cd ..
```
Alternatively, if you have existing keys, copy them to:
- ./backend/equal-shares-api-private-key.pem
- ./backend/equal-shares-api-public-key.pem
Run the following command to install and run the services:

sudo docker compose -f dev.docker-compose.yaml up --build

or For Linux / MacOS:

make serve

For Windows, run the script file windows-serve.bat

For Development

Run the following commands to install for development:

conda env create -f environment.yml
conda activate equal-shares

cd backend
poetry config virtualenvs.create false
poetry install

cd ../frontend
npm ci

Usage

For running the frontend, backend and the database run the following command:

make serve

Or

docker compose -f dev.docker-compose.yaml up --build

or For Windows run the script file windows-serve.bat

The API will run on http://localhost:8000/

In the API Dashbord (Swagger UI) you can see, manage and test the System.
admin_key is key for authentication as admin and you can have it from docker-compose.yml under the environments of backend.

And the frontend will run on http://localhost:5173/

The frontend is a simple form for voting.
For authentication the URL needs the paramters email and token.

For example: http://localhost:5173/?email=some.mail%40example.com&token=jJfGlbWO7%2BxiPCaMofLnX2zYFIGZOuBQ1Vg65zAyUd0EVAfk36y%2FIzH67UAUGsrjlvMKMsF9%2FIAlMC66Ner2g9vyP%2F%2FazBMirFpN9spDyFeHAiEk3tmz%2FlCXnQfz%2BDmayKZsxO5n%2BLf1bs4eF8TR6u8wwQumV%2BnErXvF1%2BCy4W0%3D

You can create a token using /admin/create-token in the API Dashbord.

Management

Follow these steps to set up and manage the system:

Create Database Tables:
- In the API Dashboard, run /admin/create-tables
Create and Activate Poll:
- Create a new poll via /admin/polls/create
  - Use the name parameter to specify the poll name
- Activate the poll via /admin/polls/set-active
  - Use either poll_id or name to specify which poll to activate
Configure Settings:
- Run /admin/set-settings with the following parameters:
  - max_total_points - the maximum total points a voter can give to all the projects in total
  - points_step - a number that points in votes can be divided by. For example if points_step is 100, voter cannot give 150 points to a project but can give 100 or 200 points.
  - open_for_voting - set to true to allow voting
Add Projects:
- Use /admin/add-projects to upload projects from an XLSX file
- The XLSX file should have the following columns:
  - Column 1: name of the project
  - Column 2: min points of the project
  - Column 3: max points of the project
  - Column 4: "description 2" is not in use
  - Column 5: "description 1" of the project
  - Column 6: "is fixed project" - if this column is 'v' then the project cannot be unselected.

Project Template File

A ready-to-use example file, project_template.xlsx, is available in the root directory of this project. Use this file as a template when uploading projects via the /admin/add-projects endpoint in the API dashboard.

Where to find: Root directory (project_template.xlsx)
How to use:
1. Open the file in Excel or Google Sheets.
2. Edit or add rows for your projects, following the column order below.
3. Save as .xlsx and upload it using the /admin/add-projects endpoint.
Column order (no header row):
1. Project name
2. Min points
3. Max points
4. Description 2 (can be left empty)
5. Description 1
6. Is fixed project (v for fixed, otherwise leave blank)

Additional Management Commands:

Delete all projects and votes: /admin/delete-projects-and-votes
Delete only votes: /admin/delete-votes
Get Projects and Settings as JSON: /admin/projects

Voting Rules

All votes must allocate the entire available budget exactly
The system will not accept votes where the total allocated points differ from the maximum allowed points

Development

For running the services run the following command:

make serve

or for windows run the script file windows-serve.bat

Linters and Formatters

For clean, safe and maintainable deployment exits number of Linters and Formatters.

Formaters - are tools that automatically format and fix the code.
- Backend: isort, black
- Frontend: Prettier, ESLint
Linters - are tools for check the code.
Backend: flake8, black, mypy
Frontend: ESLint

Before running the formatter and linters, make sure conda environment is activated and node version is 21.5.0
For running the formatters and linters run the following commands:

make fix-lint

For running only the formatters run the following commands:

make fix

For running only the linters run the following commands:

make lint

or for windows run the script file windows-lint.bat
Note: In windows we do not support formatters

Testing

For running the tests run the following command:

make test

Or for windows run the script file windows-test.bat

To run the backend tests inside the backend container (when using Docker Compose for local development), use:

# From the project root
docker compose -f dev.docker-compose.yaml exec backend make test

This will run all backend unit tests and doctests using the Makefile inside the container.

Examples

In backend/examples directory you can find examples of how
to run the algorithm for reasearch and debugging.
In the API call public.py that is Facade for the algorithm module and not directly the algorithm.

For running examples of the algorithm of equal shares run the following command:

make examples-run-algorithm

Production

Production Scripts

bash /app/equal-shares/scripts/restart.sh

For updating the services the following command will:

pull the last version of the repository
update files permissions
build the services
restart the services

bash /app/equal-shares/scripts/pull.sh

For reset the environment variables files by copying the files to the /app directory run the following command:

bash /app/equal-shares/scripts/rest-env-files.sh

For building the services run the following command:

bash /app/equal-shares/scripts/build.sh

For checking manually that backend can connect to the database run the following command:

bash /app/equal-shares/scripts/check-database.sh

For renewing the SSL Certificate run the following command:

bash /app/equal-shares/scripts/renew-ssl.sh

Production Monitoring and Logs

For view all the services run the following command:

docker ps

For viewing the logs (100 lines) of the backend service run the following command:

docker logs equal-shares-backend-1 --tail 100

For viewing the logs (100 lines) of the frontend service run the following command:

docker logs equal-shares-frontend-1 --tail 100

For viewing the logs (100 lines) of the database service run the following command:

docker logs equal-shares-db-1 --tail 100

Production Requirements

Linux Server with:
- Domain and Sub-Domain - for the Frontend and the API
- DNS API Token for the SSL Certificate certbot (for example DigitalOcean: dns_digitalocean_token)
SSH
RSA keys for the API (equal-shares-api-private-key.pem and equal-shares-api-public-key.pem)

The main domain will be for the websites and the sub-domain will be for the API and the admin dashbord.

Production Installation

In the next steps we will show how to install the project on new production server.

Use SSH for connecting to the server
Run the following commands for update server:

sudo apt update
sudo apt upgrade -y

For install docker and docker compose run the following commands:

sudo apt-get install -y ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

# Install Docker
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

For check the installation of docker run the following command:

sudo docker run hello-world

For installing the dependencies for SSL Certificate:

Note: this example is for DigitalOcean, you can use other providers.
You can see other providers in the Certbot Documentation

sudo apt install -y python3-certbot-dns-digitalocean

For checking the installation of certbot run the following command:

certbot --version
certbot plugins

For installing the project run the following commands:

mkdir /app
cd /app
git clone https://github.com/equal-shares/equal-shares.git

For configuring the environment variables run the following commands:

ash /app/equal-shares/scripts/rest-env-files.sh

For creating Admin Key run the following command:

python3 -c "import uuid;print(uuid.uuid4())"

Save the output of the command, this is the Admin Key for managing the API
Copy the output and paste it to the /app/backend.env as value of ADMIN_KEY using nano:

nano /app/backend.env

Use nano for editing VITE_API_HOST in /app/frontend.env
Set the value of VITE_API_HOST to URL of the sub domain of for the API / Admin Dashbord. For example url like: https://api.equal-shares.net

nano /app/frontend.env

Run the folowing commands for create directory for the Keys:

mkdir -p /app/keys

For adding the api RSA keys disconnect the SSH
Copy the RSA keys of the API to production server using the following commands:

Note: replace <server-domain> with the domain or IP of the server, and you should have the keys in your current directory

scp equal-shares-api-private-key.pem root@<server-domain>:/app/keys/equal-shares-api-private-key.pem
scp equal-shares-api-public-key.pem root@<server-domain>:/app/keys/equal-shares-api-public-key.pem

Connect to the server using SSH
The next steps will be for the SSL Certificate (11 - 12)

You can see the Lets Encrypt Tutorial - DigitalOcean for more information

Run the following command:

nano /root/certbot-creds.ini

And add the following lines to the file:

dns_digitalocean_token = <dns-digitalocean-token>

Replace <dns-digitalocean-token> with the DNS API Token for the SSL Certificate certbot And save the file

For retrieving the SSL Certificate. Run the following command:

sudo certbot certonly \
  --dns-digitalocean \
  --dns-digitalocean-credentials ~/certbot-creds.ini \
  -d '<server-domain>' \
  -d '*.<server-domain>'

Note: replace <server-domain> with the domain of the server

After the command you will have to write email.
Then press Y, then press N.

Install Nejinx for the server:

sudo apt install -y nginx

For configuring the Nginx stop the Nginx service and remove the default configuration:

systemctl stop nginx

For removing the default configuration run the following commands:

rm /etc/nginx/sites-enabled/*
rm /etc/nginx/sites-available/*

Then copy the configuration of the Nginx from the project to the server:

cp /app/equal-shares/prod/nginx/default /etc/nginx/sites-available/default

Then use nano for editing the configuration:

nano /etc/nginx/sites-available/default

And replace <server-domain> with the domain of the server for the website

And replace <sub-server-domain> with the sub domain of the server for the API / Admin Dashbord

Note: exists 6 places to replace the domain and 2 places to replace the sub domain

Then save the file and run the following commands:

cp /etc/nginx/sites-available/default /etc/nginx/sites-enabled/default

And start the Nginx service:

systemctl start nginx

Update the permissions of the directories:

chmod 744 /app/**

For build and start the services run the following commands:

bash /app/equal-shares/scripts/build.sh

For start the services run the following commands:

bash /app/equal-shares/scripts/pull.sh

For checking the services open in Browser (like Google Chrome) the domain of the server for the website and the sub domain for the Admin / API Dashbord.

For creating the database tables open the sub domain of the server for the Admin / API Dashbord
And run the route /admin/create-tables

Links

ca website
ca-frontend Original frontend - React
ca-backend Original backend - Django
equalshares.net
Final-Project For the algorithm of equal shares - Flask and React
Lets Encrypt Tutorial - DigitalOcean

Authors

Bar Nahmias
Didi Avidad
Omer Priel

Name		Name	Last commit message	Last commit date
Latest commit History 339 Commits
.github/workflows		.github/workflows
.vscode		.vscode
backend		backend
docs		docs
frontend		frontend
prod		prod
res		res
scripts		scripts
wp		wp
.editorconfig		.editorconfig
.gitignore		.gitignore
.nvmrc		.nvmrc
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
dev.docker-compose.yaml		dev.docker-compose.yaml
environment.yml		environment.yml
lint.docker-compose.yaml		lint.docker-compose.yaml
prod.docker-compose.yaml		prod.docker-compose.yaml
project_template.xlsx		project_template.xlsx
test.docker-compose.yaml		test.docker-compose.yaml
test_data.py		test_data.py
windows-lint.bat		windows-lint.bat
windows-serv.bat		windows-serv.bat
windows-test.bat		windows-test.bat

Folders and files

Latest commit

History

Repository files navigation

equal-shares

Description

Table of content

Technologies

Versions

Files Structure

Production Files Structure

Environment Variables

Backend

Frontend

Database

Diagrams

Communication Diagram

Database ERD

Requirements

For Using Locally

For Development

For Development

Usage

Management

Project Template File

Voting Rules

Development

Linters and Formatters

Testing

Examples

Production

Production Scripts

Production Monitoring and Logs

Production Requirements

Production Installation

Links

Authors

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages