Skip to content

todo.md

Latest commit

 

History

History
453 lines (298 loc) · 12.2 KB

todo.md

File metadata and controls

453 lines (298 loc) · 12.2 KB

TODO List

one-python

We don't need a lot of libraries. We just need the best ones.

Libraries

Utilities

Interactive Interpreter

  • IPython - A rich toolkit to help you make the most out of using Python interactively.

Environment

  • poetry - A tool to create isolated Python environments and manage poetry project

IDE

  • PyCharm - Commercial Python IDE based on the IntelliJ platform by JetBrains. Free community edition available.

General Machine Learning

  • scikit-learn - Simple and efficient tools for data mining and data analysis.

Machine Learning > Deep Learning

  • TensorFlow - Low-level (configurations over conventions) library for building deep learning data flow graphs.

Machine Learning > Deep Learning + Computer Vision

  • caffe - Deep learning framework made with expression, speed, and modularity in mind.

Optical Character Recognition (OCR)

Chinese Word Segmentation

  • jieba - Chinese Words Segmentation Utilities.

Concurrency and Networking

  • gevent - A coroutine-based Python networking library that uses greenlet.

HTTP Request

  • requests - Python HTTP requests for humans.

Web Crawling

  • Scrapy - A fast high-level screen scraping and web crawling framework.

Web Content Extracting

  • newspaper - News extraction, article extraction and content curation in Python.

Scientific Computing

  • scipy - An open-source software for mathematics, science, and engineering. statistics, optimization, integration, linear algebra, Fourier transforms, signal and image processing, ODE solvers, and more.

Natural Language

  • nltk - A suite of libraries and programs for symbolic and statistical natural language processing.

Markdown

  • mistune - The fastest markdown parser in pure Python with renderer features, inspired by marked.

Data Analysis

  • pandas - A software library for data manipulation and analysis. In particular, it offers data structures and operations for manipulating numerical tables and time series.

Frameworks

  • Django - The most popular full featured web framework in Python.

Issues

List of labels and corresponding titles

Label Title in Releases
enhancement, feature 🚀 Features
bug, refactoring, bugfix, fix 🔧 Fixes & Refactoring
build, ci, testing 📦 Build System & CI/CD
breaking 💥 Breaking Changes
documentation 📝 Documentation
dependencies ⬆️ Dependencies updates

Makefile usage

[Makefile](https://github.com/{{ cookiecutter.github_name }}/{{ cookiecutter.project_name }}/blob/master/Makefile) contains a lot of functions for faster development.

1. Download and remove Poetry

To download and install Poetry run:

make poetry-download

To uninstall

make poetry-remove

2. Install all dependencies and pre-commit hooks

Install requirements:

make install

Pre-commit hooks coulb be installed after git init via

make pre-commit-install

3. Codestyle

Automatic formatting uses pyupgrade, isort and black.

make codestyle

# or use synonym
make formatting

Codestyle checks only, without rewriting files:

make check-codestyle

Note: check-codestyle uses isort, black and darglint library

Update all dev libraries to the latest version using one comand

make update-dev-deps

4. Code security 

make check-safety

This command launches Poetry integrity checks as well as identifies security issues with Safety and Bandit.

make check-safety

5. Type checks

Run mypy static type checker

make mypy

6. Tests with coverage badges

Run pytest

make test

7. All linters

Of course there is a command to rule run all linters in one:

make lint

the same as:

make test && make check-codestyle && make mypy && make check-safety

8. Docker 

make docker-build

which is equivalent to:

make docker-build VERSION=latest

Remove docker image with

make docker-remove

More information [about docker](https://github.com/{{ cookiecutter.github_name }}/{{ cookiecutter.project_name }}/tree/master/docker).

9. Cleanup

Delete pycache files 

make pycache-remove

Remove package build

make build-remove

Delete .DS_STORE files

make dsstore-remove

Remove .mypycache

make mypycache-remove

Or to remove all above run:

make cleanup

🎯 What's next

  • Typer is great for creating CLI applications.
  • Rich makes it easy to add beautiful formatting in the terminal.
  • Pydantic – data validation and settings management using Python type hinting.
  • Loguru makes logging (stupidly) simple.
  • tqdm – fast, extensible progress bar for Python and CLI.
  • IceCream is a little library for sweet and creamy debugging.
  • orjson – ultra fast JSON parsing library.
  • Returns makes you function's output meaningful, typed, and safe!
  • Hydra is a framework for elegantly configuring complex applications.
  • FastAPI is a type-driven asynchronous web framework.

Todo App

Todo App is a hobby project that is primarily focused around being a playground for experimenting with different technologies.

Application Architecture

Todo App Architecture

Secrets Management

This repository uses SOPS with age for managing secrets.

See Configuration and Secrets Management for details.

Reverse Proxy

Real-Time Messaging

Workflow Engine

Monitoring Stack

Identity and Access Management

Running the App Locally

  1. Add the following entries to your /etc/hosts file:
127.0.0.1    todo-app.com
127.0.0.1    auth.todo-app.com
127.0.0.1    centrifugo.todo-app.com
127.0.0.1    temporal.todo-app.com
127.0.0.1    grafana.todo-app.com
127.0.0.1    minio.todo-app.com

  1. Make sure you have the unencrypted secrets in the env directory. See Configuration and Secrets Management for details.

  2. Run Docker Compose

> docker compose up --build -d

  1. Import the Keycloak realm. See Configuring Keycloak for details.

  2. Open the App in browser: http://todo-app.com

    • Normal user:
      • username: demouser
      • password: S3c_r3t!
    • Viewer (read-only) user:
      • username: demoviewer
      • password: S3c_r3t!

Tools

JSON Schema Generator

The app uses JSON Schema for message validation. See json-schema-generator for details.

one-python

We don't need a lot of libraries. We just need the best ones.

Interactive Interpreter

  • IPython - A rich toolkit to help you make the most out of using Python interactively.

Environment

  • poetry - A tool to create isolated Python environments and manage poetry project

IDE

  • PyCharm - Commercial Python IDE based on the IntelliJ platform by JetBrains. Free community edition available.

General Machine Learning

  • scikit-learn - Simple and efficient tools for data mining and data analysis.

Machine Learning > Deep Learning

  • TensorFlow - Low-level (configurations over conventions) library for building deep learning data flow graphs.

Machine Learning > Deep Learning + Computer Vision

  • caffe - Deep learning framework made with expression, speed, and modularity in mind.

Optical Character Recognition (OCR)

Chinese Word Segmentation

  • jieba - Chinese Words Segmentation Utilities.

Concurrency and Networking

  • gevent - A coroutine-based Python networking library that uses greenlet.

HTTP Request

  • requests - Python HTTP requests for humans.

Web Crawling

  • Scrapy - A fast high-level screen scraping and web crawling framework.

Web Content Extracting

  • newspaper - News extraction, article extraction and content curation in Python.

Scientific Computing

  • scipy - An open-source software for mathematics, science, and engineering. statistics, optimization, integration, linear algebra, Fourier transforms, signal and image processing, ODE solvers, and more.

Natural Language

  • nltk - A suite of libraries and programs for symbolic and statistical natural language processing.

Markdown

  • mistune - The fastest markdown parser in pure Python with renderer features, inspired by marked.

Data Analysis

  • pandas - A software library for data manipulation and analysis. In particular, it offers data structures and operations for manipulating numerical tables and time series.

Frameworks

  • Django - The most popular full featured web framework in Python.