Skip to content

Commit 25ead3e

Browse files
MadrakaMadraka
committed
feat: add Python SDK including core modules, tests, examples, and CI configuration.
1 parent 6447ad9 commit 25ead3e

30 files changed

Lines changed: 2463 additions & 33 deletions

‎.github/WORKFLOW.md‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
# Python SDK Workflow Organization
2+
3+
## Overview
4+
5+
The Python SDK has its own self-contained CI/CD workflow located in `sdk/python/.github/workflows/`. This allows the SDK to be managed independently and potentially moved to its own repository in the future.
6+
7+
## Workflow Structure
8+
9+
### 1. **ci.yml** - Main CI/CD Pipeline
10+
11+
**Triggers:**
12+
- Push to `main` or `develop` branches (when SDK files change)
13+
- Pull requests to `main` (when SDK files change)
14+
- Release events
15+
16+
**Jobs:**
17+
18+
#### Quality Check
19+
- Runs `cargo fmt`, `cargo clippy`, and Rust unit tests
20+
- Ensures code quality before proceeding
21+
22+
#### Python Tests
23+
- Matrix strategy across Python 3.9-3.12
24+
- Multi-platform: Linux, macOS, Windows
25+
- Runs pytest with coverage reporting
26+
- Uploads coverage to Codecov
27+
28+
#### Build Wheels
29+
- Only runs on release events
30+
- Builds native wheels for:
31+
- Linux (x86_64, aarch64)
32+
- macOS (x86_64, arm64)
33+
- Windows (x86_64)
34+
- Uploads wheels as artifacts
35+
36+
#### Publish to PyPI
37+
- Only runs for non-prerelease releases
38+
- Downloads all platform wheels
39+
- Publishes to PyPI using trusted publisher (OIDC)
40+
41+
#### Benchmarks
42+
- Runs on pushes to `main` only
43+
- Executes performance benchmarks
44+
- Uploads results as artifacts
45+
46+
## Path Filtering
47+
48+
All jobs use path filtering to only run when SDK-specific files change:
49+
```yaml
50+
paths:
51+
- 'sdk/python/**'
52+
- '.github/workflows/python-ci.yml'
53+
```
54+
55+
## Independence from Main Repo
56+
57+
This structure allows the SDK to:
58+
1. Have its own release cycle
59+
2. Be tested independently
60+
3. Maintain separate versioning
61+
4. Eventually move to its own repository with minimal changes
62+
63+
## Future Migration
64+
65+
To move the SDK to a separate repository:
66+
1. Copy `sdk/python/` to new repo root
67+
2. Update path filters in workflow (remove `sdk/python/` prefix)
68+
3. Update checkout paths
69+
4. Set up PyPI trusted publisher for new repo
70+
5. Update documentation links
71+
72+
## Environment Variables
73+
74+
The workflow uses:
75+
- `CARGO_TERM_COLOR=always` - Colored Cargo output
76+
- `RUST_BACKTRACE=1` - Full backtraces on errors
77+
78+
## Secrets Required
79+
80+
For PyPI publishing, configure:
81+
- **Trusted Publisher (Recommended)**: Configure at https://pypi.org/manage/account/publishing/
82+
- Owner: `lnmplang`
83+
- Repository: `lnmp-protocol` (or new SDK repo name)
84+
- Workflow: `ci.yml`
85+
- Environment: `pypi`
86+
87+
No manual tokens required with trusted publishers!

‎.github/workflows/ci.yml‎

Lines changed: 228 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,243 @@
1-
name: CI - Python SDK
1+
name: Python SDK CI/CD
22

33
on:
44
push:
5-
branches: [ main ]
5+
branches: [ main, develop ]
6+
paths:
7+
- 'sdk/python/**'
8+
- '.github/workflows/python-ci.yml'
69
pull_request:
710
branches: [ main ]
11+
paths:
12+
- 'sdk/python/**'
13+
release:
14+
types: [published]
15+
16+
env:
17+
CARGO_TERM_COLOR: always
18+
RUST_BACKTRACE: 1
819

920
jobs:
10-
test:
21+
# ============================================
22+
# Job 1: Code Quality & Unit Tests
23+
# ============================================
24+
quality:
25+
name: Code Quality (Rust)
1126
runs-on: ubuntu-latest
27+
defaults:
28+
run:
29+
working-directory: sdk/python
30+
31+
steps:
32+
- uses: actions/checkout@v4
33+
34+
- name: Install Rust Toolchain
35+
uses: dtolnay/rust-toolchain@stable
36+
with:
37+
components: rustfmt, clippy
38+
39+
- name: Cache Cargo Dependencies
40+
uses: actions/cache@v3
41+
with:
42+
path: |
43+
~/.cargo/bin/
44+
~/.cargo/registry/index/
45+
~/.cargo/registry/cache/
46+
~/.cargo/git/db/
47+
sdk/python/target/
48+
key: ${{ runner.os }}-cargo-${{ hashFiles('sdk/python/Cargo.lock') }}
49+
50+
- name: Run rustfmt
51+
run: cargo fmt --all -- --check
52+
53+
- name: Run clippy
54+
run: cargo clippy --all-targets --all-features -- -D warnings
55+
56+
- name: Run Rust tests
57+
run: cargo test --verbose
58+
59+
# ============================================
60+
# Job 2: Python Tests (Multi-version)
61+
# ============================================
62+
test:
63+
name: Python Tests (${{ matrix.python-version }})
64+
runs-on: ${{ matrix.os }}
65+
needs: quality
1266
strategy:
67+
fail-fast: false
1368
matrix:
14-
python: [3.11]
69+
os: [ubuntu-latest, macos-latest, windows-latest]
70+
python-version: ['3.9', '3.10', '3.11', '3.12']
71+
72+
defaults:
73+
run:
74+
working-directory: sdk/python
75+
1576
steps:
1677
- uses: actions/checkout@v4
78+
79+
- name: Set up Python ${{ matrix.python-version }}
80+
uses: actions/setup-python@v4
81+
with:
82+
python-version: ${{ matrix.python-version }}
83+
84+
- name: Install Rust Toolchain
85+
uses: dtolnay/rust-toolchain@stable
86+
87+
- name: Cache Cargo Dependencies
88+
uses: actions/cache@v3
89+
with:
90+
path: |
91+
~/.cargo/bin/
92+
~/.cargo/registry/index/
93+
~/.cargo/registry/cache/
94+
~/.cargo/git/db/
95+
sdk/python/target/
96+
key: ${{ runner.os }}-cargo-${{ hashFiles('sdk/python/Cargo.lock') }}
97+
98+
- name: Install Python Dependencies
99+
run: |
100+
python -m pip install --upgrade pip
101+
pip install maturin pytest pytest-cov
102+
103+
- name: Build Extension
104+
run: maturin develop --release
105+
106+
- name: Run Python Tests
107+
run: pytest tests/ -v --cov=lnmp --cov-report=xml
108+
109+
- name: Upload Coverage
110+
if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.11'
111+
uses: codecov/codecov-action@v3
112+
with:
113+
file: ./sdk/python/coverage.xml
114+
flags: python-sdk
115+
116+
# ============================================
117+
# Job 3: Build Wheels (Multi-platform)
118+
# ============================================
119+
build:
120+
name: Build Wheels (${{ matrix.target }})
121+
runs-on: ${{ matrix.os }}
122+
needs: test
123+
if: github.event_name == 'release'
124+
strategy:
125+
matrix:
126+
include:
127+
# Linux
128+
- os: ubuntu-latest
129+
target: x86_64-unknown-linux-gnu
130+
name: linux-x86_64
131+
- os: ubuntu-latest
132+
target: aarch64-unknown-linux-gnu
133+
name: linux-aarch64
134+
135+
# macOS
136+
- os: macos-latest
137+
target: x86_64-apple-darwin
138+
name: macos-x86_64
139+
- os: macos-latest
140+
target: aarch64-apple-darwin
141+
name: macos-arm64
142+
143+
# Windows
144+
- os: windows-latest
145+
target: x86_64-pc-windows-msvc
146+
name: windows-x86_64
147+
148+
defaults:
149+
run:
150+
working-directory: sdk/python
151+
152+
steps:
153+
- uses: actions/checkout@v4
154+
155+
- name: Set up Python
156+
uses: actions/setup-python@v4
157+
with:
158+
python-version: '3.11'
159+
160+
- name: Install Rust Toolchain
161+
uses: dtolnay/rust-toolchain@stable
162+
with:
163+
targets: ${{ matrix.target }}
164+
165+
- name: Install Maturin
166+
run: pip install maturin
167+
168+
- name: Build Wheels
169+
run: maturin build --release --target ${{ matrix.target }} --out dist
170+
171+
- name: Upload Wheels
172+
uses: actions/upload-artifact@v3
173+
with:
174+
name: wheels-${{ matrix.name }}
175+
path: sdk/python/dist/*.whl
176+
177+
# ============================================
178+
# Job 4: Publish to PyPI
179+
# ============================================
180+
publish:
181+
name: Publish to PyPI
182+
runs-on: ubuntu-latest
183+
needs: build
184+
if: github.event_name == 'release' && !github.event.release.prerelease
185+
environment:
186+
name: pypi
187+
url: https://pypi.org/p/lnmp
188+
permissions:
189+
id-token: write
190+
191+
steps:
192+
- name: Download All Wheels
193+
uses: actions/download-artifact@v3
194+
with:
195+
path: dist
196+
197+
- name: Flatten Dist Directory
198+
run: |
199+
mkdir -p final-dist
200+
find dist -name '*.whl' -exec cp {} final-dist/ \;
201+
202+
- name: Publish to PyPI
203+
uses: pypa/gh-action-pypi-publish@release/v1
204+
with:
205+
packages-dir: final-dist/
206+
skip-existing: true
207+
208+
# ============================================
209+
# Job 5: Benchmarks (Optional)
210+
# ============================================
211+
benchmark:
212+
name: Performance Benchmarks
213+
runs-on: ubuntu-latest
214+
needs: quality
215+
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
216+
defaults:
217+
run:
218+
working-directory: sdk/python
219+
220+
steps:
221+
- uses: actions/checkout@v4
222+
17223
- name: Set up Python
18224
uses: actions/setup-python@v4
19225
with:
20-
python-version: ${{ matrix.python }}
21-
- name: Install
22-
run: python -m pip install -U pip setuptools wheel
23-
- name: Run tests
24-
run: python -m pytest -q
226+
python-version: '3.11'
227+
228+
- name: Install Rust Toolchain
229+
uses: dtolnay/rust-toolchain@stable
230+
231+
- name: Install Dependencies
232+
run: |
233+
pip install maturin
234+
maturin develop --release
235+
236+
- name: Run Benchmarks
237+
run: python benchmarks/run_benchmarks.py
238+
239+
- name: Upload Benchmark Results
240+
uses: actions/upload-artifact@v3
241+
with:
242+
name: benchmarks
243+
path: sdk/python/BENCHMARKS.md

0 commit comments

Comments
 (0)