ONNX export: use dummy data
Calibration: use public reference dataset
Train: use private dataset
datasets:
- MNIST
- CIFAR10
Models:
Settings: + setting 0: 16 clients: 8 LeNet-5, 8 ResNet-2 + setting 1: 16 clients: 4 LeNet-5, 12 ResNet-2
+ setting 2: 8 clients: 2 LeNet-5, 2 ResNet-2, 2 ResNet-8, 2 ResNet-14
+ setting 3: 16 LeNet-5 on MNIST
Models:
- MNIST (10 class, channels: 1, image_size: 28)
- Resnet2: Total params: 9770, Trainable Params: 9770
- Lenet5: Total params: 44470, Trainable Params: 44470
- CIFAR10 (10 class)
Command:
-
nohup poetry run python simulate.py --data=mnist --scenario=id --model-settings=2 --rounds=200 --gpus=4 > ./logs/output.log 2>&1 &
-
ID Training: nohup poetry run python simulate.py --data=mnist --scenario=id --model-settings=16_2 --start-round=600 --rounds=1000 --gpus=4 --batch-size=2048 > ./logs/output.log 2>&1 &
-
KD Training: nohup poetry run python simulate.py --data=mnist --scenario=kdb --k=0 --kd-weight=0.5 --model-settings=2 --rounds=2000 --gpus=4 --batch-size=2048 > ./logs/output.log 2>&1 &
-
nohup poetry run python zkp.py > ./logs/output.log 2>&1 &
-
Test GPU: nohup poetry run python simulate.py --data=mnist --scenario=id --k=0 --kd-weight=0.5 --model-settings=t3 --rounds=2 --gpus=1 > ./logs/output.log 2>&1 &
-
Monitor GPU: watch -n 1 nvidia-smi EZKL Time:
-
resnet2:
- setting generation: 148.97 seconds
- calibration: 3174.76 seconds
- model compilation: 0.05 seconds
- setup:
- prove:
- lenet5:
- setting generation: 1.48 seconds
- calibration: 102.04 seconds
- model compilation: 0.04 seconds
- setup: 211.93 seconds
model resnet18
200 scenario k kd_weight mean std 0 id 0 0.0 0.689481 0.342672 1 kdb 1 0.5 0.977144 0.011034 2 kdb 2 0.5 0.978925 0.010933 3 kdb 3 0.5 0.977263 0.010265 4 kdb 4 0.5 0.978162 0.011714 5 kdb 5 0.5 0.977906 0.011919
300 scenario k kd_weight mean std 0 id 0 0.0 0.843456 0.128610 1 kdb 1 0.5 0.979081 0.009640 2 kdb 2 0.5 0.977481 0.010592 3 kdb 3 0.5 0.979044 0.010464 4 kdb 4 0.5 0.943438 0.139096 5 kdb 5 0.5 0.936987 0.161924
max round_id k kd_weight scenario mean std 5958 993 0 0.0 id 0.903275 0.076711 5827 971 1 0.5 kdb 0.983475 0.008143 3362 560 2 0.5 kdb 0.981694 0.008321 3621 603 3 0.5 kdb 0.981744 0.008865 4198 699 4 0.5 kdb 0.981869 0.008776 5885 980 5 0.5 kdb 0.982238 0.008507