Here are the current steps for builing BitNet.cpp for Raspberry Pi 5

[flatsiedatsie]

It took me a while to get this all working properly, so I thought I'd share steps that work:

#!/bin/bash


# Flash the latest Rasbperry Pi OS Lite onto an 32GB+ SD card, and run it on a Pi 5. 
# Log in via SSH, a then:


# Prepare
sudo apt update
sudo apt install python3-pip python3-dev cmake build-essential git clang 
wget -O - https://apt.llvm.org/llvm.sh | sudo bash -s 18


# Get the working version of BitNet.cpp
git clone --recursive https://github.com/microsoft/BitNet.git
cd BitNet
git reset --hard 404980e
git pull



# An alternative command would be:
# git clone --recursive --revision=c1e9a9a237f847f8aac9f7015b756cf4faf66d0e --depth=1 https://github.com/microsoft/BitNet.git
# ..but the version of Git available on Raspberry Pi OS doesn't support --revision yet


# Create and activate a new conda environment with Python 3.9
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-aarch64.sh
bash Miniconda3-latest-Linux-aarch64.sh
source ~/.bashrc
conda create -n bitnet-cpp python=3.9
conda activate bitnet-cpp


# Install Python dependencies
pip install huggingface_hub
pip install -r requirements.txt


# Download the model
hf download microsoft/BitNet-b1.58-2B-4T-gguf --local-dir models/BitNet-b1.58-2B-4T


# Build BitNet.cpp
python utils/codegen_tl1.py --model bitnet_b1_58-3B --BM 160,320,320 --BK 64,128,64 --bm 32,64,32
export CC=clang-18 CXX=clang++-18
rm -rf build && mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)
cd ..


# Test if everything works
python run_inference.py  -m "models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf"  -p "You are a helpful assistant."  -cnv

Related issues:

#195
#240
#305
#348
#411
#470

[flatsiedatsie]

WAIT

WAIT...

I just tested using the latest version, but replacing the src/ggml-bitnet-mad.cpp file with THIS MAD FILE from this PR.

It also works!

So that would mean skipping these two lines from my first post:

git reset --hard 404980e
git pull

And manually replacing that file instead.