Configuration
Host : Raspberry Pi 4 (4 cores of Cortex A-72 + DDR4 4GB)
RPC servers : 4 x Odroid C4 4GB (4 cores of Cortex A-53 2GHz + DDR4 4GB)
Network : 1Gbps ethernet
Storage : USB3-SATA SSD
Model : unsloth/Qwen 3.5-0.8B-UD-Q2_K_XL.gguf
| # of rpc servers | Prompt | Generation |
| 0 | 9.7 t/s | 4.5 t/s |
| 1 | 4.3 t/s (default: Host+RPC) | 3.4 t/s (default: Host+RPC) |
| 4.3 t/s (tensor-split 1,0: no Host) | 2.6 t/s | |
| 2 | 5.0 t/s 4.7 t/s |
3.5 t/s 3.6 t/s |
| 4 | 4.6 t/s 4.0 t/s |
2.9 t/s 2.7 t/s |
Model : unsloth/Qwen 3.5-0.8B-UD-Q4_K_XL.gguf