For reference, accountability, and posterity…
I intend to submit a proposal this semester for equipment to work on a project developing quick-deploy SBC-based genAI, whether on standard Pi5s or something cool like the NVIDIA Jetson Nano 4GB Dev Kit, using the localhost server functionality on LM Studio via my earlier work with hotpot broadcasting (the “pibrary” project).
Lots of potentially interesting use-cases both in the higher ed space and as a community resource.
I would LOVE to run one of the versions of kl3m (1st language model to obtain the Fairly Trained L Certification) with this project. Both kl3m-170m and kl3m-1.7b have been announced, and have sweet benchmarks.