BOSGAME has demonstrated distributed AI inference on its new AI MAX+ 395 7-Node Cluster using the DeepSeek-V3.1 671B model. The cluster consists of seven BOSGAME M5 AI mini PCs connected directly through USB4 cables. This hardware setup aims to offer a scalable and cost-effective approach to private AI deployment and local large language model (LLM) inference.
As AI parameter sizes grow, a single computer often lacks the VRAM capacity required for heavy local workloads. While traditional enterprise servers provide high computing power, they also come with high setup costs and power requirements. Developers are therefore exploring scale-out hardware layouts that use high-end processors like the Ryzen AI Max 395 to distribute workloads.
Also read: Spectra XSR Spire Intel Xeon Fanless Mini PC
By pooling resources across seven nodes, the mini PC cluster provides 896GB of Unified Memory. Out of this total, up to 672GB can be allocated as Unified VRAM to load large neural networks. The nodes communicate using USB4 Direct Connection, which eliminates the need for a traditional centralized server management card.
This modular scale-out architecture allows developers to start with a single mini PC and expand their computing pool as their projects grow. In addition to running local inference, the cluster supports local software development and debugging via an API service powered by Llama.cpp. Keeping the data on local hardware helps organizations maintain control over data privacy and security.
The cluster supports a wide range of workloads, including software development, debugging, Kubernetes operations, and edge AI computing applications. On the other hand, the modular expansion model helps developers align their hardware investments with actual computing demands. BOSGAME has published further setup guides and product availability details on its official website today.
Source: BOSGAME, Bosgame Product Page
