Skip to content
DataSpaceDataSpace

Local AI

In order to enable local AI capabilities an additional server is required with the following specifications:

Minimum (CPU Only):

Section titled “Minimum (CPU Only):”
  • CPU: 64 cores
  • RAM: 256GB
  • Storage: 100GB SSD

Production (GPU):

Section titled “Production (GPU):”
  • CPU: 64 cores
  • RAM: 128GB
  • GPU: 1x H100 80GB
  • Storage: 500GB NVMe SSD

We are leveraging a 120 Billion parameter open source model and therefore greatly benefit from GPU acceleration. While the system is flexible enough to handle any open source model, the smaller models don’t perform as well and often struggle with tool calling.