Ctrlk
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Local AI

In order to enable local AI capabilities an additional server is required with the following specifications:

Minimum (CPU Only):

  • CPU: 64 cores

  • RAM: 256GB

  • Storage: 100GB SSD

Production (GPU):

  • CPU: 64 cores

  • RAM: 128GB

  • GPU: 1x H100 80GB

  • Storage: 500GB NVMe SSD

We are leveraging a 120 Billion parameter open source model and therefore greatly benefit from GPU acceleration. While the system is flexible enough to handle any open source model, the smaller models don't perform as well and often struggle with tool calling.

PreviousInstallationNextOAuth

Last updated