DocsGetting StartedIkigenia PlatformLearning HubArcaniaGuidesUsing KoboldCPP

Using KoboldCpp

Learn how to set up and configure KoboldCpp as a local AI backend for Arcania

You can find the full KoboldCpp documentation here.

Installation Steps

  1. Clone the repo:

    git clone https://github.com/LostRuins/koboldcpp
cd koboldcpp

  2. Download the model: For example, we will use OpenChat 3.5 model, which is what is used on the demo instance. There are many models to choose from.

    Navigate to TheBloke/openchat_3.5-GGUF and download one of the models, such as openchat_3.5.Q5_K_M.gguf. Place this file inside the ./models directory.

  3. Build KoboldCpp:

    make

  4. Run the server:

    ./koboldcpp.py ./models/openchat_3.5.Q5_K_M.gguf

Configuration

  1. Select KoboldCpp as the backend:

    settings -> ChatBot -> ChatBot Backend -> KoboldCpp

  2. Configure KoboldCpp:

    settings -> ChatBot -> KoboldCpp

  3. Enable extra features: Inside of "Use KoboldCpp" ensure that "Use Extra" is enabled. This will allow you to use the extra features of KoboldCpp, such as streaming.

Using ElevenLabsUsing LlamaCPP