Bring Your Own LLM to StackAI
Use your own LLMs in StackAI. Set a default connection for your preferred LLM provider. Easily configure your environment for smooth integration, manage API key usage across providers, deactivate specific LLM connections, and access advanced settings for complete control over your AI setup.
This guide will walk you through how to set a default connection for your preferred LLM provider. You’ll also learn how to disable the use of StackAI API keys across providers, deactivate specific LLM providers, and manage other advanced configuration options.
Let’s start!
1. Creating a default connection
First, navigate to the Feature Access tab in Settings to create a new connection to an LLM.
Under the LLMs section, search for Local LLMs—you can follow the same steps for any other provider as well. If needed, this is also where you can disable specific LLM providers to prevent your users from accessing them.
Once you’re in the tab for a specific LLM provider, you’ll see a toggle to enable or disable the provider, as well as a “New Connection” button under Default Connection to define it as your primary connection.
Once you click the button, you can create a connection that will serve as the default for all LLMs from that provider within your workspace. This is especially useful because your users won’t need to manually enter an API key to use those models.
2. Using an LLM with a default connection in the workflow builder
Once the connection is set up, you can create a new project, choose your LLM provider, select the specific model you want to use, and start interacting with it immediately.
You won’t need to manually add the connection—your default connection will appear automatically at the bottom. If you prefer to use a different connection, you can create it directly from that section and select it for your project.
3. Connections manager
All your connections are stored in the Connections Manager tab. You can also create new connections from there, and easily select any of them when adding an LLM node to your project.
You can choose to make your connections either public or private, and you’re free to change the default connection at any time. Simply go to the Access Control tab in your Settings to update this preference.
4. Disconnecting all StackAI’s API keys usage
If you’d like to prevent any unintended usage of LLMs without your API keys, you can disable all StackAI API Keys. This ensures that only LLMs configured to send data to your own servers are allowed. To do this, navigate to Settings > Feature Access > Other > General LLM Configuration.