Skip to main content
If your organization hosts its own models or uses a private AI deployment, you can connect RelayHub to a custom LLM endpoint. RelayHub then sends the AI traffic to your endpoint. The models of the endpoint are available next to the platform models.

​
Endpoint types

Connect models that you deploy through Azure AI Foundry. RelayHub uses the OpenAI-compatible interface of Azure.You need:
  • The endpoint URL, for example https://your-resource.openai.azure.com/
  • An API key
  • The deployment names of your models

​
Add a custom endpoint

1

Open the Advanced tab

In the Admin group of the sidebar, click Providers. Then click the Advanced tab. Only administrators see this screen.
2

Click Add Provider

In the LLM Providers section, click Add Provider.
3

Fill in the form

Complete these fields:
  • Provider Name: a name for this endpoint, for example “Azure Production”.
  • Provider Type: Azure AI Foundry or OpenAI Compatible.
  • Endpoint URL: the full URL of your endpoint.
  • API Key: your credential. RelayHub encrypts it.
  • Set as Default Provider: select this option to send the traffic to this endpoint.
  • Allow Platform Fallback: this option is on by default. Clear it if no request may go to a platform key.
Click Create Provider.
4

Import the models

Expand the provider row. In Configured Models, click Discover Models. RelayHub reads the model list of your endpoint and shows Discovered Models. Select the models, then click Import Selected.
5

Check the status

When the provider is ready, the row shows Live in chat. If a step is missing, the row shows Setup incomplete and a list of what to do.

​
When the provider is ready

The row shows Live in chat only when all three of these are true:
  • The provider is enabled. To enable it, click Edit provider.
  • The provider has one or more imported models.
  • The provider is the default provider. Its models do not appear in chat until it is the default.

​
Keep all traffic on your endpoint

When the default custom provider has Allow Platform Fallback off, RelayHub sends each LLM call to your endpoint. This includes:
  • Chat conversations, in standard chat and in Dual Chat
  • The embeddings for document indexing
  • Background work, for example memory and knowledge extraction
  • Vision and image analysis
  • Utility tasks, for example summaries and classification
With fallback off, when your endpoint stops, LLM calls show an error. RelayHub does not use an external provider in its place, so your data stays on your endpoint.

​
Models

Discover Models reads the model list of your endpoint. If your endpoint does not give a model list, click Add Model and enter the model yourself:
  • Deployment Name: the model name or the deployment name on your endpoint.
  • Display Name: the name that people see in the model picker.
  • Role: the job of the model, for example Chat Default, Embedding, Utility, Vision or Reasoning.
  • Capabilities: what the model can do, for example reasoning effort, tools and images.
To change a model, click Edit model. To delete it, click Delete model.

​
Other provider controls

Each provider row has these controls:
  • Test connection: check that RelayHub can reach the endpoint.
  • Edit provider: change the fields, or enable or disable the provider.
  • Delete provider: remove the provider.
To stop the use of a custom provider for a short time, disable it with Edit provider. The traffic then goes to your own keys or to the platform keys.
Send a few chat messages through a new endpoint before your team starts to use it.