Skip to main content
The Custom provider enables you to integrate any LLM API endpoint that isn’t already supported by writing a simple Rhai script. This is perfect for proprietary APIs, custom deployments, or non-standard endpoints.

​
How It Works

The Custom provider uses a Rhai script to translate between Circuit Breaker Labs’ internal message format and your custom API’s request/response schema.
1

CBL sends messages to your script

Circuit Breaker Labs provides conversation history as an array of messages with role and content fields.
2

Your script builds the request

The build_request() function transforms messages into your API’s expected format.
3

CBL posts to your endpoint

The request body is sent as JSON to the URL you specify.
4

Your script parses the response

The parse_response() function extracts the assistant’s message from the API response.

​
Configuration Options

​
--url
string
required
Endpoint URL to POST requests to.Example: https://api.example.com/v1/chat/completions
​
--script
path
required
Path to the Rhai script file that defines request/response translation.Example: --script ./providers/my_custom_api.rhai

​
Writing a Custom Script

Your Rhai script must implement two functions:

​
build_request(messages)

Transforms the conversation history into your API’s request format. Parameters:
  • messages (array): Array of message objects, each with:
    • role (string): Either "system", "user", or "assistant"
    • content (string): The message content
Returns: A map that will be serialized to JSON and sent to your endpoint.

​
parse_response(body)

Extracts the assistant’s response from your API’s response. Parameters:
  • body (dynamic): The full deserialized JSON response from your API
Returns: String containing the assistant’s message content.

​
Example Scripts

​
OpenAI-Compatible API

// For any OpenAI-compatible endpoint
fn build_request(messages) {
    #{
        "model": "gpt-4o",
        "messages": messages
    }
}

fn parse_response(body) {
    body["choices"][0]["message"]["content"].to_string()
}

​
Ollama API

// For Ollama's chat endpoint
fn build_request(messages) {
    #{
        "model": "llama3.2",
        "messages": messages,
        "stream": false
    }
}

fn parse_response(body) {
    body["message"]["content"].to_string()
}

​
Custom API with Authentication

// Example with API key in request body
fn build_request(messages) {
    #{
        "apiKey": "your-api-key",
        "model": "custom-model-v1",
        "prompt": messages,
        "maxTokens": 2000
    }
}

fn parse_response(body) {
    body["result"]["text"].to_string()
}

​
Anthropic Claude API

// For Anthropic's Claude API
fn build_request(messages) {
    // Separate system message from conversation
    let system_msg = "";
    let conversation = [];
    
    for msg in messages {
        if msg.role == "system" {
            system_msg = msg.content;
        } else {
            conversation.push(msg);
        }
    }
    
    #{
        "model": "claude-3-5-sonnet-20241022",
        "max_tokens": 4096,
        "system": system_msg,
        "messages": conversation
    }
}

fn parse_response(body) {
    body["content"][0]["text"].to_string()
}

​
Google Gemini API

// For Google's Gemini API
fn build_request(messages) {
    let contents = [];
    
    for msg in messages {
        if msg.role != "system" {  // Gemini doesn't use system role
            let role = if msg.role == "assistant" { "model" } else { "user" };
            contents.push(#{
                "role": role,
                "parts": [#{ "text": msg.content }]
            });
        }
    }
    
    #{
        "contents": contents
    }
}

fn parse_response(body) {
    body["candidates"][0]["content"]["parts"][0]["text"].to_string()
}

​
Complete Usage Example

cbl single-turn \
    --threshold 0.5 \
    --variations 2 \
    --maximum-iteration-layers 2 \
    custom \
    --url https://api.example.com/v1/chat \
    --script ./my_provider.rhai

​
Rhai Language Basics

Rhai is a simple scripting language with JavaScript-like syntax:

​
Creating Objects (Maps)

let obj = #{
    "key": "value",
    "number": 42,
    "nested": #{
        "inner": "data"
    }
};

​
Accessing Fields

let value = obj["key"];           // Bracket notation
let nested = obj["nested"]["inner"];  // Nested access

​
Arrays

let arr = [1, 2, 3];
arr.push(4);
let first = arr[0];

​
Loops

for item in array {
    print(item);
}

​
String Conversion

let str = value.to_string();  // Convert to string
For complete Rhai documentation, visit rhai.rs/book.

​
Example Scripts Repository

Circuit Breaker Labs provides example scripts in the repository: 
ls examples/providers/
Available examples:
  • openai_completions.rhai - OpenAI chat completions API
  • ollama_chat.rhai - Ollama chat API
  • openai_responses.rhai - OpenAI with response formatting
  • ollama_completions.rhai - Ollama completions endpoint

​
Debugging Your Script

Rhai scripts can use print() and debug() functions for logging: 
fn build_request(messages) {
    print("Building request with " + messages.len() + " messages");
    debug(messages);
    
    let request = #{
        "model": "custom-model",
        "messages": messages
    };
    
    debug(request);
    return request;
}
These will output to the CBL CLI logs during evaluation runs.

​
Authentication

For APIs requiring authentication, you have several options:

​
1. Include in Request Body

fn build_request(messages) {
    #{
        "api_key": "your-key-here",
        "messages": messages
    }
}

​
2. Use HTTP Headers

Set authentication headers using environment variables or CBL configuration. The Custom provider automatically includes headers from the CBL context.

​
3. URL Parameters

// Include key in URL when creating custom provider
// --url https://api.example.com/chat?key=YOUR_KEY
Security: Avoid hardcoding API keys in scripts. Use environment variables or secure configuration management instead.

​
Error Handling

If your script encounters an error, CBL will report it with the error location: 
Error: Script error in build_request: Property not found: 'message'
  at my_provider.rhai:12
Common issues:
  • Accessing non-existent fields: Check the API response structure
  • Type mismatches: Ensure proper type conversions with .to_string()
  • Missing return values: Both functions must return a value

​
Tips

Start with Examples: Copy an existing example script from examples/providers/ and modify it for your API.
Test Your API First: Use curl or Postman to understand your API’s request/response format before writing the script.
Use Debug Logging: Add print() statements to see what data your script is working with.
Response Validation: Ensure parse_response() always returns a non-empty string, or evaluations will fail.

Rhai Language Book

Complete Rhai scripting language documentation

Example Scripts

Ready-to-use provider scripts for common APIs