llmclient

https://docs.rs/llmclient/0.3.2
Admin
A unified Rust client library for interacting with multiple LLM providers including Claude, GPT,...
Tokens:72,710
Snippets:1,011
Trust Score:9.1
Update:4 months ago
Show doc for...
Tokens:
Raw
### Gemini Client Setup

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/gemini.rs.html

Function to get a Gemini client, including fetching the API key using `gcloud auth print-access-token` and setting up the authorization header.

```rust
async fn get_gemini_client() -> Result<Client, Box<dyn std::error::Error + Send>> {
// Extract API Key information
let output = Command::new("gcloud")
.arg("auth")
.arg("print-access-token")
.output()
.expect("Failed to execute command");

let api_key: String = String::from_utf8_lossy(&output.stdout).trim().to_string();

// Create headers
let mut headers: HeaderMap = HeaderMap::new();

// Create api key header
headers.insert(
"Authorization",
HeaderValue::from_str(&format!("Bearer {}", api_key))
.map_err(|e| -> Box<dyn std::error::Error + Send> { Box::new(e) })?,
);

get_client(headers).await
}
```

--------------------------------

### Groq Dialogue Example

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/groq.rs.html

Illustrates a multi-turn dialogue with the Groq API, including system prompts and follow-up questions.

```Rust
let system = "Use a Scottish accent to answer questions";
        let mut messages = 
            vec!["How many brains does an octopus have, when they have been injured and lost a leg?".to_string()];
        let res = GroqCompletion::call(&system, &messages, 0.2, false, true).await;
        println!("{res:?}");

        messages.push(res.unwrap().to_string());
        messages.push("Is a cuttle fish similar?".to_string());

        let res = GroqCompletion::call(&system, &messages, 0.2, false, true).await;
        println!("{res:?}");
```

--------------------------------

### Example Function Definitions for GPT

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/gpt.rs.html

These are example function definitions that can be passed to the GPT model to enable function calling. The first defines an 'arithmetic' function, and the second defines an 'apple' function with 'color' and 'taste' parameters.

```rust
528// expr: An arithmetic expression
529fn arithmetic(expr)
530
```

```rust

533// Find the color of an apple and its taste pass them to this function
534// color: The color of an apple
535// taste: The taste of an apple
536fn apple(color, taste)
537
```

--------------------------------

### Gemini Dialogue Example

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/gemini.rs.html

Demonstrates a multi-turn conversation with the Gemini API, including system prompts and response handling.

```rust
    #[tokio::test]
    async fn test_call_gemini_dialogue() {
        let system = "Use a Scottish accent to answer questions";
        let mut messages = vec!["How many brains does an octopus have, when they have been injured and lost a leg?".to_string()];
        let res = GeminiCompletion::call(&system, &messages, 0.2, false, true).await;
        println!("{res:?}");

        messages.push(res.unwrap().to_string());
        messages.push("Is a cuttle fish similar?".to_string());

        let res = GeminiCompletion::call(&system, &messages, 0.2, false, true).await;
        println!("{res:?}");
    }
```

--------------------------------

### Common HTTP Client Setup

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/common.rs.html

Sets up a common HTTP client with JSON content type and accept headers, and a user agent.

```rust
pub async fn get_client(mut headers: HeaderMap) -> Result<Client, Box<dyn std::error::Error + Send>> {
    // We would like json
    headers.insert(
        "Content-Type",
        HeaderValue::from_str("appication/json; charset=utf-8")
            .map_err(|e| -> Box<dyn std::error::Error + Send> { Box::new(e) })?,
    );
    headers.insert(
        "Accept",
        HeaderValue::from_str("appication/json")
            .map_err(|e| -> Box<dyn std::error::Error + Send> { Box::new(e) })?,
    );

    // Create client
    let client: Client = Client::builder()
        .user_agent("TargetR")
        .timeout(std::time::Duration::new(120, 0))
        //.gzip(true)
        .default_headers(headers)
        .build()
        .map_err(|e| -> Box<dyn std::error::Error + Send> { Box::new(e) })?;

    Ok(client)
}
```

--------------------------------

### Gemini with Citations

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/gemini.rs.html

Example of requesting text generation with citations from the Gemini API.

```rust
    #[tokio::test]
    async fn test_call_gemini_citation() {
        let messages = vec![Content::text("user", "Give citations for the General theory of Relativity.")];
        gemini(messages).await;
    }
```

--------------------------------

### Groq Function Call Example

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/groq.rs.html

Demonstrates calling a Groq model with a function definition to perform arithmetic.

```Rust
let model: String = std::env::var("GROQ_MODEL").expect("GROQ_MODEL not found in enviroment variables");
        let messages =  vec!["The answer is (60 * 24) * 365.25".to_string()];
        let func_def =
r#"// Derive the value of the arithmetic expression
// expr: An arithmetic expression
fn arithmetic(expr)
"#;
        let functions = get_function_json("groq", &[func_def]);
        let res = GroqCompletion::call_model_function(&model, "", &messages, 0.2, false, true, functions).await;
        println!("{res:?}");

        let answer = call_actual_function(res.ok());
        println!("{answer:?}");
```

--------------------------------

### Basic Gemini Text Generation

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/gemini.rs.html

Example of calling the Gemini API to generate text based on a user prompt.

```rust
    #[tokio::test]
    async fn test_call_gemini_basic() {
        let messages = vec![Content::text("user", "What is the meaning of life?")];
        gemini(messages).await;
    }
```

--------------------------------

### Groq Test Cases

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/groq.rs.html

This section provides example test cases for the Groq client, demonstrating how to call the API with different prompts and assert the results.

```rust
#[cfg(test)]
mod tests {
    use super::*;

    async fn groq(content: Vec<GroqMessage>) {
        match call_groq(content).await {
            Ok(ret) => { println!("{ret}"); assert!(true) },
            Err(e) => { println!("{e}"); assert!(false) },
        }
    }

    #[tokio::test]
    async fn test_call_groq_basic() {
        let messages = vec![GroqMessage::text("user", "What is the meaning of life?")];
        groq(messages).await;
    }
    #[tokio::test]
    async fn test_call_groq_citation() {
        let messages = 
            vec![GroqMessage::text("user", "Give citations for the General theory of Relativity.")];
        groq(messages).await;
    }
    #[tokio::test]
    async fn test_call_groq_poem() {
        let messages = 

```

--------------------------------

### Gemini for Creative Writing

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/gemini.rs.html

Example of using Gemini for creative writing tasks, such as generating a poem with citations.

```rust
    #[tokio::test]
    async fn test_call_gemini_poem() {
        let messages = vec![Content::text("user", "Write a creative poem about the interplay of artificial intelligence and the human spirit and provide citations")];
        gemini(messages).await;
    }
```

--------------------------------

### call_my_functions Example

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/caller.rs.html

An example implementation of the call_my_functions function, demonstrating how to parse and execute functions based on JSON input. It includes logic for 'arithmetic' and 'apple' functions, with error handling for invalid formulas or function calls.

```rust
use crate::functions::*
use evalexpr::eval;

/**
  This function should be implemented for the users functions.

  This is just an example.
**/
pub fn call_my_functions(js: Result<Vec<ParseFunction>, serde_json::Error>) -> Vec<String> {
    match js {
        Ok(funcs) => 
            funcs.into_iter()
                .map(|f| {
                    if f.function == "arithmetic" && f.arguments.len() == 1 {
                        let desc = eval(&f.arguments[0].desc);

                        if let Ok(v) = desc {
                            format!("{} -> {}", f.function, v)
                        } else {
                            format!("Invalid formula for: {}", f.function)
                        }
                    } else if f.function == "apple" && f.arguments.len() == 2 {
                        format!("You have found an apple({}, {})", f.arguments[0].desc, f.arguments[1].desc)
                    } else {
                        format!("No function found : {} {:?}", f.function, f.arguments)
                    }
                }
                )
                .collect(),
        Err(e) => 
            vec![format!("{e:?}")]
    }
}

```

--------------------------------

### Gemini for Factual Questions

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/gemini.rs.html

Example of asking a factual question to the Gemini API.

```rust
    #[tokio::test]
    async fn test_call_gemini_logic() {
        let messages = vec![Content::text("user", "How many brains does an octopus have, when they have been injured and lost a leg?")];
        gemini(messages).await;
    }
```

--------------------------------

### Groq Common Function Call Example

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/groq.rs.html

Illustrates calling a Groq model with multiple function definitions for different purposes, like arithmetic and apple properties.

```Rust
let messages = vec!["a fruit that is blue with a sour tast".to_string()];
        let func_def =
r#"// Derive the value of the arithmetic expression
// expr: An arithmetic expression
fn arithmetic(expr)
"#;
        let func_def2 =
r#"// Find the color of an apple and it's taste pass them to this function.
// color: The color of an apple
// taste: The taste of an apple
fn apple(color, taste)
"#;
        let res = call_function_llm("groq", &messages, &[func_def, func_def2]).await;
        println!("{res:?}");

        let answer = call_actual_function(res.ok());
        println!("{answer:?}");
```

--------------------------------

### Common Function Calling with Gemini

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/gemini.rs.html

Example of using a common function calling mechanism with Gemini, including defining function signatures.

```rust
    #[tokio::test]
    async fn test_call_function_common_gemini() {
        let messages =  vec!["The answer is (60 * 24) * 365.25".to_string()];
        let func_def =
        r#"
// Recognize and derive the value of an arithmetic expression
// expr: An arithmetic expression
fn arithmetic(expr)
"#;
        let res = call_function_llm("gemini", &messages, &[func_def]).await;
        println!("{res:?}");

        let answer = call_actual_function(res.ok());
        println!("{answer:?}");
    }
```

--------------------------------

### Gemini with Specific Model

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/gemini.rs.html

Example of calling the Gemini API with a specific model specified via an environment variable.

```rust
    #[tokio::test]
    async fn test_call_gemini_dialogue_model() {
        let model: String = std::env::var("GEMINI_MODEL").expect("GEMINI_MODEL not found in enviroment variables");
        let messages = vec!["Hello".to_string()];
        let res = GeminiCompletion::call_model(&model, "", &messages, 0.2, false, true).await;
        println!("{res:?}");
    }
```

--------------------------------

### Groq Client Initialization

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/groq.rs.html

This snippet shows how to create a Groq client by retrieving the API key from environment variables and setting up the necessary authorization headers.

```rust
async fn get_groq_client() -> Result<Client, Box<dyn std::error::Error + Send>> {
    // Extract API Key information
    let api_key: String =
        env::var("GROQ_API_KEY").expect("GROQ_API_KEY not found in enviroment variables");

    // Create headers
    let mut headers: HeaderMap = HeaderMap::new();

    // Create api key header
    headers.insert(
        "Authorization",
        HeaderValue::from_str(&format!("Bearer {}", api_key))
            .map_err(|e| -> Box<dyn std::error::Error + Send> { Box::new(e) })?,
    );

    get_client(headers).await
}
```

--------------------------------

### Any::type_id

Source: https://docs.rs/llmclient/0.3.2/llmclient/functions/struct.Properties.html

Gets the TypeId of the Properties value.

```rust
fn type_id(&self) -> TypeId
```

--------------------------------

### GPT Client Initialization

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/gpt.rs.html

This snippet demonstrates how to initialize the GPT client by retrieving the API key from environment variables and setting up the necessary headers for authentication.

```rust
pub async fn get_gpt_client() -> Result<Client, Box<dyn std::error::Error + Send>> {
    // Extract API Key information
    let api_key: String =
        env::var("OPENAI_API_KEY").expect("OPENAI_API_KEY not found in enviroment variables");

    // Create headers
    let mut headers: HeaderMap = HeaderMap::new();

    // Create api key header
    headers.insert(
        "Authorization",
        HeaderValue::from_str(&format!("Bearer {}", api_key))
            .map_err(|e| -> Box<dyn std::error::Error + Send> { Box::new(e) })?,
    );

    get_client(headers).await
}

```

--------------------------------

### Deepseek Client Initialization

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/deepseek.rs.html

This function demonstrates how to initialize the Deepseek client by retrieving the API key from environment variables and setting up the necessary authorization headers.

```rust
async fn get_deepseek_client() -> Result<Client, Box<dyn std::error::Error + Send>> {
    // Extract API Key information
    let api_key:
        String = env::var("DEEPSEEK_API_KEY").expect("DEEPSEEK_API_KEY not found in enviroment variables");

    // Create headers
    let mut headers: HeaderMap = HeaderMap::new();

    // Create api key header
    headers.insert(
        "Authorization",
        HeaderValue::from_str(&format!("Bearer {}", api_key))
            .map_err(|e| -> Box<dyn std::error::Error + Send> { Box::new(e) })?,
    );

    get_client(headers).await
}
```

--------------------------------

### Get Functions

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/functions.rs.html

Recursively extracts function and argument data from a JSON Value based on a path.

```rust
pub fn get_functions(val: &Value, found: &Vec<String>) -> HashMap<String, Vec<String>> {
    fn getter(val: &Value, places: &str, found: &mut HashMap<String, Vec<String>>) -> HashMap<String, Vec<String>> {
        let mut v = val;
        let items: Vec<&str> = places.split(':').collect();
        let var = *items.last().unwrap();

        for (pos, i) in items.iter().enumerate() {
            if *i == var {
                if let Value::String(it) = v {
                    let key = &i[2..var.len()-1];

                    found.entry(key.to_string())
                        .and_modify(|v| v.push(it.to_string()))
                        .or_insert(vec![it.to_string()]);
                } else if let Value::Number(it) = v {
                    let key = &i[2..var.len()-1];

                    found.entry(key.to_string())
                        .and_modify(|v| v.push(it.to_string()))
                        .or_insert(vec![it.to_string()]);
                } else if let Value::Object(it) = v {
                    let key = &i[2..var.len()-1];


```

--------------------------------

### Mistral Client Initialization

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/mistral.rs.html

Function to create and configure the Mistral API client, including setting up authorization headers.

```rust
async fn get_mistral_client() -> Result<Client, Box<dyn std::error::Error + Send>> {
    // Extract API Key information
    let api_key:
        String = env::var("MISTRAL_API_KEY").expect("MISTRAL_API_KEY not found in enviroment variables");

    // Create headers
    let mut headers: HeaderMap = HeaderMap::new();

    // Create api key header
    headers.insert(
        "Authorization",
        HeaderValue::from_str(&format!("Bearer {}", api_key))
            .map_err(|e| -> Box<dyn std::error::Error + Send> { Box::new(e) })?,
    );

    get_client(headers).await
}
```

--------------------------------

### Groq Call with Specific Question

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/groq.rs.html

Demonstrates a Groq API call with a specific question about octopuses.

```Rust
let messages = 
            vec![GroqMessage::text("user", "How many brains does an octopus have, when they have been injured and lost a leg?")];
groq(messages).await;
```

--------------------------------

### Get Model Name from Environment Variable

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/common.rs.html

Retrieves the specific model name for a given LLM provider from environment variables.

```rust
fn get_model(llm: &str) -> String {
    let model = 
        match llm {
            "google" | "gemini" => {
                    std::env::var("GEMINI_MODEL")
            },
            "openai" | "gpt" => {
                    std::env::var("GPT_MODEL")
            },
            "mistral" => {
                    std::env::var("MISTRAL_MODEL")
            },
            "anthropic" | "claude" => {
                    std::env::var("CLAUDE_MODEL")
            },
            _ => {
                    std::env::var("GROQ_MODEL")
            },
        };

    model.expect("{llm} MODEL not found in enviroment variables")
}
```

--------------------------------

### Get Claude Client

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/claude.rs.html

This function retrieves the Claude client, setting up necessary headers with API key and version information from environment variables.

```rust
async fn get_claude_client() -> Result<Client, Box<dyn std::error::Error + Send>> {
    // Extract API Key information
    let api_key: String =
        env::var("ANTHROPIC_API_KEY").expect("ANTHROPIC_API_KEY not found in environment variables");
    // Date when version was available
    let version: String =
        env::var("CLAUDE_VERSION").expect("CLAUDE_VERSION not found in environment variables");

    // Create headers
    let mut headers: HeaderMap = HeaderMap::new();

    // Create api key header
    headers.insert(
        "x-api-key",
        HeaderValue::from_str(&api_key)
            .map_err(|e| -> Box<dyn std::error::Error + Send> { Box::new(e) })?,
    );
    // Create api key header
    headers.insert(
        "anthropic-version",
        HeaderValue::from_str(&version)
            .map_err(|e| -> Box<dyn std::error::Error + Send> { Box::new(e) })?,
    );

    get_client(headers).await
}
```

--------------------------------

### Basic Groq Call

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/groq.rs.html

Demonstrates a basic call to the Groq API with a user message.

```Rust
vec![GroqMessage::text("user", "Write a creative poem about the interplay of artificial intelligence and the human spirit and provide citations")];
groq(messages).await;
```

--------------------------------

### Call Deepseek Completion

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/deepseek.rs.html

Demonstrates calling the Deepseek completion API with a system message and a list of user messages, then continuing the conversation.

```rust
let res = DeepseekCompletion::call(&system, &messages, 0.2, false, true).await;
println!("{res:?}");

messages.push(res.unwrap().to_string());
messages.push("Is a cuttle fish similar?".to_string());

let res = DeepseekCompletion::call(&system, &messages, 0.2, false, true).await;
println!("{res:?}");
```

--------------------------------

### GPT Client Implementation

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/gpt.rs.html

This snippet shows the core logic for calling the GPT API, processing the response, and extracting relevant information like text, finish reason, and usage statistics.

```rust
            let text: String = 
                match res.choices {
                    Some(ref choices) if !choices.is_empty() => {
                        // For now they only return one choice!
                        let text = choices[0].message.content.clone();
                        let text = text.lines().filter(|l| !l.starts_with("`")).fold(String::new(), |s, l| s + l + "\n");

                        text
                    },
                    Some(_) | None => {
                        "None".into()
                    }
                };
            let finish_reason: String = 
                match res.choices {
                    Some(ref choices) if !choices.is_empty() => {
                        // For now they only return one choice!
                        choices[0].finish_reason.to_string().to_uppercase()
                    },
                    Some(_) | None => {
                        "None".into()
                    }
                };
            let usage: Triple = res.usage.to_triple();

            Ok(LlmReturn::new(LlmType::GPT, text, finish_reason, usage, timing, None, None))
        }
    }
}
```

--------------------------------

### Groq Dialogue with Model Specification

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/groq.rs.html

Shows how to specify a model for a Groq dialogue and retrieve it from environment variables.

```Rust
let model: String = std::env::var("GROQ_MODEL").expect("GROQ_MODEL not found in enviroment variables");
        let messages = vec!["Hello".to_string()];
        let res = GroqCompletion::call_model(&model, "", &messages, 0.2, false, true).await;
        println!("{res:?}");
```

--------------------------------

### Test Call Claude Dialogue

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/claude.rs.html

This test function illustrates a dialogue with the Claude API, starting with a system prompt to use a Scottish accent and then engaging in a multi-turn conversation about octopuses and cuttlefish.

```rust
#[tokio::test]
    #[serial]
    async fn test_call_claude_dialogue() {
        let system = "Use a Scottish accent to answer questions";
        let mut messages = 
            vec!["How many brains does an octopus have, when they have been injured and lost a leg?".to_string()];
        let res = ClaudeCompletion::call(&system, &messages, 0.2, false, true).await;
        println!("{res:?}");

        messages.push(res.unwrap().to_string());
        messages.push("Is a cuttle fish similar?".to_string());

        let res = ClaudeCompletion::call(&system, &messages, 0.2, false, true).await;
        println!("{res:?}");
    }
```

--------------------------------

### Test Call Function Common Claude

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/claude.rs.html

This test function demonstrates calling a common function with the Claude API, using a predefined arithmetic expression. It also includes a commented-out example for a function that is not yet supported by Claude.

```rust
#[tokio::test]
    #[serial]
    async fn test_call_function_common_claude() {
        let messages =  vec!["The answer is (60 * 24) * 365.25".to_string()];
        //let messages = vec!["a fruit that is red with a sweet taste".to_string()];
        let func_def =
r#"// Derive the value of the arithmetic expression
// expr: An arithmetic expression
fn arithmetic(expr)
"#;
        // This does not work in Claude yet
/*
        let func_def2 =
r#"// Find the color of an apple and its taste pass them to this function
// color: The color of an apple
// taste: The taste of an apple
fn apple(color, taste)
"#;
*/
        let res = call_function_llm("claude", &messages, &[func_def]).await;
        println!("{res:?}");

        let answer = call_actual_function(res.ok());
        println!("{answer:?}");
    }
```

--------------------------------

### Call GPT with all options

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/gpt.rs.html

Provides a comprehensive way to call the GPT API, allowing customization of messages, temperature, and JSON output.

```rust
pub async fn call_gpt_all(messages: Vec<GptMessage>, temperature: f32, is_json: bool) -> Result<LlmReturn, Box<dyn std::error::Error + Send>> {
    // Create chat completion
    let gpt_completion = GptCompletion::new(messages, temperature, is_json);

    call_gpt_completion(&gpt_completion).await
}
```

--------------------------------

### Groq API Response Handling

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/groq.rs.html

This snippet demonstrates how to handle responses from the Groq API, including parsing JSON, checking for errors, and extracting relevant information like text content, finish reason, and usage statistics. It also shows how to handle tool calls.

```rust
async fn call_groq(messages: Vec<GroqMessage>) -> Result<LlmReturn, Box<dyn std::error::Error + Send>> {
    let start = std::time::Instant::now();
    let client = get_groq_client().await?;

    let body = GroqRequest {
        messages,
        model: "llama3-8b-8192".to_string(),
        temperature: 0.1,
        max_tokens: 1024,
        top_p: 1.0,
        stop: None,
    };

    let res = client
        .post("https://api.groq.com/openai/v1/chat/completions")
        .json(&body)
        .send()
        .await
        .map_err(|e| -> Box<dyn std::error::Error + Send> { Box::new(e) })?;

    let res = res
        .map_err(|e| -> Box<dyn std::error::Error + Send> { Box::new(e) })?;
        //.json()
        .text()
        .await
        .map_err(|e| -> Box<dyn std::error::Error + Send> { Box::new(e) })?;

    let timing = start.elapsed().as_secs() as f64 + start.elapsed().subsec_millis() as f64 / 1000.0;

//println!("{res}");
    if res.contains("\"error:\"") {
        let ret: Result<LlmError,_> = serde_json::from_str(&res);

        match ret {
            Ok(res) => 
                Ok(LlmReturn::new(LlmType::GROQ_ERROR, res.error.to_string(), res.error.to_string(), (0, 0, 0), timing, None, None)),
            Err(e) => {
                eprintln!("Error: {:?}", res);

                Ok(LlmReturn::new(LlmType::GROQ_ERROR, e.to_string(), e.to_string(), (0, 0, 0), timing, None, None))
            }
        }
    } else if res.contains("\"error\"") {
        Ok(LlmReturn::new(LlmType::GROQ_ERROR, res.to_string(), res.to_string(), (0, 0, 0), timing, None, None))
    } else if res.contains("\"arguments\":") {
        let found = vec!["choices:message:tool_calls:function:arguments:${args}".to_string(),
            "choices:message:tool_calls:function:name:${func}".to_string(),
            "usage:prompt_tokens:${in}".to_string(),
            "usage:completion_tokens:${out}".to_string(),
            "usage:total_tokens:${total}".to_string(),
//            "usage:${usage}".to_string(),
            "choices:finish_reason:${finish}".to_string()];
        let f: serde_json::Value = serde_json::from_str(&res).unwrap();
        let h = get_functions(&f, &found);
        let funcs = unpack_functions(h.clone());
        let function_calls = serde_json::to_string(&funcs).unwrap();
        let (i, o, t) = (h.get("in").unwrap()[0].clone(), h.get("out").unwrap()[0].clone(), h.get("total").unwrap()[0].clone());
        let triple = (i.parse::<usize>().unwrap(), o.parse::<usize>().unwrap(), t.parse::<usize>().unwrap());
        let finish = h.get("finish").unwrap()[0].clone();

        Ok(LlmReturn::new(LlmType::GROQ_TOOLS, function_calls, finish, triple, timing, None, None))
    } else {
        let res: GroqResponse = serde_json::from_str::<GroqResponse>(&res).unwrap();

        // Send Response
        let text: String =
            match res.choices {
                Some(ref choices) if !choices.is_empty() => {
                    // For now they only return one choice!
                    let text = choices[0].message.content.clone();
                    let text = text.lines().filter(|l| !l.starts_with("`")).fold(String::new(), |s, l| s + l + "\n");

                    text
                },
                Some(_) | None => {
                    "None".into()
                }
            };
        let finish_reason: String = 
            match res.choices {
                Some(ref choices) if !choices.is_empty() => {
                    // For now they only return one choice!
                    choices[0].finish_reason.to_string().to_uppercase()
                },
                Some(_) | None => {
                    "None".into()
                }
            };
        let usage: Triple = res.usage.to_triple();

        Ok(LlmReturn::new(LlmType::GROQ, text, finish_reason, usage, timing, None, None))
    }
}
```

--------------------------------

### Usage Struct and Implementations

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/groq.rs.html

Represents token usage in Groq API responses, with methods for initialization and display.

```rust
#[derive(Debug, Deserialize, Clone)]
pub struct Usage {
    pub prompt_tokens: usize,
    pub completion_tokens: usize,
    pub total_tokens: usize,
}

impl Usage {
    pub fn new() -> Self {
        Usage { prompt_tokens: 0, completion_tokens: 0, total_tokens: 0 }
    }

    pub fn to_triple(&self) -> (usize, usize, usize) {
        (self.prompt_tokens, self.completion_tokens, self.total_tokens)
    }
}

impl std::fmt::Display for Usage {
    fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
        write!(f, "{} + {} = {}", self.prompt_tokens, self.completion_tokens, self.total_tokens)
    }
}

impl Default for Usage {
    fn default() -> Self {
        Self::new()
    }
}
```

--------------------------------

### PolicyExt::or

Source: https://docs.rs/llmclient/0.3.2/llmclient/functions/struct.Properties.html

Creates a new Policy that returns Action::Follow if either policy returns Action::Follow.

```rust
fn or<P, B, E>(self, other: P) -> Or<T, P>
where T: Policy<B, E>, P: Policy<B, E>
```

--------------------------------

### GPT Function Calling Test

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/gpt.rs.html

Demonstrates how to use GPT for function calling by defining a function signature and passing it to the API.

```rust
    #[tokio::test]
    async fn test_call_function_gpt() {
        let model: String = std::env::var("GPT_MODEL").expect("GPT_MODEL not found in enviroment variables");
        let messages =  vec!["The answer is (60 * 24) * 365.25".to_string()];
        let func_def =
r#"
// Derive the value of the arithmetic expression
// expr: An arithmetic expression
fn arithmetic(expr)
"#;
        let functions = get_function_json("gpt", &[func_def]);
        let res = GptCompletion::call_model_function(&model, "", &messages, 0.2, false, true, functions).await;
        println!("{res:?}");

        let answer = call_actual_function(res.ok());
        println!("{answer:?}");
    }

```

--------------------------------

### GPT Call Test with Citations

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/gpt.rs.html

A test case to prompt the GPT API for citations related to a specific topic.

```rust
    #[tokio::test]
    async fn test_call_gpt_citation() {
        let messages = 
            vec![GptMessage::text("user", "Give citations for the General theory of Relativity.")];
        gpt(messages).await;
    }

```

--------------------------------

### Helper functions for calling Groq API

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/groq.rs.html

Provides various convenience functions to call the Groq API with different parameter combinations.

```rust
/// Call GROQ with some messages
pub async fn call_groq(messages: Vec<GroqMessage>) -> Result<LlmReturn, Box<dyn std::error::Error + Send>> {
    call_groq_all(messages, 0.2, false).await
}

/// Call GROQ with some messages and option for Json
pub async fn call_groq_json(messages: Vec<GroqMessage>, is_json: bool) -> Result<LlmReturn, Box<dyn std::error::Error + Send>> {
    call_groq_all(messages, 0.2, is_json).await
}

/// Call GROQ with some messages and temperature
pub async fn call_groq_temperature(messages: Vec<GroqMessage>, temperature: f32) -> Result<LlmReturn, Box<dyn std::error::Error + Send>> {
    call_groq_all(messages, temperature, false).await
}

/// Call GROQ with some messages, option for Json and temperature
pub async fn call_groq_all(messages: Vec<GroqMessage>, temperature: f32, is_json: bool) -> Result<LlmReturn, Box<dyn std::error::Error + Send>> {
    // Create chat completion
    let groq_completion = GroqCompletion::new(messages, temperature, is_json);

    call_groq_completion(&groq_completion).await
}
```

--------------------------------

### Dialogue Formatting

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/gemini.rs.html

Formats a slice of strings into a dialogue with alternating user and model roles.

```rust
    /// Supply multi-String content with user and model alternating
    fn dialogue(prompts: &[String], _has_system: bool) -> Vec<Self> {
        prompts.iter()
            .enumerate()
            .map(|(i, p)| {
                let role = if i % 2 == 0 { "user" } else { "model" };

                Self::text(role, p)
            })
            .collect()
    }
```

--------------------------------

### From::from

Source: https://docs.rs/llmclient/0.3.2/llmclient/functions/struct.Properties.html

Returns the argument unchanged.

```rust
fn from(t: T) -> T
```

--------------------------------

### or Function

Source: https://docs.rs/llmclient/0.3.2/llmclient/gemini/struct.SafetySettings.html

Creates a new Policy that returns Action::Follow if either self or other returns Action::Follow.

```rust
fn or<P, B, E>(self, other: P) -> Or<T, P>
where T: Policy<B, E>, P: Policy<B, E>,
```

--------------------------------

### Function Struct and Implementation

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/functions.rs.html

Defines the core structure for a function, including its name, description, and optional parameters or input schema, with a constructor that adapts based on the LLM type.

```Rust
#[derive(Debug, Serialize, Deserialize, Clone)]
pub struct Function {
    pub name: String,
    pub description: String,
    #[serde(skip_serializing_if = "Option::is_none")]
    pub parameters: Option<Parameters>,
    #[serde(skip_serializing_if = "Option::is_none")]
    pub input_schema: Option<Parameters>,
}

impl Function {
    pub fn new(name: &str, description: &str, parameters: Parameters, is_gpt: bool) -> Self {
        Function {
            name: name.to_string(),
            description: description.to_string(),
            parameters: if is_gpt { Some(parameters.clone()) } else { None },
            input_schema: if !is_gpt { Some(parameters.clone()) } else { None },
        }
    }
```

--------------------------------

### GPT Client Functions

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/gpt.rs.html

This snippet shows the core functions for interacting with the GPT model, including calling the model directly and with function support.

```Rust
async fn call_model(model: &str, system: &str, user: &[String], temperature: f32, is_json: bool, is_chat: bool) -> Result<LlmReturn, Box<dyn std::error::Error + Send>> {
        Self::call_model_function(model, system, user, temperature, is_json, is_chat, None).await
    }

    /// Create and call llm with model/function by supplying data and common parameters
   async fn call_model_function(model: &str, system: &str, user: &[String], temperature: f32, is_json: bool, is_chat: bool, function: Option<Vec<Function>>) -> Result<LlmReturn, Box<dyn std::error::Error + Send>> {
        let mut messages = Vec::new();

        if !system.is_empty() {
            messages.push(GptMessage { role: "system".into(), content: system.into() });
        }

        user.iter()
            .enumerate()
            .for_each(|(i, c)| {
                let role = if !is_chat || i % 2 == 0 { "user" } else { "assistant" };

                messages.push(GptMessage { role: role.into(), content: c.to_string() });
            });

//println!("{:?}", function);
        let completion = GptCompletion {
            model: model.into(),
            tools: Some(FunctionCall::functions(function)),
            messages,
            temperature,
            response_format: ResponseFormat::new(is_json)
        };
//println!("-- {{:?}}", serde_json::to_string(&completion));

        call_gpt_completion(&completion).await
    }
```

--------------------------------

### Call Function with Deepseek

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/deepseek.rs.html

Illustrates calling a Deepseek model with a function definition to perform calculations.

```rust
let model: String = std::env::var("DEEPSEEK_MODEL").expect("DEEPSEEK_MODEL not found in enviroment variables");
let messages =  vec!["The answer is (60 * 24) * 365.25".to_string()];
let func_def =
r#"// Derive the value of the arithmetic expression
// expr: An arithmetic expression
fn arithmetic(expr)
"#;
let functions = get_function_json("deepseek", &[func_def]);
let res = DeepseekCompletion::call_model_function(&model, "", &messages, 0.2, false, true, functions).await;
println!("{res:?}");

let answer = call_actual_function(res.ok());
println!("{answer:?}");
```

--------------------------------

### Gemini Function Calling

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/gemini.rs.html

Demonstrates how to use Gemini for function calling, where the model can suggest calling a defined function.

```rust
    #[tokio::test]
    async fn test_call_function_gemini() {
        let model: String = std::env::var("GEMINI_MODEL").expect("GEMINI_MODEL not found in enviroment variables");
        let messages =  vec!["The answer is (60 * 24) * 365.25".to_string()];
        let func_def =
r#"
// Derive the value of the arithmetic expression
// expr: An arithmetic expression
fn arithmetic(expr)
"#;
        let functions = get_function_json("gemini", &[func_def]);
        let res = GeminiCompletion::call_model_function(&model, "", &messages, 0.2, false, true, functions).await;
        println!("{res:?}");

        let answer = call_actual_function(res.ok());
        println!("{answer:?}");
    }
```

--------------------------------

### PEG Parser Grammar for LLM Functions

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/functions.rs.html

Defines a PEG grammar for parsing LLM function definitions, including comments and arguments.

```rust
peg::parser!( grammar llmfunc() for str {
    pub rule func(func: &str, all_args: &str, mand_args: &str) -> String
        = "\n"* fc:func_comment()+ ac:arg_comment()+ "fn"? _ f:ident() _ "(" a:arg_ident() ** comma() ")" _ "\n"* _ {
            let cnt = ac.iter().enumerate()
                .filter(|(i, arg)| {
                    arg.starts_with(a[*i]) || format!("*{{arg}}").starts_with(a[*i])
                }).count();
            if ac.len() == a.len() && a.len() == cnt {
                let ma: Vec<&str> = a.iter()
                    .filter(|a| !a.starts_with('*'))
                    .map(|a| &a[..])
                    .collect();
                let a: Vec<&str> = a.iter()
                    .map(|a| if let Some(stripped) = a.strip_prefix('*') { stripped } else { a })
                    .collect();
                let mut h: HashMap<&str, String> = HashMap::new();

                h.insert("func", f.to_string());
                h.insert("func_desc", fc[0].to_string());
                h.insert("arg", a.join("|"));
                h.insert("arg_desc", ac.join("|"));
                h.insert("marg", ma.join("|"));
                h.insert("all_args", all_args.to_string());
                h.insert("mand_args", mand_args.to_string());
                h.insert("func_call", func.to_string());

                Template::new("${{func_call}}").render(&h)
            } else {
                "Error: Argument names do not match".to_string()
            }
        }

    rule _ = [' ']*

    rule comma() = "," " "* 

    rule ident() -> &'input str
        = s:$(['a'..='z'|'A'..='Z'|'0'..='9'|'_']+) { s }

    rule arg_ident() -> &'input str
        = s:$("*"? ident()) { s }

    rule func_comment() -> &'input str
        = _ "/"*<2,3> _ s:$(comment()) _ "\n"+ { s }

    rule arg_comment() -> &'input str
        = _ "/"*<2,3> _ a:$(ident() ":" comment()) _ "\n"+ { a }

    rule comment() -> &'input str
        = s:$([';'..='`'|'a'..='~'|'_'|' '..='9']+) { s }
});
```

--------------------------------

### Basic GPT Call Test

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/gpt.rs.html

A basic test case to call the GPT API with a simple user message.

```rust
    #[tokio::test]
    async fn test_call_gpt_basic() {
        let messages = vec![GptMessage::text("user", "What is the meaning of life?")];
        gpt(messages).await;
    }

```

--------------------------------

### Parameters Struct and Implementation

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/functions.rs.html

Defines the structure for function parameters, including type, properties, and required fields, with a constructor.

```Rust
#[derive(Debug, Serialize, Deserialize, Clone)]
pub struct Parameters {
    pub r#type: String,
    pub properties: Properties,
    pub required: Vec<String>
}

impl Parameters {
    pub fn new(ptype: &str, properties: Properties, required: Vec<String>) -> Self {
        Parameters {
            r#type: ptype.to_string(),
            properties,
            required
        }
    }
```

--------------------------------

### PolicyExt::and

Source: https://docs.rs/llmclient/0.3.2/llmclient/functions/struct.Properties.html

Creates a new Policy that returns Action::Follow only if both policies return Action::Follow.

```rust
fn and<P, B, E>(self, other: P) -> And<T, P>
where T: Policy<B, E>, P: Policy<B, E>
```

--------------------------------

### Usage Structure and Methods

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/deepseek.rs.html

Represents token usage statistics for Deepseek API calls, with methods for initialization, conversion to a tuple, and formatted display.

```rust
#[derive(Debug, Deserialize, Clone)]
pub struct Usage {
    pub prompt_tokens: usize,
    pub completion_tokens: usize,
    pub total_tokens: usize,
}

impl Usage {
    pub fn new() -> Self {
        Usage { prompt_tokens: 0, completion_tokens: 0, total_tokens: 0 }
    }

    pub fn to_triple(&self) -> (usize, usize, usize) {
        (self.prompt_tokens, self.completion_tokens, self.total_tokens)
    }
}

impl Default for Usage {
    fn default() -> Self {
        Self::new()
    }
}

impl std::fmt::Display for Usage {
    fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
        write!(f, "{} + {} = {}", self.prompt_tokens, self.completion_tokens, self.total_tokens)
    }
}
```

--------------------------------

### Unpack Functions

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/functions.rs.html

Parses a HashMap to extract and structure function definitions and their arguments.

```rust
pub fn unpack_functions(h: HashMap<String, Vec<String>>) -> Option<Vec<ParseFunction>> {
    let func = h.get("func");
    let args = h.get("args");

    if let Some(func) = func {
        if let Some(args) = args {
            let funcs = func.iter().zip(args.iter())
                .map(|(f, a)| {
                    // Claude has expression wrapped in String(..)
                    let a = {
                        let re = Regex::new(r"(.*)String\((.*)\)(.*)").unwrap();
                        match re.captures(a) {
                            Some(bits) => format!("{}{}{}", &bits[1], &bits[2], &bits[3]),
                            None => a.to_string()
                        }
                    };
//println!("{f}: {a} - {}", a.contains("String"));
                    if a.starts_with('{') && a.ends_with('}') {
                        let fh: Result<HashMap<String, String>, _> = serde_json::from_str(&a);
                        if let Ok(fh) = fh {
                            let args: Vec<ParseArgument> = fh.iter()
                                .map(|(pn, pv)| ParseArgument::new(pn, pv))
                                .collect();

                            ParseFunction::new(f, args)
                        } else {
                            ParseFunction::new(f, vec![])
                        }
                    } else {
                        ParseFunction::new(f, vec![])
                    }
                })
                .collect();

            return Some(funcs);
        }
    }

    None
}

```

--------------------------------

### Claude Tests

Source: https://docs.rs/llmclient/0.3.2/src/llmclient/claude.rs.html

Unit tests for the Claude client, including basic calls and calls with citations.

```rust
mod tests {
    use super::*;
    use serial_test::serial;

    async fn claude(content: Vec<ClaudeMessage>) {
        match call_claude(content).await {
            Ok(ret) => { println!("{ret}"); assert!(true) },
            Err(e) => { println!("{e}"); assert!(false) },
        }
    }

    #[tokio::test]
    #[serial]
    async fn test_call_claude_basic() {
        let messages: Vec<ClaudeMessage> = vec![ClaudeMessage { role: "user".into(), content: "What is the meaning of life?".into() }];

        claude(messages).await;
    }
    #[tokio::test]
    #[serial]
    async fn test_call_claude_citation() {
        let messages = 
            vec![ClaudeMessage::text("user", "Give citations for the General theory of Relativity.")];
        claude(messages).await;
    }
}
```
Tokens:
Raw