Azure AI Inference Plus

https://github.com/zpg6/azure-ai-inference-plus
Admin
Enhanced wrapper that makes Azure AI Inference SDK simple and reliable with automatic retry, JSON...
Tokens:1,935
Snippets:13
Trust Score:7.5
License:MIT
Update:1 year ago
Show doc for...
Tokens:
Raw
### Installing Azure AI Inference Plus

Source: https://github.com/zpg6/azure-ai-inference-plus/blob/main/README.md

This command installs the `azure-ai-inference-plus` library using pip, the Python package installer. It is the first step to using the SDK and ensures all necessary dependencies are downloaded and configured. This library supports Python 3.10 and newer versions.

```Bash
pip install azure-ai-inference-plus
```

--------------------------------

### Initializing ChatCompletionsClient with Manual Credentials in Python

Source: https://github.com/zpg6/azure-ai-inference-plus/blob/main/README.md

This example illustrates how to explicitly provide the Azure AI endpoint and API key when initializing the `ChatCompletionsClient` using `AzureKeyCredential`. This method is useful when environment variables are not preferred or available, offering direct control over authentication parameters for the client.

```Python
from azure_ai_inference_plus import ChatCompletionsClient, SystemMessage, UserMessage, AzureKeyCredential

client = ChatCompletionsClient(
    endpoint="https://your-resource.services.ai.azure.com/models",
    credential=AzureKeyCredential("your-api-key")
)
```

--------------------------------

### Performing Chat Completion with Automatic Retries in Python

Source: https://github.com/zpg6/azure-ai-inference-plus/blob/main/README.md

This example demonstrates the built-in automatic retry mechanism of the `azure-ai-inference-plus` library. By default, the client will automatically retry failed requests with exponential backoff, enhancing reliability without requiring any explicit configuration from the user.

```Python
# Automatically retries on failures - just works!
response = client.complete(
    messages=[UserMessage(content="Tell me a joke")],
    model="Phi-4"
)
```

--------------------------------

### Implementing Retry Callbacks for Observability in Python

Source: https://github.com/zpg6/azure-ai-inference-plus/blob/main/README.md

This example demonstrates how to add optional callbacks (`on_chat_retry` and `on_json_retry`) to the `RetryConfig` to gain observability into retry events. These functions are invoked when general chat completion failures or JSON validation failures trigger a retry, allowing for custom logging, monitoring, or debugging of transient issues.

```Python
from azure_ai_inference_plus import RetryConfig

def on_chat_retry(attempt, max_retries, exception, delay):
    print(f"🔄 Chat retry {attempt}/{max_retries}: {type(exception).__name__} - waiting {delay:.1f}s")

def on_json_retry(attempt, max_retries, message):
    print(f"📝 JSON retry {attempt}/{max_retries}: {message}")

# Add callbacks to your retry config
client = ChatCompletionsClient(
    retry_config=RetryConfig(
        max_retries=3,
        on_chat_retry=on_chat_retry,    # Called for general failures
        on_json_retry=on_json_retry     # Called for JSON validation failures
    )
)
```

--------------------------------

### Generating Guaranteed Valid JSON with Reasoning Separation in Python

Source: https://github.com/zpg6/azure-ai-inference-plus/blob/main/README.md

This example shows how to combine `response_format='json_object'` with `reasoning_tags` to ensure the model returns valid JSON while still capturing its internal reasoning. The `content` attribute will contain only the pure JSON, with any thinking process automatically stripped, while the `reasoning` attribute retains the model's thought process.

```Python
response = client.complete(
    messages=[
        SystemMessage(content="You are a helpful assistant that returns JSON."),
        UserMessage(content="Give me Paris info as JSON with keys: name, country, population"),
    ],
    max_tokens=2000,
    model="DeepSeek-R1",
    response_format="json_object",  # ✨ Clean JSON guaranteed
    reasoning_tags=["<think>", "</think>"]
)

# Pure JSON - reasoning automatically stripped
data = response.choices[0].message.content  # {"name": "Paris", ...}

# But reasoning is still accessible
thinking = response.choices[0].message.reasoning  # "Let me think about Paris..."
```

--------------------------------

### Setting Up Azure AI Environment Variables (Bash)

Source: https://github.com/zpg6/azure-ai-inference-plus/blob/main/README.md

This snippet shows the required environment variables (`AZURE_AI_ENDPOINT` and `AZURE_AI_API_KEY`) that need to be set, typically in a `.env` file, for authenticating and connecting to Azure AI services. These variables are automatically picked up by the `azure-ai-inference-plus` library for configuration.

```bash
AZURE_AI_ENDPOINT=https://your-resource.services.ai.azure.com/models
AZURE_AI_API_KEY=your-api-key-here
```

--------------------------------

### Manually Configuring Azure AI Inference Plus Client (Python)

Source: https://github.com/zpg6/azure-ai-inference-plus/blob/main/README.md

This snippet shows how to manually configure the `ChatCompletionsClient` by providing the Azure AI endpoint and API key directly as arguments during client initialization. This method offers an alternative to using environment variables for credential management.

```python
from azure_ai_inference_plus import ChatCompletionsClient, AzureKeyCredential

client = ChatCompletionsClient(
    endpoint="https://your-resource.services.ai.azure.com/models",
    credential=AzureKeyCredential("your-api-key")
)
```

--------------------------------

### Initializing ChatCompletionsClient with Environment Variables in Python

Source: https://github.com/zpg6/azure-ai-inference-plus/blob/main/README.md

This snippet demonstrates how to initialize the `ChatCompletionsClient` without explicit credentials, relying on `AZURE_AI_ENDPOINT` and `AZURE_AI_API_KEY` environment variables. It then shows a basic chat completion request using `SystemMessage` and `UserMessage` to interact with a specified model, printing the AI's response. The `max_tokens` parameter limits the response length.

```Python
from azure_ai_inference_plus import ChatCompletionsClient, SystemMessage, UserMessage

# Uses environment variables: AZURE_AI_ENDPOINT, AZURE_AI_API_KEY
client = ChatCompletionsClient()

response = client.complete(
    messages=[
        SystemMessage(content="You are a helpful assistant."),
        UserMessage(content="What's the capital of France?"),
    ],
    max_tokens=100,
    model="Codestral-2501"
)

print(response.choices[0].message.content)
# "The capital of France is Paris..."
```

--------------------------------

### Migrating Imports to Azure AI Inference Plus (Python)

Source: https://github.com/zpg6/azure-ai-inference-plus/blob/main/README.md

This snippet illustrates the simple import change required when migrating from the `azure.ai.inference` SDK to `azure-ai-inference-plus`. It demonstrates how to update the import statements to use the new library, while ensuring existing code remains compatible with automatic retries and JSON validation.

```python
# Before
from azure.ai.inference import ChatCompletionsClient
from azure.ai.inference.models import SystemMessage, UserMessage
from azure.core.credentials import AzureKeyCredential

# After
from azure_ai_inference_plus import ChatCompletionsClient, SystemMessage, UserMessage, AzureKeyCredential
```

--------------------------------

### Performing Chat Completion with Automatic Reasoning Separation in Python

Source: https://github.com/zpg6/azure-ai-inference-plus/blob/main/README.md

This snippet demonstrates the `reasoning_tags` feature, which automatically separates the model's internal thought process from its final output. By specifying `<think>` tags, the `content` attribute provides the clean answer, while the `reasoning` attribute exposes the step-by-step thinking, particularly useful for models like DeepSeek-R1.

```Python
response = client.complete(
    messages=[
        SystemMessage(content="You are a helpful assistant."),
        UserMessage(content="What's 2+2? Think step by step."),
    ],
    model="DeepSeek-R1",
    reasoning_tags=["<think>", "</think>"]  # ✨ Auto-separation
)

# Clean output without reasoning clutter
print(response.choices[0].message.content)
# "2 + 2 equals 4."

# Access the reasoning separately
print(response.choices[0].message.reasoning)
# "Let me think about this step by step. 2 + 2 is a basic addition..."
```

--------------------------------

### Generating Embeddings with Azure AI Inference Plus (Python)

Source: https://github.com/zpg6/azure-ai-inference-plus/blob/main/README.md

This snippet demonstrates how to initialize the `EmbeddingsClient` from `azure_ai_inference_plus` and use it to generate embeddings for a list of input strings. It specifies the model to be used for the embedding generation, such as 'text-embedding-3-large'.

```python
from azure_ai_inference_plus import EmbeddingsClient

client = EmbeddingsClient()
response = client.embed(
    input=["Hello world", "Python is great"],
    model="text-embedding-3-large"
)
```

--------------------------------

### Customizing Retry Behavior for ChatCompletionsClient in Python

Source: https://github.com/zpg6/azure-ai-inference-plus/blob/main/README.md

This snippet shows how to override the default retry behavior by providing a `RetryConfig` object during client initialization. Users can customize parameters such as `max_retries` (maximum number of attempts) and `delay_seconds` (initial delay between retries) to fine-tune the retry strategy according to specific application needs.

```Python
from azure_ai_inference_plus import RetryConfig

# Override default behavior
client = ChatCompletionsClient(
    retry_config=RetryConfig(max_retries=5, delay_seconds=2.0)
)
```

--------------------------------

### Ensuring Valid JSON Output with Automatic Validation in Python

Source: https://github.com/zpg6/azure-ai-inference-plus/blob/main/README.md

This snippet highlights the `response_format='json_object'` feature, which guarantees that the model's output will always be valid JSON. The library automatically validates the JSON and retries if necessary, eliminating the need for manual error handling or `try/catch` blocks for JSON parsing errors.

```Python
response = client.complete(
    messages=[UserMessage(content="Give me a JSON response")],
    model="Codestral-2501",
    response_format="json_object"  # ✨ Auto-validation + retry
)

# Always valid JSON, no try/catch needed!
data = response.choices[0].message.content
```
Tokens:
Raw