### OpenAI Batch Creation and Execution Example Source: https://github.com/parasail-ai/openai-batch/blob/main/README.md This example shows how to set your OpenAI API key, generate an input file for batch processing using example prompts, create a batch with a specified model (e.g., gpt-4o-mini), and then run the batch. The batch can be resumed using its ID. ```bash export OPENAI_API_KEY="" # Create an example batch input file python -m openai_batch.example_prompts | \ python -m openai_batch.create_batch --model 'gpt-4o-mini' > input.jsonl # Run this batch (resumable with `--resume `) python -m openai_batch.run input.jsonl ``` -------------------------------- ### Parasail Batch Creation and Execution Example Source: https://github.com/parasail-ai/openai-batch/blob/main/README.md This example demonstrates setting your Parasail API key, generating an input file for batch processing, creating a batch with a specified model (e.g., meta-llama/Meta-Llama-3-8B-Instruct), and then running the batch using the Parasail provider. The batch can be resumed using its ID. ```bash export PARASAIL_API_KEY="" # Create an example batch input file python -m openai_batch.example_prompts | \ python -m openai_batch.create_batch --model 'meta-llama/Meta-Llama-3-8B-Instruct' > input.jsonl # Run this batch (resumable with `--resume `) python -m openai_batch.run -p parasail input.jsonl ``` -------------------------------- ### Python: Step-by-Step Batch Job Workflow Source: https://github.com/parasail-ai/openai-batch/blob/main/README.md Provides a detailed, step-by-step workflow for managing batch jobs. This example shows creating a Batch object, adding requests, submitting the job, periodically checking its status, and downloading results. ```Python from openai_batch import Batch import time # Create a batch object batch = Batch( submission_input_file="batch_input.jsonl", output_file="batch_output.jsonl", error_file="batch_errors.jsonl" ) # Add chat completion requests to the batch objects = ["cat", "robot", "coffee mug", "spaceship", "banana"] for i in range(5): batch.add_to_batch( model="gpt-4o-mini", messages=[{"role": "user", "content": f"Tell me a joke about a {objects[i]}"}] ) # Submit the batch batch_id = batch.submit() print(f"Batch submitted with ID: {batch_id}") # Check status periodically while True: status = batch.status() print(f"Batch status: {status.status}") if status.status in ["completed", "failed", "expired", "cancelled"]: break time.sleep(60) # Check every minute # Download results once completed output_path, error_path = batch.download() print(f"Output saved to: {output_path}") print(f"Errors saved to: {error_path}") ``` -------------------------------- ### Run Batch from Input File (Command Line) Source: https://github.com/parasail-ai/openai-batch/blob/main/README.md This command-line utility allows you to run a batch job using an input file. It starts the batch process, waits for completion, and then downloads the results. Useful switches are available for controlling the execution flow. ```bash python -m openai_batch.run input.jsonl # Useful switches: # -c: Only create the batch, do not wait for completion. # --resume : Attach to an existing batch job and wait for it to finish. # --dry-run: Validate configuration without making a request. # For a full list of options: python -m openai_batch.run --help ``` -------------------------------- ### Python: Basic Embedding Batch Inference Source: https://github.com/parasail-ai/openai-batch/blob/main/README.md Illustrates how to use the openai-batch library for creating embedding batches. This example shows adding documents to a batch for embedding generation using a specified model. ```Python from openai_batch import Batch with Batch() as batch: documents = ["The quick brown fox jumps over the lazy dog", "Machine learning models can process natural language"] for doc in documents: batch.add_to_batch( model="text-embedding-3-small", # OpenAI embedding model input=doc ) result, output_path, error_path = batch.submit_wait_download() ``` -------------------------------- ### Python: Automatic Provider Selection for Batches Source: https://github.com/parasail-ai/openai-batch/blob/main/README.md Demonstrates how the library automatically selects the correct provider (OpenAI or Parasail) based on the model name. This simplifies usage by removing the need for explicit provider configuration for common models. ```Python from openai_batch import Batch # OpenAI models automatically use the OpenAI provider openai_batch = Batch() openai_batch.add_to_batch( model="gpt-4o-mini", # OpenAI model messages=[{"role": "user", "content": "Hello, world!"}] ) # Other models automatically use the Parasail provider parasail_batch = Batch() parasail_batch.add_to_batch( model="meta-llama/Meta-Llama-3-8B-Instruct", # Non-OpenAI model messages=[{"role": "user", "content": "Hello, world!"}] ) ``` -------------------------------- ### Python: Basic Chat Completion Batch Inference Source: https://github.com/parasail-ai/openai-batch/blob/main/README.md Demonstrates basic usage of the openai-batch library for submitting chat completion requests. It shows how to add multiple requests to a batch and then submit, wait for completion, and download the results. ```Python import random from openai_batch import Batch # Create a batch with random prompts with Batch() as batch: objects = ["cat", "robot", "coffee mug", "spaceship", "banana"] for i in range(100): batch.add_to_batch( model="meta-llama/Meta-Llama-3-8B-Instruct", temperature=0.7, max_completion_tokens=1000, messages=[{"role": "user", "content": f"Tell me a joke about a {random.choice(objects)}"}] ) # Submit, wait for completion, and download results result, output_path, error_path = batch.submit_wait_download() print(f"Batch completed with status {result.status} and stored in {output_path}") ``` -------------------------------- ### Python: Explicit Provider Selection for Batches Source: https://github.com/parasail-ai/openai-batch/blob/main/README.md Shows how to explicitly specify a provider when creating a Batch object. This is useful for ensuring the correct provider is used, especially when dealing with custom configurations or non-standard model names. ```Python from openai_batch import Batch from openai_batch.providers import get_provider_by_name # Get a specific provider provider = get_provider_by_name("parasail") # Create a batch with this provider batch = Batch(provider=provider) batch.add_to_batch( model="meta-llama/Meta-Llama-3-8B-Instruct", messages=[{"role": "user", "content": "Hello, world!"}] ) ``` -------------------------------- ### Python: Resuming an Existing Batch Job Source: https://github.com/parasail-ai/openai-batch/blob/main/README.md Illustrates how to resume an existing batch job by providing its batch ID to the Batch constructor. This allows for checking the status or downloading results of a previously submitted batch. ```Python from openai_batch import Batch import time # Resume an existing batch batch = Batch(batch_id="batch_abc123") ``` -------------------------------- ### Monitor Batch Status and Download Results (Python) Source: https://github.com/parasail-ai/openai-batch/blob/main/README.md This Python snippet demonstrates how to monitor the status of a batch job in a loop. It checks the status periodically, prints updates, and downloads the output files once the batch is completed. It also handles terminal states like failed, expired, or cancelled. ```python import time # Assuming 'batch' is an initialized batch object # Example: batch = openai_batch.Batch(batch_id='batch_abc123') while True: status = batch.status() print(f"Batch status: {status.status}") if status.status == "completed": output_path, error_path = batch.download() print(f"Output saved to: {output_path}") break elif status.status in ["failed", "expired", "cancelled"]: print(f"Batch ended with status: {status.status}") break time.sleep(60) # Check every minute ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.