### Install EcoLogits with LiteLLM extras Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/litellm.md Install EcoLogits with the necessary dependencies for LiteLLM compatibility using pip. ```shell pip install ecologits[litellm] ``` -------------------------------- ### Install EcoLogits with Google GenAI Support Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/google_genai.md Install EcoLogits with the necessary dependencies for Google Gemini compatibility using pip. ```shell pip install ecologits[google-genai] ``` -------------------------------- ### Install EcoLogits with OpenAI Support Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/openai.md Install EcoLogits with the necessary dependencies for OpenAI compatibility using pip. ```shell pip install ecologits[openai] ``` -------------------------------- ### Install EcoLogits with Cohere dependencies Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/cohere.md Install EcoLogits with the necessary dependencies for Cohere compatibility using pip. ```shell pip install ecologits[cohere] ``` -------------------------------- ### Install EcoLogits with Hugging Face Hub Support Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/huggingface_hub.md Install EcoLogits with the necessary dependencies for Hugging Face Hub compatibility using pip. ```shell pip install ecologits[huggingface-hub] ``` -------------------------------- ### Install EcoLogits Source: https://github.com/mlco2/ecologits/blob/main/README.md Install the EcoLogits package using pip. For specific provider integrations, append the provider name in brackets. ```shell pip install ecologits ``` -------------------------------- ### Install EcoLogits with Anthropic Dependencies Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/anthropic.md Use this command to install EcoLogits with the necessary dependencies for Anthropic client compatibility. ```shell pip install ecologits[anthropic] ``` -------------------------------- ### Install Development Dependencies Source: https://github.com/mlco2/ecologits/blob/main/docs/contributing.md Installs the necessary development dependencies for EcoLogits using uv. Ensure you have cloned your fork and navigated into the repository directory first. ```shell git clone git@github.com:/ecologits.git cd ecologits # Install ecologits development dependencies with uv make install ``` -------------------------------- ### Install EcoLogits with OpenTelemetry Support Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/opentelemetry.md Install EcoLogits with the 'opentelemetry' extra-dependency to include necessary libraries for OpenTelemetry compatibility. Ensure to include your specific provider. ```shell pip install ecologits[opentelemetry] # (1)! ``` -------------------------------- ### Install EcoLogits with Mistral AI dependencies Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/mistralai.md Use this command to install EcoLogits with the necessary dependencies for Mistral AI client compatibility. ```shell pip install ecologits[mistralai] ``` -------------------------------- ### Initialize EcoLogits for Impact Assessment Source: https://github.com/mlco2/ecologits/blob/main/docs/why.md Initialize the EcoLogits library to begin assessing environmental impacts. This setup is required before making requests to supported providers. ```python from ecologits import EcoLogits EcoLogits.init() ``` -------------------------------- ### Streaming Example - Sync Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/cohere.md Retrieve environmental impacts for synchronous chat completions in streaming mode. Impacts are calculated on the last chunk. ```python from cohere import Client from ecologits import EcoLogits # Initialize EcoLogits EcoLogits.init(providers=["cohere"]) client = Client(api_key="") stream = client.chat_stream( message="Tell me a funny joke!", max_tokens=100 ) for chunk in stream: if chunk.event_type == "stream-end": # Get estimated environmental impacts of the inference print(chunk.impacts) ``` -------------------------------- ### Streaming Example - Async Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/cohere.md Retrieve environmental impacts for asynchronous chat completions in streaming mode. Impacts are calculated on the last chunk. ```python import asyncio from cohere import AsyncClient from ecologits import EcoLogits # Initialize EcoLogits EcoLogits.init(providers=["cohere"]) client = AsyncClient(api_key="") async def main() -> None: stream = client.chat_stream( message="Tell me a funny joke!", max_tokens=100 ) async for chunk in stream: if chunk.event_type == "stream-end": # Get estimated environmental impacts of the inference print(chunk.impacts) asyncio.run(main()) ``` -------------------------------- ### Configure OpenTelemetry Endpoint in EcoLogits Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/opentelemetry.md Initialize EcoLogits with the OpenTelemetry endpoint URL. This setup allows EcoLogits to automatically log metrics for all subsequent requests made using the configured client. ```python from ecologits import EcoLogits from openai import OpenAI # Configure OpenTelemetry endpoint in EcoLogits EcoLogits.init( providers=["openai"], opentelemetry_endpoint='https://localhost:4318/v1/metrics' ) client = OpenAI() # All requests will log ecologits metrics response = client.chat.completions.create( model="gpt-4o-mini", messages=[{"role": "user", "content": "Tell me a funny joke!"}] ) ``` -------------------------------- ### Async Chat Completions with Google GenAI Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/google_genai.md Utilize the asynchronous client for non-blocking content generation and impact retrieval. This example also demonstrates compatibility with the chat API. ```python import asyncio from ecologits import EcoLogits from google import genai # Initialize EcoLogits EcoLogits.init(providers=["google_genai"]) # Make generate content request async def main() -> None: client = genai.Client() response = await client.aio.models.generate_content( # (1)! model="gemini-2.0-flash-001", contents="Tell me a joke!" ) # Get estimated environmental impacts of the inference print(response.impacts) asyncio.run(main()) ``` ```python chat = client.aio.chats.create(model='gemini-2.0-flash-001') response = await chat.send_message('Tell me a joke!') print(response.impacts) ``` -------------------------------- ### Chat Completions - Sync Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/cohere.md Use this snippet to get environmental impacts for synchronous chat completions with Cohere. Ensure EcoLogits is initialized. ```python from cohere import Client from ecologits import EcoLogits # Initialize EcoLogits EcoLogits.init(providers=["cohere"]) client = Client(api_key="") response = client.chat( message="Tell me a funny joke!", max_tokens=100 ) # Get estimated environmental impacts of the inference print(response.impacts) ``` -------------------------------- ### Synchronous Chat Completion with EcoLogits Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/anthropic.md Integrate EcoLogits with the synchronous Anthropic client to get environmental impacts alongside chat responses. Ensure EcoLogits is initialized with the 'anthropic' provider. ```python from anthropic import Anthropic from ecologits import EcoLogits # Initialize EcoLogits EcoLogits.init(providers=["anthropic"]) client = Anthropic(api_key="") response = client.messages.create( max_tokens=100, messages=[{"role": "user", "content": "Tell me a funny joke!"}], model="claude-3-haiku-20240307", ) # Get estimated environmental impacts of the inference print(response.impacts) ``` -------------------------------- ### Chat Completions - Async Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/cohere.md Use this snippet to get environmental impacts for asynchronous chat completions with Cohere. Ensure EcoLogits is initialized. ```python import asyncio from cohere import AsyncClient from ecologits import EcoLogits # Initialize EcoLogits EcoLogits.init(providers=["cohere"]) client = AsyncClient(api_key="") async def main() -> None: response = await client.chat( message="Tell me a funny joke!", max_tokens=100 ) # Get estimated environmental impacts of the inference print(response.impacts) asyncio.run(main()) ``` -------------------------------- ### Example Impact Value Structure (GWP) Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/impacts.md Illustrates the structure of individual impact objects like GWP, showing their type, name, unit, and value. This class inherits from BaseImpact. ```python from ecologits.impacts.modeling import BaseImpact from ecologits.utils.range_value import RangeValue class GWP(BaseImpact): # (1)! type: str = "GWP" name: str = "Global Warming Potential" unit: str = "kgCO2eq" value: float | RangeValue = 0.34 ``` -------------------------------- ### Sync OpenAI Response with EcoLogits Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/openai.md Use this snippet to get a synchronous response from OpenAI and log its environmental impact using EcoLogits. Ensure EcoLogits is initialized with the 'openai' provider. ```python from ecologits import EcoLogits from openai import OpenAI # Initialize EcoLogits EcoLogits.init(providers=["openai"]) client = OpenAI(api_key="") response = client.responses.create( model="gpt-4o-mini", input="Tell me a funny joke!" ) # Get estimated environmental impacts of the inference print(response.impacts) ``` -------------------------------- ### Serve Documentation Locally Source: https://github.com/mlco2/ecologits/blob/main/docs/contributing.md Serves the project documentation locally at http://localhost:8000. This allows you to preview documentation changes before committing. ```shell # Serve the documentation at localhost:8000 uv run mkdocs serve ``` -------------------------------- ### Get AWS Instance Average Power with BoaviztAPI Source: https://github.com/mlco2/ecologits/blob/main/docs/methodology/llm_inference.md Use this curl command to query the BoaviztAPI for the average power consumption of a specific AWS instance type. Ensure you have jq installed to parse the JSON output. ```shell curl -X 'POST' \ 'https://api.boavizta.org/v1/cloud/instance?verbose=true' \ -H 'accept: application/json' \ -H 'Content-Type: application/json' \ -d '{ \ "provider": "aws", \ "instance_type": "p5.48xlarge" \ }' | jq .verbose.avg_power ``` -------------------------------- ### Initialize EcoLogits with OpenAI Provider Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/openai.md Initialize EcoLogits to trace OpenAI requests. Ensure you have the 'openai' provider enabled. ```python import os from ecologits import EcoLogits from openai import AzureOpenAI # Initialize EcoLogits EcoLogits.init(providers=["openai"]) client = AzureOpenAI( azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT"), api_key=os.getenv("AZURE_OPENAI_API_KEY"), api_version="2024-02-01" ) response = client.chat.completions.create( model="gpt-35-turbo", messages=[ {"role": "user", "content": "Tell me a funny joke!"} ] ) # Get estimated environmental impacts of the inference print(response.impacts) ``` -------------------------------- ### Build Project Documentation Source: https://github.com/mlco2/ecologits/blob/main/docs/contributing.md Builds the project's documentation. This command is essential if you have modified any documentation files, including docstrings, that affect the API documentation. ```shell # Build documentation make docs # If you have changed the documentation, make sure it builds successfully. ``` -------------------------------- ### Initialize EcoLogits with Providers Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/index.md Initialize the EcoLogits client tracers to intercept and enrich API responses. Supports multiple providers like OpenAI and Anthropic. Note that disabling a provider at runtime is not supported. ```python from ecologits import EcoLogits # Example for OpenAI and Anthropic EcoLogits.init(providers=["openai", "anthropic"]) ``` -------------------------------- ### Example Impact with RangeValue Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/impacts.md Shows how an impact value can be represented as a RangeValue object, indicating a minimum and maximum estimate. This is used for high-confidence approximation intervals. ```python >>> response.impacts.gwp.value RangeValue(min=0.16, max=0.48) # in kgCO2eq (1) ``` -------------------------------- ### Initialize EcoLogits and Track OpenAI API Usage Source: https://github.com/mlco2/ecologits/blob/main/README.md Initialize EcoLogits with the desired providers and then use the OpenAI client to make API calls. The environmental impacts are accessible via the `impacts` attribute of the response. ```python from ecologits import EcoLogits from openai import OpenAI # Initialize EcoLogits EcoLogits.init(providers=["openai"]) client = OpenAI(api_key="") response = client.chat.completions.create( model="gpt-4o-mini", messages=[ {"role": "user", "content": "Tell me a funny joke!"} ] ) # Get estimated environmental impacts of the inference print(f"Energy consumption: {response.impacts.energy.value.mean} kWh") print(f"GHG emissions: {response.impacts.gwp.value.mean} kgCO2eq") ``` -------------------------------- ### Deploy Versioned Docs to gh-pages Source: https://github.com/mlco2/ecologits/blob/main/RELEASE.md This command deploys versioned documentation using `mike`. It updates aliases and pushes to the `gh-pages` branch. Ensure you fetch `gh-pages` and check `versions.json` afterwards. ```shell git fetch origin gh-pages git branch -f gh-pages origin/gh-pages uv run mike deploy -b gh-pages latest --update-aliases --push git fetch origin gh-pages git show origin/gh-pages:versions.json ``` -------------------------------- ### Select Electricity Mix Zone Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/impacts.md Initialize EcoLogits with a specific electricity mix zone, such as France ('FRA'), to use regional energy production data for calculations. The default is 'WOR' (World). ```python from ecologits import EcoLogits # Select the electricity mix of France EcoLogits.init(electricity_mix_zone="FRA") ``` -------------------------------- ### Async Streaming with Google GenAI Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/google_genai.md Handle streaming responses asynchronously, calculating cumulative environmental impacts in the final chunk. This example also supports the chat API. ```python import asyncio from ecologits import EcoLogits from google import genai # Initialize EcoLogits EcoLogits.init(providers=["google_genai"]) # Ask something to Google Gemini in streaming and async mode async def main() -> None: # Make generate content request client = genai.Client() stream = await client.aio.models.generate_content_stream( # (1)! model="gemini-2.0-flash-001", contents="Tell me a joke!" ) # Get cumulative estimated environmental impacts of the inference async for chunk in stream: if chunk.impacts is not None: print(chunk.impacts) asyncio.run(main()) ``` ```python chat = client.aio.chats.create(model='gemini-2.0-flash-001') stream = await chat.send_message_stream("Tell me long a joke!") async for chunk in stream: if chunk.impacts is not None: print(chunk.impacts) ``` -------------------------------- ### Async OpenAI Response with EcoLogits Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/openai.md This snippet demonstrates how to handle asynchronous OpenAI responses and log their environmental impact with EcoLogits. It requires initializing EcoLogits with the 'openai' provider. ```python import asyncio from ecologits import EcoLogits from openai import AsyncOpenAI # Initialize EcoLogits EcoLogits.init(providers=["openai"]) client = AsyncOpenAI(api_key="") async def main() -> None: response = await client.responses.create( model="gpt-4o-mini", input="Tell me a funny joke!" ) # Get estimated environmental impacts of the inference print(response.impacts) asyncio.run(main()) ``` -------------------------------- ### Calculate Video Impacts with Defaults Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/video_generation.md Use `video_impacts` with default provider assumptions for model deployment location, PUE, and WUE when specific information is unknown. ```python impacts = video_impacts( model_name="bytedance/seedance-1.5-pro", resolution="720p", duration=5, ) ``` -------------------------------- ### Sync Streaming with Google GenAI Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/google_genai.md Process streaming responses synchronously, where environmental impacts are calculated for the entire request in the last chunk. This example is also applicable to the chat API. ```python from ecologits import EcoLogits from google import genai # Initialize EcoLogits EcoLogits.init(providers=["google_genai"]) # Make generate content request client = genai.Client() stream = client.models.generate_content_stream( # (1)! model="gemini-2.0-flash-001", contents="Tell me a joke!" ) # Get cumulative estimated environmental impacts of the inference for chunk in stream: if chunk.impacts is not None: print(chunk.impacts) ``` ```python chat = client.chats.create(model='gemini-2.0-flash-001') stream = chat.send_message_stream("Tell me long a joke!") for chunk in stream: if chunk.impacts is not None: print(chunk.impacts) ``` -------------------------------- ### Specify Video Resolution Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/video_generation.md Set the desired video resolution using named presets like '720p', '1080p', or '4k', or provide an explicit 'widthxheight' string. ```python video_impacts( model_name="google/veo-3.1", resolution="720p", duration=8, ) ``` ```python video_impacts( model_name="runway/gen-4.5", resolution="1024x576", duration=5, ) ``` -------------------------------- ### Check for Warnings and Errors in EcoLogits Response Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/warnings_and_errors.md This code demonstrates how to check if a request resulted in any warnings or errors and how to print them. Ensure EcoLogits is initialized before making requests. ```python from ecologits import EcoLogits EcoLogits.init() response = ... # Request code goes here if response.impacts.has_warnings: for w in response.impacts.warnings: print(w) if response.impacts.has_errors: for e in response.impacts.errors: print(e) ``` -------------------------------- ### Commit, Tag, and Push Changes for PyPI Release Source: https://github.com/mlco2/ecologits/blob/main/RELEASE.md This snippet shows the Git commands required to commit changes, push to the origin, tag the release, and push the tags. Ensure CI jobs succeed before pushing tags. ```shell git add . git commit -m "chore: bump version to x.y.z" git push origin # wait for all CI jobs to succeed git tag x.y.z git push origin --tags ``` -------------------------------- ### Run Pre-commit Hooks Source: https://github.com/mlco2/ecologits/blob/main/docs/contributing.md Executes all pre-commit hooks to check and lint your code before committing. This ensures code quality and consistency. ```shell # Run all checks before commit make pre-commit ``` -------------------------------- ### Configure Electricity Mix for EcoLogits Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/index.md Change the electricity mix for server-side computation to adjust impact factors based on geographic location. Available zones are listed in electricity_mixes.json. The default zone is 'WOR' (World). ```python from ecologits import EcoLogits # Select the electricity mix of France EcoLogits.init(providers=[...], electricity_mix_zone="FRA") ``` -------------------------------- ### Streaming Chat Completions with EcoLogits (Sync) Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/mistralai.md Utilize EcoLogits with the Mistral AI client for synchronous streaming chat completions. Impacts are calculated incrementally and the last chunk contains cumulative data. ```python from ecologits import EcoLogits from mistralai import Mistral # Initialize EcoLogits EcoLogits.init(providers=["mistralai"]) client = Mistral(api_key="") stream = client.chat.stream( messages=[ {"role": "user", "content": "Tell me a funny joke!"} ], model="mistral-tiny" ) for chunk in stream: # Get cumulative estimated environmental impacts of the inference print(chunk.data.impacts) ``` -------------------------------- ### Asynchronous Streaming Chat Completion with EcoLogits Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/anthropic.md Implement EcoLogits with the asynchronous Anthropic client for streaming chat completions. Impacts are computed on the final chunk. Ensure EcoLogits is initialized with the 'anthropic' provider. ```python import asyncio from anthropic import AsyncAnthropic from ecologits import EcoLogits # Initialize EcoLogits EcoLogits.init(providers=["anthropic"]) client = AsyncAnthropic(api_key="") async def main() -> None: async with client.messages.stream( max_tokens=100, messages=[{"role": "user", "content": "Tell me a funny joke!"}], model="claude-3-haiku-20240307", ) as stream: async for text in stream.text_stream: pass # Get estimated environmental impacts of the inference print(stream.impacts) asyncio.run(main()) ``` -------------------------------- ### Async Chat Completion with EcoLogits Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/litellm.md Perform an asynchronous chat completion using LiteLLM and retrieve the estimated environmental impacts. Ensure your provider's API key is set in an .env file and use `litellm.acompletion`. ```python import asyncio import litellm from ecologits import EcoLogits # Initialize EcoLogits EcoLogits.init(providers=["litellm"]) async def main() -> None: response = await litellm.acompletion( model="gpt-3.5-turbo", messages=[ {"role": "user", "content": "Tell me a funny joke!"} ] ) # Get estimated environmental impacts of the inference print(response.impacts) asyncio.run(main()) ``` -------------------------------- ### Sync Chat Completion with EcoLogits Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/litellm.md Perform a synchronous chat completion using LiteLLM and access the estimated environmental impacts. Ensure your provider's API key is set in an .env file and use `litellm.completion`. ```python from ecologits import EcoLogits import litellm # Initialize EcoLogits EcoLogits.init(providers=["litellm"]) response = litellm.completion( model="gpt-4o-2024-05-13", messages=[{ "content": "Hello, how are you?","role": "user"}] ) # Get estimated environmental impacts of the inference print(response.impacts) ``` -------------------------------- ### Asynchronous Chat Completion with EcoLogits Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/anthropic.md Integrate EcoLogits with the asynchronous Anthropic client for environmental impact tracking. Initialize EcoLogits with the 'anthropic' provider before making calls. ```python import asyncio from anthropic import AsyncAnthropic from ecologits import EcoLogits # Initialize EcoLogits EcoLogits.init(providers=["anthropic"]) client = AsyncAnthropic(api_key="") async def main() -> None: response = await client.messages.create( max_tokens=100, messages=[{"role": "user", "content": "Tell me a funny joke!"}], model="claude-3-haiku-20240307", ) # Get estimated environmental impacts of the inference print(response.impacts) asyncio.run(main()) ``` -------------------------------- ### Sync Streaming Chat Completion with EcoLogits Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/litellm.md Process a synchronous streaming chat completion with LiteLLM and access the cumulative environmental impacts from the last chunk. Ensure your provider's API key is set in an .env file and use `litellm.completion` with `stream=True`. ```python from ecologits import EcoLogits import litellm # Initialize EcoLogits EcoLogits.init(providers=["litellm"]) stream = litellm.completion( model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hello World!"}], stream=True ) for chunk in stream: # Get cumulative estimated environmental impacts of the inference print(chunk.impacts) ``` -------------------------------- ### Streaming Chat Completions with EcoLogits (Async) Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/mistralai.md Utilize EcoLogits with the Mistral AI client for asynchronous streaming chat completions. Impacts are calculated incrementally and the last chunk contains cumulative data. ```python import asyncio from ecologits import EcoLogits from mistralai import Mistral # Initialize EcoLogits EcoLogits.init(providers=["mistralai"]) client = Mistral(api_key="") async def main() -> None: stream = await client.chat.stream_async( messages=[ {"role": "user", "content": "Tell me a funny joke!"} ], model="mistral-tiny" ) async for chunk in stream: # Get cumulative estimated environmental impacts of the inference print(chunk.data.impacts) asyncio.run(main()) ``` -------------------------------- ### Estimate Video Generation Impacts with ByteDance Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/bytedance.md Use this snippet to estimate energy and GWP for ByteDance video models. Ensure you use the same model, resolution, and duration values as provided to the generation service. ```python from ecologits.estimations import video_impacts model = "bytedance/seedance-1.5-pro" resolution = "1080p" duration = 5 impacts = video_impacts( model_name=model, resolution=resolution, duration=duration, ) print(impacts.energy.value) print(impacts.gwp.value) ``` -------------------------------- ### Async Chat Completion with Hugging Face Hub Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/huggingface_hub.md Perform an asynchronous chat completion using Hugging Face Hub's AsyncInferenceClient and retrieve estimated environmental impacts. ```python import asyncio from ecologits import EcoLogits from huggingface_hub import AsyncInferenceClient # Initialize EcoLogits EcoLogits.init(providers=["huggingface_hub"]) client = AsyncInferenceClient(model="meta-llama/Meta-Llama-3.1-8B") async def main() -> None: response = await client.chat_completion( messages=[{"role": "user", "content": "Tell me a funny joke!"}], max_tokens=15 ) # Get estimated environmental impacts of the inference print(response.impacts) asyncio.run(main()) ``` -------------------------------- ### Sync Streaming Chat Completion with Hugging Face Hub Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/huggingface_hub.md Handle streaming chat completions synchronously with Hugging Face Hub's InferenceClient, where impacts are cumulatively calculated. ```python from ecologits import EcoLogits from huggingface_hub import InferenceClient # Initialize EcoLogits EcoLogits.init(providers=["huggingface_hub"]) client = InferenceClient(model="meta-llama/Meta-Llama-3.1-8B") stream = client.chat_completion( messages=[{"role": "user", "content": "Tell me a funny joke!"}], max_tokens=15, stream=True ) for chunk in stream: # Get cumulative estimated environmental impacts of the inference print(chunk.impacts) ``` -------------------------------- ### Sync Chat Completion with Hugging Face Hub Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/huggingface_hub.md Perform a synchronous chat completion using Hugging Face Hub's InferenceClient and retrieve estimated environmental impacts. ```python from ecologits import EcoLogits from huggingface_hub import InferenceClient # Initialize EcoLogits EcoLogits.init(providers=["huggingface_hub"]) client = InferenceClient(model="meta-llama/Meta-Llama-3.1-8B") response = client.chat_completion( messages=[{"role": "user", "content": "Tell me a funny joke!"}], max_tokens=15 ) # Get estimated environmental impacts of the inference print(response.impacts) ``` -------------------------------- ### Chat Completions with EcoLogits (Async) Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/mistralai.md Integrate EcoLogits with the Mistral AI client for asynchronous chat completions. EcoLogits adds an 'Impacts' object to the response containing environmental data. ```python import asyncio from ecologits import EcoLogits from mistralai import Mistral # Initialize EcoLogits EcoLogits.init(providers=["mistralai"]) client = Mistral(api_key="") async def main() -> None: response = await client.chat.complete_async( messages=[ {"role": "user", "content": "Tell me a funny joke!"} ], model="mistral-tiny" ) # Get estimated environmental impacts of the inference print(response.impacts) asyncio.run(main()) ``` -------------------------------- ### Async Streaming Chat Completion with EcoLogits Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/litellm.md Handle an asynchronous streaming chat completion with LiteLLM and retrieve the cumulative environmental impacts from the last chunk. Ensure your provider's API key is set in an .env file and use `litellm.acompletion` with `stream=True`. ```python import asyncio import litellm from ecologits import EcoLogits # Initialize EcoLogits EcoLogits.init(providers=["litellm"]) async def main() -> None: stream = await litellm.acompletion( model="gpt-3.5-turbo", messages=[ {"role": "user", "content": "Tell me a funny joke!"} ], stream=True ) async for chunk in stream: # Get cumulative estimated environmental impacts of the inference print(chunk.impacts) asyncio.run(main()) ``` -------------------------------- ### Sync Streaming OpenAI Response with EcoLogits Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/openai.md Use this for synchronous streaming responses from OpenAI. Environmental impacts are only available on the 'response.completed' event. Initialize EcoLogits with the 'openai' provider. ```python from ecologits import EcoLogits from openai import OpenAI # Initialize EcoLogits EcoLogits.init(providers=["openai"]) client = OpenAI(api_key="") stream = client.responses.create( model="gpt-4o-mini", input="Tell me a funny joke!", stream=True ) for event in stream: if event.type == "response.completed": # Get estimated environmental impacts of the inference print(event.impacts) ``` -------------------------------- ### Chat Completions with EcoLogits (Sync) Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/mistralai.md Integrate EcoLogits with the Mistral AI client for synchronous chat completions. EcoLogits adds an 'Impacts' object to the response containing environmental data. ```python from ecologits import EcoLogits from mistralai import Mistral # Initialize EcoLogits EcoLogits.init(providers=["mistralai"]) client = Mistral(api_key="") response = client.chat.complete( messages=[ {"role": "user", "content": "Tell me a funny joke!"} ], model="mistral-tiny" ) # Get estimated environmental impacts of the inference print(response.impacts) ``` -------------------------------- ### Estimate Video Generation Impacts with Runway Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/runway.md Use the `video_impacts` function to estimate energy and GWP for Runway video generation. Ensure the `ecologits` library is imported. ```python from ecologits.estimations import video_impacts model = "runway/gen-4.5" resolution = "1080p" duration = 10 impacts = video_impacts( model_name=model, resolution=resolution, duration=duration, ) print(impacts.energy.value) print(impacts.gwp.value) ``` -------------------------------- ### Track Hugging Face Hub LLM Environmental Impacts Source: https://github.com/mlco2/ecologits/blob/main/docs/get_started.md Initialize EcoLogits with the Hugging Face Hub provider and use the InferenceClient for chat completion. Environmental impacts are accessible via the response object. ```python from ecologits import EcoLogits from huggingface_hub import InferenceClient # Initialize EcoLogits EcoLogits.init(providers=["huggingface_hub"]) client = InferenceClient(model="meta-llama/Meta-Llama-3.1-8B") response = client.chat_completion( # (1)! messages=[{"role": "user", "content": "Tell me a funny joke!"}], max_tokens=15 ) # Get estimated environmental impacts of the inference print(f"Energy consumption: {response.impacts.energy.value} kWh") # (2)! print(f"GHG emissions: {response.impacts.gwp.value} kgCO2eq") # (3)! # Get potential warnings if response.impacts.has_warnings: for w in response.impacts.warnings: print(w) # (4)! # Get potential errors if response.impacts.has_errors: for w in response.impacts.errors: print(w) # (5)! ``` -------------------------------- ### Pass PUE and WUE as Ranges Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/video_generation.md Use `RangeValue` to specify minimum and maximum values for `datacenter_pue` and `datacenter_wue` when dealing with variable environments. ```python from ecologits.utils.range_value import RangeValue impacts = video_impacts( model_name="runway/gen-4.5", resolution="1080p", duration=10, datacenter_pue=RangeValue(min=1.09, max=1.14), datacenter_wue=RangeValue(min=0.13, max=0.999), ) ``` -------------------------------- ### Record Tests with VCR.py Source: https://github.com/mlco2/ecologits/blob/main/docs/contributing.md Records new test cassettes when adding a new provider. After recording, ensure cassette files in `tests/cassettes/` do not contain sensitive information and update `conftest.py` if necessary. ```shell make test-record ``` -------------------------------- ### Async Streaming Chat Completion with Hugging Face Hub Source: https://github.com/mlco2/ecologits/blob/main/docs/tutorial/providers/huggingface_hub.md Handle streaming chat completions asynchronously with Hugging Face Hub's AsyncInferenceClient, where impacts are cumulatively calculated. ```python import asyncio from ecologits import EcoLogits from huggingface_hub import AsyncInferenceClient # Initialize EcoLogits EcoLogits.init(providers=["huggingface_hub"]) client = AsyncInferenceClient(model="meta-llama/Meta-Llama-3.1-8B") async def main() -> None: stream = await client.chat_completion( messages=[{"role": "user", "content": "Tell me a funny joke!"}], max_tokens=15, stream=True ) async for chunk in stream: # Get cumulative estimated environmental impacts of the inference print(chunk.impacts) asyncio.run(main()) ```