### Setup dependencies and pipeline

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Gemini_and_Stable_Diffusion.ipynb

Installs required libraries and initializes the Stable Diffusion pipeline using SDXL Turbo.

```python
#@title Setup dependencies & pipeline - this takes a bit

%pip install --quiet --upgrade diffusers accelerate mediapy

import mediapy as media, random, sys, torch
from diffusers import AutoPipelineForText2Image

pipe = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/sdxl-turbo",
    torch_dtype=torch.float16,
    use_safetensors=True,
    variant="fp16",
    )

pipe = pipe.to("cuda")
```

--------------------------------

### Print Hello World in Colab Slideshow

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/How_to_Use_Slideshow_Mode.ipynb

A basic Python example demonstrating a 'Hello World' output within Colab's slideshow mode. This snippet is directly executable.

```python
# @title Hello World Example
print("Hello World")
```

--------------------------------

### Install Keras 3 and TensorFlow dependencies

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Gemma_Distributed_Fine_tuning_on_TPU.ipynb

Use this command to install the specified versions of TensorFlow, Keras, and NLP libraries. Ensure Keras 3 is installed last to maintain compatibility.

```bash
!pip install tensorflow-cpu~=2.16.0 keras-nlp==0.8.2 tensorflow-hub==0.16.1 keras==3.0.5 tensorflow-text==2.16.1
```

--------------------------------

### Installation output log

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Gemma_Distributed_Fine_tuning_on_TPU.ipynb

The console output generated after running the pip installation command.

```text
Collecting tensorflow-cpu~=2.16.0
  Downloading tensorflow_cpu-2.16.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (214.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 214.0/214.0 MB 2.9 MB/s eta 0:00:00
[?25hCollecting keras-nlp==0.8.2
  Downloading keras_nlp-0.8.2-py3-none-any.whl (465 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 465.3/465.3 kB 17.9 MB/s eta 0:00:00
[?25hRequirement already satisfied: tensorflow-hub==0.16.1 in /usr/local/lib/python3.10/dist-packages (0.16.1)
Collecting keras==3.0.5
  Downloading keras-3.0.5-py3-none-any.whl (1.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 16.7 MB/s eta 0:00:00
[?25hCollecting tensorflow-text==2.16.1
  Downloading tensorflow_text-2.16.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.2/5.2 MB 17.0 MB/s eta 0:00:00
[?25hCollecting keras-core (from keras-nlp==0.8.2)
  Downloading keras_core-0.1.7-py3-none-any.whl (950 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 950.8/950.8 kB 1.9 MB/s eta 0:00:00
[?25hRequirement already satisfied: absl-py in /usr/local/lib/python3.10/dist-packages (from keras-nlp==0.8.2) (1.4.0)
Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from keras-nlp==0.8.2) (1.25.2)
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from keras-nlp==0.8.2) (24.0)
Requirement already satisfied: regex in /usr/local/lib/python3.10/dist-packages (from keras-nlp==0.8.2) (2023.12.25)
Requirement already satisfied: rich in /usr/local/lib/python3.10/dist-packages (from keras-nlp==0.8.2) (13.7.1)
Requirement already satisfied: dm-tree in /usr/local/lib/python3.10/dist-packages (from keras-nlp==0.8.2) (0.1.8)
Requirement already satisfied: kagglehub in /usr/local/lib/python3.10/dist-packages (from keras-nlp==0.8.2) (0.2.2)
Requirement already satisfied: protobuf>=3.19.6 in /usr/local/lib/python3.10/dist-packages (from tensorflow-hub==0.16.1) (3.20.3)
Requirement already satisfied: tf-keras>=2.14.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow-hub==0.16.1) (2.15.1)
Collecting namex (from keras==3.0.5)
  Downloading namex-0.0.7-py3-none-any.whl (5.8 kB)
Requirement already satisfied: h5py in /usr/local/lib/python3.10/dist-packages (from keras==3.0.5) (3.10.0)
Requirement already satisfied: ml-dtypes in /usr/local/lib/python3.10/dist-packages (from keras==3.0.5) (0.2.0)
Collecting tensorflow<2.17,>=2.16.1 (from tensorflow-text==2.16.1)
  Downloading tensorflow-2.16.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (589.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 589.8/589.8 MB 1.1 MB/s eta 0:00:00
[?25hRequirement already satisfied: astunparse>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow-cpu~=2.16.0) (1.6.3)
Requirement already satisfied: flatbuffers>=23.5.26 in /usr/local/lib/python3.10/dist-packages (from tensorflow-cpu~=2.16.0) (24.3.25)
Requirement already satisfied: gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow-cpu~=2.16.0) (0.5.4)
Requirement already satisfied: google-pasta>=0.1.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow-cpu~=2.16.0) (0.2.0)
Requirement already satisfied: libclang>=13.0.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow-cpu~=2.16.0) (18.1.1)
Collecting ml-dtypes (from keras==3.0.5)
  Downloading ml_dtypes-0.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.2/2.2 MB 4.0 MB/s eta 0:00:00
[?25hRequirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.10/dist-packages (from tensorflow-cpu~=2.16.0) (3.3.0)
Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow-cpu~=2.16.0) (2.31.0)
Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from tensorflow-cpu~=2.16.0) (67.7.2)
Requirement already satisfied: six>=1.12.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow-cpu~=2.16.0) (1.16.0)
Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow-cpu~=2.16.0) (2.4.0)
Requirement already satisfied: typing-extensions>=3.6.6 in /usr/local/lib/python3.10/dist-packages (from tensorflow-cpu~=2.16.0) (4.11.0)
```

--------------------------------

### Setup Speech-to-Text Functions and Configuration

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Talk_to_Gemini_with_Google's_Speech_to_Text_API.ipynb

Sets up Python functions for speech-to-text conversion using the Google Cloud Speech API and configures recognition parameters. It also includes setup for browser audio recording.

```python
#@title Setup

# noting here that a lot of this code is forked from https://codelabs.developers.google.com/codelabs/cloud-speech-text-python3#0

# set up cloud speech detection functions

from google.cloud import speech

def speech_to_text(
    config: speech.RecognitionConfig,
    audio: speech.RecognitionAudio,
) -> speech.RecognizeResponse:
    client = speech.SpeechClient()

    # Synchronous speech recognition request
    response = client.recognize(config=config, audio=audio)

    return response

def print_response(response: speech.RecognizeResponse):
    for result in response.results:
        print_result(result)

def print_result(result: speech.SpeechRecognitionResult):
    best_alternative = result.alternatives[0]
    print("-" * 80)
    print(f"language_code: {result.language_code}")
    print(f"transcript:    {best_alternative.transcript}")
    print(f"confidence:    {best_alternative.confidence:.0%}")

# config for speech recognition; modify language here & other params
config = speech.RecognitionConfig(
    language_code="en",
    enable_automatic_punctuation=True,
)

# required set up to enable recording audio in your browser

!pip install ipywebrtc
import io
from ipywebrtc import AudioRecorder, CameraStream

# required in Colab to enable 3rd party widgets
from google.colab import output
output.enable_custom_widget_manager()
```

--------------------------------

### Configure OpenAI API Key

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Learning_with_Gemini_and_ChatGPT.ipynb

Installs required packages and initializes the OpenAI client using a secret stored in Colab.

```python
#@title Configure OpenAI API key

# access your OpenAI API key

# installing llmx first isn't necessary but avoids a confusing error when installing openai
!pip install -q llmx
!pip install -q openai
from openai import OpenAI

openai_api_secret_name = 'OPENAI_API_KEY'  # @param {type: "string"}

try:
  OPENAI_API_KEY=userdata.get(openai_api_secret_name)
  client = OpenAI(
    api_key=OPENAI_API_KEY
  )
except userdata.SecretNotFoundError as e:
   print(f'''Secret not found

This expects you to create a secret named {openai_api_secret_name} in Colab

Visit https://platform.openai.com/api-keys to create an API key

Store that in the secrets section on the left side of the notebook (key icon)

Name the secret {openai_api_secret_name}''')
   raise e
except userdata.NotebookAccessError as e:
  print(f'''You need to grant this notebook access to the {openai_api_secret_name} secret in order for the notebook to access Gemini on your behalf.''')
  raise e
except Exception as e:
  # unknown error
  print(f"There was an unknown error. Ensure you have a secret {openai_api_secret_name} stored in Colab and it's a valid key from https://platform.openai.com/api-keys")
  raise e
```

--------------------------------

### Install Google Cloud Speech Library

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Talk_to_Gemini_with_Google's_Speech_to_Text_API.ipynb

Installs the necessary Google Cloud Speech library for Colab. This is a prerequisite for using the Speech-to-Text API.

```python
#@title Install Google Cloud's speech library

!pip install -q google-cloud-speech
from google.cloud import speech
```

--------------------------------

### Generate Image Prompt with Gemini

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Generate_images_with_Gemini_and_Vertex.ipynb

Uses the Gemini API to generate a detailed image prompt for a specified item. The prompt is designed to be compelling for online sales. It starts a chat session and sends a message requesting a verbose prompt.

```python
#@title Use Gemini to generate an image prompt for your item

item_selling = 'lemonade' #@param {type: "string"}

model = genai.GenerativeModel('gemini-pro')
chat = model.start_chat(history=[])

prompttext = f"""
  I'm selling {item_selling} online, and I need to generate an image of it.
  I need the image to be compelling and interesting to convince people to buy.
  Can you create a prompt I can use to generate an image of {item_selling} with Vertex?
  Respond with only the prompt, no other text. Be as verbose as possible.
  """

response = chat.send_message(prompttext)

response.text
```

--------------------------------

### Analyze a Demo Image

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Classify_an_image_using_Gemini.ipynb

Downloads a sample image, analyzes it using the configured Gemini model, and displays the image along with its description in an HTML table. Ensure the 'gemini-pro-vision' model is initialized.

```python
#@title Demo image analysis

import PIL.Image, base64, io

from IPython.display import HTML, display

!curl -o image.jpg https://t0.gstatic.com/licensed-image?q=tbn:ANd9GcQ_Kevbk21QBRy-PgB4kQpS79brbmmEG7m3VOTShAn4PecDU5H5UxrJxE3Dw1JiaG17V88QIol19-3TM2wCHw

img = PIL.Image.open('image.jpg')
response = model.generate_content(img)

# convert the image to Base64 for embedding in HTML
buffered = io.BytesIO()
img.save(buffered, format="JPEG")
img_str = base64.b64encode(buffered.getvalue()).decode()

# construct a table to render the view
table_html = f"""
<table>
<tr>
    <td><img src='data:image/jpeg;base64,{img_str}' style='width:500px; height:auto;'></td>
</tr>
<tr>
    <td style='vertical-align: bottom;'>{response.text}</td>
</tr>
</table>
"""

# display the table
display(HTML(table_html))
```

--------------------------------

### Import JAX and Check TPU Devices

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Gemma_Distributed_Fine_tuning_on_TPU.ipynb

Imports the JAX library and lists available TPU devices. This is a sanity check to ensure TPUs are accessible.

```python
import jax

jax.devices()
```

--------------------------------

### Enable Google Cloud Speech-to-Text API

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Talk_to_Gemini_with_Google's_Speech_to_Text_API.ipynb

Enables the Google Cloud Speech-to-Text API for your project. This command should be run once per project.

```bash
#@title [Run once per project] Enable the Google Cloud speech-to-text API

!gcloud services enable speech.googleapis.com
```

--------------------------------

### Define a Python list

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/How_to_Use_Slideshow_Mode.ipynb

Initializes a list of strings for demonstration purposes.

```python
students = ['Mikayla', 'Julianne', 'Golnar']
```

--------------------------------

### Authenticate with Service Account

Source: https://context7.com/googlecolab/colabtools/llms.txt

Authenticates using a service account key file. Prompts for key file upload if not already authenticated, then verifies the service account and project.

```python
from google.colab import auth

# Authenticate with a service account
# This will prompt for key file upload if not already authenticated
auth.authenticate_service_account()

# After authentication, verify the service account
import google.auth
creds, project = google.auth.default()
print(f"Authenticated as: {creds.service_account_email}")
print(f"Project: {project}")
```

--------------------------------

### Record speech with CameraStream

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Talk_to_Gemini_with_Google's_Speech_to_Text_API.ipynb

Initializes a microphone stream and audio recorder widget for capturing user input.

```python
#@title Record your speech

# create a microphone stream
camera = CameraStream(constraints={'audio': True, 'video':False})

# create an audio recorder that uses the microphone stream
recorder = AudioRecorder(stream=camera)

# display the recorder widget
recorder
```

--------------------------------

### Create Tab Interface with TabBar Widget

Source: https://context7.com/googlecolab/colabtools/llms.txt

Organizes output into separate tabs using the TabBar widget. Requires imports for widgets, matplotlib, and numpy. Each tab is populated using output_to.

```python
from google.colab import widgets
import matplotlib.pyplot as plt
import numpy as np

# Create a tab bar with named tabs
tabs = widgets.TabBar(['Data', 'Visualization', 'Summary'])

# Populate each tab
with tabs.output_to('Data'):
    import pandas as pd
    df = pd.DataFrame({
        'x': np.random.randn(100),
        'y': np.random.randn(100)
    })
    print(df.head(10))

with tabs.output_to('Visualization'):
    plt.figure(figsize=(8, 6))
    plt.scatter(df['x'], df['y'], alpha=0.5)
    plt.xlabel('X')
    plt.ylabel('Y')
    plt.title('Scatter Plot')
    plt.show()

with tabs.output_to('Summary', select=False):  # Don't auto-select
    print(df.describe())

# Clear a specific tab
tabs.clear_tab('Data')

# Iterate through tabs
for i in tabs:
    print(f"Content for tab {i}")
```

--------------------------------

### Ask Question and Compare Responses

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Learning_with_Gemini_and_ChatGPT.ipynb

Sends a prompt to both Gemini and ChatGPT, then renders the differences between the responses using difflib.

```python
#@title Ask a question!

text = 'Write a python function that calculates the distance between any two latitudes and longitudes on earth' # @param {type:"string"}

# ask Gemini
model = genai.GenerativeModel('gemini-pro')
chat = model.start_chat(history=[])

response = chat.send_message('%s -- Please answer as concisely as you can, avoiding any extra conversation or text' % text)
gemini_response = response.text

# ask ChatGPT
completion = client.chat.completions.create(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "user", "content": '%s -- Please answer as concisely as you can, avoiding any extra conversation or text' % text}
  ]
)

openai_response = completion.choices[0].message.content

# render the diff

# importing these every execution is unnecessary but avoids another notebook cell
from IPython.display import HTML
import difflib

# omit the legend to slim down the UI
difflib.HtmlDiff._legend = ''

HTML(difflib.HtmlDiff().make_file(gemini_response.splitlines(), openai_response.splitlines(), 'Gemini', 'ChatGPT'))
```

--------------------------------

### Verify Gemma Model Partitioning

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Gemma_Distributed_Fine_tuning_on_TPU.ipynb

Inspects the partitioning of a specific decoder block in the Gemma model. This helps verify how the model weights are distributed across devices.

```python
decoder_block_1 = gemma_lm.backbone.get_layer('decoder_block_1')
print(type(decoder_block_1))
for variable in decoder_block_1.weights:
  print(f'{variable.path:<58}  {str(variable.shape):<16}  {str(variable.value.sharding.spec)}')

```

--------------------------------

### Authenticate with Google Cloud

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Generate_images_with_Gemini_and_Vertex.ipynb

Authenticates the user with Google Cloud and initializes Vertex AI. Requires a Google Cloud project ID.

```python
#@title Authenticate with Google Cloud and your project ID

import vertexai
from vertexai.preview.vision_models import Image, ImageGenerationModel

from google.colab import auth

gcp_project_id = '' # @param {type: "string"}

auth.authenticate_user(project_id=gcp_project_id)

vertexai.init(project=gcp_project_id)
```

--------------------------------

### Use Tags for Selective Output Clearing

Source: https://context7.com/googlecolab/colabtools/llms.txt

Demonstrates using output.use_tags to apply tags to output sections for later selective clearing. Imports are required.

```python
from google.colab import output
import time

# Tag different sections of output
with output.use_tags('header'):
    print("=" * 50)
    print("My Application")
    print("=" * 50)

with output.use_tags('status'):
    for i in range(5):
        output.clear(output_tags='status')
        print(f"Processing step {i+1}/5...")
        time.sleep(1)

with output.use_tags('results'):
    print("Final Results:")
    print("- Accuracy: 95%")
    print("- Time: 5 seconds")

# Clear only the status messages
output.clear(output_tags='status')
```

--------------------------------

### Import Spreadsheet Data and Define Sales Topic

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Sell_lemonade_with_Gemini_and_Sheets.ipynb

Imports data from a specified Google Sheets URL and defines the sales topic. It retrieves all values from the first sheet and prints the first five rows. Ensure the spreadsheet URL is valid and accessible.

```python
#@title Enter the url of a spreadsheet to import and subject to sell

spreadsheet_url = "https://docs.google.com/spreadsheets/d/1E28W9EUvGn90zui8tG13PtnTaVbC_4VkbY1_NEaA2Ec/edit#gid=0" #@param {type:"string"}
sales_topic = "Lemonade" #@param {type:"string"}

worksheet = gc.open_by_url(spreadsheet_url).sheet1

# get_all_values gives a list of rows.
rows = worksheet.get_all_values()
print(rows[0:5])
```

--------------------------------

### Import Keras and Keras NLP

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Gemma_Distributed_Fine_tuning_on_TPU.ipynb

Imports the Keras and Keras NLP libraries after setting the backend.

```python
import keras
import keras_nlp
```

--------------------------------

### Display Gemma Model Summary

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Gemma_Distributed_Fine_tuning_on_TPU.ipynb

Display a summary of the Gemma model architecture, including layers, output shapes, and parameter counts. This helps in understanding the model's structure.

```python
gemma_lm.summary()
```

--------------------------------

### Generate Text with Gemma Model

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Gemma_Distributed_Fine_tuning_on_TPU.ipynb

Generate text using the fine-tuned Gemma model. Specify a prompt and the maximum length of the generated output. This demonstrates the model's text generation capabilities.

```python
gemma_lm.generate("Best comedy movies: ", max_length=256)
```

--------------------------------

### Set Kaggle Credentials for Gemma Access

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Gemma_Distributed_Fine_tuning_on_TPU.ipynb

Configure environment variables with Kaggle username and API key obtained from Kaggle settings. This is required to access Gemma models hosted on Kaggle.

```python
import os
from google.colab import userdata
os.environ["KAGGLE_USERNAME"] = userdata.get('KAGGLE_USERNAME')
os.environ["KAGGLE_KEY"] = userdata.get('KAGGLE_KEY')
```

--------------------------------

### Generate text prompt with Gemini

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Gemini_and_Stable_Diffusion.ipynb

Uses the Gemini model to embellish a user-provided text description into a prompt suitable for Stable Diffusion.

```python
#@title Use Gemini to create a text prompt with to feed to Stable Diffusion

model = genai.GenerativeModel('gemini-pro')

text = 'Draw a kitty cat typing on a computer' # @param {type:"string"}

prompt = "You are creating a prompt for Stable Diffusion to generate an image. Please generate a text prompt for %s. Only respond with the prompt itself, but embellish it as needed but keep it under 80 tokens." % text
response = model.generate_content(prompt)
response.text
```

--------------------------------

### Create and Populate Grid from Data

Source: https://context7.com/googlecolab/colabtools/llms.txt

Uses the create_grid function for a convenient way to generate and populate a grid from data. Requires numpy and create_grid from google.colab.widgets.

```python
from google.colab.widgets import create_grid
import numpy as np

# Create distance matrix visualization
vectors = [np.array([1, 0]), np.array([0, 1]), np.array([1, 1])]
labels = ['v1', 'v2', 'v3']

grid = create_grid(
    row_data=list(zip(labels, vectors)),
    col_data=list(zip(labels, vectors)),
    render=lambda r, c: f"{np.linalg.norm(r[1] - c[1]):.2f}",
    header_render=lambda x: x[0],
    header_row=True,
    header_column=True
)
```

--------------------------------

### Use Polars DataFrame with InteractiveSheet

Source: https://context7.com/googlecolab/colabtools/llms.txt

Demonstrates how to initialize an InteractiveSheet with a Polars DataFrame.

```python
import polars as pl
pl_df = pl.DataFrame({'x': [1, 2, 3], 'y': [4, 5, 6]})
sheet = InteractiveSheet(df=pl_df, backend='polars')
```

--------------------------------

### Set Up Model Parallel Distribution

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Gemma_Distributed_Fine_tuning_on_TPU.ipynb

Configures ModelParallel to shard model weights and activation tensors across devices according to the defined LayoutMap. This enables distributed loading of models like Gemma 7B across multiple TPU cores.

```python
model_parallel = keras.distribution.ModelParallel(
    device_mesh, layout_map, batch_dim_name="batch")

keras.distribution.set_distribution(model_parallel)

```

--------------------------------

### Generate image with Stable Diffusion

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Gemini_and_Stable_Diffusion.ipynb

Executes the Stable Diffusion pipeline using the prompt generated by Gemini and displays the resulting image.

```python
#@title Generate the image with Stable Diffusion

prompt = response.text
seed = random.randint(0, sys.maxsize)

num_inference_steps = 20

images = pipe(
    prompt = prompt,
    guidance_scale = 0.0,
    num_inference_steps = num_inference_steps,
    generator = torch.Generator("cuda").manual_seed(seed),
    ).images

media.show_images(images)
```

--------------------------------

### Generate Image with Vertex AI

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Generate_images_with_Gemini_and_Vertex.ipynb

Generates an image using Vertex AI's image generation model based on a provided prompt. The generated image is saved locally as 'gen-img1.png' and then displayed. Requires the 'response.text' from the previous Gemini prompt generation step.

```python
#@title Use Vertex to generate an image

from IPython.display import Image

model = ImageGenerationModel.from_pretrained("imagegeneration@005")
images = model.generate_images(prompt=response.text)

images[0].save(location="./gen-img1.png", include_generation_parameters=True)

Image('./gen-img1.png', height=500)
```

--------------------------------

### Configure Fine-tuning Hyperparameters

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Gemma_Distributed_Fine_tuning_on_TPU.ipynb

Sets the sequence length and optimizer for the fine-tuning process.

```python
# Fine-tune on the IMDb movie reviews dataset.

# Limit the input sequence length to 128 to control memory usage.
gemma_lm.preprocessor.sequence_length = 128
# Use AdamW (a common optimizer for transformer models).
optimizer = keras.optimizers.AdamW(
    learning_rate=5e-5,
    weight_decay=0.01,
)
```

--------------------------------

### Generate Text with AI Models

Source: https://context7.com/googlecolab/colabtools/llms.txt

Shows basic text generation, streaming responses, using specific models, and generating code.

```python
from google.colab import ai

# Basic text generation
response = ai.generate_text("What is the capital of France?")
print(response)
```

```python
# Streaming response for long outputs
stream = ai.generate_text(
    "Write a short story about a robot learning to paint.",
    stream=True
)
for chunk in stream:
    print(chunk, end='', flush=True)
```

```python
# Use a different model
response = ai.generate_text(
    "Explain quantum entanglement in simple terms.",
    model_name="google/gemini-2.5-flash"
)
print(response)
```

```python
# Code generation example
code_prompt = """
Write a Python function that:
1. Takes a list of numbers
2. Returns the top 3 largest numbers
3. Handles edge cases
"""
code = ai.generate_text(code_prompt)
print(code)
```

--------------------------------

### Set Keras Backend to JAX

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Gemma_Distributed_Fine_tuning_on_TPU.ipynb

Sets the Keras backend to JAX and configures XLA to not preallocate memory. This is necessary for using JAX as the backend for Keras.

```python
import os

# The Keras 3 distribution API is only implemented for the JAX backend for now
os.environ["KERAS_BACKEND"] = "jax"
os.environ["XLA_PYTHON_CLIENT_PREALLOCATE"] = "false"
```

--------------------------------

### Simple Text Generation

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Getting_started_with_google_colab_ai.ipynb

Perform basic text-to-text generation by providing a prompt to the `generate_text` function. This is suitable for straightforward queries and tasks.

```python
# @title Simple batch generation example
# Only text-to-text input/output is supported
from google.colab import ai

response = ai.generate_text("What is the capital of France?")
print(response)
```

--------------------------------

### Download Gemma 7B Model

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Gemma_Distributed_Fine_tuning_on_TPU.ipynb

Downloads the Gemma 7B instruct model using KerasNLP. Ensure you have the necessary permissions and storage.

```python
gemma_lm = keras_nlp.models.GemmaCausalLM.from_preset("gemma_instruct_7b_en")
```

--------------------------------

### Configure Gemini API Key

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Learning_with_Gemini_and_ChatGPT.ipynb

Retrieves the Gemini API key from Colab secrets and initializes the GenerativeModel.

```python
#@title Configure Gemini API key

# access your Gemini API key

import google.generativeai as genai
from google.colab import userdata

gemini_api_secret_name = 'GOOGLE_API_KEY'  # @param {type: "string"}

try:
  GOOGLE_API_KEY=userdata.get(gemini_api_secret_name)
  genai.configure(api_key=GOOGLE_API_KEY)
except userdata.SecretNotFoundError as e:
   print(f'''Secret not found

This expects you to create a secret named {gemini_api_secret_name} in Colab

Visit https://makersuite.google.com/app/apikey to create an API key

Store that in the secrets section on the left side of the notebook (key icon)

Name the secret {gemini_api_secret_name}''')
   raise e
except userdata.NotebookAccessError as e:
  print(f'''You need to grant this notebook access to the {gemini_api_secret_name} secret in order for the notebook to access Gemini on your behalf.''')
  raise e
except Exception as e:
  # unknown error
  print(f"There was an unknown error. Ensure you have a secret {gemini_api_secret_name} stored in Colab and it's a valid key from https://makersuite.google.com/app/apikey")
  raise e

model = genai.GenerativeModel('gemini-pro')
```

--------------------------------

### Manage Output Stream Formatting

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Getting_started_with_google_colab_ai.ipynb

Handles conditional spacing based on punctuation and manages line breaks when the maximum line length is exceeded.

```python
                            "...", "…"                          # Ellipses
                        }
                        if current_word not in no_leading_space_punctuation:
                            sys.stdout.write(' ')
                            self.current_line_length += 1
                    sys.stdout.write(current_word)
                    self.current_line_length += len(current_word)

            if delimiter == '\n':
                sys.stdout.write('\n')
                self.current_line_length = 0
            elif delimiter == ' ':
                # If line is full and a space delimiter arrives, it implies a wrap.
                if self.current_line_length >= self.max_length:
                    sys.stdout.write('\n')
                    self.current_line_length = 0

        sys.stdout.flush()
```

--------------------------------

### Create Grid Layout with Grid Widget

Source: https://context7.com/googlecolab/colabtools/llms.txt

Arranges independent output cells in an NxM grid using the Grid widget. Requires imports for widgets, matplotlib, and numpy. Cells are populated using output_to with row and column indices.

```python
from google.colab import widgets
import matplotlib.pyplot as plt
import numpy as np

# Create a 2x3 grid
grid = widgets.Grid(2, 3)

# Populate individual cells
with grid.output_to(0, 0):
    print("Cell (0,0)")

with grid.output_to(0, 1):
    plt.figure(figsize=(3, 2))
    plt.plot([1, 2, 3], [1, 4, 9])
    plt.title('Line')
    plt.show()

with grid.output_to(0, 2):
    plt.figure(figsize=(3, 2))
    plt.bar(['A', 'B', 'C'], [3, 7, 5])
    plt.title('Bar')
    plt.show()

with grid.output_to(1, 0):
    print("Statistics:")
    print("Mean: 4.67")

with grid.output_to(1, 1):
    plt.figure(figsize=(3, 2))
    plt.pie([30, 40, 30], labels=['X', 'Y', 'Z'])
    plt.show()

# Clear a specific cell
grid.clear_cell(1, 0)

# Iterate through all cells
for row, col in grid:
    print(f"({row},{col})")
```

--------------------------------

### Define Markdown display helper

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Talk_to_Gemini_with_Google's_Speech_to_Text_API.ipynb

Formats text as Markdown blockquotes for cleaner output in Colab.

```python
from IPython.display import Markdown
import textwrap

def to_markdown(text):
  text = text.replace('•', '  *')
  return Markdown(textwrap.indent(text, '> ', predicate=lambda _: True))
```

--------------------------------

### Formatted Streaming Text Generation

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Getting_started_with_google_colab_ai.ipynb

Use this snippet to generate text and display it chunk by chunk as it becomes available. Ensure the LineWrapper class is defined or imported for proper formatting.

```python
from google.colab import ai

wrapper = LineWrapper()
for chunk in ai.generate_text('Give me a long winded description about the evolution of the Roman Empire.', model_name='google/gemini-2.0-flash', stream=True):
  wrapper.print(chunk)
```

--------------------------------

### Generate Text with Gemma Model

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Gemma_Distributed_Fine_tuning_on_TPU.ipynb

Generates text based on a given prompt using the Gemma model. Adjust `max_length` to control the output size.

```python
gemma_lm.generate("Best comedy movies: ", max_length=64)
```

--------------------------------

### Create DeviceMesh for Distributed Computation

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Gemma_Distributed_Fine_tuning_on_TPU.ipynb

Initializes a DeviceMesh with a (1, 8) shape to distribute computation across 8 TPU cores. This is a fundamental step for enabling distributed training with Keras 3's distribution API.

```python
# Create a device mesh with (1, 8) shape so that the weights are sharded across
# all 8 TPUs.
device_mesh = keras.distribution.DeviceMesh(
    (1, 8),
    ["batch", "model"],
    devices=keras.distribution.list_devices())

```

--------------------------------

### List Available AI Models

Source: https://context7.com/googlecolab/colabtools/llms.txt

Retrieves and prints a list of all AI models available for text generation in Colab.

```python
from google.colab import ai

# Get list of available models
models = ai.list_models()
for model in models:
    print(model)
```

--------------------------------

### Generate and Display Sample Visualization in Colab

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/How_to_Use_Slideshow_Mode.ipynb

This Python snippet generates a sample line plot using Matplotlib and displays it as a PNG image within Colab's slideshow mode. It requires imports for numpy, IPython.display, matplotlib, io, and base64.

```python
# @title A More Interesting Output
import numpy as np
import IPython.display as display
from matplotlib import pyplot as plt
import io
import base64

ys = 200 + np.random.randn(100)
x = [x for x in range(len(ys))]

fig = plt.figure(figsize=(4, 3), facecolor='w')
plt.plot(x, ys, '-')
plt.fill_between(x, ys, 195, where=(ys > 195), facecolor='g', alpha=0.6)
plt.title("Sample Visualization", fontsize=10)

data = io.BytesIO()
plt.savefig(data)
image = F"data:image/png;base64,{base64.b64encode(data.getvalue()).decode()}"
alt = "Sample Visualization"
display.display(display.Markdown(F"![{alt}]({image})"))
plt.close(fig)
```

--------------------------------

### Download Kaggle Dataset with Colab Cache

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Colab_Kagglehub_Dataset_Caching.ipynb

Use this snippet to download a Kaggle dataset that is cached by Colab. The download will be nearly instantaneous.

```python
# Copyright 2025 Google LLC.
# SPDX-License-Identifier: Apache-2.0
import kagglehub

dataset_path = kagglehub.dataset_download("elmadafri/the-wildfire-dataset/versions/3")

print("The files are present at:", dataset_path)
```

--------------------------------

### Configure LayoutMap for Tensor Sharding

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Gemma_Distributed_Fine_tuning_on_TPU.ipynb

Defines a LayoutMap to specify how model weights and tensors are sharded across devices. This map uses regex to match tensor paths and assign sharding strategies, such as sharding across 8 TPUs for model dimensions.

```python
model_dim = "model"

layout_map = keras.distribution.LayoutMap(device_mesh)

# Weights that match 'token_embedding/embeddings' will be sharded on 8 TPUs
layout_map["token_embedding/embeddings"] = (None, model_dim)

# Regex to match against the query, key and value matrices in the decoder
# attention layers
layout_map["decoder_block.*attention.*(query|key|value).*kernel"] = (
    None, model_dim, None)
layout_map["decoder_block.*attention_output.*kernel"] = (
    None, None, model_dim)
layout_map["decoder_block.*ffw_gating.*kernel"] = (model_dim, None)
layout_map["decoder_block.*ffw_linear.*kernel"] = (None, model_dim)

```

--------------------------------

### Check for GPU availability

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Gemini_and_Stable_Diffusion.ipynb

Verifies that an NVIDIA GPU is accessible in the current runtime environment.

```python
#@title Check for a GPU
#todo fail if this fails

import os

if os.system('nvidia-smi'):
  raise Exception("No GPU found. Access a GPU through Runtime > Change runtime type and try again.")
```

--------------------------------

### Authenticate with Google Cloud

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Talk_to_Gemini_with_Google's_Speech_to_Text_API.ipynb

Authenticates the user with Google Cloud and specifies the project ID. Ensure you have a Google Cloud account, project, and billing set up.

```python
#@title Authenticate with Google Cloud and your project ID

from google.colab import auth

gcp_project_id = '' # @param {type: "string"}

auth.authenticate_user(project_id=gcp_project_id)
```

--------------------------------

### List Available Models

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Getting_started_with_google_colab_ai.ipynb

Use this function to see all the language models accessible through the google.colab.ai library. This helps in choosing the right model for your task.

```python
# @title List available models
from google.colab import ai

ai.list_models()
```

--------------------------------

### Access List Elements by Index in Python

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/How_to_Use_Slideshow_Mode.ipynb

Demonstrates how to access individual elements of a Python list using zero-based indexing. This is a fundamental list operation.

```python
students = ['Mikayla', 'Julianne', 'Golnar']

# This prints 'Mikayla'
print(students[0])

# This prints 'Julianne'
print(students[1])

# This prints 'Golnar'
print(students[2])
```

--------------------------------

### Markdown for Open in Colab Badge

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/colab-github-demo.ipynb

Use this markdown to create a badge that links to a Colab notebook hosted on GitHub. Replace the notebook URL with your desired notebook.

```markdown
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/googlecolab/colabtools/blob/main/notebooks/colab-github-demo.ipynb)
```

--------------------------------

### Generate Sales Pitches with Gemini

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Sell_lemonade_with_Gemini_and_Sheets.ipynb

Uses Gemini to generate personalized sales pitches based on spreadsheet data. It appends a 'Pitch' column to the data and iterates through each row to create a prompt for Gemini, then formats the responses for display.

```python
#@title Use Gemini to suggest what to say to pitch them

rows[0].append('Pitch')

#skip the title row
for row in rows[1:]:
  prompt = "I'm selling %s. Can you suggest a short paragraph of text for how I might best pitch them on buying? I'm writing a note to %s my %s. Here's some context about them: %s." % (sales_topic, row[0], row[1], row[2])

  response = model.generate_content(prompt)
  row.append(response.text)

#print out the results nicely
from IPython.display import Markdown
import textwrap

def to_markdown(text):
  text = text.replace('•', '  *')
  return Markdown(textwrap.indent(text, '> ', predicate=lambda _: True))

to_markdown(''.join([row[3]+'\n\n' for row in rows[1:]]))
```

--------------------------------

### files.view

Source: https://context7.com/googlecolab/colabtools/llms.txt

Opens a file in Colab's built-in file viewer or opens a directory in the file browser.

```APIDOC
## files.view

### Description
Opens a file in Colab's built-in file viewer, or opens a directory in the file browser.

### Parameters
#### Path Parameters
- **path** (string) - Required - The file path or directory path to view.
```

--------------------------------

### Temporary Output Context Manager

Source: https://context7.com/googlecolab/colabtools/llms.txt

Uses output.temporary to automatically clear all output within the context when exiting. Imports are required.

```python
from google.colab import output
import time

print("This stays visible")

with output.temporary():
    print("Loading...")
    time.sleep(2)
    print("Still loading...")
    time.sleep(2)
# All loading messages are automatically cleared

print("Done! Loading messages are gone.")
```

--------------------------------

### Configure Gemini API Key

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Sell_lemonade_with_Gemini_and_Sheets.ipynb

Configures the Gemini API key for use in the Colab notebook. It expects a secret named 'GOOGLE_API_KEY' to be stored in Colab's secrets manager. Provides instructions on how to create the API key and store it.

```python
#@title Configure Gemini API key

#Access your Gemini API key

import google.generativeai as genai
from google.colab import userdata

gemini_api_secret_name = 'GOOGLE_API_KEY'  # @param {type: "string"}

try:
  GOOGLE_API_KEY=userdata.get(gemini_api_secret_name)
  genai.configure(api_key=GOOGLE_API_KEY)
except userdata.SecretNotFoundError as e:
   print(f'Secret not found\n\nThis expects you to create a secret named {gemini_api_secret_name} in Colab\n\nVisit https://makersuite.google.com/app/apikey to create an API key\n\nStore that in the secrets section on the left side of the notebook (key icon)\n\nName the secret {gemini_api_secret_name}')
   raise e
except userdata.NotebookAccessError as e:
  print(f'You need to grant this notebook access to the {gemini_api_secret_name} secret in order for the notebook to access Gemini on your behalf.')
  raise e
except Exception as e:
  # unknown error
  print(f"There was an unknown error. Ensure you have a secret {gemini_api_secret_name} stored in Colab and it's a valid key from https://makersuite.google.com/app/apikey")
  raise e

model = genai.GenerativeModel('gemini-pro')
```

--------------------------------

### Download File from Colab Runtime

Source: https://context7.com/googlecolab/colabtools/llms.txt

Downloads a file from the Colab runtime to the user's local machine via browser download. Useful for saving results, models, or processed data.

```python
from google.colab import files
import pandas as pd

# Create and save a file
df = pd.DataFrame({
    'name': ['Alice', 'Bob', 'Charlie'],
    'score': [95, 87, 92]
})
df.to_csv('results.csv', index=False)

# Download the file to local machine
files.download('results.csv')

# Download a trained model
import pickle
model = {'weights': [0.1, 0.2, 0.3]}
with open('model.pkl', 'wb') as f:
    pickle.dump(model, f)
files.download('model.pkl')
```

--------------------------------

### Load IMDB Reviews Dataset

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Gemma_Distributed_Fine_tuning_on_TPU.ipynb

Loads the IMDB reviews dataset using TensorFlow Datasets. It configures the dataset for supervised learning and then drops the labels for further processing.

```python
import tensorflow_datasets as tfds

imdb_train = tfds.load(
    "imdb_reviews",
    split="train",
    as_supervised=True,
    batch_size=2,
)
imdb_train = imdb_train.map(lambda x, y: x)

imdb_train.unbatch().take(1).get_single_element().numpy()
```

--------------------------------

### Import spreadsheet data

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Prepare_Christmas_cards_with_Gemini_and_Sheets.ipynb

Loads data from a specified Google Sheet URL into a list of rows.

```python
#@title Enter the name of a spreadsheet to import

spreadsheet_url = "https://docs.google.com/spreadsheets/d/1ZX5Q3BvhOegE33LaIA_zETouvJdQE7AgflkF6lyriCU/edit?usp=sharing" #@param {type:"string"}

worksheet = gc.open_by_url(spreadsheet_url).sheet1

# get_all_values gives a list of rows.
rows = worksheet.get_all_values()
print(rows[0:5])
```

--------------------------------

### Train Gemma Model

Source: https://github.com/googlecolab/colabtools/blob/main/notebooks/Gemma_Distributed_Fine_tuning_on_TPU.ipynb

Train the Gemma model on the IMDb dataset for a specified number of epochs. Ensure the dataset and optimizer are correctly configured before calling fit.

```python
gemma_lm.fit(imdb_train, epochs=1)
```

--------------------------------

### AI List Models API

Source: https://context7.com/googlecolab/colabtools/llms.txt

Lists all available AI models for text generation.

```APIDOC
## GET /ai/list_models

### Description
Lists all available AI models for text generation.

### Method
GET

### Endpoint
/ai/list_models

### Response
#### Success Response (200)
- **models** (array of strings) - A list of available model names.

#### Response Example
```json
{
  "models": [
    "google/gemini-2.5-flash",
    "google/gemini-2.0-flash",
    "other/model"
  ]
}
```
```

--------------------------------

### Unassign Runtime

Source: https://context7.com/googlecolab/colabtools/llms.txt

Programmatically disconnects the notebook from its runtime to free up resources after completing tasks like training and saving models.

```python
from google.colab import runtime

# Train a model
print("Training complete!")
print("Saving results...")

# Save your work
import pickle
with open('model.pkl', 'wb') as f:
    pickle.dump(trained_model, f)

# Download results
from google.colab import files
files.download('model.pkl')

# Free up resources when done
print("Disconnecting runtime to save resources...")
runtime.unassign()
```