### Install Sample Dependencies Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/googlesamples/assistant/grpc/README.rst Install system-level audio libraries and Python requirements. ```bash sudo apt-get install portaudio19-dev libffi-dev libssl-dev pip install --upgrade -r requirements.txt ``` -------------------------------- ### Install Google Assistant SDK Packages Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt Install the gRPC bindings and the SDK with samples. Also includes commands for installing audio dependencies on Debian/Ubuntu. ```bash # Install the gRPC bindings pip install --upgrade google-assistant-grpc ``` ```bash # Install the SDK with samples pip install --upgrade google-assistant-sdk[samples] ``` ```bash # Install audio dependencies on Debian/Ubuntu sudo apt-get install portaudio19-dev libffi-dev libssl-dev ``` -------------------------------- ### Install Audio Test Sample Dependencies Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/README.rst Install necessary system dependencies for the audio test sample, including portaudio development files and libffi. Then, install the Python package using pip. ```bash sudo apt-get install portaudio19-dev libffi-dev libssl-dev pip install --upgrade google-assistant-sdk[samples] ``` -------------------------------- ### Install Google Assistant SDK with Samples Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/README.rst Install the Google Assistant SDK and its associated samples using pip. This command upgrades the package to the latest version. ```bash pip install --upgrade google-assistant-sdk[samples] ``` -------------------------------- ### Verify Audio Setup Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/googlesamples/assistant/grpc/README.rst Run the audio helper module to record and play back a 5-second sample. ```bash # Record a 5 sec sample and play it back python -m audio_helpers ``` -------------------------------- ### Install automation tools Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-grpc/MAINTAINER.md Install the nox tool to manage project automation tasks. ```bash pip install --upgrade nox ``` -------------------------------- ### Install Google Assistant gRPC package Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-grpc/README.rst Use pip to install the required library for the Google Assistant Service. ```bash pip install --upgrade google-assistant-grpc ``` -------------------------------- ### Install package with local modifications Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-grpc/MAINTAINER.md Install the current package in editable mode. ```bash pip install -e . ``` -------------------------------- ### Install and Generate OAuth2 Credentials Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt Install the OAuth tool and generate device credentials using the google-oauthlib-tool. Replace the placeholder path with your actual client secrets file. ```bash # Install the OAuth tool pip install --upgrade google-auth-oauthlib[tool] ``` ```bash # Generate credentials (replace with your client secrets file) google-oauthlib-tool --client-secrets path/to/client_secret_.json \ --scope https://www.googleapis.com/auth/assistant-sdk-prototype \ --save --headless ``` -------------------------------- ### Generate Device Credentials Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/googlesamples/assistant/grpc/README.rst Install the OAuth library and generate credentials using the client secrets file. ```bash pip install --upgrade google-auth-oauthlib[tool] google-oauthlib-tool --client-secrets path/to/client_secret_.json --scope https://www.googleapis.com/auth/assistant-sdk-prototype --save --headless ``` -------------------------------- ### Verify ALSA Audio Setup Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/googlesamples/assistant/grpc/README.rst Use ALSA command-line tools to test playback and recording capabilities. ```bash # Play a test sound speaker-test -t wav # Record and play back some audio using ALSA command-line tools arecord --format=S16_LE --duration=5 --rate=16000 --file-type=raw out.raw aplay --format=S16_LE --rate=16000 --file-type=raw out.raw ``` -------------------------------- ### Generate and load device credentials Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-grpc/README.rst Install the OAuth tool, generate credentials, and load them into the application. ```bash pip install --upgrade google-auth-oauthlib[tool] google-oauthlib-tool --client-secrets path/to/client_secret_.json --scope https://www.googleapis.com/auth/assistant-sdk-prototype --save --headless ``` ```python import io import google.oauth2.credentials with io.open('/path/to/credentials.json', 'r') as f: credentials = google.oauth2.credentials.Credentials(token=None, **json.load(f)) ``` -------------------------------- ### Register New Device Model and Instance Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/README.rst Register a new device model and a corresponding device instance using the devicetool. Replace placeholder values with your specific project ID and device details. This example registers a light device with OnOff trait. ```bash googlesamples-assistant-devicetool --project-id PROJECT_ID register --model 'my-model-identifier' \ --type LIGHT --trait action.devices.traits.OnOff \ --manufacturer 'Assistant SDK developer' \ --product-name 'Assistant SDK light' \ --description 'Assistant SDK light device' \ --device 'my-device-identifier' \ --nickname 'My Assistant Light' ``` -------------------------------- ### Install Python 3 on Ubuntu/Debian Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/googlesamples/assistant/grpc/README.rst Use these commands to install Python 3 and the virtual environment module on Debian-based systems. ```bash sudo apt-get update sudo apt-get install python3 python3-venv ``` -------------------------------- ### Install Package with Local Modifications Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/MAINTAINER.md Install the package in editable mode with sample dependencies. This is useful for development and testing local changes. ```bash pip install -e .[samples] ``` -------------------------------- ### Run Text Input Sample Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt Start an interactive text-based conversation with the Assistant. ```bash # Start interactive text conversation python -m googlesamples.assistant.grpc.textinput \ --device-model-id 'my-model-id' \ --device-id 'my-device-id' ``` -------------------------------- ### Audio Test Utility - Default Recording Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt Run the audio test utility to verify your audio recording and playback setup. This command records for the default duration and plays it back. ```bash googlesamples-assistant-audiotest ``` -------------------------------- ### Define Device Command Handler Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt Register a function to handle specific device commands using the DeviceRequestHandler decorator. This example shows how to handle the 'action.devices.commands.OnOff' command. ```python device_handler = device_helpers.DeviceRequestHandler('my-device') @device_handler.command('action.devices.commands.OnOff') def onoff(on): print('Device turned', 'on' if on else 'off') ``` -------------------------------- ### Initialize and Run Assistant Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt Instantiate the SampleAssistant with necessary configurations and enter a loop to continuously listen for user input and trigger Assistant interactions. ```python assistant = SampleAssistant('en-US', 'my-model', 'my-device', conversation_stream, channel, 185, device_handler) while True: input('Press Enter to speak...') assistant.assist() ``` -------------------------------- ### Create and Configure Virtual Environment Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/googlesamples/assistant/grpc/README.rst Standard procedure to initialize a virtual environment and update core packaging tools. ```bash python3 -m venv env env/bin/python -m pip install --upgrade pip setuptools wheel source env/bin/activate ``` -------------------------------- ### Run Push-to-Talk Sample Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/googlesamples/assistant/grpc/README.rst Execute the push-to-talk sample with various configurations for verbose output, file input, and file output. ```bash python -m pushtotalk --device-id 'my-device-identifier' --device-model-id 'my-model-identifier' ``` ```bash python -m pushtotalk --device-id 'my-device-identifier' --device-model-id 'my-model-identifier' -v ``` ```bash python -m pushtotalk --device-id 'my-device-identifier' --device-model-id 'my-model-identifier' -i in.wav ``` ```bash python -m pushtotalk --device-id 'my-device-identifier' --device-model-id 'my-model-identifier' -o out.wav ``` -------------------------------- ### Register Model and Instance Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt Perform full device registration in a single command. ```bash googlesamples-assistant-devicetool --project-id PROJECT_ID register \ --model 'my-project-my-led1' \ --type LIGHT \ --trait action.devices.traits.OnOff \ --manufacturer 'Assistant SDK developer' \ --product-name 'Assistant SDK light' \ --description 'Assistant SDK light device' \ --device 'my-device-identifier' \ --nickname 'My Assistant Light' \ --client-type SERVICE ``` -------------------------------- ### Run Push-to-Talk Sample Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt Execute voice-based conversations with various input/output configurations. ```bash # Basic usage with microphone input and speaker output googlesamples-assistant-pushtotalk \ --device-model-id 'my-model-id' \ --device-id 'my-device-id' # Run with verbose logging to see gRPC communication googlesamples-assistant-pushtotalk \ --device-model-id 'my-model-id' \ --device-id 'my-device-id' \ -v # Use file input instead of microphone googlesamples-assistant-pushtotalk \ --device-model-id 'my-model-id' \ --device-id 'my-device-id' \ -i input_query.wav # Save response to file googlesamples-assistant-pushtotalk \ --device-model-id 'my-model-id' \ --device-id 'my-device-id' \ -o response.wav # Adjust audio settings for limited throughput devices googlesamples-assistant-pushtotalk \ --device-model-id 'my-model-id' \ --device-id 'my-device-id' \ --audio-block-size=3200 \ --audio-flush-size=6400 ``` -------------------------------- ### Run Push-to-Talk Sample Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/README.rst Execute the push-to-talk sample to interact with the Google Assistant. Requires device model and ID. ```bash googlesamples-assistant-pushtotalk --device-model-id 'my-device-model' --device-id 'my-device-identifier' ``` -------------------------------- ### Run Push-to-Talk Sample in Verbose Mode Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/README.rst Run the push-to-talk sample with verbose output to inspect gRPC communication. Requires device model and ID. ```bash googlesamples-assistant-pushtotalk --device-model-id 'my-device-model' --device-id 'my-device-identifier' -v ``` -------------------------------- ### Run Text and Audio File Input Samples Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/googlesamples/assistant/grpc/README.rst Execute samples that process text input or convert local audio files via the Assistant. ```bash python -m textinput --device-id 'my-device-identifier' --device-model-id 'my-model-identifier' ``` ```bash python -m audiofileinput --device-id 'my-device-identifier' --device-model-id 'my-model-identifier' -i in.wav -o out.wav ``` -------------------------------- ### Record and Playback Audio Samples Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/README.rst Use the googlesamples-assistant-audiotest tool to record 10 seconds of audio and play it back, verifying audio input and output. ```bash googlesamples-assistant-audiotest --record-time 10 ``` -------------------------------- ### Create a new release Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-grpc/MAINTAINER.md Run the release nox session to generate the distribution package. ```bash nox -s release ``` -------------------------------- ### List All Registered Device Models and Instances Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/README.rst List all registered device models or instances associated with your project using the devicetool. Use the --model flag to list models, or omit it to list instances. ```bash googlesamples-assistant-devicetool --project-id PROJECT_ID list --model ``` -------------------------------- ### Configure text-based Assistant request Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt Sets up an AssistConfig object for text queries, including audio output settings and device identification. ```python config = embedded_assistant_pb2.AssistConfig( audio_out_config=embedded_assistant_pb2.AudioOutConfig( encoding='LINEAR16', sample_rate_hertz=16000, volume_percentage=0, # Set to 0 for text-only responses ), dialog_state_in=embedded_assistant_pb2.DialogStateIn( language_code='en-US', conversation_state=conversation_state, is_new_conversation=False, ), device_config=embedded_assistant_pb2.DeviceConfig( device_id='my-device-identifier', device_model_id='my-model-identifier', ), text_query="What time is it?", # Text query instead of audio ) # Enable screen output for visual responses config.screen_out_config.screen_mode = embedded_assistant_pb2.ScreenOutConfig.PLAYING # Single request for text query request = embedded_assistant_pb2.AssistRequest(config=config) ``` -------------------------------- ### Implement SampleAssistant class for voice interaction Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt This class handles the gRPC communication with the Google Assistant API, managing audio streams and device requests. It requires a configured gRPC channel and a device handler for processing actions. ```python import json import logging import grpc import google.auth.transport.grpc import google.auth.transport.requests import google.oauth2.credentials from google.assistant.embedded.v1alpha2 import ( embedded_assistant_pb2, embedded_assistant_pb2_grpc ) from googlesamples.assistant.grpc import audio_helpers, device_helpers ASSISTANT_API_ENDPOINT = 'embeddedassistant.googleapis.com' END_OF_UTTERANCE = embedded_assistant_pb2.AssistResponse.END_OF_UTTERANCE DIALOG_FOLLOW_ON = embedded_assistant_pb2.DialogStateOut.DIALOG_FOLLOW_ON class SampleAssistant: def __init__(self, language_code, device_model_id, device_id, conversation_stream, channel, deadline_sec, device_handler): self.language_code = language_code self.device_model_id = device_model_id self.device_id = device_id self.conversation_stream = conversation_stream self.conversation_state = None self.is_new_conversation = True self.assistant = embedded_assistant_pb2_grpc.EmbeddedAssistantStub(channel) self.deadline = deadline_sec self.device_handler = device_handler def assist(self): """Send a voice request and play back the response.""" continue_conversation = False self.conversation_stream.start_recording() for resp in self.assistant.Assist(self._gen_assist_requests(), self.deadline): if resp.event_type == END_OF_UTTERANCE: self.conversation_stream.stop_recording() if resp.speech_results: print('You said:', ' '.join(r.transcript for r in resp.speech_results)) if len(resp.audio_out.audio_data) > 0: if not self.conversation_stream.playing: self.conversation_stream.start_playback() self.conversation_stream.write(resp.audio_out.audio_data) if resp.dialog_state_out.conversation_state: self.conversation_state = resp.dialog_state_out.conversation_state if resp.dialog_state_out.microphone_mode == DIALOG_FOLLOW_ON: continue_conversation = True if resp.device_action.device_request_json: device_request = json.loads(resp.device_action.device_request_json) self.device_handler(device_request) self.conversation_stream.stop_playback() return continue_conversation def _gen_assist_requests(self): config = embedded_assistant_pb2.AssistConfig( audio_in_config=embedded_assistant_pb2.AudioInConfig( encoding='LINEAR16', sample_rate_hertz=16000), audio_out_config=embedded_assistant_pb2.AudioOutConfig( encoding='LINEAR16', sample_rate_hertz=16000, volume_percentage=self.conversation_stream.volume_percentage), dialog_state_in=embedded_assistant_pb2.DialogStateIn( language_code=self.language_code, conversation_state=self.conversation_state, is_new_conversation=self.is_new_conversation), device_config=embedded_assistant_pb2.DeviceConfig( device_id=self.device_id, device_model_id=self.device_model_id) ) self.is_new_conversation = False yield embedded_assistant_pb2.AssistRequest(config=config) for data in self.conversation_stream: yield embedded_assistant_pb2.AssistRequest(audio_in=data) ``` -------------------------------- ### Register Device Models Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt Manage device model lifecycle using the devicetool CLI. ```bash # Register a new device model googlesamples-assistant-devicetool --project-id PROJECT_ID register-model \ --model 'my-project-my-led1' \ --type LIGHT \ --trait action.devices.traits.OnOff \ --manufacturer 'Assistant SDK developer' \ --product-name 'Assistant SDK light' \ --description 'Assistant SDK light device' # Get device model details googlesamples-assistant-devicetool --project-id PROJECT_ID get --model 'my-model-id' # List all device models googlesamples-assistant-devicetool --project-id PROJECT_ID list --model # Delete a device model googlesamples-assistant-devicetool --project-id PROJECT_ID delete --model 'my-model-id' ``` -------------------------------- ### Register Device Instances Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt Manage device instances associated with specific models. ```bash # Register a new device instance googlesamples-assistant-devicetool --project-id PROJECT_ID register-device \ --device 'my-device-identifier' \ --model 'my-model-identifier' \ --nickname 'My Assistant Light' \ --client-type SERVICE # Get device instance details googlesamples-assistant-devicetool --project-id PROJECT_ID get --device 'my-device-id' # List all device instances googlesamples-assistant-devicetool --project-id PROJECT_ID list --device # Delete a device instance googlesamples-assistant-devicetool --project-id PROJECT_ID delete --device 'my-device-id' ``` -------------------------------- ### Show Device Tool CLI Usage Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/README.rst Display the help message for the googlesamples-assistant-devicetool to understand its available commands and options. ```bash googlesamples-assistant-devicetool --help ``` -------------------------------- ### Run Unit Tests with Nox Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/MAINTAINER.md Run the unit tests for the project using nox. This command executes the test suite. ```bash nox -s unittest ``` -------------------------------- ### Configure AssistConfig for Text Query Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt Prepare AssistConfig for text-based conversations. This configuration omits audio input settings. ```python from google.assistant.embedded.v1alpha2 import embedded_assistant_pb2 ``` -------------------------------- ### Perform File-Based Audio I/O Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt Use WaveSource and WaveSink to process audio from and to WAV files. ```python from googlesamples.assistant.grpc.audio_helpers import WaveSource, WaveSink, ConversationStream # Read audio from a WAV file audio_source = WaveSource( fp=open('input_query.wav', 'rb'), sample_rate=16000, sample_width=2 ) # Write audio to a WAV file audio_sink = WaveSink( fp=open('assistant_response.wav', 'wb'), sample_rate=16000, sample_width=2 ) # Create stream for file-based conversation conversation_stream = ConversationStream( source=audio_source, sink=audio_sink, iter_size=3200, sample_width=2 ) # Process the conversation conversation_stream.start_recording() for audio_chunk in conversation_stream: # Send to Assistant pass conversation_stream.stop_recording() conversation_stream.start_playback() # Write response audio conversation_stream.write(response_audio_data) conversation_stream.stop_playback() conversation_stream.close() ``` -------------------------------- ### Initialize gRPC stubs Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-grpc/README.rst Create the EmbeddedAssistantStub using the established gRPC channel. ```python import google.assistant.embedded.v1alpha1.embedded_assistant_pb2_grpc assistant = embedded_assistant_pb2.EmbeddedAssistantStub(channel) ``` -------------------------------- ### Enable HTML Display Output Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt Use this command to enable HTML display output for text input interactions with the Assistant SDK. Ensure you replace 'my-model-id' and 'my-device-id' with your actual device credentials. ```bash python -m googlesamples.assistant.grpc.textinput \ --device-model-id 'my-model-id' \ --device-id 'my-device-id' \ --display ``` -------------------------------- ### Configure AssistConfig for Audio Input Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt Set up AssistConfig for voice-based conversations, specifying audio input and output formats, language, and device information. The first request should contain the config, followed by requests with audio data. ```python from google.assistant.embedded.v1alpha2 import embedded_assistant_pb2 # Create configuration for audio-based request config = embedded_assistant_pb2.AssistConfig( audio_in_config=embedded_assistant_pb2.AudioInConfig( encoding='LINEAR16', # Supports: LINEAR16, FLAC sample_rate_hertz=16000, ), audio_out_config=embedded_assistant_pb2.AudioOutConfig( encoding='LINEAR16', # Supports: LINEAR16, MP3, OPUS_IN_OGG sample_rate_hertz=16000, volume_percentage=50, ), dialog_state_in=embedded_assistant_pb2.DialogStateIn( language_code='en-US', conversation_state=None, # Pass previous state for follow-up queries is_new_conversation=True, ), device_config=embedded_assistant_pb2.DeviceConfig( device_id='my-device-identifier', device_model_id='my-model-identifier', ) ) # First request contains config only yield embedded_assistant_pb2.AssistRequest(config=config) # Subsequent requests contain audio data for audio_chunk in acquire_audio_data(): yield embedded_assistant_pb2.AssistRequest(audio_in=audio_chunk) ``` -------------------------------- ### Initialize audio device stream Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt Configures the SoundDeviceStream for managing audio input and output parameters. ```python from googlesamples.assistant.grpc.audio_helpers import ( ConversationStream, SoundDeviceStream, WaveSource, WaveSink, DEFAULT_AUDIO_SAMPLE_RATE, DEFAULT_AUDIO_SAMPLE_WIDTH, DEFAULT_AUDIO_ITER_SIZE, DEFAULT_AUDIO_DEVICE_BLOCK_SIZE, DEFAULT_AUDIO_DEVICE_FLUSH_SIZE ) # Create audio device stream audio_device = SoundDeviceStream( sample_rate=16000, sample_width=2, block_size=6400, flush_size=25600 ) ``` -------------------------------- ### Manage Conversation Stream Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt Initialize and control audio recording and playback streams for the Assistant. ```python # Create conversation stream for recording and playback conversation_stream = ConversationStream( source=audio_device, sink=audio_device, iter_size=3200, sample_width=2, ) # Conversation flow conversation_stream.start_recording() for audio_data in conversation_stream: yield embedded_assistant_pb2.AssistRequest(audio_in=audio_data) conversation_stream.stop_recording() # Play back response conversation_stream.start_playback() conversation_stream.write(response.audio_out.audio_data) conversation_stream.stop_playback() # Set volume (1-100%) conversation_stream.volume_percentage = 75 # Clean up conversation_stream.close() ``` -------------------------------- ### List Devices with Assistant SDK Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/README.rst Use this command to list devices associated with your Google Assistant SDK project. ```bash googlesamples-assistant-devicetool --project-id PROJECT_ID list --device ``` -------------------------------- ### Run Lint Tool with Nox Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/MAINTAINER.md Execute the linting process using nox. This command checks code quality and style. ```bash nox -s lint ``` -------------------------------- ### Adjust Audio Block and Flush Sizes Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/README.rst Configure the audio block size and flush size for the audio test tool to optimize performance on soundcards with limited throughput. ```bash googlesamples-assistant-audiotest --record-time 10 --audio-block-size=3200 --audio-flush-size=6400 ``` -------------------------------- ### Audio Test Utility - Adjust Audio Parameters Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt For devices with limited throughput, adjust the audio block and flush sizes using the --audio-block-size and --audio-flush-size flags to optimize performance. ```bash googlesamples-assistant-audiotest \ --record-time 10 \ --audio-block-size=3200 \ --audio-flush-size=6400 ``` -------------------------------- ### Create EmbeddedAssistant gRPC Stub and Call Assist Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt Create a gRPC stub for the EmbeddedAssistant service and initiate a call to the Assist streaming method. This method takes a generator of requests and returns a generator of responses. ```python from google.assistant.embedded.v1alpha2 import ( embedded_assistant_pb2, embedded_assistant_pb2_grpc ) # Create the gRPC stub assistant = embedded_assistant_pb2_grpc.EmbeddedAssistantStub(grpc_channel) # Call the Assist streaming method # Takes a generator of AssistRequest and returns a generator of AssistResponse assist_responses = assistant.Assist(assist_requests_generator, deadline_sec) for response in assist_responses: # Process each response pass ``` -------------------------------- ### Regenerate python gRPC stubs Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-grpc/MAINTAINER.md Clone the googleapis repository and run the protoc nox session to update stubs. ```bash git clone https://github.com/googleapis/googleapis nox -s protoc ``` -------------------------------- ### Authenticate and Authorize Channel Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt This snippet demonstrates how to load credentials, refresh them, and establish a secure gRPC channel for interacting with the Assistant API. Ensure 'credentials.json' is available and contains valid OAuth2 tokens. ```python with open('credentials.json', 'r') as f: credentials = google.oauth2.credentials.Credentials(token=None, **json.load(f)) http_request = google.auth.transport.requests.Request() credentials.refresh(http_request) channel = google.auth.transport.grpc.secure_authorized_channel( credentials, http_request, ASSISTANT_API_ENDPOINT) ``` -------------------------------- ### Configure Audio Stream for Conversation Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt Set up audio streams for recording and playback using SoundDeviceStream and ConversationStream. This configuration is essential for voice-based interactions. ```python audio_device = audio_helpers.SoundDeviceStream( sample_rate=16000, sample_width=2, block_size=6400, flush_size=25600) conversation_stream = audio_helpers.ConversationStream( source=audio_device, sink=audio_device, iter_size=3200, sample_width=2) ``` -------------------------------- ### Verify Device Registration Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/README.rst Verify that a specific device model or instance has been correctly registered by retrieving its details using the devicetool. ```bash googlesamples-assistant-devicetool --project-id PROJECT_ID get --model 'my-model-identifier' googlesamples-assistant-devicetool --project-id PROJECT_ID get --device 'my-device-identifier' ``` -------------------------------- ### Call the Assist streaming method Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-grpc/README.rst Initiate the Assist streaming request using a generator. ```python assist_responses_generator = assistant.Assist(assist_requests_generator) start_acquiring_audio() ``` -------------------------------- ### Register or Overwrite Device Model Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/README.rst Register or update a device model with specified traits using the devicetool. The model identifier must be globally unique. ```bash googlesamples-assistant-devicetool --project-id PROJECT_ID register-model --model 'my-model-identifier' \ --type LIGHT --trait action.devices.traits.OnOff \ --manufacturer 'Assistant SDK developer' \ --product-name 'Assistant SDK light' \ --description 'Assistant SDK light device' ``` -------------------------------- ### Adjust Audio Block Size Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/googlesamples/assistant/grpc/README.rst Configure the audio block size to resolve choppy audio issues based on the hardware output. ```bash # If using a USB speaker or dedicated soundcard, set block size to "0" # to automatically adjust the buffer size python -m audio_helpers --audio-block-size=0 # If using the line-out 3.5mm audio jack on the device, set block size # to a value larger than the `ConverseResponse` audio payload size python -m audio_helpers --audio-block-size=3200 # Run the Assistant sample using the best block size value found above python -m pushtotalk --audio-block-size=value ``` -------------------------------- ### Adjust Audio Flush Size Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/googlesamples/assistant/grpc/README.rst Configure the flush size to prevent audio truncation. ```bash # Set flush size to a value larger than the audio block size. You can # run the sample using the --audio-flush-size flag as well. python -m audio_helpers --audio-block-size=3200 --audio-flush-size=6400 ``` -------------------------------- ### Load OAuth2 Credentials and Create gRPC Channel Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt Load stored OAuth2 credentials from a JSON file and create an authorized gRPC channel for communication with the Google Assistant Service API. ```python import json import google.oauth2.credentials import google.auth.transport.grpc import google.auth.transport.requests # Load OAuth 2.0 credentials from file with open('/path/to/credentials.json', 'r') as f: credentials = google.oauth2.credentials.Credentials(token=None, **json.load(f)) http_request = google.auth.transport.requests.Request() credentials.refresh(http_request) # Create an authorized gRPC channel ASSISTANT_API_ENDPOINT = 'embeddedassistant.googleapis.com' grpc_channel = google.auth.transport.grpc.secure_authorized_channel( credentials, http_request, ASSISTANT_API_ENDPOINT ) ``` -------------------------------- ### Set Language for Text Input Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt This command allows you to specify a different language for text input interactions. Replace 'my-model-id' and 'my-device-id' with your device credentials and 'ja-JP' with the desired language code. ```bash python -m googlesamples.assistant.grpc.textinput \ --device-model-id 'my-model-id' \ --device-id 'my-device-id' \ --lang 'ja-JP' ``` -------------------------------- ### Register or Overwrite Device Instance Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-sdk/README.rst Register or update a device instance, associating it with a previously registered device model using the devicetool. The device instance identifier should be unique within your Google Developer Project. ```bash googlesamples-assistant-devicetool --project-id PROJECT_ID register-device --device 'my-device-identifier' \ --model 'my-model-identifier' \ --nickname 'My Assistant Light' ``` -------------------------------- ### Process AssistResponse messages Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt Handles incoming responses from the Assistant, including audio playback, transcripts, conversation state, and device actions. ```python from google.assistant.embedded.v1alpha2 import embedded_assistant_pb2 END_OF_UTTERANCE = embedded_assistant_pb2.AssistResponse.END_OF_UTTERANCE DIALOG_FOLLOW_ON = embedded_assistant_pb2.DialogStateOut.DIALOG_FOLLOW_ON CLOSE_MICROPHONE = embedded_assistant_pb2.DialogStateOut.CLOSE_MICROPHONE conversation_state = None continue_conversation = False for response in assistant.Assist(request_generator, deadline=185): # Detect end of user speech if response.event_type == END_OF_UTTERANCE: print('End of audio request detected') stop_recording() # Get user speech transcript if response.speech_results: transcript = ' '.join(r.transcript for r in response.speech_results) print(f'User said: {transcript}') # Play back Assistant audio response if len(response.audio_out.audio_data) > 0: playback_audio_data(response.audio_out.audio_data) # Get text response from Assistant if response.dialog_state_out.supplemental_display_text: print(f'Assistant: {response.dialog_state_out.supplemental_display_text}') # Save conversation state for follow-up queries if response.dialog_state_out.conversation_state: conversation_state = response.dialog_state_out.conversation_state # Check if Assistant expects follow-up if response.dialog_state_out.microphone_mode == DIALOG_FOLLOW_ON: continue_conversation = True elif response.dialog_state_out.microphone_mode == CLOSE_MICROPHONE: continue_conversation = False # Handle volume changes if response.dialog_state_out.volume_percentage != 0: set_volume(response.dialog_state_out.volume_percentage) # Handle screen output (HTML) if response.screen_out.data: display_html(response.screen_out.data) # Handle device actions if response.device_action.device_request_json: device_request = json.loads(response.device_action.device_request_json) handle_device_action(device_request) ``` -------------------------------- ### Register custom device action handlers Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt Uses the DeviceRequestHandler decorator to map Assistant commands to local device functions. ```python from googlesamples.assistant.grpc.device_helpers import DeviceRequestHandler import json import time # Create handler for specific device device_handler = DeviceRequestHandler('my-device-identifier') # Register handler for built-in OnOff trait @device_handler.command('action.devices.commands.OnOff') def onoff(on): if on: print('Turning device on') # Control your hardware here else: print('Turning device off') # Register handler for custom action (defined in actions.json) @device_handler.command('com.example.commands.BlinkLight') def blink(speed, number): print(f'Blinking device {number} times') delay = {'SLOWLY': 2, 'NORMALLY': 1, 'QUICKLY': 0.5}.get(speed, 1) for i in range(int(number)): print('Device is blinking.') time.sleep(delay) # Process device action from AssistResponse if response.device_action.device_request_json: device_request = json.loads(response.device_action.device_request_json) futures = device_handler(device_request) # Wait for all device actions to complete if futures: concurrent.futures.wait(futures) ``` -------------------------------- ### Handle AssistResponse stream Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-grpc/README.rst Process incoming responses to manage recording state, transcriptions, and audio output. ```python for assist_response in assist_response_generator: if resp.event_type == END_OF_UTTERANCE: stop_acquiring_audio() if resp.speech_results: print(' '.join(r.transcript for r in resp.speech_results) if resp.dialog_state_out.supplemental_display_text: print(resp.dialog_state_out.supplemental_display_text) if len(resp.audio_out.audio_data) > 0: playback_audio_data(resp.audio_out.audio_data) ``` -------------------------------- ### Generate AssistRequest messages Source: https://github.com/googlesamples/assistant-sdk-python/blob/master/google-assistant-grpc/README.rst Define a generator function to yield configuration and audio data requests. ```python import google.assistant.embedded.v1alpha1.embedded_assistant_pb2 def generate_assist_requests(): yield embedded_assistant_pb2.AssistConfig( audio_in_config=embedded_assistant_pb2.AudioInConfig( encoding='LINEAR16', sample_rate_hertz=16000, ), audio_out_config=embedded_assistant_pb2.AudioOutConfig( encoding='LINEAR16', sample_rate_hertz=16000, ), device_config=embedded_assistant_pb2.DeviceConfig( device_id=device_id, device_model_id=device_model_id, ) ) for data in acquire_audio_data(): yield embedded_assistant_pb2.AssistRequest(audio_in=data) ``` -------------------------------- ### Custom Actions Definition - actions.json Source: https://context7.com/googlesamples/assistant-sdk-python/llms.txt This JSON defines custom device actions for your Google Assistant device. It includes manifest details, action intents, query patterns, and fulfillment logic. ```json { "manifest": { "displayName": "Blinky light", "invocationName": "Blinky light", "category": "PRODUCTIVITY" }, "actions": [ { "name": "com.example.actions.BlinkLight", "availability": { "deviceClasses": [ { "assistantSdkDevice": {} } ] }, "intent": { "name": "com.example.intents.BlinkLight", "parameters": [ { "name": "number", "type": "SchemaOrg_Number" }, { "name": "speed", "type": "Speed" } ], "trigger": { "queryPatterns": [ "blink ($Speed:speed)? $SchemaOrg_Number:number times", "blink $SchemaOrg_Number:number times ($Speed:speed)?" ] } }, "fulfillment": { "staticFulfillment": { "templatedResponse": { "items": [ { "simpleResponse": { "textToSpeech": "Blinking $number times" } }, { "deviceExecution": { "command": "com.example.commands.BlinkLight", "params": { "speed": "$speed", "number": "$number" } } } ] } } } } ], "types": [ { "name": "$Speed", "entities": [ {"key": "SLOWLY", "synonyms": ["slowly", "slow"]}, {"key": "NORMALLY", "synonyms": ["normally", "regular"]}, {"key": "QUICKLY", "synonyms": ["quickly", "fast", "quick"]} ] } ] } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.