### Install Nginx Source: https://docs.mentraglass.com/app-devs/getting-started/deployment/ubuntu-deployment Commands to install Nginx and ensure it starts on boot. ```bash # Install Nginx sudo apt install -y nginx # Enable Nginx to start on boot and start it now sudo systemctl enable --now nginx # Verify Nginx is running sudo systemctl status nginx ``` -------------------------------- ### Install dependencies and run the app Source: https://docs.mentraglass.com/app-devs/getting-started/quickstart Commands to install dependencies, set up environment variables, and run the development server. ```bash cd bun install ``` ```bash cp .env.example .env ``` ```bash bun run dev ``` ```bash ngrok http --url= 3000 ``` -------------------------------- ### Quick Start Source: https://docs.mentraglass.com/app-devs/core-concepts/simple-storage Demonstrates basic usage of setting, getting, and checking for keys in simple storage. ```typescript protected async onSession(session: AppSession, sessionId: string, userId: string) { // Store data await session.simpleStorage.set('theme', 'dark'); // Retrieve data const theme = await session.simpleStorage.get('theme'); // Check if exists const hasTheme = await session.simpleStorage.hasKey('theme'); } ``` -------------------------------- ### startLivestream() Example Source: https://docs.mentraglass.com/app-devs/reference/managers/camera Examples demonstrating how to start a managed livestream using default WebRTC or with restream destinations for SRT/HLS/DASH. ```typescript // Default: WebRTC for low-latency playback const result = await session.camera.startLivestream(); console.log('WebRTC URL:', result.webrtcUrl); // With restream destinations: SRT + HLS/DASH const result = await session.camera.startLivestream({ restreamDestinations: [ { url: 'rtmp://a.rtmp.youtube.com/live2/YOUR-KEY', name: 'YouTube' } ], video: { frameRate: 30 }, audio: { sampleRate: 48000 } }); console.log('HLS URL:', result.hlsUrl); ``` -------------------------------- ### Setting Up the Cloud Backend Source: https://docs.mentraglass.com/os-devs/contributing/overview Instructions for setting up the MentraOS cloud backend, including dependency installation and starting the development environment. ```bash cd MentraOS/cloud # Install dependencies bun install # Setup Docker network bun run dev:setup-network # Start development environment ./scripts/docker-setup.sh # or bun run setup-deps bun run dev ``` -------------------------------- ### Example of starting unmanaged livestreams Source: https://docs.mentraglass.com/app-devs/reference/managers/camera Demonstrates how to start unmanaged livestreams using different protocols and advanced configurations. ```typescript // SRT stream await session.camera.startLocalLivestream({ streamUrl: 'srt://my-server.com:4201?streamid=my-key' }); // RTMP stream await session.camera.startLocalLivestream({ streamUrl: 'rtmp://live.example.com/stream/key' }); // WHIP (WebRTC) stream await session.camera.startLocalLivestream({ streamUrl: 'https://whip.example.com/ingest/endpoint' }); // Advanced configuration await session.camera.startLocalLivestream({ streamUrl: 'srt://my-server.com:4201?streamid=key', video: { width: 1920, height: 1080, bitrate: 5000000, frameRate: 30 }, audio: { bitrate: 128000, sampleRate: 44100 }, stream: { durationLimit: 1800 } }); ``` -------------------------------- ### Setting Up the Manager App Source: https://docs.mentraglass.com/os-devs/contributing/overview Instructions for cloning the repository, installing dependencies, and starting the development server for the MentraOS Manager app. ```bash # Clone the repository git clone https://github.com/Mentra-Community/MentraOS.git cd MentraOS/mobile # Install dependencies npm install # For iOS cd ios && pod install && cd .. # Start the development server npm start # Run on Android/iOS npm run android # or npm run ios ``` -------------------------------- ### get Method Example Source: https://docs.mentraglass.com/app-devs/reference/managers/settings-manager Illustrates how to retrieve a setting value using the get method, including examples with and without a default value. ```typescript // Get with default value const fontSize = session.settings.get('font_size', 16); // Get without default (returns undefined if not found) const theme = session.settings.get('theme'); ``` -------------------------------- ### Environment Setup Source: https://docs.mentraglass.com/os-devs/contributing/mentraos-asg-client-guidelines Copy the example environment file and configure local development settings for the MentraOS host, port, and security. ```bash # Copy environment file cp .env.example .env # For local development, modify .env: mentraos_HOST=localhost # or your computer's IP mentraos_PORT=8002 mentraos_SECURE=false ``` -------------------------------- ### Enable and Start Systemd Service Source: https://docs.mentraglass.com/app-devs/getting-started/deployment/ubuntu-deployment Commands to reload systemd, enable the service to start on boot, and start it immediately. ```bash # Reload systemd to recognize the new service sudo systemctl daemon-reload # Enable the service to start on boot and start it now sudo systemctl enable --now myapp # Check service status sudo systemctl status myapp ``` -------------------------------- ### Install Bun Runtime Source: https://docs.mentraglass.com/app-devs/getting-started/deployment/ubuntu-deployment Downloads and installs the Bun JavaScript runtime, reloads the shell, and verifies the installation. ```bash # Download and install Bun curl -fsSL https://bun.sh/install | bash # Reload your shell to update PATH exec $SHELL # Verify installation bun --version ``` -------------------------------- ### Building and Installing APK Source: https://docs.mentraglass.com/os-devs/asg-client/mentra-live Commands to build the debug APK for the ASG Client and install it on the connected glasses. ```bash # Navigate to asg_client directory cd /path/to/AugmentOS/asg_client # Build debug APK ./gradlew assembleDebug # Install on connected glasses adb install -r app/build/outputs/apk/debug/app-debug.apk ``` -------------------------------- ### Update Pairing Guide Utility Source: https://docs.mentraglass.com/os-devs/contributing/add-new-glasses-support Example of updating the getPairingGuide utility to return the custom pairing guide component for the new glasses. ```javascript case "Your Glasses Model Name": return ; ``` -------------------------------- ### Example Get MentraOS Setting Source: https://docs.mentraglass.com/app-devs/reference/managers/settings-manager Examples demonstrating how to get MentraOS settings with and without default values. ```typescript const metricEnabled = session.settings.getMentraosSetting( 'metricSystemEnabled', false ); const userLocale = session.settings.getMentraosSetting( 'locale', 'en-US' ); ``` -------------------------------- ### Create Pairing Guide Component Source: https://docs.mentraglass.com/os-devs/contributing/add-new-glasses-support Example of creating a React component for the pairing guide specific to the new glasses. ```javascript const YourGlassesPairingGuide = () => ( How to pair Your Glasses: 1. Turn on your glasses{"\n"} 2. Enable Bluetooth pairing mode{"\n"} 3. Select your glasses from the list ) ``` -------------------------------- ### checkExistingStream() Example Source: https://docs.mentraglass.com/app-devs/reference/managers/camera An example demonstrating how to check for an existing stream and take appropriate action, such as joining a managed stream or starting a new one. ```typescript const streamCheck = await session.camera.checkExistingStream(); if (streamCheck.hasActiveStream) { if (streamCheck.streamInfo?.type === 'managed') { console.log('HLS URL:', streamCheck.streamInfo.hlsUrl); // Join existing managed stream const result = await session.camera.startLivestream(); } else { console.log('Active stream:', streamCheck.streamInfo?.streamUrl); console.log('Requested by:', streamCheck.streamInfo?.requestingAppId); } } else { const result = await session.camera.startLivestream(); } ``` -------------------------------- ### Start your app locally Source: https://docs.mentraglass.com/app-devs/getting-started/deployment/local-development Commands to install dependencies and start the development server for your MentraOS app. ```bash # In your app directory bun install bun run dev # Your app is now running on http://localhost:3000 ``` -------------------------------- ### Starting a Livestream Source: https://docs.mentraglass.com/app-devs/core-concepts/camera/README Example of how to start a managed livestream, which provides a WebRTC playback URL. ```typescript const result = await session.camera.startLivestream(); console.log('WebRTC playback URL:', result.webrtcUrl); ``` -------------------------------- ### SelectSetting Example Source: https://docs.mentraglass.com/app-devs/reference/interfaces/setting-types Example of a SelectSetting. ```json { "type": "select", "key": "theme", "label": "Color Theme", "options": [ { "label": "Light", "value": "light" }, { "label": "Dark", "value": "dark" }, { "label": "Auto", "value": "auto" } ], "defaultValue": "auto" } ``` -------------------------------- ### Quick Start: Subscribing to Events Source: https://docs.mentraglass.com/app-devs/core-concepts/app-session/events-and-data This example demonstrates how to subscribe to voice transcription and button press events from a smart glass session. ```typescript protected async onSession(session: AppSession, sessionId: string, userId: string) { // Subscribe to voice transcription session.events.onTranscription((data) => { session.logger.info('User said:', data.text); }); // Subscribe to button presses session.events.onButtonPress((data) => { session.logger.info('Button pressed:', data.button); }); } ``` -------------------------------- ### SelectWithSearchSetting Example Source: https://docs.mentraglass.com/app-devs/reference/interfaces/setting-types Example of a SelectWithSearchSetting. ```json { "type": "select_with_search", "key": "country", "label": "Country", "options": [ { "label": "United States", "value": "US" }, { "label": "United Kingdom", "value": "UK" }, { "label": "Canada", "value": "CA" } // ... many more options ], "defaultValue": "US" } ``` -------------------------------- ### Local Server Development Setup Source: https://docs.mentraglass.com/os-devs/asg-client/mentra-live Configuration for testing with a local MentraOS server by forwarding ports and setting environment variables. ```bash # Forward local port to glasses # This makes localhost:8002 on glasses connect to your PC adb reverse tcp:8002 tcp:8002 # Configure .env file: MENTRAOS_HOST=localhost MENTRAOS_PORT=8002 MENTRAOS_SECURE=false ``` -------------------------------- ### TextSetting Example Source: https://docs.mentraglass.com/app-devs/reference/interfaces/setting-types Example of a TextSetting. ```json { "type": "text", "key": "user_name", "label": "Your Name", "defaultValue": "User" } ``` -------------------------------- ### DNS A Record Example Source: https://docs.mentraglass.com/app-devs/getting-started/deployment/ubuntu-deployment Example of an A record configuration for pointing a domain to a server IP address. ```text myapp.example.org → 203.555.113.42 (your server IP) ``` -------------------------------- ### Complete Example Source: https://docs.mentraglass.com/app-devs/core-concepts/simple-storage A full example of using Simple Storage in an app. ```typescript class NotesApp extends AppServer { protected async onSession(session: AppSession, sessionId: string, userId: string) { // Load saved notes const notesJson = await session.simpleStorage.get('notes'); const notes = notesJson ? JSON.parse(notesJson) : []; session.layouts.showTextWall(`You have ${notes.length} notes`); // Listen for voice input to create notes session.events.onTranscription(async (data) => { if (data.isFinal && data.text.startsWith('note:')) { const noteText = data.text.replace('note:', '').trim(); // Add note notes.push({ text: noteText, timestamp: Date.now() }); // Save await session.simpleStorage.set('notes', JSON.stringify(notes)); session.layouts.showTextWall('Note saved!'); } }); } } ``` -------------------------------- ### start() Source: https://docs.mentraglass.com/app-devs/reference/app-server Starts the app server, making it listen for incoming webhook requests. ```typescript start(): Promise ``` -------------------------------- ### SliderSetting Example Source: https://docs.mentraglass.com/app-devs/reference/interfaces/setting-types Example of a SliderSetting. ```json { "type": "slider", "key": "volume", "label": "Volume", "min": 0, "max": 100, "defaultValue": 50 } ``` -------------------------------- ### Import and Usage Example Source: https://docs.mentraglass.com/app-devs/reference/managers/settings-manager Demonstrates how to import AppServer and AppSession, and how to access and use the SettingsManager within an AppSession, including getting a setting and listening for changes. ```typescript import { AppServer, AppSession } from '@mentra/sdk'; export class MyAppServer extends AppServer { protected async onSession(session: AppSession, sessionId: string, userId: string): Promise { // get the settings manager object const settingsManager = session.settings; // Get a specific setting value const language = settingsManager.get('transcribe_language', 'English'); // Listen for setting changes settingsManager.onValueChange('line_width', (newValue, oldValue) => { console.log(`Line width changed from ${oldValue} to ${newValue}`); this.updateDisplay(newValue); }); } } ``` -------------------------------- ### Best Practice: Type Your Settings Source: https://docs.mentraglass.com/app-devs/reference/managers/settings-manager Example demonstrating type safety by using generics when getting settings. ```typescript // Good: Use generics for type safety const enabled = session.settings.get('feature_enabled', false); const theme = session.settings.get<'light' | 'dark'>('theme', 'light'); // Avoid: Using 'any' loses type safety const value = session.settings.get('some_setting'); ``` -------------------------------- ### ToggleSetting Example Source: https://docs.mentraglass.com/app-devs/reference/interfaces/setting-types Example of a ToggleSetting. ```json { "type": "toggle", "key": "enable_notifications", "label": "Enable Notifications", "defaultValue": true } ``` -------------------------------- ### MultiselectSetting Example Source: https://docs.mentraglass.com/app-devs/reference/interfaces/setting-types Example of a MultiselectSetting. ```json { "type": "multiselect", "key": "features", "label": "Enabled Features", "options": [ { "label": "Auto-save", "value": "autosave" }, { "label": "Spell Check", "value": "spellcheck" }, { "label": "Dark Mode", "value": "darkmode" } ], "defaultValue": ["autosave"] } ``` -------------------------------- ### Streaming Control Source: https://docs.mentraglass.com/app-devs/core-concepts/app-session/device-control This example shows how to start and stop video streaming based on voice commands, including checking for camera availability and providing user feedback. ```typescript class StreamingApp extends AppServer { private isRecording = false; protected async onSession(session: AppSession, sessionId: string, userId: string) { if (!session.capabilities?.hasCamera) { session.layouts.showTextWall('Camera not available'); return; } session.layouts.showTextWall('Say "start" or "stop"'); session.events.onTranscription(async (data) => { if (!data.isFinal) return; const text = data.text.toLowerCase(); if (text.includes('start') && !this.isRecording) { await this.startRecording(session); } else if (text.includes('stop') && this.isRecording) { await this.stopRecording(session); } }); // Monitor stream status session.camera.onLocalLivestreamStatus((status) => { session.dashboard.content.writeToMain(`Stream: ${status.status}`); }); } private async startRecording(session: AppSession) { try { const result = await session.camera.startLivestream({ video: { resolution: '1280x720', framerate: 30 } }); this.isRecording = true; session.layouts.showTextWall('Recording started'); await session.audio.speak('Recording'); session.logger.info('Stream URL:', result.viewerUrl); } catch (error) { session.logger.error('Failed to start stream:', error); await session.audio.speak('Failed to start recording'); } } private async stopRecording(session: AppSession) { await session.camera.stopLivestream(); this.isRecording = false; session.layouts.showTextWall('Recording stopped'); await session.audio.speak('Recording stopped'); } } ``` -------------------------------- ### getAll Method Example Source: https://docs.mentraglass.com/app-devs/reference/managers/settings-manager Shows how to retrieve all settings and provides an example of filtering settings by type. ```typescript const allSettings = session.settings.getAll(); console.log(`Total settings: ${allSettings.length}`); // Filter specific types const toggles = allSettings.filter(s => s.type === 'toggle'); ``` -------------------------------- ### Install Certbot Source: https://docs.mentraglass.com/app-devs/getting-started/deployment/ubuntu-deployment Commands to install Certbot for obtaining SSL certificates. ```bash # Install snapd if not already installed sudo snap install core && sudo snap refresh core # Install certbot sudo snap install --classic certbot # Create symbolic link for easy access sudo ln -s /snap/bin/certbot /usr/bin/certbot ``` -------------------------------- ### Example Fetch Settings Source: https://docs.mentraglass.com/app-devs/reference/managers/settings-manager Example demonstrating how to manually fetch settings and handle potential errors. ```typescript try { const settings = await session.settings.fetch(); console.log('Settings refreshed:', settings); } catch (error) { console.error('Failed to fetch settings:', error); } ``` -------------------------------- ### Audio Playback Example Source: https://docs.mentraglass.com/app-devs/core-concepts/speakers/playing-audio-files Example demonstrating how to play an audio file and handle its result. ```typescript const result = await session.audio.playAudio({ audioUrl: 'https://example.com/sound.mp3' }); console.log('Success:', result.success); console.log('Duration:', result.duration); if (!result.success) { console.log('Error:', result.error); } ``` -------------------------------- ### Camera Access API Source: https://docs.mentraglass.com/os-devs/asg-client/mentra-live Java code examples for taking a picture and starting video recording using the CameraNeo API. ```java // Take photo CameraNeo.takePictureWithCallback(context, filePath, callback); // Start video recording CameraNeo.startVideoRecording(context, filePath, callback); ``` -------------------------------- ### showDoubleTextWall Example Source: https://docs.mentraglass.com/app-devs/reference/managers/layout-manager Example usage of the showDoubleTextWall method. ```typescript // Show a title and content appSession.layouts.showDoubleTextWall( 'Weather Forecast', 'Partly cloudy, 72°F, 10% chance of rain', { durationMs: 3000 } ); ``` -------------------------------- ### AppSettings Example Source: https://docs.mentraglass.com/app-devs/reference/interfaces/config-types Example of an AppSettings array. ```typescript const settings: AppSettings = [ { key: "enableNotifications", type: AppSettingType.TOGGLE, label: "Enable Notifications", defaultValue: true, value: false // User has changed from default }, { key: "refreshInterval", type: AppSettingType.SELECT, label: "Refresh Interval", options: [ { label: "1 minute", value: 60 }, { label: "5 minutes", value: 300 }, { label: "15 minutes", value: 900 } ], defaultValue: 300, value: 60 // User has selected 1 minute } ]; ``` -------------------------------- ### Install Dependencies and Test Source: https://docs.mentraglass.com/app-devs/getting-started/deployment/ubuntu-deployment Navigates to the app directory, installs dependencies using Bun, copies the environment file, and prompts to edit it. ```bash # Change to your app directory cd /opt/myapp # Install all dependencies bun install # Create your environment file from the example cp .env.example .env # Edit the environment file with your production values nano .env ``` -------------------------------- ### GroupSetting Example Source: https://docs.mentraglass.com/app-devs/reference/interfaces/setting-types An example of a JSON object representing a GroupSetting. ```json { "type": "group", "title": "Display Settings" } ``` -------------------------------- ### extractTokenFromUrl() Example Source: https://docs.mentraglass.com/app-devs/reference/token-utils Example demonstrating how to use extractTokenFromUrl to get a token from a URL. ```typescript // In a webview handling incoming requests const incomingUrl = "https://my-app.example.com/webview?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."; const token = TokenUtils.extractTokenFromUrl(incomingUrl); if (token) { // Validate the token and process the webview request } ``` -------------------------------- ### Starting the AppServer Source: https://docs.mentraglass.com/app-devs/core-concepts/app-server Code snippet to create and start the AppServer instance. ```typescript // Create server instance const server = new MyApp({ packageName: 'com.example.myapp', apiKey: process.env.API_KEY, port: 3000 }); // Start listening server.start(); console.log('Server running on port 3000'); ``` -------------------------------- ### Configure .env file Source: https://docs.mentraglass.com/app-devs/getting-started/deployment/ubuntu-deployment Example configuration for the .env file, including PORT, PACKAGE_NAME, MENTRAOS_API_KEY, and NODE_ENV. ```env PORT=3000 PACKAGE_NAME=com.yourname.yourapp MENTRAOS_API_KEY=your_api_key_from_console NODE_ENV=production ``` -------------------------------- ### Troubleshooting: App Not Starting - Check service status Source: https://docs.mentraglass.com/app-devs/getting-started/deployment/ubuntu-deployment Commands to check the status of the application service and view recent logs. ```bash sudo systemctl status myapp sudo journalctl -u myapp --since "10 minutes ago" ``` -------------------------------- ### Example usage of on() Source: https://docs.mentraglass.com/app-devs/reference/managers/event-manager Examples demonstrating how to subscribe to specific stream types using the on() method. ```typescript import {StreamType} from "@mentra/sdk" // Subscribe to a specific stream type appSession.events.on(StreamType.LOCATION_UPDATE, data => { console.log(`Location update: ${data.lat}, ${data.lng}`) }) // Using a language-specific stream const transcriptionStream = createTranscriptionStream("en-US") appSession.events.on(transcriptionStream, data => { console.log(`English transcription: ${data.text}`) }) ``` -------------------------------- ### Best Practices - Do This Source: https://docs.mentraglass.com/app-devs/core-concepts/simple-storage Examples of recommended practices for using Simple Storage. ```typescript // Use descriptive keys await storage.set('user.theme', 'dark'); await storage.set('app.lastSyncTime', Date.now().toString()); // Provide defaults const theme = await storage.get('theme') || 'light'; // Handle JSON errors try { const data = JSON.parse(await storage.get('config') || '{}'); } catch (e) { session.logger.error('Invalid JSON in storage'); } // Store environment variables for API keys elsewhere const apiKey = process.env.EXTERNAL_API_KEY; // ✅ ``` -------------------------------- ### ReferenceCard Example Source: https://docs.mentraglass.com/app-devs/reference/interfaces/layout-types Example of using the LayoutManager to show a ReferenceCard. ```typescript // Using the LayoutManager appSession.layouts.showReferenceCard( "Recipe: Chocolate Chip Cookies", "Ingredients:\n- 2 cups flour\n- 1 cup sugar\n- 1/2 cup butter\n..." ); ``` -------------------------------- ### TitleValueSetting Example Source: https://docs.mentraglass.com/app-devs/reference/interfaces/setting-types An example of a JSON object representing a TitleValueSetting. ```json { "type": "titleValue", "label": "App Version", "value": "2.1.0" } ``` -------------------------------- ### onSettingsUpdate() example Source: https://docs.mentraglass.com/app-devs/reference/managers/event-manager Example of using the onSettingsUpdate event handler to log updated application settings. ```typescript appSession.events.onSettingsUpdate(settings => { console.log("Settings updated:", settings) }) ``` -------------------------------- ### Streaming to Your Own Server Source: https://docs.mentraglass.com/app-devs/core-concepts/camera/README Examples of how to start a local livestream to a custom endpoint using SRT or RTMP protocols. ```typescript // SRT await session.camera.startLocalLivestream({ streamUrl: 'srt://your-server.com:4201?streamid=key' }); // RTMP await session.camera.startLocalLivestream({ streamUrl: 'rtmp://your-server.com/live/key' }); ``` -------------------------------- ### AppServer Configuration Example Source: https://docs.mentraglass.com/app-devs/core-concepts/app-server Shows how to instantiate AppServer with essential configuration options. ```typescript const server = new MyApp({ packageName: 'com.example.myapp', // From developer console apiKey: process.env.API_KEY, // From developer console port: 3000, // Your server port webhookPath: '/webhook', // Optional, defaults to '/webhook' mentraOSWebsocketUrl: 'wss://...' // Optional, defaults to production }); ``` -------------------------------- ### getCapabilities Method Example Source: https://docs.mentraglass.com/app-devs/reference/managers/led-manager Example of how to retrieve and log the available LED capabilities of the device, and conditionally use them. ```typescript const capabilities = session.led.getCapabilities(); console.log('Available LEDs:', capabilities); // Check if device has RGB LEDs const hasRGB = capabilities.some(led => led.isFullColor); if (hasRGB) { await session.led.turnOn({ color: 'blue', ontime: 1000 }); } ``` -------------------------------- ### Install Essentials Source: https://docs.mentraglass.com/app-devs/getting-started/deployment/ubuntu-deployment Updates package lists, upgrades existing packages, and installs essential packages like curl, unzip, git, build-essential, and ufw. ```bash # Update package lists and upgrade existing packages sudo apt update && sudo apt upgrade -y # Install essential packages sudo apt install -y curl unzip git build-essential ufw ``` -------------------------------- ### Best Practice: Provide Defaults Source: https://docs.mentraglass.com/app-devs/reference/managers/settings-manager Example illustrating the importance of providing default values when getting settings to prevent undefined errors. ```typescript // Good: Always provide a default const lineWidth = session.settings.get('line_width', 30); // Avoid: No default can cause undefined errors const lineWidth = session.settings.get('line_width'); // Could be undefined! ``` -------------------------------- ### Complete Example Application Source: https://docs.mentraglass.com/app-devs/core-concepts/webviews/react-webviews A full React application example that includes authentication context and a basic content structure. ```javascript import React from 'react'; import { MentraAuthProvider } from '@mentra/react-auth'; /** * Content component that displays the main application UI. * This component is intended to be wrapped by an authentication provider. * * @returns {React.JSX.Element} The content component */ function Content(): React.JSX.Element { return (

App Content

This is where the main application content would go.

{/* Example of conditional rendering based on authentication status could go here */}
        {/* Placeholder for dynamic content or logs */}
      
) } /** * Root App component that provides authentication context to the entire application. * This component wraps the Content component with the MentraAuthProvider * to enable authentication functionality throughout the app. * * @returns {React.JSX.Element} The main application component */ function App(): React.JSX.Element { return (

MentraOS React Test App

) } export default App ``` -------------------------------- ### Install @mentra/react Source: https://docs.mentraglass.com/app-devs/core-concepts/webviews/react-webviews Install the React authentication library in your webview project using npm, yarn, or bun. ```bash npm install @mentra/react # or yarn add @mentra/react # or bun add @mentra/react ``` -------------------------------- ### Configure the Firewall Source: https://docs.mentraglass.com/app-devs/getting-started/deployment/ubuntu-deployment Sets up basic firewall rules to allow SSH, HTTP, and HTTPS traffic, then enables the firewall and checks its status. ```bash # Allow SSH connections sudo ufw allow OpenSSH # Allow HTTP and HTTPS traffic for web accss sudo ufw allow 'Nginx Full' # Enable the firewall sudo ufw enable # Check firewall status sudo ufw status verbose ``` -------------------------------- ### ADB Connection Example Source: https://docs.mentraglass.com/os-devs/asg-client/mentra-live Connect to Mentra Live glasses via ADB over WiFi using the IP address obtained from the MentraOS app. ```bash # Connect using the IP from the app adb connect [IP_ADDRESS]:5555 # Example adb connect 192.168.1.123:5555 # Output should show: # connected to 192.168.1.123:5555 # Verify connection adb devices # Should list: 192.168.1.123:5555 device ``` -------------------------------- ### Best Practices - Always check capabilities Source: https://docs.mentraglass.com/app-devs/core-concepts/hardware-capabilities/overview Example of good practice for checking capabilities before use. ```typescript // ✅ Good if (session.capabilities?.hasCamera) { await session.camera.requestPhoto(); } // ❌ Avoid - may crash await session.camera.requestPhoto(); ``` -------------------------------- ### Example Command Processing Source: https://docs.mentraglass.com/os-devs/contributing/mentraos-asg-client-guidelines Illustrates how the ASG Client Service processes incoming JSON commands from the cloud for actions like taking photos, starting streams, and displaying text. ```java // Example command processing in AsgClientService switch (type) { case "take_photo": String requestId = dataToProcess.optString("requestId", ""); mMediaCaptureService.takePhotoAndUpload(photoFilePath, requestId); break; case "start_stream": String streamId = dataToProcess.optString("streamId", ""); String streamUrl = dataToProcess.optString("streamUrl", ""); // Protocol auto-detected from URL scheme (srt://, rtmp://, https://) StreamCommandHandler.handleCommand("start_stream", dataToProcess); break; case "display_text": // For glasses with displays showTextOnDisplay(text); break; } ``` -------------------------------- ### Set Up Log Monitoring Source: https://docs.mentraglass.com/app-devs/getting-started/deployment/ubuntu-deployment Commands to monitor application and Nginx logs using journalctl and tail. ```bash sudo journalctl -u myapp --since "1 hour ago" # Follow live logs with timestamps sudo journalctl -u myapp -f --output=short-iso # View Nginx access logs sudo tail -f /var/log/nginx/access.log # View Nginx error logs sudo tail -f /var/log/nginx/error.log ``` -------------------------------- ### Quick Start Source: https://docs.mentraglass.com/app-devs/core-concepts/led/overview Basic LED control, patterns, and turning off LEDs. ```typescript import { AppServer, AppSession } from "@mentra/sdk"; class LEDApp extends AppServer { protected async onSession(session: AppSession, sessionId: string, userId: string): Promise { // Always check LED capabilities first if (!session.capabilities?.hasLight) { session.layouts.showTextWall("No LED support on this device"); return; } // Basic LED control await session.led.turnOn({ color: 'red', ontime: 2000 }); // LED patterns await session.led.blink('green', 500, 500, 3); await session.led.solid('white', 5000); // Turn off LEDs await session.led.turnOff(); } } ``` -------------------------------- ### API reference - Start Source: https://docs.mentraglass.com/app-devs/core-concepts/camera/streaming.md Start managed streaming. ```typescript // Start session.camera.startLivestream(options?: ManagedStreamOptions): Promise ``` -------------------------------- ### Basic AppServer Setup Source: https://docs.mentraglass.com/app-devs/core-concepts/app-server Demonstrates the basic structure of extending the AppServer class and implementing the onSession method. ```typescript import { AppServer, AppSession } from '@mentraos/sdk'; class MyApp extends AppServer { protected async onSession(session: AppSession, sessionId: string, userId: string) { // Your app logic goes here } } ``` -------------------------------- ### ServerHeartbeatWebhookRequest Example Source: https://docs.mentraglass.com/app-devs/reference/interfaces/webhook-types Example of a ServerHeartbeatWebhookRequest. ```json { "type": "server_heartbeat", "sessionId": "heartbeat-123456", "userId": "system", "timestamp": "2023-04-01T14:30:00Z", "registrationId": "reg-78901234" } ``` -------------------------------- ### SessionRecoveryWebhookRequest Example Source: https://docs.mentraglass.com/app-devs/reference/interfaces/webhook-types Example of a SessionRecoveryWebhookRequest. ```json { "type": "session_recovery", "sessionId": "session-123456", "userId": "user-789012", "timestamp": "2023-04-01T14:15:30Z", "mentraOSWebsocketUrl": "wss://api.mentra.glass/app-ws/session-123456" } ``` -------------------------------- ### Example App - Session Handling Source: https://docs.mentraglass.com/app-devs/core-concepts/hardware-capabilities/display-glasses An example app server demonstrating session handling for display glasses, including capability checks and input methods. ```typescript class DisplayGlassesApp extends AppServer { protected async onSession(session: AppSession, sessionId: string, userId: string) { const caps = session.capabilities; if (!caps?.hasDisplay) { session.logger.warn('No display - audio only'); await session.audio.speak('Welcome'); return; } // Show welcome on display session.layouts.showTextWall('Voice Assistant Ready'); // Use microphone if available if (caps.hasMicrophone) { session.events.onTranscription((data) => { if (data.isFinal) { this.handleCommand(session, data.text); } }); } else { // Phone microphone still works session.events.onTranscription((data) => { if (data.isFinal) { this.handleCommand(session, data.text); } }); } } private handleCommand(session: AppSession, text: string) { session.layouts.showTextWall(`You said: ${text}`); // Speak response if no speaker on glasses if (!session.capabilities?.hasSpeaker) { session.audio.speak('Processing'); } } } ``` -------------------------------- ### ServerRegistrationWebhookRequest Example Source: https://docs.mentraglass.com/app-devs/reference/interfaces/webhook-types Example of a ServerRegistrationWebhookRequest. ```json { "type": "server_registration", "sessionId": "registration-123456", "userId": "system", "timestamp": "2023-04-01T10:00:00Z", "registrationId": "reg-78901234", "packageName": "org.company.myapp", "serverUrls": ["https://myapp-app.example.com/webhook"] } ``` -------------------------------- ### StopWebhookRequest Example Source: https://docs.mentraglass.com/app-devs/reference/interfaces/webhook-types Example of a StopWebhookRequest. ```json { "type": "stop_request", "sessionId": "session-123456", "userId": "user-789012", "timestamp": "2023-04-01T13:45:30Z", "reason": "user_disabled" } ``` -------------------------------- ### SessionWebhookRequest Example Source: https://docs.mentraglass.com/app-devs/reference/interfaces/webhook-types Example of a SessionWebhookRequest. ```json { "type": "session_request", "sessionId": "session-123456", "userId": "user-789012", "timestamp": "2023-04-01T12:30:45Z", "mentraOSWebsocketUrl": "wss://api.mentra.glass/app-ws/session-123456" } ``` -------------------------------- ### Complete Example Source: https://docs.mentraglass.com/app-devs/core-concepts/app-server A complete example of an `AppServer` implementation for a Weather App, demonstrating session handling, event listening, and layout interactions. ```typescript import { AppServer, AppSession } from '@mentraos/sdk'; class WeatherApp extends AppServer { protected async onSession(session: AppSession, sessionId: string, userId: string) { session.logger.info(`User ${userId} connected`); // Show welcome message session.layouts.showTextWall('Weather App Ready'); // Listen for voice commands session.events.onTranscription((data) => { if (data.isFinal && data.text.includes('weather')) { this.showWeather(session); } }); } protected async onStop(sessionId: string, userId: string, reason: string) { console.log(`Session ended: ${reason}`); } private async showWeather(session: AppSession) { session.layouts.showReferenceCard('Weather', 'Sunny, 72°F'); } } // Start server const server = new WeatherApp({ packageName: 'com.example.weather', apiKey: process.env.MENTRAOS_API_KEY!, port: 3000 }); server.start(); ``` -------------------------------- ### TextNoSaveButtonSetting Example Source: https://docs.mentraglass.com/app-devs/reference/interfaces/setting-types Example of a TextNoSaveButtonSetting. ```json { "type": "text_no_save_button", "key": "notes", "label": "Personal Notes", "defaultValue": "", "maxLines": 5 } ``` -------------------------------- ### Create Systemd Service File Source: https://docs.mentraglass.com/app-devs/getting-started/deployment/ubuntu-deployment Creates the systemd service file for the application. ```bash sudo nano /etc/systemd/system/myapp.service ``` -------------------------------- ### Local Testing: Start App Source: https://docs.mentraglass.com/cookbook/dashboard-note-app Command to start the development server for local testing. ```bash bun run dev ``` -------------------------------- ### solid Method Examples Source: https://docs.mentraglass.com/app-devs/reference/managers/led-manager Examples of using the solid method to keep an LED on for a specified duration, useful for indicators. ```typescript // Solid red LED for 5 seconds await session.led.solid('red', 5000); // Recording indicator await session.led.solid('white', 30000); // Status indicator await session.led.solid('green', 2000); ``` -------------------------------- ### showDashboardCard Example Source: https://docs.mentraglass.com/app-devs/reference/managers/layout-manager Example usage of the showDashboardCard method. ```typescript // Show current temperature in the dashboard appSession.layouts.showDashboardCard('Temperature', '72°F'); // Show stock price in the main view appSession.layouts.showDashboardCard('AAPL', '$178.72', { view: ViewType.MAIN }); ``` -------------------------------- ### showReferenceCard Example Source: https://docs.mentraglass.com/app-devs/reference/managers/layout-manager Example usage of the showReferenceCard method. ```typescript // Show a reference card with a recipe appSession.layouts.showReferenceCard( 'Chocolate Chip Cookies', '2 cups flour\n1 cup sugar\n1/2 cup butter\n2 eggs\n1 tsp vanilla\n2 cups chocolate chips\n\nMix ingredients. Bake at 350°F for 10-12 minutes.' ); ``` -------------------------------- ### Adapting Your App Source: https://docs.mentraglass.com/app-devs/core-concepts/hardware-capabilities/overview Choose interaction modes and enable features based on available hardware capabilities. ```typescript protected async onSession(session: AppSession, sessionId: string, userId: string) { const caps = session.capabilities; if (!caps) return; // Choose interaction mode if (caps.hasDisplay) { this.useVisualMode(session); } else if (caps.hasSpeaker) { this.useAudioMode(session); } // Enable features based on hardware if (caps.hasCamera) { this.enablePhotoFeatures(session); } if (caps.hasButton) { this.enableButtonControls(session); } } ``` -------------------------------- ### showTextWall Example Source: https://docs.mentraglass.com/app-devs/reference/managers/layout-manager Example usage of the showTextWall method. ```typescript import { ViewType } from '@mentra/sdk'; // Simple usage appSession.layouts.showTextWall('Hello, MentraOS!'); // With options appSession.layouts.showTextWall('This is an important message', { view: ViewType.MAIN, durationMs: 5000 // Show for 5 seconds }); ``` -------------------------------- ### Voice Assistant Example Source: https://docs.mentraglass.com/app-devs/core-concepts/microphone/speech-to-text A complete example of a `VoiceAssistant` class that extends `AppServer` and handles voice commands using speech-to-text. ```typescript class VoiceAssistant extends AppServer { protected async onSession(session: AppSession, sessionId: string, userId: string) { session.layouts.showTextWall('Voice Assistant Ready\nSay "help" for commands'); session.events.onTranscription(async (data) => { if (!data.isFinal) { // Show interim results session.layouts.showTextWall(`Listening: ${data.text}...`); return; } // Process final command const command = data.text.toLowerCase().trim(); session.logger.info('Command:', command, 'Confidence:', data.confidence); if (command.includes('help')) { await this.showHelp(session); } else if (command.includes('weather')) { await this.showWeather(session); } else if (command.includes('time')) { await this.showTime(session); } else if (command.includes('reminder')) { await this.setReminder(session, command); } else { session.layouts.showTextWall(`Unknown: ${data.text}`); await session.audio.speak('I did not understand that. Say help for commands.'); } }); } private async showHelp(session: AppSession) { const helpText = 'Commands:\n- Weather\n- Time\n- Set reminder'; session.layouts.showReferenceCard('Help', helpText); await session.audio.speak('You can ask about weather, time, or set a reminder'); } private async showWeather(session: AppSession) { // Fetch weather data const weather = await this.fetchWeather(); session.layouts.showReferenceCard('Weather', `${weather.condition}, ${weather.temp}°F`); await session.audio.speak(`The weather is ${weather.condition} and ${weather.temp} degrees`); } private async showTime(session: AppSession) { const time = new Date().toLocaleTimeString(); session.layouts.showTextWall(`Time: ${time}`); await session.audio.speak(`The time is ${time}`); } private async setReminder(session: AppSession, command: string) { // Parse reminder from command // "set reminder to call mom at 3pm" session.layouts.showTextWall('Reminder set!'); await session.audio.speak('Reminder set'); } } ``` -------------------------------- ### Basic Usage Source: https://docs.mentraglass.com/app-devs/core-concepts/hardware-capabilities/overview Adapt to available hardware by checking capabilities. ```typescript protected async onSession(session: AppSession, sessionId: string, userId: string) { const caps = session.capabilities; if (!caps) return; // Adapt to available hardware if (caps.hasDisplay) { session.layouts.showTextWall('Welcome!'); } else { await session.audio.speak('Welcome!'); } if (caps.hasCamera) { session.logger.info('Camera available'); } } ``` -------------------------------- ### getSetting Method Example Source: https://docs.mentraglass.com/app-devs/reference/managers/settings-manager Illustrates how to find a specific setting object by its key and access its properties. ```typescript const setting = session.settings.getSetting('font_size'); if (setting && setting.type === 'slider') { console.log(`Font size: ${setting.value} (${setting.min}-${setting.max})`); } ``` -------------------------------- ### GroupSetting Example Source: https://docs.mentraglass.com/app-devs/reference/interfaces/config-types Example of a GroupSetting object. ```typescript const groupSetting: GroupSetting = { type: "group", title: "Display Preferences" }; ``` -------------------------------- ### AppConfig Example Source: https://docs.mentraglass.com/app-devs/reference/interfaces/config-types Example of an AppConfig object. ```typescript const appConfig: AppConfig = { name: "Weather App", description: "Displays real-time weather information for your current location", settings: [ { type: "group", title: "General Settings" }, { key: "useCelsius", type: AppSettingType.TOGGLE, label: "Use Celsius", defaultValue: false }, { key: "updateFrequency", type: AppSettingType.SELECT, label: "Update Frequency", defaultValue: "hourly", options: [ { label: "Hourly", value: "hourly" }, { label: "Every 3 hours", value: "3hours" }, { label: "Daily", value: "daily" } ] } ] }; ``` -------------------------------- ### isLanguageStream Example Source: https://docs.mentraglass.com/app-devs/reference/utilities Example usage of the isLanguageStream function. ```typescript if (isLanguageStream("transcription:en-US")) { // Handle language-specific stream } else { // Handle standard stream type } ``` -------------------------------- ### getBaseStreamType Example Source: https://docs.mentraglass.com/app-devs/reference/utilities Example usage of the getBaseStreamType function. ```typescript const baseType = getBaseStreamType("transcription:en-US"); // Returns StreamType.TRANSCRIPTION const standardType = getBaseStreamType(StreamType.BUTTON_PRESS); // Returns StreamType.BUTTON_PRESS ``` -------------------------------- ### Create Nginx Virtual Host File Source: https://docs.mentraglass.com/app-devs/getting-started/deployment/ubuntu-deployment Command to create the Nginx site configuration file. ```bash sudo nano /etc/nginx/sites-available/myapp ``` -------------------------------- ### Basic Text-to-Speech Example Source: https://docs.mentraglass.com/app-devs/reference/managers/audio-manager Demonstrates the basic usage of the speak method for text-to-speech conversion. ```typescript // Basic text-to-speech const result = await session.audio.speak('Welcome to Mentra OS!'); ``` -------------------------------- ### isValidStreamType Example Source: https://docs.mentraglass.com/app-devs/reference/utilities Example usage of the isValidStreamType function. ```typescript if (isValidStreamType(StreamType.BUTTON_PRESS)) { // Valid standard stream type } if (isValidStreamType("transcription:en-US")) { // Valid language-specific stream } ``` -------------------------------- ### createTranslationStream Example Source: https://docs.mentraglass.com/app-devs/reference/utilities Example usage of the createTranslationStream function. ```typescript // Subscribe to Spanish-to-English translation const streamId = createTranslationStream("es-ES", "en-US"); session.subscribe(streamId); ``` -------------------------------- ### Sound Library Example Source: https://docs.mentraglass.com/app-devs/core-concepts/speakers/playing-audio-files An example TypeScript class demonstrating how to play different sound effects using the session.audio.playAudio method. ```typescript class SoundLibrary { private baseUrl = 'https://cdn.example.com/sounds'; private sounds = { click: 'click.mp3', success: 'success.mp3', error: 'error.mp3', notification: 'notification.mp3', confirm: 'confirm.mp3' }; async play(session: AppSession, sound: keyof typeof this.sounds, volume = 0.8) { const url = `${this.baseUrl}/${this.sounds[sound]}`; try { const result = await session.audio.playAudio({ audioUrl: url, volume }); if (!result.success) { session.logger.warn(`Sound ${sound} failed:`, result.error); } return result.success; } catch (error) { session.logger.error(`Error playing ${sound}:`, error); return false; } } } // Usage const sounds = new SoundLibrary(); session.events.onButtonPress(async (data) => { await sounds.play(session, 'click'); // Handle button action }); ``` -------------------------------- ### createTranscriptionStream Example Source: https://docs.mentraglass.com/app-devs/reference/utilities Example usage of the createTranscriptionStream function. ```typescript // Subscribe to English transcription const streamId = createTranscriptionStream("en-US"); session.subscribe(streamId); ``` -------------------------------- ### parseLanguageStream Example Source: https://docs.mentraglass.com/app-devs/reference/utilities Example usage of the parseLanguageStream function. ```typescript const info = parseLanguageStream("transcription:en-US"); if (info) { console.log(info.type); // StreamType.TRANSCRIPTION console.log(info.baseType); // "transcription" console.log(info.transcribeLanguage); // "en-US" } ``` -------------------------------- ### LightCapabilities Example Usage Source: https://docs.mentraglass.com/app-devs/reference/interfaces/capabilities Shows how to access light capabilities and log details about each light, including color support. ```typescript if (session.capabilities.hasLight && session.capabilities.light) { const lights = session.capabilities.light console.log(`Controllable lights: ${lights.count || 0}`) lights.lights?.forEach((light, index) => { console.log(`Light ${index}: `) if (light.isFullColor) { console.log(` Full RGB color support`) } else { console.log(` Fixed color: ${light.color || "unknown"}`) } }) } ``` -------------------------------- ### TranscriptionData Example Source: https://docs.mentraglass.com/app-devs/reference/interfaces/event-types Example of subscribing to transcription events. ```typescript // Subscribe to transcription events appSession.events.onTranscription((data) => { console.log(`Transcription: ${data.text}`); console.log(`Final: ${data.isFinal}`); if (data.isFinal) { // Process complete utterance } }); ``` -------------------------------- ### Build and Install Source: https://docs.mentraglass.com/os-devs/contributing/mentraos-asg-client-guidelines Commands to build the debug APK and install it on the glasses, including setting up local development server port forwarding. ```bash # Build the APK ./gradlew assembleDebug # Install on glasses adb install app/build/outputs/apk/debug/app-debug.apk # For local development server, forward ports: adb reverse tcp:8002 tcp:8002 ``` -------------------------------- ### Select Setting Example Source: https://docs.mentraglass.com/app-devs/reference/interfaces/config-types Example of a select setting. ```typescript const selectSetting: AppSetting = { key: "theme", type: AppSettingType.SELECT, label: "Theme", options: [ { label: "Light", value: "light" }, { label: "Dark", value: "dark" }, { label: "System Default", value: "system" } ], defaultValue: "system" }; ``` -------------------------------- ### Text Setting Example Source: https://docs.mentraglass.com/app-devs/reference/interfaces/config-types Example of a text setting. ```typescript const textSetting: AppSetting = { key: "username", type: AppSettingType.TEXT, label: "Username", defaultValue: "" }; ```