### Run CrabGrab examples Source: https://github.com/augmendtech/crabgrab/blob/main/README.md Commands to execute example applications from the repository. ```bash cargo run --example ``` ```bash cargo run --example --feature ``` -------------------------------- ### Create and Start Windows Audio Capture Stream Source: https://github.com/augmendtech/crabgrab/blob/main/docs/windows_docs/src/crabgrab/platform/windows/audio_capture_stream.rs.html Initializes and starts the audio capture stream. Ensure CoInitializeEx is called before creating the stream if not already initialized. ```rust pub fn create_stream( should_couninit: bool, audio_client: AudioClient ) -> Result { audio_client.Start() .map_err(|_| WindowsAudioCaptureStreamCreateError::StreamStartFailed)?; Ok(WindowsAudioCaptureStream { should_couninit, audio_client }) } ``` -------------------------------- ### Create and Start SCStream for Window Capture Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/src/crabgrab/platform/macos/capture_stream.rs.html Initializes and starts an `SCStream` for capturing a specific window. Handles potential errors during stream creation. ```rust let mut sc_stream = SCStream::new(filter, config, handler_queue, handler) .map_err(|error| StreamCreateError::Other(error))?; sc_stream.start(); Ok(MacosCaptureStream { stopped_flag, shared_callback, stream: MacosCaptureStreamInternal::Window(sc_stream), #[cfg(feature = "metal")] metal_device, #[cfg(feature = "wgpu")] wgpu_device }) ``` -------------------------------- ### Start CGDisplayStream Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/src/crabgrab/platform/macos/objc_wrap.rs.html Starts the display stream capture. Returns Ok(()) on success, or Err(()) if an error occurred during stream initiation. ```rust pub fn start(&self) -> Result<(), ()> { let error_code = unsafe { CGDisplayStreamStart(self.stream_ref) }; if error_code == 0 { Ok(()) } else { Err(()) } } ``` -------------------------------- ### Capture Stream Example Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/src/crabgrab/lib.rs.html This example demonstrates capturing frames from a window using CrabGrab. It initializes a runtime, finds a window, starts a capture stream, waits for a duration, stops the stream, and then shuts down the runtime. Ensure the necessary imports and setup are in place before running. ```rust //! _ => {} //! } //! }, //! Err(error) => { //! println!("Stream error: {:?}", error); //! } //! } //! }).unwrap(); //! // wait for a while to capture some frames //! tokio::task::block_in_place(|| std::thread::sleep(Duration::from_millis(4000))); //! stream.stop().unwrap(); //! }, //! None => { println!("Failed to find window"); } //! } //! }); //! // wait for the future to complete //! runtime.block_on(future).unwrap(); //! // shutdown the async runtime //! runtime.shutdown_timeout(Duration::from_millis(10000)); //! ``` -------------------------------- ### Start Capture Stream Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/src/crabgrab/capture_stream.rs.html Starts a new capture stream with the provided access token, configuration, and a callback function for stream events. ```APIDOC ## POST /api/capture/new_stream ### Description Starts a new capture stream with the given stream callback. ### Method POST ### Endpoint /api/capture/new_stream ### Parameters #### Request Body - **token** (object) - Required - The access token obtained from `request_access`. - **impl_capture_access_token** (string) - The access token string. - **config** (object) - Required - The configuration for the capture stream. - **callback** (function) - Required - A callback function that handles `StreamEvent` or `StreamError`. ### Request Example ```json { "token": { "impl_capture_access_token": "some_access_token_string" }, "config": { "width": 1920, "height": 1080, "pixel_format": "RGB32" }, "callback": "function(event) { console.log(event); }" } ``` ### Response #### Success Response (200) - **Stream** (object) - An object representing the active capture stream. #### Response Example ```json { "message": "Stream started successfully" } ``` #### Error Response (400) - **StreamCreateError** (string) - An error message if the stream creation fails. ``` -------------------------------- ### Screen Capture Example in Rust Source: https://github.com/augmendtech/crabgrab/blob/main/docs/windows_docs/crabgrab/index.html This example demonstrates how to capture screen content using the CrabGrab library. It includes setting up the Tokio runtime, requesting necessary permissions, filtering for specific application windows (like Finder or Explorer), and processing video frames. Ensure you have the required privileges to capture content. ```rust use std::time::Duration; use crabgrab::prelude::*; // spin up the async runtime let runtime = tokio::runtime::Builder::new_multi_thread().build().unwrap(); // run our capture code in an async context let future = runtime.spawn(async { // ensure we have priveleges to capture content let token = match CaptureStream::test_access(false) { Some(token) => token, None => CaptureStream::request_access(false).await.expect("Expected capture access") }; // filter to normal windows let filter = CapturableContentFilter::NORMAL_WINDOWS; // get capturable content matching the filter let content = CapturableContent::new(filter).await.unwrap(); // find the window we want to capture let window = content.windows().filter(|window| { let app_identifier = window.application().identifier(); app_identifier.to_lowercase().contains("finder") || app_identifier.to_lowercase().contains("explorer") }).next(); match window { Some(window) => { println!("capturing window: {}", window.title()); // create a captuere config using the first supported pixel format let config = CaptureConfig::with_window(window, CaptureStream::supported_pixel_formats()[0]).unwrap(); // create a capture stream with an event handler callback let mut stream = CaptureStream::new(token, config, |stream_event| { match stream_event { Ok(event) => { match event { StreamEvent::Video(frame) => { println!("Got frame: {}", frame.frame_id()); }, _ => {} } }, Err(error) => { println!("Stream error: {:?}", error); } } }).unwrap(); // wait for a while to capture some frames tokio::task::block_in_place(|| std::thread::sleep(Duration::from_millis(4000))); stream.stop().unwrap(); }, None => { println!("Failed to find window"); } } }); // wait for the future to complete runtime.block_on(future).unwrap(); // shutdown the async runtime runtime.shutdown_timeout(Duration::from_millis(10000)); ``` -------------------------------- ### Capture Stream Example Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/src/crabgrab/lib.rs.html Demonstrates initializing a capture stream, requesting access, filtering content, and handling video frame events. ```rust use std::time::Duration; use crabgrab::prelude::*; // spin up the async runtime let runtime = tokio::runtime::Builder::new_multi_thread().build().unwrap(); // run our capture code in an async context let future = runtime.spawn(async { // ensure we have priveleges to capture content let token = match CaptureStream::test_access(false) { Some(token) => token, None => CaptureStream::request_access(false).await.expect("Expected capture access") }; // filter to normal windows let filter = CapturableContentFilter::NORMAL_WINDOWS; // get capturable content matching the filter let content = CapturableContent::new(filter).await.unwrap(); // find the window we want to capture let window = content.windows().filter(|window| { let app_identifier = window.application().identifier(); app_identifier.to_lowercase().contains("finder") || app_identifier.to_lowercase().contains("explorer") }).next(); match window { Some(window) => { println!("capturing window: {}", window.title()); // create a captuere config using the first supported pixel format let config = CaptureConfig::with_window(window, CaptureStream::supported_pixel_formats()[0]).unwrap(); // create a capture stream with an event handler callback let mut stream = CaptureStream::new(token, config, |stream_event| { match stream_event { Ok(event) => { match event { StreamEvent::Video(frame) => { println!("Got frame: {}", frame.frame_id()); }, ``` -------------------------------- ### CaptureStream::new Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/crabgrab/capture_stream/struct.CaptureStream.html Starts a new capture stream with the provided configuration and callback. ```APIDOC ## new ### Description Starts a new capture stream with the given stream callback. ### Parameters - **token** (CaptureAccessToken) - Required - The access token for the capture. - **config** (CaptureConfig) - Required - The configuration for the capture stream. - **callback** (FnMut) - Required - A callback function that receives Result. ### Response - **Result** - Returns the new CaptureStream instance or an error if creation fails. ``` -------------------------------- ### Initialize and Start MacosCaptureStream Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/src/crabgrab/platform/macos/capture_stream.rs.html Creates a new CGDisplayStream instance and initializes the MacosCaptureStream wrapper with necessary state and device handles. ```rust let display_stream = CGDisplayStream::new(stream_callback, display_id, size, pixel_format, options_dict, dispatch_queue); display_stream.start().map_err(|_| StreamCreateError::Other("Stream failed to start".into()))?; Ok(MacosCaptureStream { stream: MacosCaptureStreamInternal::Display(display_stream), stopped_flag, shared_callback, #[cfg(feature = "metal")] metal_device, #[cfg(feature = "wgpu")] wgpu_device }) } } } ``` -------------------------------- ### Get Window Application Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/src/crabgrab/platform/macos/capturable_content.rs.html Returns a `MacosCapturableApplication` struct representing the application that owns the window. ```rust pub fn application(&self) -> MacosCapturableApplication { MacosCapturableApplication { running_application: self.window.owning_application() } } ``` -------------------------------- ### Initialize Capture Stream Source: https://github.com/augmendtech/crabgrab/blob/main/docs/windows_docs/src/crabgrab/lib.rs.html Demonstrates setting up an async runtime and initializing a capture stream for a specific window. ```rust use std::time::Duration; use crabgrab::prelude::*; // spin up the async runtime let runtime = tokio::runtime::Builder::new_multi_thread().build().unwrap(); // run our capture code in an async context let future = runtime.spawn(async { // ensure we have priveleges to capture content let token = match CaptureStream::test_access(false) { Some(token) => token, None => CaptureStream::request_access(false).await.expect("Expected capture access") }; // filter to normal windows let filter = CapturableContentFilter::NORMAL_WINDOWS; // get capturable content matching the filter let content = CapturableContent::new(filter).await.unwrap(); // find the window we want to capture let window = content.windows().filter(|window| { let app_identifier = window.application().identifier(); app_identifier.to_lowercase().contains("finder") || app_identifier.to_lowercase().contains("explorer") }).next(); match window { Some(window) => { println!("capturing window: {}", window.title()); // create a captuere config using the first supported pixel format let config = CaptureConfig::with_window(window, CaptureStream::supported_pixel_formats()[0]).unwrap(); // create a capture stream with an event handler callback let mut stream = CaptureStream::new(token, config, |stream_event| { ``` -------------------------------- ### Get Audio Frame Sample Rate Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/src/crabgrab/frame.rs.html Retrieves the sample rate of the captured audio frame. No specific setup is required beyond having an `AudioFrame` instance. ```rust pub fn sample_rate(&self) -> AudioSampleRate { self.impl_audio_frame.sample_rate() } ``` -------------------------------- ### Get Audio Frame Origin Time Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/src/crabgrab/frame.rs.html Returns the time since the start of the stream when this audio frame begins. Note that this implementation appears to be identical to `duration()`. ```rust pub fn origin_time(&self) -> Duration { self.impl_audio_frame.duration() } ``` -------------------------------- ### Initialize MacosAudioCaptureConfig Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/src/crabgrab/platform/macos/capture_stream.rs.html Creates a new `MacosAudioCaptureConfig` with default settings. By default, audio from the current process is not excluded. ```rust impl MacosAudioCaptureConfig { pub fn new() -> Self { Self { exclude_current_process_audio: false, } } } ``` -------------------------------- ### Get Video Frame Origin Time Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/src/crabgrab/frame.rs.html Returns the time elapsed since the stream's start when this video frame was generated. This indicates the frame's position within the stream timeline. ```rust pub fn origin_time(&self) -> Duration { self.impl_video_frame.origin_time() } ``` -------------------------------- ### Get Wgpu Device Instance Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/crabgrab/feature/wgpu/trait.WgpuCaptureStream.html Retrieves the device instance wrapper as supplied to `CaptureConfig::with_wgpu_device(..)`. This method is part of the WgpuCaptureStream trait. ```rust fn get_wgpu_device_instance( &self ) -> Option + Send + Sync + 'static>>; ``` -------------------------------- ### Configure WGPU Device for Platform Capture Source: https://github.com/augmendtech/crabgrab/blob/main/docs/windows_docs/src/crabgrab/feature/wgpu/mod.rs.html Handles platform-specific device initialization for screen capture, including Metal device extraction on macOS and DXGI adapter retrieval on Windows. ```rust Some(device.raw_device().lock().clone()) } else { None } }).expect("Expected metal device underneath wgpu"); Ok(Self { impl_capture_config: MacosCaptureConfig { metal_device: device, wgpu_device: Some(wgpu_device.clone()), ..self.impl_capture_config }, ..self }) } } #[cfg(target_os = "windows")] { unsafe { let mut dxgi_adapter_result = Err("Unimplemented for this wgpu backend".to_string()); AsRef::::as_ref(&*wgpu_device).as_hal::(|device| { device.map(|device| { //device.raw_device().AddRef(); let raw_device_ptr = device.raw_device().as_mut_ptr() as *mut c_void; let raw_queue_ptr = device.raw_queue().as_mut_ptr() as *mut c_void; let d3d12_device = ID3D12Device::from_raw(raw_device_ptr); let d3d12_queue = ID3D12CommandQueue::from_raw(raw_queue_ptr); let adapter_luid = d3d12_device.GetAdapterLuid(); let dxgi_factory: IDXGIFactory5 = match CreateDXGIFactory() { Err(error) => { dxgi_adapter_result = Err(format!("Failed to create dxgi factory: {}", error.to_string())); return; }, Ok(factory) => factory, }; dxgi_adapter_result = dxgi_factory.EnumAdapterByLuid(adapter_luid) .map_err(|error| format!("Failed to find matching dxgi adapter for wgpu device: {}", error.to_string())) .map(|dxgi_adapter: IDXGIAdapter4| (dxgi_adapter, d3d12_device, d3d12_queue)); }) }); let (dxgi_adapter, _d3d12_device, _d3d12_queue) = dxgi_adapter_result?; let dxgi_adapter = dxgi_adapter.cast::().unwrap(); let mut d3d11_device = None; D3D11CreateDevice ( &dxgi_adapter, D3D_DRIVER_TYPE_UNKNOWN, None, D3D11_CREATE_DEVICE_BGRA_SUPPORT | D3D11_CREATE_DEVICE_DEBUG, Some(&[D3D_FEATURE_LEVEL_11_0]), D3D11_SDK_VERSION, Some(&mut d3d11_device), None, None ).map_err(|error| format!("Failed to create d3d11 device from dxgi adapter: {}", error.to_string()))?; let d3d11_device = d3d11_device.unwrap(); Ok(Self { impl_capture_config: WindowsCaptureConfig { d3d11_device: Some(d3d11_device), wgpu_device: Some(wgpu_device), dxgi_adapter: Some(dxgi_adapter), ..self.impl_capture_config }, ..self }) } } } } ``` -------------------------------- ### Constructor: new Source: https://github.com/augmendtech/crabgrab/blob/main/docs/windows_docs/crabgrab/platform/windows/struct.ImplAudioCaptureConfig.html Creates a new instance of ImplAudioCaptureConfig. ```APIDOC ## fn new ### Description Creates a new instance of the ImplAudioCaptureConfig struct. ### Method Rust Function ### Response - **Self** (struct) - A new instance of ImplAudioCaptureConfig. ``` -------------------------------- ### Enumerate Windows and Get Process Information Source: https://github.com/augmendtech/crabgrab/blob/main/docs/windows_docs/src/crabgrab/platform/windows/capturable_content.rs.html This snippet uses `EnumWindows` to iterate through all top-level windows and `GetWindowThreadProcessId` to retrieve the process ID associated with each window. It also uses `GetModuleFileNameExW` to get the executable file name for a given process. ```rust unsafe { EnumWindows(Some(enum_windows_proc), LPARAM(0)).as_bool().unwrap(); } ``` ```rust unsafe { let mut process_id: u32 = 0; GetWindowThreadProcessId(hwnd, Some(&mut process_id)); let process_handle = OpenProcess(PROCESS_QUERY_INFORMATION | PROCESS_VM_READ, FALSE, process_id).unwrap(); let file_name = GetModuleFileNameExW(process_handle, None).unwrap(); CloseHandle(process_handle); println!("Process Name: {}", file_name.to_string_lossy()); } ``` -------------------------------- ### Create new AudioCaptureConfig Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/crabgrab/capture_stream/struct.AudioCaptureConfig.html Initializes a new configuration instance with default settings of 24000 Hz and mono. ```rust pub fn new() -> Self ``` -------------------------------- ### Get DX11 Texture from Video Frame Source: https://github.com/augmendtech/crabgrab/blob/main/docs/windows_docs/src/crabgrab/feature/dx11/mod.rs.html Retrieves the ID3D11Texture2D interface from a video frame, which is necessary for direct manipulation or rendering with DirectX 11. This method first obtains the surface and then casts it to the DXGI interface access to get the texture. ```rust fn get_dx11_texture(&self) -> Result<(ID3D11Texture2D, DirectXPixelFormat), WindowsDx11VideoFrameError> { let (surface, pixel_format) = self.get_dx11_surface()?; let dxgi_interface_access = surface.cast::() .map_err(|e| WindowsDx11VideoFrameError::Other(format!("Failed to cast surface to dxgi interface access: {}", e.to_string()))) ?; let texture = unsafe { dxgi_interface_access.GetInterface::() } .map_err(|e| WindowsDx11VideoFrameError::Other(format!("Failed to get ID3D11Texture interface {}", e.to_string()))) ?; Ok((texture, pixel_format)) } ``` -------------------------------- ### Get CVPixelBuffer Height Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/src/crabgrab/platform/macos/objc_wrap.rs.html Returns the height of the CVPixelBuffer in pixels. ```rust pub fn get_height(&self) -> usize { unsafe { CVPixelBufferGetHeight(self.0) } } ``` -------------------------------- ### Initialize Windows Audio Capture Stream Source: https://github.com/augmendtech/crabgrab/blob/main/docs/windows_docs/src/crabgrab/platform/windows/audio_capture_stream.rs.html Constructor for WindowsAudioCaptureStream that initializes COM and sets up the audio client. ```rust impl WindowsAudioCaptureStream { pub fn new(config: AudioCaptureConfig, mut callback: Box FnMut(Result, WindowsAudioCaptureStreamError>) + Send + 'static>) -> Result { unsafe { let should_couninit = CoInitializeEx(None, COINIT_MULTITHREADED).is_ok(); ``` -------------------------------- ### Initialize Capture Stream Source: https://github.com/augmendtech/crabgrab/blob/main/docs/windows_docs/src/crabgrab/platform/windows/capture_stream.rs.html Configures the capture session, sets up frame and audio handlers, and returns the stream output structure. ```rust device: callback_direct3d_device.clone(), frame, frame_id, frame_size: (width, height), pixel_format, dpi, t_capture, t_origin, duration, #[cfg(feature = "wgpu")] wgpu_device: callback_wgpu_device.clone(), }; let video_frame = VideoFrame { impl_video_frame }; (*callback)(Ok(StreamEvent::Video(video_frame))); Ok(()) }); frame_pool.FrameArrived(&frame_handler).map_err(|_| StreamCreateError::Other("Failed to listen to FrameArrived event".into()))?; graphics_capture_item.Closed(&close_handler).map_err(|_| StreamCreateError::Other("Failed to listen to Closed event".into()))?; let capture_session = frame_pool.CreateCaptureSession(&graphics_capture_item) .map_err(|_| StreamCreateError::Other("Failed to create GraphicsCaptureSession".into()))?; let _ = capture_session.SetIsBorderRequired(!config.impl_capture_config.borderless); let _ = capture_session.SetIsCursorCaptureEnabled(config.show_cursor); let audio_stream = if let Some(audio_config) = config.capture_audio { let handler_config = audio_config.clone(); let audio_handler = Box::new(move |audio_result: Result, WindowsAudioCaptureStreamError>| { if audio_handler_data.closed.load(atomic::Ordering::Acquire) { return; } match audio_result { Ok(packet) => { let audio_frame_id = audio_handler_data.audio_frame_id_counter.fetch_add(1, atomic::Ordering::AcqRel); let event = StreamEvent::Audio(AudioFrame { impl_audio_frame: WindowsAudioFrame { data: packet.data.to_owned().into_boxed_slice(), channel_count: handler_config.channel_count, sample_rate: handler_config.sample_rate, duration: packet.duration, origin_time: packet.origin_time, frame_id: audio_frame_id } }); (*audio_handler_data.callback.lock())(Ok(event)); }, Err(_) => { (*audio_handler_data.callback.lock())(Err(StreamError::Other("Audio stream error".to_string()))); } } }); match WindowsAudioCaptureStream::new(audio_config, audio_handler) { Ok(audio_stream) => { Some(audio_stream) }, Err(_) => { return Err(StreamCreateError::Other("Failed to create audio stream".into())) } } } else { None }; Ok( StreamCreateOutput { auto_com, audio_stream, capture_session, dxgi_adapter: dxgi_adapter.map(|adapter| adapter.cast().unwrap()), dxgi_adapter_error, d3d11_device, #[cfg(feature = "wgpu")] wgpu_device, dxgi_device, frame_pool, shared_handler_data } ) } pub fn new(token: WindowsCaptureAccessToken, config: CaptureConfig, callback: Box) + Send + 'static>) -> Result { let auto_com = AutoCom::new(COINIT_APARTMENTTHREADED); let (init_tx, init_rx) = std::sync::mpsc::channel(); std::thread::spawn(move || { match Self::create_capture_stream(token, config, callback) { Err(error) => { _ = init_tx.send(Err(error)); return; }, Ok(stream_create_output) => { let StreamCreateOutput { dxgi_adapter, dxgi_adapter_error, dxgi_device, d3d11_device, #[cfg(feature = "wgpu")] wgpu_device, frame_pool, capture_session, ``` -------------------------------- ### Get CVPixelBuffer Width Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/src/crabgrab/platform/macos/objc_wrap.rs.html Returns the width of the CVPixelBuffer in pixels. ```rust pub fn get_width(&self) -> usize { unsafe { CVPixelBufferGetWidth(self.0) } } ``` -------------------------------- ### Get Window Title Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/src/crabgrab/platform/macos/capturable_content.rs.html Retrieves the title of the macOS window. ```rust pub fn title(&self) -> String { self.window.title() } ``` -------------------------------- ### Initialize CaptureStream with Callback Source: https://context7.com/augmendtech/crabgrab/llms.txt Starts a capture stream and processes incoming video, audio, and status events via a closure. Requires explicit capture access permissions. ```rust use std::time::Duration; use crabgrab::prelude::*; #[tokio::main] async fn main() { let token = CaptureStream::test_access(false) .or(CaptureStream::request_access(false).await) .expect("Capture access required"); let content = CapturableContent::new(CapturableContentFilter::DISPLAYS) .await .unwrap(); let display = content.displays().next().unwrap(); let config = CaptureConfig::with_display(display, CapturePixelFormat::Bgra8888); // Create capture stream with callback let mut stream = CaptureStream::new(token, config, |result| { match result { Ok(event) => match event { StreamEvent::Video(frame) => { println!("Frame #{}: size={:?}, dpi={:.1}, time={:?}", frame.frame_id(), frame.size(), frame.dpi(), frame.origin_time() ); // frame.content_rect() returns the actual content bounds // frame.capture_time() returns the Instant when delivered }, StreamEvent::Audio(audio) => { println!("Audio: {:?} rate, {:?} channels, duration={:?}", audio.sample_rate(), audio.channel_count(), audio.duration() ); }, StreamEvent::Idle => println!("Stream idle (window minimized?)"), StreamEvent::End => println!("Stream ended"), }, Err(error) => eprintln!("Stream error: {:?}", error), } }).expect("Failed to create capture stream"); // Capture for 5 seconds std::thread::sleep(Duration::from_secs(5)); // Stop the capture stream.stop().expect("Failed to stop stream"); } ``` -------------------------------- ### Initialize and Capture Windows Audio Stream Source: https://github.com/augmendtech/crabgrab/blob/main/docs/windows_docs/src/crabgrab/platform/windows/audio_capture_stream.rs.html Configures the audio endpoint and initializes the capture client for loopback recording. The capture loop runs in a separate thread to process audio packets. ```rust let mm_device_enumerator: IMMDeviceEnumerator = CoCreateInstance(&MMDeviceEnumerator, None, CLSCTX_ALL) .map_err(|e| WindowsAudioCaptureStreamCreateError::Other(format!("Failed to create MMDeviceEnumerator: {}", e.to_string())))?; let device = mm_device_enumerator.GetDefaultAudioEndpoint(eRender, eConsole) .map_err(|_| WindowsAudioCaptureStreamCreateError::EndpointEnumerationFailed)?; let audio_client: IAudioClient = device.Activate(CLSCTX_ALL, None) .map_err(|_| WindowsAudioCaptureStreamCreateError::AudioClientActivationFailed)?; let mut format = WAVEFORMATEX::default(); format.wFormatTag = WAVE_FORMAT_PCM as u16; format.nSamplesPerSec = match config.sample_rate { AudioSampleRate::Hz8000 => 8000, AudioSampleRate::Hz16000 => 16000, AudioSampleRate::Hz24000 => 24000, AudioSampleRate::Hz48000 => 48000, }; format.wBitsPerSample = 16; format.nChannels = match config.channel_count { AudioChannelCount::Mono => 1, AudioChannelCount::Stereo => 2, }; format.nBlockAlign = format.nChannels * 2; format.nAvgBytesPerSec = format.nSamplesPerSec * format.nBlockAlign as u32; format.cbSize = 0; let callback_format = format.clone(); let buffer_size = 512; let buffer_time = buffer_size as i64 * 10000000i64 / format.nSamplesPerSec as i64; let buffer_duration = Duration::from_nanos(buffer_time as u64 * 100); let half_buffer_duration = buffer_duration / 2; audio_client.Initialize(AUDCLNT_SHAREMODE_SHARED, AUDCLNT_STREAMFLAGS_LOOPBACK, buffer_time, buffer_time, &format as *const _, None) .map_err(|_| WindowsAudioCaptureStreamCreateError::AudioClientInitializeFailed)?; let capture_client : IAudioCaptureClient = audio_client.GetService() .map_err(|_| WindowsAudioCaptureStreamCreateError::AudioCaptureCreationFailed)?; let capture_client_send = SendCaptureClient::from_iaudiocaptureclient(capture_client); std::thread::spawn(move || { { let should_couninit = CoInitializeEx(None, COINIT_MULTITHREADED).is_ok(); let mut last_device_position = 0u64; let mut sample_count = 0u64; let capture_client = capture_client_send.into_iaudiocaptureclient(); loop { std::thread::sleep(half_buffer_duration); let _buffered_count = match capture_client.GetNextPacketSize() { Ok(count) => count, Err(_) => { (callback)(Err(WindowsAudioCaptureStreamError::Other(format!("Stream failed - couldn't fetch packet size")))); break; } }; let mut data_ptr: *mut u8 = std::ptr::null_mut(); let mut num_frames = 0u32; let mut flags = 0u32; let mut device_position = 0u64; match capture_client.GetBuffer(&mut data_ptr as *mut _, &mut num_frames as *mut _, &mut flags as *mut _, Some(&mut device_position as *mut _), None) { Ok(_) => { let packet = WindowsAudioCaptureStreamPacket { data: std::slice::from_raw_parts(data_ptr as *const i16, num_frames as usize * 2), channel_count: callback_format.nChannels as u32, origin_time: Duration::from_nanos(device_position as u64 * 100), duration: Duration::from_nanos((device_position - last_device_position) as u64), sample_index: sample_count }; (callback)(Ok(packet)); let _ = capture_client.ReleaseBuffer(num_frames); last_device_position = device_position; sample_count += num_frames as u64; }, Err(_) => { (callback)(Err(WindowsAudioCaptureStreamError::GetBufferFailed)); break; ``` -------------------------------- ### GET /capturable-content/displays Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/src/crabgrab/capturable_content.rs.html Retrieves an iterator over the currently capturable displays. ```APIDOC ## GET /capturable-content/displays ### Description Returns an iterator over the capturable displays available within the content instance. ### Method GET ### Endpoint /capturable-content/displays ### Response #### Success Response (200) - **iterator** (CapturableDisplayIterator) - An iterator yielding CapturableDisplay objects. ``` -------------------------------- ### GET /capturable-content/windows Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/src/crabgrab/capturable_content.rs.html Retrieves an iterator over the currently capturable windows. ```APIDOC ## GET /capturable-content/windows ### Description Returns an iterator over the capturable windows available within the content instance. ### Method GET ### Endpoint /capturable-content/windows ### Response #### Success Response (200) - **iterator** (CapturableWindowIterator) - An iterator yielding CapturableWindow objects. ``` -------------------------------- ### Initialize MacosCaptureConfig Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/src/crabgrab/platform/macos/capture_stream.rs.html Creates a new `MacosCaptureConfig` with default values. Defaults include `scale_to_fit` set to true and `maximum_fps` to None. ```rust impl MacosCaptureConfig { pub fn new() -> Self { Self { scale_to_fit: true, maximum_fps: None, resolution_type: MacosCaptureResolutionType::Nominal, #[cfg(feature = "metal")] metal_device: None, #[cfg(feature = "wgpu")] wgpu_device: None, } } } ``` -------------------------------- ### Get raw IOSurfaceRef Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/crabgrab/feature/iosurface/struct.IoSurface.html Retrieves the underlying raw IOSurfaceRef pointer. ```rust pub fn get_raw(&self) -> *const c_void ``` -------------------------------- ### Create AVAudioPCMBuffer Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/src/crabgrab/platform/macos/objc_wrap.rs.html Initializes an AVAudioPCMBuffer using an existing AVAudioFormat and an AudioBufferList without copying data. This method returns a Result, indicating success or failure during initialization. ```rust impl AVAudioPCMBuffer { pub fn new_with_format_buffer_list_no_copy_deallocator(format: AVAudioFormat, buffer_list_no_copy: *const AudioBufferList) -> Result { unsafe { let id: *mut AnyObject = msg_send![class!(AVAudioPCMBuffer), alloc]; let result: *mut AnyObject = msg_send![id, initWithPCMFormat: format bufferListNoCopy: buffer_list_no_copy]; if result.is_null() { let _: () = msg_send![id, release]; Err(()) } else { Ok(Self(id)) } } } } ``` -------------------------------- ### Implement WindowsDx11CaptureStream for CaptureStream Source: https://github.com/augmendtech/crabgrab/blob/main/docs/windows_docs/crabgrab/feature/dx11/trait.WindowsDx11CaptureStream.html Example implementation of the WindowsDx11CaptureStream trait for the CaptureStream struct. ```rust impl WindowsDx11CaptureStream for CaptureStream ``` -------------------------------- ### Method: with_wgpu_device Source: https://github.com/augmendtech/crabgrab/blob/main/docs/windows_docs/crabgrab/feature/wgpu/trait.WgpuCaptureConfigExt.html Configures a capture object with a specific Wgpu device. ```APIDOC ## fn with_wgpu_device ### Description Associates a Wgpu device with the current capture configuration. This method is required for implementations of the WgpuCaptureConfigExt trait. ### Parameters - **self** (Sized) - Required - The capture configuration instance. - **device** (Arc + Send + Sync + 'static>) - Required - An atomically reference-counted Wgpu device instance. ### Response - **Result** - Returns the configured instance on success, or an error string if the configuration fails. ``` -------------------------------- ### Redirect to Platform Windows Configuration Source: https://github.com/augmendtech/crabgrab/blob/main/docs/windows_docs/crabgrab/platform/windows/capture_stream/struct.WindowsAudioCaptureConfig.html Performs a client-side redirect to the Windows audio capture configuration structure documentation. ```javascript location.replace("../../../../crabgrab/platform/windows/struct.ImplAudioCaptureConfig.html" + location.search + location.hash); ``` -------------------------------- ### CGDisplay Functions Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/src/crabgrab/platform/macos/objc_wrap.rs.html Functions to get the main display ID and screen size. ```APIDOC ## CGDisplay Functions ### Description Functions to retrieve the main display's identifier and its physical dimensions. ### Method N/A (Functions) ### Endpoint N/A ### Parameters N/A ### Request Example N/A ### Response #### Success Response (Various) - `CGMainDisplayID`: Returns the unique identifier for the main display. - `CGDisplayScreenSize`: Returns the physical size of a given display. #### Response Example N/A ``` -------------------------------- ### Get Frame ID (Rust) Source: https://github.com/augmendtech/crabgrab/blob/main/docs/windows_docs/src/crabgrab/platform/windows/frame.rs.html Returns the unique identifier for the audio frame. ```rust fn frame_id(&self) -> u64 { self.frame_id } ``` -------------------------------- ### Get Origin Time (Rust) Source: https://github.com/augmendtech/crabgrab/blob/main/docs/windows_docs/src/crabgrab/platform/windows/frame.rs.html Returns the origin time of the audio buffer. ```rust fn origin_time(&self) -> std::time::Duration { self.origin_time } ``` -------------------------------- ### Windows Capture Configuration with WGPU and D3D11 Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/src/crabgrab/feature/wgpu/mod.rs.html Configures Windows capture using WGPU and D3D11. This snippet handles the creation of D3D11 devices and DXGI adapters, essential for screen capture on Windows platforms. ```rust wgpu_device: Some(wgpu_device.clone()), ..self.impl_capture_config }, ..self }) } } #[cfg(target_os = "windows")] { unsafe { let mut dxgi_adapter_result = Err("Unimplemented for this wgpu backend".to_string()); AsRef::::as_ref(&*wgpu_device).as_hal::(|device| { device.map(|device| { //device.raw_device().AddRef(); let raw_device_ptr = device.raw_device().as_mut_ptr() as *mut c_void; let raw_queue_ptr = device.raw_queue().as_mut_ptr() as *mut c_void; let d3d12_device = ID3D12Device::from_raw(raw_device_ptr); let d3d12_queue = ID3D12CommandQueue::from_raw(raw_queue_ptr); let adapter_luid = d3d12_device.GetAdapterLuid(); let dxgi_factory: IDXGIFactory5 = match CreateDXGIFactory() { Err(error) => { dxgi_adapter_result = Err(format!("Failed to create dxgi factory: {}", error.to_string())); return; }, Ok(factory) => factory, }; dxgi_adapter_result = dxgi_factory.EnumAdapterByLuid(adapter_luid) .map_err(|error| format!("Failed to find matching dxgi adapter for wgpu device: {}", error.to_string())) .map(|dxgi_adapter: IDXGIAdapter4| (dxgi_adapter, d3d12_device, d3d12_queue)); }) }); let (dxgi_adapter, _d3d12_device, _d3d12_queue) = dxgi_adapter_result?; let dxgi_adapter = dxgi_adapter.cast::().unwrap(); let mut d3d11_device = None; D3D11CreateDevice ( &dxgi_adapter, D3D_DRIVER_TYPE_UNKNOWN, None, D3D11_CREATE_DEVICE_BGRA_SUPPORT | D3D11_CREATE_DEVICE_DEBUG, Some(&[D3D_FEATURE_LEVEL_11_0]), D3D11_SDK_VERSION, Some(&mut d3d11_device), None, None ).map_err(|error| format!("Failed to create d3d11 device from dxgi adapter: {}", error.to_string()))?; let d3d11_device = d3d11_device.unwrap(); Ok(Self { impl_capture_config: WindowsCaptureConfig { d3d11_device: Some(d3d11_device), wgpu_device: Some(wgpu_device), dxgi_adapter: Some(dxgi_adapter), ..self.impl_capture_config }, ..self }) } } } } ``` -------------------------------- ### WindowsAudioCaptureStream::new Source: https://github.com/augmendtech/crabgrab/blob/main/docs/windows_docs/src/crabgrab/platform/windows/audio_capture_stream.rs.html Creates a new WindowsAudioCaptureStream instance. This function initializes the necessary Windows audio components and sets up the audio capture client. ```APIDOC ## WindowsAudioCaptureStream::new ### Description Initializes a new audio capture stream for Windows. This involves COM initialization, device enumeration, and audio client setup. ### Method `pub fn new(config: AudioCaptureConfig, callback: Box FnMut(Result, WindowsAudioCaptureStreamError>) + Send + 'static>) -> Result` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **config** (AudioCaptureConfig) - Configuration for the audio capture stream, including sample rate and channel count. - **callback** (Box) - A closure that will be called with captured audio data packets or errors. ### Request Example ```rust // Example usage (conceptual) let config = AudioCaptureConfig { sample_rate: 44100, channel_count: 2 }; let stream = WindowsAudioCaptureStream::new(config, Box::new(|result| { match result { Ok(packet) => { // Process audio packet }, Err(e) => { // Handle error } } })); ``` ### Response #### Success Response (Ok) - **Self** (WindowsAudioCaptureStream) - A new instance of the audio capture stream. #### Error Response (Err) - **WindowsAudioCaptureStreamCreateError** - An enum variant indicating the type of error encountered during creation (e.g., `EndpointEnumerationFailed`, `AudioClientActivationFailed`). #### Response Example ```rust // Success example let stream: Result = ...; // Error example let error: Result = Err(WindowsAudioCaptureStreamCreateError::AudioClientInitializeFailed); ``` ``` -------------------------------- ### Get Audio Duration (Rust) Source: https://github.com/augmendtech/crabgrab/blob/main/docs/windows_docs/src/crabgrab/platform/windows/frame.rs.html Returns the total duration of the audio buffer. ```rust fn duration(&self) -> std::time::Duration { self.duration } ``` -------------------------------- ### VideoFrame - type_id Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/crabgrab/frame/struct.VideoFrame.html Gets the TypeId of the VideoFrame. This is a standard Rust trait method. ```APIDOC ## VideoFrame - type_id ### Description Gets the `TypeId` of `self`. ### Method `type_id` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example None ### Response #### Success Response (200) - `TypeId`: The unique identifier for the type of the `VideoFrame`. #### Response Example None ``` -------------------------------- ### CaptureConfig Methods Source: https://github.com/augmendtech/crabgrab/blob/main/docs/windows_docs/crabgrab/capture_stream/struct.CaptureConfig.html Methods for initializing a CaptureConfig instance for windows or displays. ```APIDOC ## with_window ### Description Create a capture configuration for a given capturable window. ### Parameters - **window** (CapturableWindow) - Required - The window to capture. - **pixel_format** (CapturePixelFormat) - Required - The pixel format for the capture stream. ### Response - **Result** - Returns a new CaptureConfig or an error if configuration fails. ## with_display ### Description Create a capture configuration for a given capturable display. ### Parameters - **display** (CapturableDisplay) - Required - The display to capture. - **pixel_format** (CapturePixelFormat) - Required - The pixel format for the capture stream. ### Response - **CaptureConfig** - Returns a new CaptureConfig instance. ``` -------------------------------- ### Complete Window Capture Example in Rust Source: https://context7.com/augmendtech/crabgrab/llms.txt This Rust code snippet demonstrates a complete application for capturing screen frames of a specific window. It includes requesting capture permissions, finding a target window by application identifier, configuring the capture stream, and continuously capturing frames with basic error handling and logging. Ensure you have the necessary dependencies and permissions set up. ```rust use std::time::Duration; use crabgrab::prelude::*; use crabgrab::feature::bitmap::VideoFrameBitmap; fn main() { let runtime = tokio::runtime::Builder::new_multi_thread().build().unwrap(); runtime.block_on(async { // Step 1: Get capture permission let token = match CaptureStream::test_access(true) { Some(token) => { println!("Capture access already granted"); token }, None => { println!("Requesting capture permission..."); CaptureStream::request_access(true) .await .expect("Capture permission denied") } }; // Step 2: Enumerate windows let filter = CapturableContentFilter::NORMAL_WINDOWS; let content = CapturableContent::new(filter).await.unwrap(); // Step 3: Find target window let window = content.windows() .filter(|w| { let id = w.application().identifier().to_lowercase(); !w.title().is_empty() && ( id.contains("terminal") || id.contains("explorer") || id.contains("finder") ) }) .next(); match window { Some(window) => { println!("Found window: \"{}\"", window.title()); println!(" Application: {}", window.application().name()); println!(" Size: {:?}", window.rect().size); // Step 4: Create capture config let pixel_format = CaptureStream::supported_pixel_formats()[0]; let config = CaptureConfig::with_window(window, pixel_format) .expect("Invalid window") .with_show_cursor(true) .with_buffer_count(3); // Step 5: Start capture stream let mut frame_count = 0u64; let mut stream = CaptureStream::new(token, config, move |result| { match result { Ok(StreamEvent::Video(frame)) => { frame_count += 1; if frame_count % 30 == 0 { println!("Captured {} frames, latest: {:?}", frame_count, frame.size()); } }, Ok(StreamEvent::Idle) => { println!("Stream idle - window may be minimized"); }, Ok(StreamEvent::End) => { println!("Stream ended"); }, Ok(_) => {}, // Ignore other stream events Err(e) => { eprintln!("Stream error: {:?}", e); } } }).expect("Failed to create stream"); println!("Capturing for 10 seconds..."); tokio::time::sleep(Duration::from_secs(10)).await; // Step 6: Clean up stream.stop().expect("Failed to stop stream"); println!("Capture complete!"); }, None => { println!("No matching window found. Available windows:"); for w in content.windows() { println!(" - {} ({})", w.title(), w.application().identifier()); } } } }); runtime.shutdown_timeout(Duration::from_secs(5)); } ``` -------------------------------- ### CGDisplayStream Functions Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/src/crabgrab/platform/macos/objc_wrap.rs.html Functions for creating, starting, and stopping a display stream for screen recording. ```APIDOC ## CGDisplayStream Functions ### Description Functions to create a display stream for capturing screen content, start the stream, and stop it. ### Method N/A (Functions) ### Endpoint N/A ### Parameters N/A ### Request Example N/A ### Response #### Success Response (Various) - `CGDisplayStreamCreateWithDispatchQueue`: Creates a display stream with a specified dispatch queue and handler. - `CGDisplayStreamStart`: Starts the display stream. - `CGDisplayStreamStop`: Stops the display stream. #### Response Example N/A ``` -------------------------------- ### fn with_wgpu_device Source: https://github.com/augmendtech/crabgrab/blob/main/docs/macos_docs/crabgrab/capture_stream/struct.CaptureConfig.html Configures a capture stream with a Wgpu device to enable texture generation from video frames. ```APIDOC ## fn with_wgpu_device ### Description Supply a Wgpu device to the config, allowing the generation of Wgpu textures from video frames. ### Parameters #### Request Body - **wgpu_device** (Arc + Send + Sync + 'static>) - Required - The Wgpu device to be used for texture generation. ### Response #### Success Response (200) - **Result** - Returns the configured instance or an error string if the configuration fails. ```