### Implement Video Capture and Recording using NRSDK in C#
Source: https://docs.xreal.com/2.4.1/Input%20and%20Camera/Access%20RGB%20Camera
This C# example illustrates how to implement video capture and record it to local storage using the NRVideoCapture class from the NRSDK. It handles video configuration, starting and stopping capture, and setting preview textures. Dependencies include NRPreviewer, VideoRecordConfigPanel, and potentially audio recording permissions.
```csharp
// A video Capture Example
public class VideoCapture2LocalExample : MonoBehaviour
{
/// The previewer.
public NRPreviewer Previewer;
public VideoRecordConfigPanel m_ConfigPanel;
/// Save the video to Application.persistentDataPath.
/// The full pathname of the video save file.
public string VideoSavePath
{
get
{
string timeStamp = Time.time.ToString().Replace(".", "").Replace(":", "");
string filename = string.Format("Nreal_Record_{0}.mp4", timeStamp);
return Path.Combine(Application.persistentDataPath, filename);
}
}
/// The video capture.
NRVideoCapture m_VideoCapture = null;
/// Starts this object.
void Start()
{
CreateVideoCaptureTest();
}
/// Tests create video capture.
void CreateVideoCaptureTest()
{
NRVideoCapture.CreateAsync(false, delegate (NRVideoCapture videoCapture)
{
NRDebugger.Info("Created VideoCapture Instance!");
if (videoCapture != null)
{
m_VideoCapture = videoCapture;
}
else
{
NRDebugger.Error("Failed to create VideoCapture Instance!");
}
});
}
/// Starts video capture.
public void StartVideoCapture()
{
if (m_VideoCapture != null)
{
CameraParameters cameraParameters = new CameraParameters();
if (m_ConfigPanel == null)
{
Resolution cameraResolution = NRVideoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
cameraParameters.hologramOpacity = 0.0f;
cameraParameters.frameRate = cameraResolution.refreshRate;
cameraParameters.cameraResolutionWidth = cameraResolution.width;
cameraParameters.cameraResolutionHeight = cameraResolution.height;
cameraParameters.pixelFormat = CapturePixelFormat.BGRA32;
// Set the blend mode.
cameraParameters.blendMode = BlendMode.Blend;
// Set audio state, audio record needs the permission of "android.permission.RECORD_AUDIO",
// Add it to your "AndroidManifest.xml" file in "Assets/Plugin".
cameraParameters.audioState = NRVideoCapture.AudioState.MicAudio;
}
else
{
cameraParameters = m_ConfigPanel.GetRecordConfigration();
}
m_VideoCapture.StartVideoModeAsync(cameraParameters, OnStartedVideoCaptureMode);
}
}
/// Stops video capture.
public void StopVideoCapture()
{
if (m_VideoCapture == null)
{
return;
}
NRDebugger.Info("Stop Video Capture!");
m_VideoCapture.StopRecordingAsync(OnStoppedRecordingVideo);
Previewer.SetData(m_VideoCapture.PreviewTexture, false);
}
/// Executes the 'started video capture mode' action.
/// The result.
void OnStartedVideoCaptureMode(NRVideoCapture.VideoCaptureResult result)
{
if (!result.success)
{
NRDebugger.Info("Started Video Capture Mode faild!");
return;
}
NRDebugger.Info("Started Video Capture Mode!");
m_VideoCapture.StartRecordingAsync(VideoSavePath, OnStartedRecordingVideo);
// Set preview texture.
Previewer.SetData(m_VideoCapture.PreviewTexture, true);
}
/// Executes the 'started recording video' action.
/// The result.
void OnStartedRecordingVideo(NRVideoCapture.VideoCaptureResult result)
{
if (!result.success)
{
NRDebugger.Info("Started Recording Video Faild!");
return;
}
NRDebugger.Info("Started Recording Video!");
if (m_ConfigPanel != null && m_ConfigPanel.UseGreenBackground)
{
// Set green background color.
m_VideoCapture.GetContext().GetBehaviour().SetBackGroundColor(Color.green);
}
}
/// Executes the 'stopped recording video' action.
/// The result.
```
--------------------------------
### Getting RGB Camera's Intrinsic Parameters
Source: https://docs.xreal.com/2.4.1/Miscellaneous/NRSDK%20Coordinate%20Systems
An example showing how to retrieve the intrinsic matrix and distortion parameters for the RGB camera using NRFrame.
```APIDOC
## Getting RGB Camera's Intrinsic Parameters
### Description
This example demonstrates how to fetch the intrinsic matrix and distortion parameters for the RGB camera using the `NRFrame` class.
### Method
N/A (Code Example)
### Endpoint
N/A
### Parameters
None
### Request Example
```csharp
// Get the rgb camera's intrinsic matrix
NativeMat3f mat = NRFrame.GetDeviceIntrinsicMatrix(NativeDevice.RGB_CAMERA);
// Get the rgb camera's distortion coeffcients
NRDistortionParams distort = NRFrame.GetDeviceDistortion(NativeDevice.RGB_CAMERA);
```
### Response
N/A
### Response Example
N/A
```
--------------------------------
### Install CoastersTrackingModule in Unity
Source: https://docs.xreal.com/2.4.1/Image%20Tracking/Marker
Instructions for installing the CoastersTrackingModule within a Unity project using the NRSDK. This module is essential for utilizing the Coasters Image Tracking feature.
```csharp
NRSDK.CoastersTrackingModule.Install();
```
--------------------------------
### Get Camera Extrinsics and Convert to OpenCV Format
Source: https://docs.xreal.com/2.4.1/Miscellaneous/NRSDK%20Coordinate%20Systems
This example demonstrates how to retrieve the extrinsic transformation between the left and right grayscale cameras from the head pose. It then converts these transformations from Unity's coordinate system to OpenCV's convention for further processing. Dependencies include NRFrame, NativeDevice, Matrix4x4, and UnityToCVMatrix functions. The input is the device pose, and the output is the extrinsic matrix in OpenCV format.
```csharp
// Get Extrinsic Left Grayscale Camera From Head
Pose lCamPos = NRFrame.GetDevicePoseFromHead(NativeDevice.LEFT_GRAYSCALE_CAMERA);
Matrix4x4 Head_T_Lcam = Matrix4x4.TRS(lCamPos.position, lCamPos.rotation, Vector3.one);
// Get Extrinsic Right Grayscale Camera From Head
Pose rCamPos = NRFrame.GetDevicePoseFromHead(NativeDevice.RIGHT_GRAYSCALE_CAMERA);
Matrix4x4 Head_T_Rcam = Matrix4x4.TRS(rCamPos.position, rCamPos.rotation, Vector3.one);
// Calculate Extrinsic Right Camera From Left
Matrix4x4 unityLcam_T_unityRcam = Head_T_Lcam.inverse * Head_T_Rcam;
// Convert Unity Extrinsic to CV
Matrix4x4 cvLcam_T_cvRcam = UnityToCVMatrix(unityLcam_T_unityRcam);
// Transform a vector from right camera to left camera
Vector3 pInRCam = new Vector3(1, 0, 0);
Vector3 pInLCam = cvLcam_T_cvRcam.MultiplyPoint(pInRCam);
```
--------------------------------
### Start Mesh Classification Process (C#)
Source: https://docs.xreal.com/Depth%20Mesh/MeshClassification
Initiates the mesh scanning and classification process. This function is called after the AR environment has been initialized and surfaces are being detected. The scanned meshes are then automatically categorized based on the defined semantic labels.
```csharp
public void StartClassification()
{
// Begin scanning and classifying the environment
m_MeshClassifier.StartScanning();
}
```
--------------------------------
### Control Image Tracking with C# in Unity
Source: https://docs.xreal.com/2.4.1/Image%20Tracking/intro
This script manages the lifecycle of image tracking, including enabling, disabling, and updating visualizers for tracked images. It uses the NRKernal API to get trackable images and instantiate/destroy corresponding visualizer prefabs. Dependencies include Unity Engine and NRKernal namespaces.
```csharp
namespace NRKernal.NRExamples
{
using System.Collections.Generic;
using UnityEngine;
public class TrackingImageExampleController : MonoBehaviour
{
public TrackingImageVisualizer TrackingImageVisualizerPrefab;
public GameObject FitToScanOverlay;
private Dictionary m_Visualizers
= new Dictionary();
private List m_TempTrackingImages = new List();
public void Update()
{
#if !UNITY_EDITOR
if (NRFrame.SessionStatus != SessionState.Running)
{
return;
}
#endif
NRFrame.GetTrackables(m_TempTrackingImages, NRTrackableQueryFilter.New);
foreach (var image in m_TempTrackingImages)
{
TrackingImageVisualizer visualizer = null;
m_Visualizers.TryGetValue(image.GetDataBaseIndex(), out visualizer);
if (image.GetTrackingState() != TrackingState.Stopped && visualizer == null)
{
visualizer = Instantiate(TrackingImageVisualizerPrefab, image.GetCenterPose().position, image.GetCenterPose().rotation);
visualizer.Image = image;
visualizer.transform.parent = transform;
m_Visualizers.Add(image.GetDataBaseIndex(), visualizer);
}
else if (image.GetTrackingState() == TrackingState.Stopped && visualizer != null)
{
m_Visualizers.Remove(image.GetDataBaseIndex());
Destroy(visualizer.gameObject);
}
FitToScanOverlay.SetActive(false);
}
}
public void EnableImageTracking()
{
var config = NRSessionManager.Instance.NRSessionBehaviour.SessionConfig;
config.ImageTrackingMode = TrackableImageFindingMode.ENABLE;
NRSessionManager.Instance.SetConfiguration(config);
}
public void DisableImageTracking()
{
var config = NRSessionManager.Instance.NRSessionBehaviour.SessionConfig;
config.ImageTrackingMode = TrackableImageFindingMode.DISABLE;
NRSessionManager.Instance.SetConfiguration(config);
}
}
}
```
--------------------------------
### Query Controller Button State with NRInput
Source: https://docs.xreal.com/2.4.1/Input%20and%20Camera/NRInput
Demonstrates how to query the current, pressed, and released state of controller buttons using NRInput's Get, GetDown, and GetUp methods. This is fundamental for reacting to user input from the XREAL controller.
```csharp
//returns true if a Trigger button is currently being pressed
NRInput.GetButton(ControllerButton.TRIGGER);
//returns true if a Trigger button was pressed down this frame
NRInput.GetButtonDown(ControllerButton.TRIGGER);
//returns true if a Trigger button was released this frame
NRInput.GetButtonUp(ControllerButton.TRIGGER);
```
--------------------------------
### Upload and Download Files with Aliyun OSS .NET SDK in Unity
Source: https://docs.xreal.com/Spatial%20Anchor/Tutorial-Sharing%20Anchors/Aliyun
This C# script demonstrates how to integrate the Aliyun OSS .NET SDK into a Unity project. It provides methods for uploading files to and downloading files from an Aliyun OSS bucket, requiring Aliyun AccessKey ID, AccessKey Secret, endpoint, and bucket name for configuration. Ensure the Aliyun.OSS.dll is placed in the Assets > Plugins folder.
```csharp
using Aliyun.OSS;
using System;
using System.IO;
using System.Threading.Tasks;
using UnityEngine;
public class AliyunOSSManager
{
private static AliyunOSSManager _instance;
public static AliyunOSSManager Instance
{
get
{
if (_instance == null)
{
_instance = new AliyunOSSManager();
}
return _instance;
}
}
private string accessKeyId = "your-access-key-id";
private string accessKeySecret = "your-access-key-secret";
private string endpoint = "your-endpoint";//similar to:oss-cn-beijing.aliyuncs.com
private string bucketName = "your-bucket-name";
private OssClient client;
public AliyunOSSManager()
{
client = new OssClient(endpoint, accessKeyId, accessKeySecret);
}
public Task UploadFile(string objectName, string localFilename)
{
TaskCompletionSource tcs = new TaskCompletionSource();
try
{
client.PutObject(bucketName, objectName, localFilename);
Debug.Log("File uploaded successfully.");
tcs.SetResult(true); // Set the task as successfully completed.
}
catch (Exception e)
{
Debug.LogError("Failed to upload file: " + e.Message);
tcs.SetResult(false); // Set the task to complete, but failed to return.
}
return tcs.Task;
}
public Task DownloadFile(string objectName, string localFilename)
{
TaskCompletionSource tcs = new TaskCompletionSource();
try
{
OssObject ossObject = client.GetObject(bucketName, objectName);
using (var requestStream = ossObject.Content)
{
byte[] buf = new byte[1024];
var fs = File.Open(localFilename, FileMode.OpenOrCreate);
var len = 0;
while ((len = requestStream.Read(buf, 0, 1024)) != 0)
{
fs.Write(buf, 0, len);
}
fs.Close();
}
Debug.Log("File downloaded successfully.");
tcs.SetResult(true); // Set the task to be completed successfully.
}
catch (Exception e)
{
Debug.LogError("Failed to download file: " + e.Message);
tcs.SetResult(false); // Set the task to complete, but the return failed.
}
return tcs.Task;
}
}
```
--------------------------------
### Getting Extrinsics Between Grayscale Cameras
Source: https://docs.xreal.com/2.4.1/Miscellaneous/NRSDK%20Coordinate%20Systems
This example demonstrates how to calculate the extrinsic transformation between the left and right grayscale cameras and convert it to the OpenCV coordinate system.
```APIDOC
## Getting Extrinsics Between Grayscale Cameras
### Description
Calculates the extrinsic transformation between the left and right grayscale cameras and converts it to the OpenCV coordinate system.
### Method
N/A (Code Example)
### Endpoint
N/A
### Parameters
None
### Request Example
```csharp
// Get Extrinsic Left Grayscale Camera From Head
Pose lCamPos = NRFrame.GetDevicePoseFromHead(NativeDevice.LEFT_GRAYSCALE_CAMERA);
Matrix4x4 Head_T_Lcam = Matrix4x4.TRS(lCamPos.position, lCamPos.rotation, Vector3.one);
// Get Extrinsic Right Grayscale Camera From Head
Pose rCamPos = NRFrame.GetDevicePoseFromHead(NativeDevice.RIGHT_GRAYSCALE_CAMERA);
Matrix4x4 Head_T_Rcam = Matrix4x4.TRS(rCamPos.position, rCamPos.rotation, Vector3.one);
// Calculate Extrinsic Right Camera From Left
Matrix4x4 unityLcam_T_unityRcam = Head_T_Lcam.inverse * Head_T_Rcam;
// Convert Unity Extrinsic to CV
Matrix4x4 cvLcam_T_cvRcam = UnityToCVMatrix(unityLcam_T_unityRcam);
// Transform a vector from right camera to left camera
Vector3 pInRCam = new Vector3(1, 0, 0);
Vector3 pInLCam = cvLcam_T_cvRcam.MultiplyPoint(pInRCam);
```
### Response
N/A
### Response Example
N/A
```
--------------------------------
### Unity C# Script for Photon Room Joining
Source: https://docs.xreal.com/2.4.1/Spatial%20Anchor/Tutorial-Sharing%20Anchors/SettingUpPhoton
This script allows a Unity application to connect to the Photon master server, join a specific room, or create one if it doesn't exist. It uses callbacks to provide feedback on connection and room status, and updates a UI element to indicate successful room joining. It requires Photon Unity Networking and Photon Realtime libraries.
```csharp
using UnityEngine;
using Photon.Pun;
using Photon.Realtime;
using UnityEngine.UI;
public class JoinRoomButton : MonoBehaviourPunCallbacks
{
public Button joinedRoomIndicator;
public void OnButtonClick()
{
// Connect to the Photon master server
if (!PhotonNetwork.IsConnected)
{
PhotonNetwork.ConnectUsingSettings();
}
else
{
TryJoinRoom();
}
}
public override void OnConnectedToMaster()
{
Debug.Log("Connected to Photon master server");
TryJoinRoom();
}
private void TryJoinRoom()
{
// Try to join the room "MyRoom"
PhotonNetwork.JoinRoom("MyRoom");
}
public override void OnJoinedRoom()
{
Debug.Log("Joined room: " + PhotonNetwork.CurrentRoom.Name);
joinedRoomIndicator.image.color = Color.green;
}
public override void OnJoinRoomFailed(short returnCode, string message)
{
Debug.Log("Failed to join room");
// If we failed to join the room, it might not exist, so let's try to create it
PhotonNetwork.CreateRoom("MyRoom");
}
public override void OnCreatedRoom()
{
Debug.Log("Created room: " + PhotonNetwork.CurrentRoom.Name);
}
public override void OnCreateRoomFailed(short returnCode, string message)
{
Debug.Log("Failed to create room");
}
}
```
--------------------------------
### Getting Raw RGB Camera Image Data
Source: https://docs.xreal.com/2.4.1/Miscellaneous/NRSDK%20Coordinate%20Systems
Provides an example of how to obtain the raw byte array of an image from the RGB camera using Texture2D and GetRawTextureData.
```APIDOC
## Getting Raw RGB Camera Image Data
### Description
This example shows how to retrieve the raw image data from the RGB camera as a byte array using `Texture2D.GetRawTextureData()` accessed via `NRRGBCamTexture`.
### Method
N/A (Code Example)
### Endpoint
N/A
### Parameters
None
### Request Example
```csharp
public class CameraCaptureController : MonoBehaviour
{
// Save the reference for Texture2D from NRRGBCamTexture.
Texture2D mTex2d;
// The instance of NRRGBCamTexture.
NRRGBCamTexture mCamTex;
void Start()
{
// Create an instance of NRRGBCamTexture
mCamTex = new NRRGBCamTexture();
// Get Texture2D target and save it.
mTex2d = mCamTex.GetTexture();
mCamTex.Play();
}
void LateUpdate()
{
// Get raw data from Texture2D per frame.
byte[] rawData = mTex2d.GetRawTextureData();
}
}
```
### Response
N/A
### Response Example
N/A
```
--------------------------------
### Upload and Download Files with Aliyun OSS SDK in Unity C#
Source: https://docs.xreal.com/2.4.1/Spatial%20Anchor/Tutorial-Sharing%20Anchors/Aliyun
This C# script utilizes the Aliyun OSS .NET SDK to enable file upload and download functionality within a Unity project. It requires Aliyun OSS credentials (AccessKey ID, Secret, endpoint, bucket name) to interact with the Alibaba Cloud Object Storage Service. The script provides asynchronous methods for uploading and downloading files, handling potential exceptions and logging the outcomes.
```csharp
using Aliyun.OSS;
using System;
using System.IO;
using System.Threading.Tasks;
using UnityEngine;
public class AliyunOSSManager
{
private static AliyunOSSManager _instance;
public static AliyunOSSManager Instance
{
get
{
if (_instance == null)
{
_instance = new AliyunOSSManager();
}
return _instance;
}
}
private string accessKeyId = "your-access-key-id";
private string accessKeySecret = "your-access-key-secret";
private string endpoint = "your-endpoint"; //similar to:oss-cn-beijing.aliyuncs.com
private string bucketName = "your-bucket-name";
private OssClient client;
public AliyunOSSManager()
{
client = new OssClient(endpoint, accessKeyId, accessKeySecret);
}
public Task UploadFile(string objectName, string localFilename)
{
TaskCompletionSource tcs = new TaskCompletionSource();
try
{
client.PutObject(bucketName, objectName, localFilename);
Debug.Log("File uploaded successfully.");
tcs.SetResult(true); // Set the task as successfully completed.
}
catch (Exception e)
{
Debug.LogError("Failed to upload file: " + e.Message);
tcs.SetResult(false); // Set the task to complete, but failed to return.
}
return tcs.Task;
}
public Task DownloadFile(string objectName, string localFilename)
{
TaskCompletionSource tcs = new TaskCompletionSource();
try
{
OssObject ossObject = client.GetObject(bucketName, objectName);
using (var requestStream = ossObject.Content)
{
byte[] buf = new byte[1024];
var fs = File.Open(localFilename, FileMode.OpenOrCreate);
var len = 0;
while ((len = requestStream.Read(buf, 0, 1024)) != 0)
{
fs.Write(buf, 0, len);
}
fs.Close();
}
Debug.Log("File downloaded successfully.");
tcs.SetResult(true); // Set the task to be completed successfully.
}
catch (Exception e)
{
Debug.LogError("Failed to download file: " + e.Message);
tcs.SetResult(false); // Set the task to complete, but the return failed.
}
return tcs.Task;
}
}
```
--------------------------------
### Get Tracked Image List in NRSDK
Source: https://docs.xreal.com/2.4.1/Image%20Tracking/Marker
Example code snippet demonstrating how to retrieve the current list of tracked images using the NRFrame.GetTrackables method in the NRSDK. This is useful for applications that need to monitor recognized images.
```csharp
NRFrame.GetTrackables(m_TempTrackingImages, NRTrackableQueryFilter.All);
```
--------------------------------
### Joining and Creating Photon Rooms in Unity
Source: https://docs.xreal.com/Spatial%20Anchor/Tutorial-Sharing%20Anchors/SettingUpPhoton
This C# script allows a Unity application to connect to the Photon master server, join a specific room named 'MyRoom', or create the room if it doesn't exist. It uses Photon's PUN2 library and provides visual feedback by changing the color of a UI button upon successful room joining. Ensure Photon Unity Networking and Photon Realtime assemblies are referenced in your project's assembly definitions.
```csharp
using UnityEngine;
using Photon.Pun;
using Photon.Realtime;
using UnityEngine.UI;
public class JoinRoomButton : MonoBehaviourPunCallbacks
{
public Button joinedRoomIndicator;
public void OnButtonClick()
{
// Connect to the Photon master server
if (!PhotonNetwork.IsConnected)
{
PhotonNetwork.ConnectUsingSettings();
}
else
{
TryJoinRoom();
}
}
public override void OnConnectedToMaster()
{
Debug.Log("Connected to Photon master server");
TryJoinRoom();
}
private void TryJoinRoom()
{
// Try to join the room "MyRoom"
PhotonNetwork.JoinRoom("MyRoom");
}
public override void OnJoinedRoom()
{
Debug.Log("Joined room: " + PhotonNetwork.CurrentRoom.Name);
joinedRoomIndicator.image.color = Color.green;
}
public override void OnJoinRoomFailed(short returnCode, string message)
{
Debug.Log("Failed to join room");
// If we failed to join the room, it might not exist, so let's try to create it
PhotonNetwork.CreateRoom("MyRoom");
}
public override void OnCreatedRoom()
{
Debug.Log("Created room: " + PhotonNetwork.CurrentRoom.Name);
}
public override void OnCreateRoomFailed(short returnCode, string message)
{
Debug.Log("Failed to create room");
}
}
```
--------------------------------
### Control Hand Tracking Input Source
Source: https://docs.xreal.com/2.4.1/Hand%20Tracking
Demonstrates how to switch the input source to hand tracking or controller and check if hand tracking is currently running. It also shows how to retrieve the right-hand NRHand object and check for system gestures. Assumes NRInput and HandEnum are available.
```csharp
//returns true if input source switch to hand tracking success
bool switchToHandTracking = NRInput.SetInputSource(InputSourceEnum.Hands);
//returns true if input source switch to controller success bool
switchToController = NRInput.SetInputSource(InputSourceEnum.Controller);
//returns true if hand tracking is running bool isRunning =
NRInput.Hands.IsRunning;
//returns the NRHand of right-handness NRHand hand =
NRInput.Hands.GetHand(HandEnum.RightHand);
//returns true if user is now performing system gesture bool
isPerformingSystemGesture = NRInput.Hands.IsPerformingSystemGesture();
```
--------------------------------
### Get RGB Camera Intrinsic Parameters
Source: https://docs.xreal.com/2.4.1/Miscellaneous/NRSDK%20Coordinate%20Systems
This example demonstrates how to retrieve the intrinsic matrix and distortion parameters for the RGB camera. It uses the NRFrame class to access these fundamental camera calibration properties. The inputs are the NativeDevice enum specifying the RGB camera, and the outputs are NativeMat3f for the intrinsic matrix and NRDistortionParams for the distortion coefficients. This information is crucial for accurate camera pose estimation and 3D reconstruction.
```csharp
// Get the rgb camera's intrinsic matrix
NativeMat3f mat = NRFrame.GetDeviceIntrinsicMatrix(NativeDevice.RGB_CAMERA);
// Get the rgb camera's distortion coeffcients
NRDistortionParams distort = NRFrame.GetDeviceDistortion(NativeDevice.RGB_CAMERA);
```
--------------------------------
### Get Raw RGB Camera Image Data as Byte Array
Source: https://docs.xreal.com/2.4.1/Miscellaneous/NRSDK%20Coordinate%20Systems
This example shows how to acquire the raw image data from the RGB camera as a byte array. It utilizes Texture2D and NRRGBCamTexture to access the frame's pixel data, which is stored row-wise according to the OpenCV convention. The input is a Texture2D obtained from NRRGBCamTexture, and the output is a byte array containing the image pixels. This method is useful for low-level image processing.
```csharp
// Here are parts of the example code for using RGB camera, you can find the
// complete code in the CameraCaptureController file of NRSDK.
public class CameraCaptureController : MonoBehaviour
{
// Save the reference for Texture2D from NRRGBCamTexture.
Texture2D mTex2d;
// The instance of NRRGBCamTexture.
NRRGBCamTexture mCamTex;
void Start()
{
// Create an instance of NRRGBCamTexture
mCamTex = new NRRGBCamTexture();
// Get Texture2D target and save it.
mTex2d = mCamTex.GetTexture();
mCamTex.Play();
}
void LateUpdate()
{
// Get raw data from Texture2D per frame.
byte[] rawData = mTex2d.GetRawTextureData();
}
}
```
--------------------------------
### Use XREALInput Compatibility Layer for NRInput Calls
Source: https://docs.xreal.com/MigratingFromNRSDKToXREALSDK/intro
Demonstrates how to use the XREALInput compatibility layer to maintain code similarity with existing NRInput implementations. This involves updating namespace references and replacing NRInput calls with XREALInput equivalents.
```csharp
// Old code
using NRInput;
// New code
using Unity.XR.XREAL.Samples;
```
```csharp
// Old code
if (NRInput.GetButtonDown(ControllerButton.TRIGGER)) { }
Vector3 position = NRInput.GetPosition();
NRInput.TriggerHapticVibration(duration, amplitude);
// New code
if (XREALInput.GetButtonDown(ControllerButton.TRIGGER)) { }
Vector3 position = XREALInput.GetPosition();
XREALInput.TriggerHapticVibration(duration, amplitude);
```
--------------------------------
### Visualize Virtual Objects on Tracked Images with C#
Source: https://docs.xreal.com/2.4.1/Image%20Tracking/intro
This script visualizes virtual objects corresponding to tracked images. It instantiates specific GameObjects based on the image's database index and updates their position and rotation to match the tracked image. Dependencies include Unity Engine and NRKernal namespaces.
```csharp
namespace NRKernal.NRExamples
{
using UnityEngine;
public class TrackingImageVisualizer : MonoBehaviour
{
public NRTrackableImage Image;
// Configure virtual objects for different images
public GameObject VirtualObjectForImageA;
public GameObject VirtualObjectForImageB;
private GameObject instantiatedObject;
public void Update()
{
if (Image == null || Image.GetTrackingState() != TrackingState.Tracking)
{
if (instantiatedObject != null)
{
instantiatedObject.SetActive(false);
}
return;
}
if (instantiatedObject == null)
{
// Select the virtual object to instantiate based on the image
if (Image.GetDataBaseIndex() == 0) // Assuming index 0 is for ImageA
{
instantiatedObject = Instantiate(VirtualObjectForImageA, transform);
}
else if (Image.GetDataBaseIndex() == 1) // Assuming index 1 is for ImageB
{
instantiatedObject = Instantiate(VirtualObjectForImageB, transform);
}
}
if (instantiatedObject != null)
{
instantiatedObject.transform.position = Image.GetCenterPose().position;
instantiatedObject.transform.rotation = Image.GetCenterPose().rotation;
instantiatedObject.SetActive(true);
}
}
}
}
```
--------------------------------
### Get Image ID
Source: https://docs.xreal.com/2.4.1/Image%20Tracking/Marker
Retrieves the ID of a Marker image. The ID ranges from 0 to 10.
```APIDOC
## GET NRTrackableImage.GetCoastersDataBaseIndex
### Description
Retrieves the ID of a Marker image. In Marker, the value range of ID is [0,10], corresponding to different states of Marker cards.
### Method
GET
### Endpoint
/websites/xreal/NRTrackableImage/GetCoastersDataBaseIndex
### Parameters
#### Path Parameters
None
#### Query Parameters
None
### Request Example
```json
{
"example": "NRTrackableImage.GetCoastersDataBaseIndex()"
}
```
### Response
#### Success Response (200)
- **imageID** (integer) - The ID of the Marker image.
#### Response Example
```json
{
"imageID": 5
}
```
```
--------------------------------
### Implement CloudLoad in localMapExample (C#)
Source: https://docs.xreal.com/2.4.1/Spatial%20Anchor/Tutorial-Sharing%20Anchors/CloudSaveandLoad
Downloads a specified file from the cloud server using UUID and loads it into the scene via LoadwithUUID. It handles downloading the anchor-to-object dictionary and then the specific anchor file, updating the NRWorldAnchorStore and instantiating the anchor in the scene.
```csharp
public const string CloudAnchor2ObjectFile = "cloud_anchor2object.json";
public async Task CloudLoad(string uuid)
{
if (m_NRWorldAnchorStore == null)
{
Debug.Log("[m_NRWorldAnchorStore] is null");
return;
}
// Download the cloud anchor to object dictionary from the cloud
string path = Path.Combine(m_NRWorldAnchorStore.MapPath, CloudAnchor2ObjectFile);
//await ossManager.DownloadFile(CloudAnchor2ObjectFile, path);
await storageManager.DownloadFile(CloudAnchor2ObjectFile, path);
// Read the dictionary from the downloaded JSON file
Dictionary m_Anchor2ObjectDict = new Dictionary();
if (File.Exists(path))
{
string json = File.ReadAllText(path);
m_Anchor2ObjectDict = LitJson.JsonMapper.ToObject>(json);
}
// Get the UserDefinedKey for the given uuid
string UserDefinedKey;
if (m_Anchor2ObjectDict.TryGetValue(uuid, out UserDefinedKey))
{
// Download the anchor file from the cloud
string anchorPath = Path.Combine(m_NRWorldAnchorStore.MapPath, uuid);
//await ossManager.DownloadFile(uuid, anchorPath);
await storageManager.DownloadFile(uuid, anchorPath);
Debug.Log("anchorPath: "+anchorPath);
Debug.Log("uuid: "+uuid);
Debug.Log("UserDefinedKey: "+UserDefinedKey);
// Create a new dictionary for the specific UUID and Key
Dictionary specificAnchorDict = new Dictionary();
specificAnchorDict[uuid] = UserDefinedKey;
// Save the downloaded dictionary to the NRWorldAnchorStore
m_NRWorldAnchorStore.SetAnchor2ObjectDict(specificAnchorDict);
// Load the anchor
m_NRWorldAnchorStore.LoadwithUUID(uuid, (UInt64 handle) =>
{
var go = Instantiate(m_AnchorPrefabDict[UserDefinedKey]);
Debug.Log("cloud load: successful! ");
#if UNITY_EDITOR
go.transform.position = UnityEngine.Random.insideUnitSphere + Vector3.forward * 2;
#else
go.transform.position = Vector3.forward * 10000;
#endif
NRWorldAnchor anchor = go.AddComponent();
anchor.UserDefinedKey = UserDefinedKey;
anchor.UUID = uuid;
anchor.BindAnchor(handle);
go.SetActive(true);
NRDebugger.Info("[NRWorldAnchorStore] CloudLoadwithUUID: {0}, UserDefinedKey: {1} Handle: {2}", uuid, UserDefinedKey, handle);
});
}
else
{
Debug.LogError("No UserDefinedKey found for uuid: " + uuid);
}
}
```
--------------------------------
### Get Tracking Status
Source: https://docs.xreal.com/2.4.1/Image%20Tracking/Marker
Retrieves the current tracking state of an image. The states are Tracking and Paused.
```APIDOC
## GET NRTrackableImage.GetTrackingState
### Description
Retrieves the current tracking state of an image. In Marker, the `GetTrackingState` method will only return two states: **Tracking** and **Paused**.
### Method
GET
### Endpoint
/websites/xreal/NRTrackableImage/GetTrackingState
### Parameters
#### Path Parameters
None
#### Query Parameters
None
### Request Example
```json
{
"example": "NRTrackableImage.GetTrackingState()"
}
```
### Response
#### Success Response (200)
- **trackingState** (enum) - The current tracking state of the image. Possible values: Tracking, Paused, Stopped.
#### Response Example
```json
{
"trackingState": "Tracking"
}
```
```
--------------------------------
### Old NRSDK Image Tracking Code
Source: https://docs.xreal.com/MigratingFromNRSDKToXREALSDK/Migration_ImageTracking
This C# script demonstrates the old image tracking implementation using NRSDK. It manually instantiates visualizers for detected images. Dependencies include MonoBehaviour and NRSDK specific classes.
```csharp
public class TrackingImageExampleController : MonoBehaviour
{
public TrackingImageVisualizer TrackingImageVisualizerPrefab;
private Dictionary m_Visualizers = new Dictionary();
private List m_TempTrackingImages = new List();
public void Update()
{
NRFrame.GetTrackables(m_TempTrackingImages, NRTrackableQueryFilter.New);
foreach (var image in m_TempTrackingImages)
{
TrackingImageVisualizer visualizer = null;
m_Visualizers.TryGetValue(image.GetDataBaseIndex(), out visualizer);
if (image.GetTrackingState() != TrackingState.Stopped && visualizer == null)
{
visualizer = Instantiate(TrackingImageVisualizerPrefab, image.GetCenterPose().position, image.GetCenterPose().rotation);
visualizer.Image = image;
m_Visualizers.Add(image.GetDataBaseIndex(), visualizer);
}
}
}
}
```
--------------------------------
### Create CarManager Script for Car Initialization (C#)
Source: https://docs.xreal.com/2.4.1/Plane%20Detection/Add%20a%20car
This C# script manages the instantiation and initialization of the car prefab. It checks for user input (trigger tap) and the presence of a detected plane to spawn the car at the reticle's location. It requires NRKernal for input detection and Unity for game object manipulation. The script takes CarPrefab, ReticleBehaviour, and CarBehaviour as public dependencies.
```csharp
using System.Collections;
using System.Collections.Generic;
using NRKernal;
using UnityEngine;
public class CarManager : MonoBehaviour
{
public GameObject CarPrefab;
public ReticleBehaviour Reticle;
public CarBehaviour Car;
private GameObject LockedPlane;
private void Update()
{
if (Car == null && WasTapped() && Reticle.CurrentPlane != null)
{
// Spawn our car at the reticle location.
var obj = Instantiate(CarPrefab);
Car = obj.GetComponent();
Car.Reticle = Reticle;
Car.transform.position = Reticle.transform.position;
}
}
private bool WasTapped()
{
return NRInput.GetButtonDown(ControllerButton.TRIGGER);
}
}// Some code
```
--------------------------------
### Get Image Pose and Size Information
Source: https://docs.xreal.com/2.4.1/Image%20Tracking/Marker
Provides access to an image's dimensions (ExtentX, ExtentZ) and its center pose.
```APIDOC
## GET NRTrackableImage.ImagePoseAndSize
### Description
Provides access to an image's dimensions (ExtentX, ExtentZ) and its center pose.
### Method
GET
### Endpoint
/websites/xreal/NRTrackableImage/ImagePoseAndSize
### Parameters
#### Path Parameters
None
#### Query Parameters
None
### Request Example
```json
{
"example": "NRTrackableImage.ExtentX, NRTrackableImage.ExtentZ, NRTrackableImage.GetCenterPose()"
}
```
### Response
#### Success Response (200)
- **extentX** (float) - The length in the X direction of the image.
- **extentZ** (float) - The length in the Z direction of the image.
- **centerPose** (object) - The pose of the center of the image.
#### Response Example
```json
{
"extentX": 0.5,
"extentZ": 0.3,
"centerPose": {
"position": {"x": 0.1, "y": 0.2, "z": 0.3},
"rotation": {"x": 0.0, "y": 0.0, "z": 0.0, "w": 1.0}
}
}
```
```
--------------------------------
### Simulate 6DoF Head Pose Movement in Unity
Source: https://docs.xreal.com/2.4.1/Tools/Emulator
This C# code snippet demonstrates how to simulate the 6 Degrees of Freedom (6DoF) head pose movement in Unity using keyboard and mouse input. It translates player input into positional and rotational data, then uses the NREmulatorManager to send this data to the native emulator API. Requires Unity's Input system and a reference to a CameraTarget transform.
```csharp
void UpdateHeadPosByInput()
{
float mouse_x = Input.GetAxis("Mouse X") * HeadRotateSpeed;
float mouse_y = Input.GetAxis("Mouse Y") * HeadRotateSpeed;
Vector3 mouseMove = new Vector3(m_CameraTarget.transform.eulerAngles.x - mouse_y, m_CameraTarget.transform.eulerAngles.y + mouse_x, 0);
Quaternion q = Quaternion.Euler(mouseMove);
m_CameraTarget.transform.rotation = q;
Vector3 p = GetBaseInput();
p = p * HeadMoveSpeed * Time.deltaTime;
Vector3 pos = p + m_CameraTarget.transform.position;
m_CameraTarget.transform.position = pos;
// Call api to simulate the headpose movement
NREmulatorManager.Instance.NativeEmulatorApi.SetHeadTrackingPose(pos, q);
}
```