### Python Face Detection on Static Images with MediaPipe

Source: https://github.com/google-ai-edge/mediapipe/blob/master/docs/solutions/face_detection.md

Processes static images to detect faces using MediaPipe. It reads images, converts them to RGB, performs face detection, and draws bounding boxes and key points on the detected faces. Requires OpenCV and MediaPipe.

```python
import cv2
import mediapipe as mp

mp_face_detection = mp.solutions.face_detection
mp_drawing = mp.solutions.drawing_utils

IMAGE_FILES = [] # Replace with actual image file paths
with mp_face_detection.FaceDetection(
    model_selection=1, min_detection_confidence=0.5) as face_detection:
  for idx, file in enumerate(IMAGE_FILES):
    image = cv2.imread(file)
    # Convert the BGR image to RGB and process it with MediaPipe Face Detection.
    results = face_detection.process(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))

    # Draw face detections of each face.
    if not results.detections:
      continue
    annotated_image = image.copy()
    for detection in results.detections:
      print('Nose tip:')
      print(mp_face_detection.get_key_point(
          detection, mp_face_detection.FaceKeyPoint.NOSE_TIP))
      mp_drawing.draw_detection(annotated_image, detection)
    cv2.imwrite('/tmp/annotated_image' + str(idx) + '.png', annotated_image)
```

--------------------------------

### Detect Faces with MediaPipe Face Detector

Source: https://github.com/google-ai-edge/mediapipe/blob/master/mediapipe/tasks/web/vision/README.md

Initializes the MediaPipe Face Detector task using a WASM fileset and a TFLite model. It takes an HTMLImageElement as input and returns face detection results.

```javascript
const vision = await FilesetResolver.forVisionTasks(
    "https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision/wasm"
);
const faceDetector = await FaceDetector.createFromModelPath(vision,
    "https://storage.googleapis.com/mediapipe-models/face_detector/blaze_face_short_range/float16/1/blaze_face_short_range.tflite"
);
const image = document.getElementById("image") as HTMLImageElement;
const detections = faceDetector.detect(image);
```

--------------------------------

### Face Detection Configuration Options

Source: https://github.com/google-ai-edge/mediapipe/blob/master/docs/solutions/face_detection.md

Configuration options for the MediaPipe Face Detection solution, including model selection, selfie mode, and minimum detection confidence.

```APIDOC
## MediaPipe Face Detection Configuration

### Configuration Options

#### model_selection
- **Type**: Integer
- **Values**: `0` (short-range model, faces within 2 meters), `1` (full-range model, faces within 5 meters).
- **Default**: `0`
- **Note**: Not available for JavaScript (use "model" instead).

#### model
- **Type**: String
- **Values**: "short" (short-range model), "full" (full-range model).
- **Default**: ""
- **Note**: Valid only for JavaScript solution.

#### selfie_mode
- **Type**: Boolean
- **Default**: `false`
- **Note**: Valid only for JavaScript solution.

#### min_detection_confidence
- **Type**: Float
- **Range**: `[0.0, 1.0]`
- **Default**: `0.5`
- **Description**: Minimum confidence value for a detection to be considered successful.

### Output

#### detections
- **Type**: Collection of detection proto messages
- **Description**: Each detection contains a bounding box (`xmin`, `width`, `ymin`, `height` - all normalized) and 6 key points (`x`, `y` - normalized).
```

--------------------------------

### Python Face Detection on Webcam Feed with MediaPipe

Source: https://github.com/google-ai-edge/mediapipe/blob/master/docs/solutions/face_detection.md

Performs real-time face detection on a webcam feed using MediaPipe. It captures frames, converts them to RGB, processes them for face detection, and displays the annotated video feed. Requires OpenCV and MediaPipe.

```python
import cv2
import mediapipe as mp

mp_face_detection = mp.solutions.face_detection
mp_drawing = mp.solutions.drawing_utils

cap = cv2.VideoCapture(0)
with mp_face_detection.FaceDetection(
    model_selection=0, min_detection_confidence=0.5) as face_detection:
  while cap.isOpened():
    success, image = cap.read()
    if not success:
      print("Ignoring empty camera frame.")
      # If loading a video, use 'break' instead of 'continue'.
      continue

    # To improve performance, optionally mark the image as not writeable to
    # pass by reference.
    image.flags.writeable = False
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    results = face_detection.process(image)

    # Draw the face detection annotations on the image.
    image.flags.writeable = True
    image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
    if results.detections:
      for detection in results.detections:
        mp_drawing.draw_detection(image, detection)
    # Flip the image horizontally for a selfie-view display.
    cv2.imshow('MediaPipe Face Detection', cv2.flip(image, 1))
    if cv2.waitKey(5) & 0xFF == 27:
      break
cap.release()
```

--------------------------------

### Python Face Detection API Usage

Source: https://github.com/google-ai-edge/mediapipe/blob/master/docs/solutions/face_detection.md

Example usage of the MediaPipe Face Detection API in Python, demonstrating initialization and basic setup.

```APIDOC
## Python Face Detection API

### Description
This section provides an example of how to use the MediaPipe Face Detection API in Python. It assumes you have already installed the MediaPipe Python package.

### Supported Configuration Options

*   `model_selection`
*   `min_detection_confidence`

### Usage Example

```python
import cv2
import mediapipe as mp

mp_face_detection = mp.solutions.face_detection
mp_drawing = mp.solutions.drawing_utils

# Initialize Face Detection
with mp_face_detection.FaceDetection(
    model_selection=0, min_detection_confidence=0.5) as face_detection:
    
    # Load image or capture video frame
    image = cv2.imread("path/to/your/image.jpg")
    if image is None:
        print("Error: Could not load image.")
    else:
        # Convert the BGR image to RGB
        results = face_detection.process(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))

        # Draw face detections on the image
        if results.detections:
            annotated_image = image.copy()
            mp_drawing.draw_detection(
                annotated_image, results.detections[0])
            cv2.imshow('Face Detection', annotated_image)
            cv2.waitKey(0)
            cv2.destroyAllWindows()
        else:
            print("No faces detected.")
```

### Request Body

This API does not typically use a request body for initialization. Configuration is passed as arguments to the `FaceDetection` class constructor.

### Response

#### Success Response (Object)

- **detections** (List[DetectionProto]): A list of detected faces, where each `DetectionProto` contains bounding box and keypoint information.
```

--------------------------------

### Android Camera Input and Face Detection with OpenGL

Source: https://github.com/google-ai-edge/mediapipe/blob/master/docs/solutions/face_detection.md

This Java code snippet demonstrates setting up MediaPipe Face Detection on Android. It configures face detection options, initializes camera input, connects it to the face detection solution, and sets up an OpenGL surface view for rendering results. The code also shows how to extract specific facial landmarks like the nose tip and log their normalized coordinates. It handles camera starting and rendering requests.

```java
// For camera input and result rendering with OpenGL.
FaceDetectionOptions faceDetectionOptions = 
    FaceDetectionOptions.builder()
        .setStaticImageMode(false)
        .setModelSelection(0).build();
FaceDetection faceDetection = new FaceDetection(this, faceDetectionOptions);
faceDetection.setErrorListener(
    (message, e) -> Log.e(TAG, "MediaPipe Face Detection error:" + message));

// Initializes a new CameraInput instance and connects it to MediaPipe Face Detection Solution.
CameraInput cameraInput = new CameraInput(this);
cameraInput.setNewFrameListener(
    textureFrame -> faceDetection.send(textureFrame));

// Initializes a new GlSurfaceView with a ResultGlRenderer<FaceDetectionResult> instance
// that provides the interfaces to run user-defined OpenGL rendering code.
// See mediapipe/examples/android/solutions/facedetection/src/main/java/com/google/mediapipe/examples/facedetection/FaceDetectionResultGlRenderer.java
// as an example.
SolutionGlSurfaceView<FaceDetectionResult> glSurfaceView =
    new SolutionGlSurfaceView<>(
        this, faceDetection.getGlContext(), faceDetection.getGlMajorVersion());
glSurfaceView.setSolutionResultRenderer(new FaceDetectionResultGlRenderer());
glSurfaceView.setRenderInputImage(true);
faceDetection.setResultListener(
    faceDetectionResult -> {
      if (faceDetectionResult.multiFaceDetections().isEmpty()) {
        return;
      }
      RelativeKeypoint noseTip =
          faceDetectionResult
              .multiFaceDetections()
              .get(0)
              .getLocationData()
              .getRelativeKeypoints(FaceKeypoint.NOSE_TIP);
      Log.i(
          TAG,
          String.format(
              "MediaPipe Face Detection nose tip normalized coordinates (value range: [0, 1]): x=%f, y=%f",
              noseTip.getX(), noseTip.getY()));
      // Request GL rendering.
      glSurfaceView.setRenderData(faceDetectionResult);
      glSurfaceView.requestRender();
    });

// The runnable to start camera after the GLSurfaceView is attached.
glSurfaceView.post(
    () ->
        cameraInput.start(
            this,
            faceDetection.getGlContext(),
            CameraInput.CameraFacing.FRONT,
            glSurfaceView.getWidth(),
            glSurfaceView.getHeight()));

```

--------------------------------

### JavaScript Face Detection in Web Browser with MediaPipe

Source: https://github.com/google-ai-edge/mediapipe/blob/master/docs/solutions/face_detection.md

Implements real-time face detection in a web browser using MediaPipe and JavaScript. It utilizes HTML video and canvas elements, along with MediaPipe's drawing utilities, to display detected faces. Requires loading MediaPipe and utility scripts from CDN.

```html
<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/camera_utils/camera_utils.js" crossorigin="anonymous"></script>
  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/control_utils/control_utils.js" crossorigin="anonymous"></script>
  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/drawing_utils/drawing_utils.js" crossorigin="anonymous"></script>
  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/face_detection/face_detection.js" crossorigin="anonymous"></script>
</head>

<body>
  <div class="container">
    <video class="input_video"></video>
    <canvas class="output_canvas" width="1280px" height="720px"></canvas>
  </div>
</body>
</html>
```

```javascript
<script type="module">
const videoElement = document.getElementsByClassName('input_video')[0];
const canvasElement = document.getElementsByClassName('output_canvas')[0];
const canvasCtx = canvasElement.getContext('2d');
const drawingUtils = window;

function onResults(results) {
  // Draw the overlays.
  canvasCtx.save();
  canvasCtx.clearRect(0, 0, canvasElement.width, canvasElement.height);
  canvasCtx.drawImage(
      results.image, 0, 0, canvasElement.width, canvasElement.height);
  if (results.detections.length > 0) {
    drawingUtils.drawRectangle(
        canvasCtx, results.detections[0].boundingBox,
        {color: 'blue', lineWidth: 4, fillColor: '#00000000'});
    drawingUtils.drawLandmarks(canvasCtx, results.detections[0].landmarks, {
      color: 'red',
      radius: 5,
    });
  }
  canvasCtx.restore();
}

const faceDetection = new FaceDetection({
    locateFile: (file) => {
  return `https://cdn.jsdelivr.net/npm/@mediapipe/face_detection@0.0/${file}`;
}});
faceDetection.setOptions({
  model: 'short',
  minDetectionConfidence: 0.5
});
faceDetection.onResults(onResults);

const camera = new Camera(videoElement, {
  onFrame: async () => {
    await faceDetection.send({image: videoElement});
  },
  width: 1280,
  height: 720
});
camera.start();
</script>
```

--------------------------------

### Detect Face Landmarks with MediaPipe Face Landmarker

Source: https://github.com/google-ai-edge/mediapipe/blob/master/mediapipe/tasks/web/vision/README.md

Initializes the MediaPipe Face Landmarker task using a WASM fileset and a TFLite model. It takes an HTMLImageElement as input and returns face landmark results.

```javascript
const vision = await FilesetResolver.forVisionTasks(
    "https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision/wasm"
);
const faceLandmarker = await FaceLandmarker.createFromModelPath(vision,
    "https://storage.googleapis.com/mediapipe-models/face_landmarker/face_landmarker/float16/1/face_landmarker.task"
);
const image = document.getElementById("image") as HTMLImageElement;
const landmarks = faceLandmarker.detect(image);
```

--------------------------------

### Initialize and Use MediaPipe Face Detection with Image Input (Java)

Source: https://github.com/google-ai-edge/mediapipe/blob/master/docs/solutions/face_detection.md

This snippet shows how to set up MediaPipe Face Detection for static images, retrieve an image from the gallery using ActivityResultLauncher, and send the bitmap to the face detection model. It includes options for model selection and displays the nose tip coordinates upon successful detection.

```java
// For reading images from gallery and drawing the output in an ImageView.
FaceDetectionOptions faceDetectionOptions = 
    FaceDetectionOptions.builder()
        .setStaticImageMode(true)
        .setModelSelection(0).build();
FaceDetection faceDetection = new FaceDetection(this, faceDetectionOptions);

// Connects MediaPipe Face Detection Solution to the user-defined ImageView
// instance that allows users to have the custom drawing of the output landmarks
// on it. See mediapipe/examples/android/solutions/facedetection/src/main/java/com/google/mediapipe/examples/facedetection/FaceDetectionResultImageView.java
// as an example.
FaceDetectionResultImageView imageView = new FaceDetectionResultImageView(this);
faceDetection.setResultListener(
    faceDetectionResult -> {
      if (faceDetectionResult.multiFaceDetections().isEmpty()) {
        return;
      }
      int width = faceDetectionResult.inputBitmap().getWidth();
      int height = faceDetectionResult.inputBitmap().getHeight();
      RelativeKeypoint noseTip =
          faceDetectionResult
              .multiFaceDetections()
              .get(0) 
              .getLocationData()
              .getRelativeKeypoints(FaceKeypoint.NOSE_TIP);
      Log.i(
          TAG,
          String.format(
              "MediaPipe Face Detection nose tip coordinates (pixel values): x=%f, y=%f",
              noseTip.getX() * width, noseTip.getY() * height));
      // Request canvas drawing.
      imageView.setFaceDetectionResult(faceDetectionResult);
      runOnUiThread(() -> imageView.update());
    });
faceDetection.setErrorListener(
    (message, e) -> Log.e(TAG, "MediaPipe Face Detection error:" + message));

// ActivityResultLauncher to get an image from the gallery as Bitmap.
ActivityResultLauncher<Intent> imageGetter = 
    registerForActivityResult(
        new ActivityResultContracts.StartActivityForResult(),
        result -> {
          Intent resultIntent = result.getData();
          if (resultIntent != null && result.getResultCode() == RESULT_OK) {
            Bitmap bitmap = null;
            try {
              bitmap = 
                  MediaStore.Images.Media.getBitmap(
                      this.getContentResolver(), resultIntent.getData());
              // Please also rotate the Bitmap based on its orientation.
            } catch (IOException e) {
              Log.e(TAG, "Bitmap reading error:" + e);
            }
            if (bitmap != null) {
              faceDetection.send(bitmap);
            }
          }
        });
Intent pickImageIntent = new Intent(Intent.ACTION_PICK);
pickImageIntent.setDataAndType(MediaStore.Images.Media.INTERNAL_CONTENT_URI, "image/*");
imageGetter.launch(pickImageIntent);

```

--------------------------------

### Real-time Webcam Face Mesh Detection with Python

Source: https://github.com/google-ai-edge/mediapipe/blob/master/docs/solutions/face_mesh.md

This code snippet captures video from a webcam, processes each frame using MediaPipe Face Mesh to detect facial landmarks, and draws these landmarks onto the image. It requires OpenCV and MediaPipe libraries. The output is a real-time video feed with facial mesh annotations displayed in a window.

```python
import cv2
import mediapipe as mp

mp_drawing = mp.solutions.drawing_utils
mp_drawing_styles = mp.solutions.drawing_styles
mp_face_mesh = mp.solutions.face_mesh

# For webcam input:
drawing_spec = mp_drawing.DrawingSpec(thickness=1, circle_radius=1)
cap = cv2.VideoCapture(0)
with mp_face_mesh.FaceMesh(
    max_num_faces=1,
    refine_landmarks=True,
    min_detection_confidence=0.5,
    min_tracking_confidence=0.5) as face_mesh:
  while cap.isOpened():
    success, image = cap.read()
    if not success:
      print("Ignoring empty camera frame.")
      # If loading a video, use 'break' instead of 'continue'.
      continue

    # To improve performance, optionally mark the image as not writeable to
    # pass by reference.
    image.flags.writeable = False
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    results = face_mesh.process(image)

    # Draw the face mesh annotations on the image.
    image.flags.writeable = True
    image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
    if results.multi_face_landmarks:
      for face_landmarks in results.multi_face_landmarks:
        mp_drawing.draw_landmarks(
            image=image,
            landmark_list=face_landmarks,
            connections=mp_face_mesh.FACEMESH_TESSELATION,
            landmark_drawing_spec=None,
            connection_drawing_spec=mp_drawing_styles
            .get_default_face_mesh_tesselation_style())
        mp_drawing.draw_landmarks(
            image=image,
            landmark_list=face_landmarks,
            connections=mp_face_mesh.FACEMESH_CONTOURS,
            landmark_drawing_spec=None,
            connection_drawing_spec=mp_drawing_styles
            .get_default_face_mesh_contours_style())
        mp_drawing.draw_landmarks(
            image=image,
            landmark_list=face_landmarks,
            connections=mp_face_mesh.FACEMESH_IRISES,
            landmark_drawing_spec=None,
            connection_drawing_spec=mp_drawing_styles
            .get_default_face_mesh_iris_connections_style())
    # Flip the image horizontally for a selfie-view display.
    cv2.imshow('MediaPipe Face Mesh', cv2.flip(image, 1))
    if cv2.waitKey(5) & 0xFF == 27:
      break
cap.release()
```

--------------------------------

### Android Face Detection with Video Input using Java

Source: https://github.com/google-ai-edge/mediapipe/blob/master/docs/solutions/face_detection.md

This Java code snippet demonstrates how to use MediaPipe Face Detection with video input on Android. It initializes the FaceDetection model, sets up video input handling, configures OpenGL rendering for results, and includes an ActivityResultLauncher to pick a video file from the device. The code logs the normalized coordinates of the nose tip for the first detected face.

```java
// For video input and result rendering with OpenGL.
FaceDetectionOptions faceDetectionOptions = 
    FaceDetectionOptions.builder()
        .setStaticImageMode(false)
        .setModelSelection(0).build();
FaceDetection faceDetection = new FaceDetection(this, faceDetectionOptions);
faceDetection.setErrorListener(
    (message, e) -> Log.e(TAG, "MediaPipe Face Detection error:" + message));

// Initializes a new VideoInput instance and connects it to MediaPipe Face Detection Solution.
VideoInput videoInput = new VideoInput(this);
videoInput.setNewFrameListener(
    textureFrame -> faceDetection.send(textureFrame));

// Initializes a new GlSurfaceView with a ResultGlRenderer<FaceDetectionResult> instance
// that provides the interfaces to run user-defined OpenGL rendering code.
// See mediapipe/examples/android/solutions/facedetection/src/main/java/com/google/mediapipe/examples/facedetection/FaceDetectionResultGlRenderer.java
// as an example.
SolutionGlSurfaceView<FaceDetectionResult> glSurfaceView =
    new SolutionGlSurfaceView<>(
        this, faceDetection.getGlContext(), faceDetection.getGlMajorVersion());
glSurfaceView.setSolutionResultRenderer(new FaceDetectionResultGlRenderer());
glSurfaceView.setRenderInputImage(true);

faceDetection.setResultListener(
    faceDetectionResult -> {
      if (faceDetectionResult.multiFaceDetections().isEmpty()) {
        return;
      }
      RelativeKeypoint noseTip =
          faceDetectionResult
              .multiFaceDetections()
              .get(0)
              .getLocationData()
              .getRelativeKeypoints(FaceKeypoint.NOSE_TIP);
      Log.i(
          TAG,
          String.format(
              "MediaPipe Face Detection nose tip normalized coordinates (value range: [0, 1]): x=%f, y=%f",
              noseTip.getX(), noseTip.getY()));
      // Request GL rendering.
      glSurfaceView.setRenderData(faceDetectionResult);
      glSurfaceView.requestRender();
    });

ActivityResultLauncher<Intent> videoGetter =
    registerForActivityResult(
        new ActivityResultContracts.StartActivityForResult(),
        result -> {
          Intent resultIntent = result.getData();
          if (resultIntent != null) {
            if (result.getResultCode() == RESULT_OK) {
              glSurfaceView.post(
                  () ->
                      videoInput.start(
                          this,
                          resultIntent.getData(),
                          faceDetection.getGlContext(),
                          glSurfaceView.getWidth(),
                          glSurfaceView.getHeight()));
            }
          }
        });
Intent pickVideoIntent = new Intent(Intent.ACTION_PICK);
pickVideoIntent.setDataAndType(MediaStore.Video.Media.INTERNAL_CONTENT_URI, "video/*");
videoGetter.launch(pickVideoIntent);

```

--------------------------------

### JavaScript Face Mesh Integration and Real-time Processing

Source: https://github.com/google-ai-edge/mediapipe/blob/master/docs/solutions/face_mesh.md

This JavaScript code integrates the MediaPipe Face Mesh solution to detect and draw facial landmarks in real-time. It initializes the FaceMesh model, sets configuration options, defines a callback for processing results, and starts a camera feed to send frames for analysis. Dependencies include the MediaPipe Face Mesh library and camera utilities.

```javascript
<script type="module">
const videoElement = document.getElementsByClassName('input_video')[0];
const canvasElement = document.getElementsByClassName('output_canvas')[0];
const canvasCtx = canvasElement.getContext('2d');

function onResults(results) {
  canvasCtx.save();
  canvasCtx.clearRect(0, 0, canvasElement.width, canvasElement.height);
  canvasCtx.drawImage(
      results.image, 0, 0, canvasElement.width, canvasElement.height);
  if (results.multiFaceLandmarks) {
    for (const landmarks of results.multiFaceLandmarks) {
      drawConnectors(canvasCtx, landmarks, FACEMESH_TESSELATION,
                     {color: '#C0C0C070', lineWidth: 1});
      drawConnectors(canvasCtx, landmarks, FACEMESH_RIGHT_EYE, {color: '#FF3030'});
      drawConnectors(canvasCtx, landmarks, FACEMESH_RIGHT_EYEBROW, {color: '#FF3030'});
      drawConnectors(canvasCtx, landmarks, FACEMESH_RIGHT_IRIS, {color: '#FF3030'});
      drawConnectors(canvasCtx, landmarks, FACEMESH_LEFT_EYE, {color: '#30FF30'});
      drawConnectors(canvasCtx, landmarks, FACEMESH_LEFT_EYEBROW, {color: '#30FF30'});
      drawConnectors(canvasCtx, landmarks, FACEMESH_LEFT_IRIS, {color: '#30FF30'});
      drawConnectors(canvasCtx, landmarks, FACEMESH_FACE_OVAL, {color: '#E0E0E0'});
      drawConnectors(canvasCtx, landmarks, FACEMESH_LIPS, {color: '#E0E0E0'});
    }
  }
  canvasCtx.restore();
}

const faceMesh = new FaceMesh({locateFile: (file) => {
  return `https://cdn.jsdelivr.net/npm/@mediapipe/face_mesh/${file}`;
}});
faceMesh.setOptions({
  maxNumFaces: 1,
  refineLandmarks: true,
  minDetectionConfidence: 0.5,
  minTrackingConfidence: 0.5
});
faceMesh.onResults(onResults);

const camera = new Camera(videoElement, {
  onFrame: async () => {
    await faceMesh.send({image: videoElement});
  },
  width: 1280,
  height: 720
});
camera.start();
</script>
```

--------------------------------

### Import MediaPipe Face Mesh in Python

Source: https://github.com/google-ai-edge/mediapipe/blob/master/docs/getting_started/python.md

Imports the MediaPipe library and specifically the face mesh solution within a Python interpreter session. This is a prerequisite for using MediaPipe's face detection capabilities.

```python
import mediapipe as mp
mp_face_mesh = mp.solutions.face_mesh
```

--------------------------------

### Image Input with MediaPipe Face Mesh (Java)

Source: https://github.com/google-ai-edge/mediapipe/blob/master/docs/solutions/face_mesh.md

This snippet demonstrates how to process a static image using MediaPipe Face Mesh on Android. It sets up Face Mesh to work in static image mode, connects it to a custom ImageView for drawing results, and uses ActivityResultLauncher to pick an image from the gallery. Dependencies include MediaPipe, Android's MediaStore, and ActivityResultContracts.

```java
// For reading images from gallery and drawing the output in an ImageView.
FaceMeshOptions faceMeshOptions =
    FaceMeshOptions.builder()
        .setStaticImageMode(true)
        .setRefineLandmarks(true)
        .setMaxNumFaces(1)
        .setRunOnGpu(true).build();
FaceMesh faceMesh = new FaceMesh(this, faceMeshOptions);

// Connects MediaPipe Face Mesh Solution to the user-defined ImageView instance
// that allows users to have the custom drawing of the output landmarks on it.
// See mediapipe/examples/android/solutions/facemesh/src/main/java/com/google/mediapipe/examples/facemesh/FaceMeshResultImageView.java
// as an example.
FaceMeshResultImageView imageView = new FaceMeshResultImageView(this);
faceMesh.setResultListener(
    faceMeshResult -> {
      int width = faceMeshResult.inputBitmap().getWidth();
      int height = faceMeshResult.inputBitmap().getHeight();
      NormalizedLandmark noseLandmark =
          result.multiFaceLandmarks().get(0).getLandmarkList().get(1);
      Log.i(
          TAG,
          String.format(
              "MediaPipe Face Mesh nose coordinates (pixel values): x=%f, y=%f",
              noseLandmark.getX() * width, noseLandmark.getY() * height));
      // Request canvas drawing.
      imageView.setFaceMeshResult(faceMeshResult);
      runOnUiThread(() -> imageView.update());
    });
faceMesh.setErrorListener(
    (message, e) -> Log.e(TAG, "MediaPipe Face Mesh error:" + message));

// ActivityResultLauncher to get an image from the gallery as Bitmap.
ActivityResultLauncher<Intent> imageGetter =
    registerForActivityResult(
        new ActivityResultContracts.StartActivityForResult(),
        result -> {
          Intent resultIntent = result.getData();
          if (resultIntent != null && result.getResultCode() == RESULT_OK) {
            Bitmap bitmap = null;
            try {
              bitmap =
                  MediaStore.Images.Media.getBitmap(
                      this.getContentResolver(), resultIntent.getData());
              // Please also rotate the Bitmap based on its orientation.
            } catch (IOException e) {
              Log.e(TAG, "Bitmap reading error:" + e);
            }
            if (bitmap != null) {
              faceMesh.send(bitmap);
            }
          }
        });
Intent pickImageIntent = new Intent(Intent.ACTION_PICK);
pickImageIntent.setDataAndType(MediaStore.Images.Media.INTERNAL_CONTENT_URI, "image/*");
imageGetter.launch(pickImageIntent);

```

--------------------------------

### HTML Setup for MediaPipe Face Mesh in JavaScript

Source: https://github.com/google-ai-edge/mediapipe/blob/master/docs/solutions/face_mesh.md

This HTML snippet sets up the basic structure for a web page using MediaPipe Face Mesh. It includes essential script tags for MediaPipe utility libraries and the Face Mesh solution itself, along with video and canvas elements for input and output.

```html
<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/camera_utils/camera_utils.js" crossorigin="anonymous"></script>
  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/control_utils/control_utils.js" crossorigin="anonymous"></script>
  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/drawing_utils/drawing_utils.js" crossorigin="anonymous"></script>
  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/face_mesh/face_mesh.js" crossorigin="anonymous"></script>
</head>

<body>
  <div class="container">
    <video class="input_video"></video>
    <canvas class="output_canvas" width="1280px" height="720px"></canvas>
  </div>
</body>
</html>
```

--------------------------------

### Python Face Detector for Image Analysis

Source: https://context7.com/google-ai-edge/mediapipe/llms.txt

Detects human faces in images using a TFLite model, providing bounding boxes and confidence scores. Optimized for on-device real-time detection. Requires Mediapipe Tasks.

```python
from mediapipe.tasks import python
from mediapipe.tasks.python import vision
from mediapipe.python._framework_bindings import image as mp_image

# Initialize face detector
base_options = python.BaseOptions(model_asset_path='face_detector.tflite')
options = vision.FaceDetectorOptions(
    base_options=base_options,
    running_mode=vision.RunningMode.IMAGE,
    min_detection_confidence=0.5,
    min_suppression_threshold=0.3
)
detector = vision.FaceDetector.create_from_options(options)

# Detect faces
image = mp_image.Image.create_from_file('faces.jpg')
detection_result = detector.detect(image)

# Extract face locations
for detection in detection_result.detections:
    bbox = detection.bounding_box
    print(f"Face detected with confidence: {detection.categories[0].score:.2f}")
    print(f"Location: x={bbox.origin_x}, y={bbox.origin_y}, "
          f"w={bbox.width}, h={bbox.height}")

detector.close()
```

--------------------------------

### Android Face Mesh Detection with Video Input

Source: https://github.com/google-ai-edge/mediapipe/blob/master/docs/solutions/face_mesh.md

This Java code demonstrates how to set up MediaPipe Face Mesh for real-time video input on Android. It configures FaceMeshOptions, initializes VideoInput and SolutionGlSurfaceView for rendering, and handles video selection and processing. The code processes video frames, detects face landmarks, and renders the results using OpenGL.

```java
        // For video input and result rendering with OpenGL.
        FaceMeshOptions faceMeshOptions = 
            FaceMeshOptions.builder()
                .setStaticImageMode(false)
                .setRefineLandmarks(true)
                .setMaxNumFaces(1)
                .setRunOnGpu(true).build();
        FaceMesh faceMesh = new FaceMesh(this, faceMeshOptions);
        faceMesh.setErrorListener(
            (message, e) -> Log.e(TAG, "MediaPipe Face Mesh error:" + message));

        // Initializes a new VideoInput instance and connects it to MediaPipe Face Mesh Solution.
        VideoInput videoInput = new VideoInput(this);
        videoInput.setNewFrameListener(
            textureFrame -> faceMesh.send(textureFrame));

        // Initializes a new GlSurfaceView with a ResultGlRenderer<FaceMeshResult> instance
        // that provides the interfaces to run user-defined OpenGL rendering code.
        // See mediapipe/examples/android/solutions/facemesh/src/main/java/com/google/mediapipe/examples/facemesh/FaceMeshResultGlRenderer.java
        // as an example.
        SolutionGlSurfaceView<FaceMeshResult> glSurfaceView =
            new SolutionGlSurfaceView<>(
                this, faceMesh.getGlContext(), faceMesh.getGlMajorVersion());
        glSurfaceView.setSolutionResultRenderer(new FaceMeshResultGlRenderer());
        glSurfaceView.setRenderInputImage(true);

        faceMesh.setResultListener(
            faceMeshResult -> {
              NormalizedLandmark noseLandmark =
                  result.multiFaceLandmarks().get(0).getLandmarkList().get(1);
              Log.i(
                  TAG,
                  String.format(
                      "MediaPipe Face Mesh nose normalized coordinates (value range: [0, 1]): x=%f, y=%f",
                      noseLandmark.getX(), noseLandmark.getY()));
              // Request GL rendering.
              glSurfaceView.setRenderData(faceMeshResult);
              glSurfaceView.requestRender();
            });

        ActivityResultLauncher<Intent> videoGetter =
            registerForActivityResult(
                new ActivityResultContracts.StartActivityForResult(),
                result -> {
                  Intent resultIntent = result.getData();
                  if (resultIntent != null) {
                    if (result.getResultCode() == RESULT_OK) {
                      glSurfaceView.post(
                          () ->
                              videoInput.start(
                                  this,
                                  resultIntent.getData(),
                                  faceMesh.getGlContext(),
                                  glSurfaceView.getWidth(),
                                  glSurfaceView.getHeight()));
                    }
                  }
                });
        Intent pickVideoIntent = new Intent(Intent.ACTION_PICK);
        pickVideoIntent.setDataAndType(MediaStore.Video.Media.INTERNAL_CONTENT_URI, "video/*");
        videoGetter.launch(pickVideoIntent);
```

--------------------------------

### Python Face Mesh Landmark Detection and Drawing

Source: https://github.com/google-ai-edge/mediapipe/blob/master/docs/solutions/face_mesh.md

This Python snippet utilizes MediaPipe to detect and draw face mesh landmarks on images. It processes images, extracts face landmarks, and visualizes them using drawing utilities. Dependencies include OpenCV and MediaPipe. It takes an image as input and outputs an annotated image with landmarks drawn.

```python
import cv2
import mediapipe as mp
mp_drawing = mp.solutions.drawing_utils
mp_drawing_styles = mp.solutions.drawing_styles
mp_face_mesh = mp.solutions.face_mesh

# For static images:
IMAGE_FILES = []
drawing_spec = mp_drawing.DrawingSpec(thickness=1, circle_radius=1)
with mp_face_mesh.FaceMesh(
    static_image_mode=True,
    max_num_faces=1,
    refine_landmarks=True,
    min_detection_confidence=0.5) as face_mesh:
  for idx, file in enumerate(IMAGE_FILES):
    image = cv2.imread(file)
    # Convert the BGR image to RGB before processing.
    results = face_mesh.process(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))

    # Print and draw face mesh landmarks on the image.
    if not results.multi_face_landmarks:
      continue
    annotated_image = image.copy()
    for face_landmarks in results.multi_face_landmarks:
      print('face_landmarks:', face_landmarks)
      mp_drawing.draw_landmarks(
          image=annotated_image,
          landmark_list=face_landmarks,
          connections=mp_face_mesh.FACEMESH_TESSELATION,
          landmark_drawing_spec=None,
          connection_drawing_spec=mp_drawing_styles
          .get_default_face_mesh_tesselation_style()) 
      mp_drawing.draw_landmarks(
          image=annotated_image,
          landmark_list=face_landmarks,
          connections=mp_face_mesh.FACEMESH_CONTOURS,
          landmark_drawing_spec=None,
          connection_drawing_spec=mp_drawing_styles
          .get_default_face_mesh_contours_style())
      mp_drawing.draw_landmarks(
          image=annotated_image,
          landmark_list=face_landmarks,
          connections=mp_face_mesh.FACEMESH_IRISES,
          landmark_drawing_spec=None,
          connection_drawing_spec=mp_drawing_styles
          .get_default_face_mesh_iris_connections_style())
    cv2.imwrite('/tmp/annotated_image' + str(idx) + '.png', annotated_image)
```

--------------------------------

### Camera Input with MediaPipe Face Mesh (Java)

Source: https://github.com/google-ai-edge/mediapipe/blob/master/docs/solutions/face_mesh.md

This snippet shows how to use MediaPipe Face Mesh with live camera input on Android. It initializes Face Mesh, sets up a CameraInput to capture frames, and renders the results using SolutionGlSurfaceView and FaceMeshResultGlRenderer. Dependencies include MediaPipe libraries, camera APIs, and OpenGL.

```java
// For camera input and result rendering with OpenGL.
FaceMeshOptions faceMeshOptions = 
    FaceMeshOptions.builder()
        .setStaticImageMode(false)
        .setRefineLandmarks(true)
        .setMaxNumFaces(1)
        .setRunOnGpu(true).build();
FaceMesh faceMesh = new FaceMesh(this, faceMeshOptions);
faceMesh.setErrorListener(
    (message, e) -> Log.e(TAG, "MediaPipe Face Mesh error:" + message));

// Initializes a new CameraInput instance and connects it to MediaPipe Face Mesh Solution.
CameraInput cameraInput = new CameraInput(this);
cameraInput.setNewFrameListener(
    textureFrame -> faceMesh.send(textureFrame));

// Initializes a new GlSurfaceView with a ResultGlRenderer<FaceMeshResult> instance
// that provides the interfaces to run user-defined OpenGL rendering code.
// See mediapipe/examples/android/solutions/facemesh/src/main/java/com/google/mediapipe/examples/facemesh/FaceMeshResultGlRenderer.java
// as an example.
SolutionGlSurfaceView<FaceMeshResult> glSurfaceView =
    new SolutionGlSurfaceView<>(
        this, faceMesh.getGlContext(), faceMesh.getGlMajorVersion());
glSurfaceView.setSolutionResultRenderer(new FaceMeshResultGlRenderer());
glSurfaceView.setRenderInputImage(true);

faceMesh.setResultListener(
    faceMeshResult -> {
      NormalizedLandmark noseLandmark =
          result.multiFaceLandmarks().get(0).getLandmarkList().get(1);
      Log.i(
          TAG,
          String.format(
              "MediaPipe Face Mesh nose normalized coordinates (value range: [0, 1]): x=%f, y=%f",
              noseLandmark.getX(), noseLandmark.getY()));
      // Request GL rendering.
      glSurfaceView.setRenderData(faceMeshResult);
      glSurfaceView.requestRender();
    });

// The runnable to start camera after the GLSurfaceView is attached.
glSurfaceView.post(
    () ->
        cameraInput.start(
            this,
            faceMesh.getGlContext(),
            CameraInput.CameraFacing.FRONT,
            glSurfaceView.getWidth(),
            glSurfaceView.getHeight()));

```

--------------------------------

### Compile Face Detection Coral Example (Bash)

Source: https://github.com/google-ai-edge/mediapipe/blob/master/mediapipe/examples/coral/README.md

Compiles the face detection Coral example using Bazel with specific flags for Edge TPU support and optimization. Requires libusb for USB devices.

```bash
bazel build \
  --compilation_mode=opt \
  --define darwinn_portable=1 \
  --define MEDIAPIPE_DISABLE_GPU=1 \
  --define MEDIAPIPE_EDGE_TPU=usb \
  --linkopt=-l:libusb-1.0.so \
  mediapipe/examples/coral:face_detection_tpu build
```

--------------------------------

### Initialize Face Detection and Drawing Utilities (Python)

Source: https://github.com/google-ai-edge/mediapipe/blob/master/docs/solutions/face_detection.md

This code snippet initializes the necessary MediaPipe modules for face detection and drawing visualizations. It imports the `cv2` library for image processing and the `mediapipe` library, specifically accessing the `face_detection` and `drawing_utils` submodules.

```python
import cv2
import mediapipe as mp
mp_face_detection = mp.solutions.face_detection
mp_drawing = mp.solutions.drawing_utils

```

--------------------------------

### Build Face Detection MediaPipe AAR with Bazel (Bash)

Source: https://github.com/google-ai-edge/mediapipe/blob/master/docs/getting_started/android_archive_library.md

This Bash command specifically builds the MediaPipe Face Detection AAR. It uses the same comprehensive set of Bazel build flags as the general AAR build command, ensuring optimized performance and a lean AAR file for Android integration.

```Bash
bazel build -c opt --strip=ALWAYS \
    --host_crosstool_top=@bazel_tools//tools/cpp:toolchain \
    --fat_apk_cpu=arm64-v8a,armeabi-v7a \
    --legacy_whole_archive=0 \
    --features=-legacy_whole_archive \
    --copt=-fvisibility=hidden \
    --copt=-ffunction-sections \
    --copt=-fdata-sections \
    --copt=-fstack-protector \
    --copt=-Oz \
    --copt=-fomit-frame-pointer \
    --copt=-DABSL_MIN_LOG_LEVEL=2 \
    --linkopt=-Wl,--gc-sections,--strip-all \
    //mediapipe/examples/android/src/java/com/google/mediapipe/apps/aar_example:mediapipe_face_detection.aar
```

--------------------------------

### Compile Face and Object Detection Examples (Bash)

Source: https://github.com/google-ai-edge/mediapipe/blob/master/mediapipe/examples/coral/README.md

Compiles both the face detection and object detection Coral examples using the `make docker` command, specifying the ARM platform for cross-compilation. This command assumes a Linux target system with `apt-get`.

```bash
# Face detection
make -C mediapipe/examples/coral \
     PLATFORM=armhf \
     DOCKER_COMMAND="make -C mediapipe/examples/coral BAZEL_TARGET=mediapipe/examples/coral:face_detection_tpu build" \
     docker

# Object detection
make -C mediapipe/examples/coral \
     PLATFORM=armhf \
     DOCKER_COMMAND="make -C mediapipe/examples/coral BAZEL_TARGET=mediapipe/examples/coral:object_detection_tpu build" \
     docker
```

--------------------------------

### Compile Face Detection for ARM32/ARM64 (Bazel)

Source: https://github.com/google-ai-edge/mediapipe/blob/master/mediapipe/examples/coral/README.md

Compiles MediaPipe's face detection example for ARM32 or ARM64 architectures within a preconfigured Docker environment. Uses crosstool for toolchains and specifies the CPU architecture.

```bash
# For ARM32 (e.g. Raspberry Pi)
bazel build \
    --crosstool_top=@crosstool//:toolchains \
    --compiler=gcc \
    --cpu=armv7a \
    --define darwinn_portable=1 \
    --define MEDIAPIPE_DISABLE_GPU=1 \
    --define MEDIAPIPE_EDGE_TPU=usb \
    --linkopt=-l:libusb-1.0.so \
    mediapipe/examples/coral:face_detection_tpu build

# For ARM64 (e.g. Coral Dev Board)
bazel build \
    --crosstool_top=@crosstool//:toolchains \
    --compiler=gcc \
    --cpu=aarch64 \
    --define darwinn_portable=1 \
    --define MEDIAPIPE_DISABLE_GPU=1 \
    --define MEDIAPIPE_EDGE_TPU=usb \
    --linkopt=-l:libusb-1.0.so \
    mediapipe/examples/coral:face_detection_tpu build
```

--------------------------------

### Start Camera with Configurable Facing in Android

Source: https://github.com/google-ai-edge/mediapipe/blob/master/docs/getting_started/hello_world_android.md

Adds logic to the `startCamera()` function to determine camera facing based on a boolean value in the application's metadata. If 'cameraFacingFront' is true, it uses the front camera; otherwise, it defaults to the back camera.

```Java
CameraHelper.CameraFacing cameraFacing = 
    applicationInfo.metaData.getBoolean("cameraFacingFront", false)
        ? CameraHelper.CameraFacing.FRONT
        : CameraHelper.CameraFacing.BACK;
cameraHelper.startCamera(this, cameraFacing, /*unusedSurfaceTexture=*/ null);
```

--------------------------------

### Detect Body Landmarks with MediaPipe Holistic Landmarker

Source: https://github.com/google-ai-edge/mediapipe/blob/master/mediapipe/tasks/web/vision/README.md

Initializes the MediaPipe Holistic Landmarker task using a WASM fileset and a TFLite model. It combines pose, face, and hand landmark detection and takes an HTMLImageElement as input.

```javascript
const vision = await FilesetResolver.forVisionTasks(
    "https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision/wasm"
);
const holisticLandmarker = await HolisticLandmarker.createFromModelPath(vision,
    "https://storage.googleapis.com/mediapipe-models/holistic_landmarker/holistic_landmarker/float16/1/hand_landmark.task"
);
const image = document.getElementById("image") as HTMLImageElement;
const landmarks = holisticLandmarker.detect(image);
```

--------------------------------

### Android Manifest: Configure Camera Facing Metadata

Source: https://github.com/google-ai-edge/mediapipe/blob/master/docs/getting_started/hello_world_android.md

This XML snippet adds a meta-data tag to the AndroidManifest.xml file to specify the camera facing (front or back). The value is dynamically set via build variables, allowing different camera configurations without code changes. This setting is read using ApplicationInfo.

```xml
      ...
      <meta-data android:name="cameraFacingFront" android:value="${cameraFacingFront}"/>
  </application>
</manifest>
```