Merge pull request #7 from google/staging

Project import generated by Copybara.

Author: stoman

.github/workflows/scorecards.yml

@@ -34,9 +34,6 @@ jobs:
         with:
           results_file: results.sarif
           results_format: sarif
-          # Read-only PAT token. Necessary for the Branch-Protection check.
-          # On creating/modifying the PAT token, see https://github.com/ossf/scorecard-action#authentication-with-pat.
-          repo_token: ${{ secrets.SCORECARD_READ_TOKEN }}
 
           # Publish the results for public repositories to enable scorecard badges. For more details, see
           # https://github.com/ossf/scorecard-action#publishing-results.

CHANGELOG

@@ -3,7 +3,11 @@
 ## [Unreleased]
 
 ### Added
--
+
+## [1.0.1]
+
+### Added
+- Overlay graphs to display both base detection and tracked detections.
 
 ### Changed
 - Replace absl::make_unique with std::make_unique (introduced in C++14).

docs/changelog.md

@@ -9,6 +9,11 @@ nav_order: 4
 This document contains a summary of all changes to the Magritte open-source
 library.
 
+## 1.0.1
+
+*   Add overlay graphs to display both base detection and tracked detections.
+*   Replace absl::make_unique with std::make_unique (introduced in C++14).
+
 ## 1.0.0
 
 *   First release to GitHub.

docs/technical_guide/graphs.md

@@ -345,6 +345,89 @@ files in the test_data directory.
 
 **Code:** [source code](https://github.com/google/magritte/blob/master/magritte/graphs/face_sticker_redaction_offline_cpu.pbtxt)
 
+#### FaceTrackingOverlayLiveGpu
+
+A graph that detects and tracks faces to draw debug information.
+
+It draws in red the raw detection output (detection box, keypoints, score),
+and in blue the tracked detection output (detection box, keypoints, score,
+when present).
+
+This graph is specialized for GPU architectures. It also optimized for live
+streaming environments by throttling the input stream and by applying only
+full-range face detection.
+
+**Input streams:**
+
+*   `input_video`: A GpuBuffer stream containing the image on which detection
+  models are run.
+
+**Output streams:**
+
+*   `output_video`: A GpuBuffer stream containing the image annotated with
+  detections data.
+
+**Build targets:**
+
+*   Graph `cc_library`:
+
+    ```
+    @magritte//magritte/graphs:face_tracking_overlay_live_gpu
+    ```
+*   Text proto file:
+
+    ```
+    @magritte//magritte/graphs:face_tracking_overlay_live_gpu.pbtxt
+    ```
+*   Binary graph:
+
+    ```
+    @magritte//magritte/graphs:face_tracking_overlay_live_gpu_graph
+    ```
+
+**Code:** [source code](https://github.com/google/magritte/blob/master/magritte/graphs/face_tracking_overlay_live_gpu.pbtxt)
+
+#### FaceTrackingOverlayOfflineCpu
+
+A graph that detects and tracks faces to draw debug information.
+
+It draws in red the raw detection output (detection box, keypoints, score),
+and in blue the tracked detection output (detection box, keypoints, score,
+when present).
+
+This graph is specialized for CPU architectures and offline environments
+(no throttling is applied).
+
+**Input streams:**
+
+*   `input_video`: An ImageFrame stream containing the image on which detection
+  models are run.
+
+**Output streams:**
+
+*   `output_video`: An ImageFrame stream containing the image annotated with
+  detections data.
+
+**Build targets:**
+
+*   Graph `cc_library`:
+
+    ```
+    @magritte//magritte/graphs:face_tracking_overlay_offline_cpu
+    ```
+*   Text proto file:
+
+    ```
+    @magritte//magritte/graphs:face_tracking_overlay_offline_cpu.pbtxt
+    ```
+*   Binary graph:
+
+    ```
+    @magritte//magritte/graphs:face_tracking_overlay_offline_cpu_graph
+    ```
+
+**Code:** [source code](https://github.com/google/magritte/blob/master/magritte/graphs/face_tracking_overlay_offline_cpu.pbtxt)
+
 ## Subgraphs
 
 ### Detection
@@ -879,6 +962,78 @@ the ovals will be white.
 
 **Code:** [source code](https://github.com/google/magritte/blob/master/magritte/graphs/redaction/detection_to_mask/face_detection_to_mask_gpu.pbtxt)
 
+#### DetectionTrackingOverlaySubgraphCpu
+
+Subgraph to draw debug information at the locations specified by two incoming
+detection streams. For each detection stream, it will draw all data available
+from detections, including the bounding box, keypoints and score.
+
+**Input streams:**
+
+*   `IMAGE`: An ImageFrame containing the image to draw the overlays on.
+*   `[0]`: A std::vector<mediapipe::Detection>, will be rendered in red.
+*   `[1]`: A std::vector<mediapipe::Detection>, will be rendered in blue.
+
+**Output streams:**
+
+*   `IMAGE`: The resulting image.
+
+**Build targets:**
+
+*   Graph `cc_library`:
+
+    ```
+    @magritte//magritte/graphs/redaction:detection_tracking_overlay_cpu
+    ```
+*   Text proto file:
+
+    ```
+    @magritte//magritte/graphs/redaction:detection_tracking_overlay_cpu.pbtxt
+    ```
+*   Binary graph:
+
+    ```
+    @magritte//magritte/graphs/redaction:detection_tracking_overlay_cpu_graph
+    ```
+
+**Code:** [source code](https://github.com/google/magritte/blob/master/magritte/graphs/redaction/detection_tracking_overlay_cpu.pbtxt)
+
+#### DetectionTrackingOverlaySubgraphGpu
+
+Subgraph to draw debug information at the locations specified by two incoming
+detection streams. For each detection stream, it will draw all data available
+from detections, including the bounding box, keypoints and score.
+
+**Input streams:**
+
+*   `IMAGE_GPU`: A GpuBuffer containing the image to draw the overlays on.
+*   `[0]`: A std::vector<mediapipe::Detection>, will be rendered in red.
+*   `[1]`: A std::vector<mediapipe::Detection>, will be rendered in blue.
+
+**Output streams:**
+
+*   `IMAGE_GPU`: The resulting image.
+
+**Build targets:**
+
+*   Graph `cc_library`:
+
+    ```
+    @magritte//magritte/graphs/redaction:detection_tracking_overlay_gpu
+    ```
+*   Text proto file:
+
+    ```
+    @magritte//magritte/graphs/redaction:detection_tracking_overlay_gpu.pbtxt
+    ```
+*   Binary graph:
+
+    ```
+    @magritte//magritte/graphs/redaction:detection_tracking_overlay_gpu_graph
+    ```
+
+**Code:** [source code](https://github.com/google/magritte/blob/master/magritte/graphs/redaction/detection_tracking_overlay_gpu.pbtxt)
+
 #### FaceDetectionOverlaySubgraphCpu
 
 Subgraph to draw debug information at the locations specified by incoming

magritte/calculators/BUILD

@@ -111,7 +111,6 @@ mediapipe_proto_library(
 cc_library(
     name = "pixelization_calculator_gpu",
     srcs = ["pixelization_calculator_gpu.cc"],
-    visibility = ["//visibility:public"],
     deps = [
         ":pixelization_calculator_cc_proto",
         "@mediapipe//mediapipe/framework:calculator_framework",
@@ -154,7 +153,6 @@ cc_library(
     name = "blend_calculator",
     srcs = ["blend_calculator.cc"],
     hdrs = ["blend_calculator.h"],
-    visibility = ["//visibility:public"],
     deps = [
         "@mediapipe//mediapipe/framework:calculator_framework",
         "@mediapipe//mediapipe/framework/formats:image_frame",

magritte/calculators/rois_to_sprite_list_calculator.cc

@@ -45,9 +45,9 @@ using ::mediapipe::GlTexture;
 using ::mediapipe::GpuBuffer;
 #endif  // !MEDIAPIPE_DISABLE_GPU
 using ::mediapipe::MakePacket;
-using ::mediapipe::NormalizedRect;
 using ::mediapipe::formats::MatView;
 using ::mediapipe::ImageFormat;
+using ::mediapipe::NormalizedRect;
 using NormalizedRects = std::vector<NormalizedRect>;
 using Size = std::pair<int, int>;
 

magritte/calculators/rois_to_sprite_list_calculator_test.cc

@@ -44,10 +44,10 @@ using ::mediapipe::CalculatorGraphConfig;
 using ::mediapipe::CalculatorRunner;
 using ::mediapipe::ImageFormat;
 using ::mediapipe::ImageFrame;
-using ::mediapipe::NormalizedRect;
 using ::mediapipe::Packet;
 using ::mediapipe::Timestamp;
 using ::mediapipe::formats::MatView;
+using ::mediapipe::NormalizedRect;
 using SizeFloat = std::pair<float, float>;
 using SizeInt = std::pair<int, int>;
 

magritte/examples/desktop/BUILD

@@ -50,6 +50,7 @@ _top_level_graph_targets = [
     "//magritte/graphs:face_blur_with_tracking_live_cpu",
     "//magritte/graphs:face_blur_with_tracking_offline_cpu",
     "//magritte/graphs:face_overlay_offline_cpu",
+    "//magritte/graphs:face_tracking_overlay_offline_cpu",
     "//magritte/graphs:face_pixelization_offline_cpu",
     "//magritte/graphs:face_sticker_redaction_offline_cpu",
 ] + select({

magritte/graphs/BUILD

@@ -44,6 +44,17 @@ magritte_graph(
     ],
 )
 
+magritte_graph(
+    name = "face_tracking_overlay_offline_cpu",
+    graph = "face_tracking_overlay_offline_cpu.pbtxt",
+    register_as = "FaceTrackingOverlayOfflineCpu",
+    deps = [
+        "//magritte/graphs/detection:face_detection_short_and_full_range_cpu",
+        "//magritte/graphs/redaction:detection_tracking_overlay_cpu",
+        "//magritte/graphs/tracking:tracking_cpu",
+    ],
+)
+
 magritte_graph(
     name = "face_blur_with_tracking_offline_cpu",
     graph = "face_blur_with_tracking_offline_cpu.pbtxt",
@@ -89,6 +100,18 @@ magritte_graph(
     ],
 )
 
+magritte_graph(
+    name = "face_tracking_overlay_live_gpu",
+    graph = "face_tracking_overlay_live_gpu.pbtxt",
+    register_as = "FaceTrackingOverlayLiveGpu",
+    deps = [
+        "//magritte/graphs/detection:face_detection_full_range_gpu",
+        "//magritte/graphs/redaction:detection_tracking_overlay_gpu",
+        "//magritte/graphs/tracking:tracking_gpu",
+        "@mediapipe//mediapipe/calculators/core:flow_limiter_calculator",
+    ],
+)
+
 magritte_graph(
     name = "face_sticker_redaction_offline_cpu",
     graph = "face_sticker_redaction_offline_cpu.pbtxt",

magritte/graphs/face_tracking_overlay_live_gpu.pbtxt

@@ -0,0 +1,71 @@
+#
+# Copyright 2020-2022 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+package: "magritte"
+type: "FaceTrackingOverlayLiveGpu"
+
+# A graph that detects and tracks faces to draw debug information.
+#
+# It draws in red the raw detection output (detection box, keypoints, score),
+# and in blue the tracked detection output (detection box, keypoints, score,
+# when present).
+#
+# This graph is specialized for GPU architectures. It also optimized for live
+# streaming environments by throttling the input stream and by applying only
+# full-range face detection.
+#
+# Inputs:
+# - input_video: A GpuBuffer stream containing the image on which detection
+#   models are run.
+#
+# Outputs:
+# - output_video: A GpuBuffer stream containing the image annotated with
+#   detections data.
+
+input_stream: "input_video"
+output_stream: "output_video"
+
+node {
+  calculator: "FlowLimiterCalculator"
+  input_stream: "input_video"
+  input_stream: "FINISHED:output_video"
+  input_stream_info: {
+    tag_index: "FINISHED"
+    back_edge: true
+  }
+  output_stream: "throttled_input_video"
+}
+
+node {
+  calculator: "FaceDetectionFullRangeGpu"
+  input_stream: "IMAGE:throttled_input_video"
+  output_stream: "DETECTIONS:detections"
+}
+
+node {
+  calculator: "TrackingSubgraphGpu"
+  input_stream: "IMAGE:throttled_input_video"
+  input_stream: "DETECTIONS:detections"
+  output_stream: "DETECTIONS:tracked_detections"
+}
+
+node {
+  calculator: "DetectionTrackingOverlaySubgraphGpu"
+  input_stream: "IMAGE_GPU:throttled_input_video"
+  input_stream: "detections"
+  input_stream: "tracked_detections"
+  output_stream: "IMAGE_GPU:output_video"
+}
+