eat: add Exclude Upper Teeth and Show Mouth Outline options in auto mouth expresion#222
Merged
Conversation
- AutoMouthExcludeUpperTeethToggle: keeps swapped upper teeth visible by trimming the inner-mouth FaceParser mask using brightness-based detection - AutoMouthShowDebugOutlineToggle: draws green/cyan contours on preview frame - Default AutoMouthLowerLipParserSlider 17 -> 8
chyanbo
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends the auto-mouth feature set by adding an option to preserve swapped upper teeth (by trimming the inner-mouth parser mask) and a preview-only debug outline for the mouth region. It also adjusts auto-mouth parser defaults and makes RetinaFace/SCRFD detection input sizing adapt to fixed-shape ONNX exports to avoid ONNX Runtime size warnings.
Changes:
- Add UI toggles for upper-teeth exclusion and mouth-region debug outline; adjust the default lower-lip parser dilation.
- In the frame worker, propagate the new toggles into mask-generation parameters and render optional mouth-region contours on the preview frame.
- In face mask generation, implement an upper-teeth keep-mask derived from brightness and expose debug masks for preview; in face detectors, resolve effective detector input size from the ORT session.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
| app/ui/widgets/common_layout_data.py | Adds new Auto Mouth toggles and updates the lower-lip parser default. |
| app/processors/workers/frame_worker.py | Applies new auto-mouth mask flags and draws optional debug contours in the preview output. |
| app/processors/face_masks.py | Implements upper-teeth keep-mask logic and emits mouth debug masks used by the preview outline. |
| app/processors/face_detectors.py | Resolves RetinaFace/SCRFD input size from ORT session metadata to avoid fixed-shape warnings. |
Comment on lines
+4069
to
+4070
| _was_auto_active = bool(getattr(target_fb, "_auto_mouth_prev_active", False)) | ||
| target_fb._auto_mouth_prev_active = _auto_active |
Comment on lines
4099
to
+4107
| _mouth_val = int(params.get("AutoMouthMouthParserSlider", 1)) | ||
| _upper_val = int(params.get("AutoMouthUpperLipParserSlider", 3)) | ||
| _lower_val = int(params.get("AutoMouthLowerLipParserSlider", 17)) | ||
| if _mouth_val > 0 or _upper_val > 0 or _lower_val > 0: | ||
| if _mouth_val > 0 or _upper_val > 0 or _lower_val > 0 or _exclude_upper_teeth: | ||
| _p["FaceParserEnableToggle"] = True | ||
| _p["MouthParserSlider"] = _mouth_val | ||
| _p["UpperLipParserSlider"] = _upper_val | ||
| _p["LowerLipParserSlider"] = _lower_val | ||
| _p["AutoMouthUpperTeethExcludeActive"] = _exclude_upper_teeth |
Comment on lines
+546
to
+551
| # Debug mouth-region mask (inner mouth + lips), used only for preview contour. | ||
| if labels_swap is not None: | ||
| mouth_debug = self._mask_from_labels_lut(labels_swap, [11, 12, 13]) | ||
| result["mouth_debug"] = ( | ||
| resize_to_target(mouth_debug.unsqueeze(0)).clamp(0, 1).squeeze() | ||
| ) |
Comment on lines
+643
to
+652
| # Recompute mouth-only dilated mask to reflect actual slider values in debug outline. | ||
| _fp_mouth = torch.zeros((512, 512), device=device, dtype=torch.float32) | ||
| for _cls, _pname in [(11, "MouthParserSlider"), (12, "UpperLipParserSlider"), (13, "LowerLipParserSlider")]: | ||
| _val = int(parameters.get(_pname, 0)) | ||
| if _val > 0: | ||
| _m = self._mask_from_labels_lut(labels_swap, [_cls]) | ||
| _m = self._dilate_binary(_m, _val, mode) | ||
| _fp_mouth = torch.maximum(_fp_mouth, _m) | ||
| if _fp_mouth.sum() > 0: | ||
| result["mouth_debug"] = resize_to_target(_fp_mouth.unsqueeze(0)).clamp(0, 1).squeeze() |
Comment on lines
596
to
+642
| @@ -609,8 +629,28 @@ def resize_to_target(tensor): | |||
| comb = torch.maximum(m1, m2) | |||
| else: | |||
| comb = m1 | |||
|
|
|||
| if upper_teeth_keep_mask is not None and 11 in classes: | |||
| comb = comb * (1.0 - upper_teeth_keep_mask) | |||
|
|
|||
| fp = torch.maximum(fp, comb) | |||
|
|
|||
| # Enforce keep mask after all classes so dilated lips cannot | |||
| # reclaim the teeth area. | |||
| if upper_teeth_keep_mask is not None: | |||
| fp = fp * (1.0 - upper_teeth_keep_mask) | |||
|
|
|||
Comment on lines
+554
to
+556
| args["input_size"] = self._resolve_detector_input_size( | ||
| detect_mode, input_size, ort_session | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AutoMouthExcludeUpperTeethToggle: keeps swapped upper teeth visible by trimming the inner-mouth FaceParser mask using brightness-based detection.