feat: add blazeface support#1187
Open
chmjkb wants to merge 5 commits into
Open
Conversation
benITo47
requested changes
May 26, 2026
| /// Affine transform from model-input pixel coords back to source-image coords: | ||
| /// `x_src = x_model * scaleX + offsetX`. Covers both plain stretch (offsets | ||
| /// zero) and letterbox (offsets carry the centre-pad). | ||
| struct BoxTransform { |
Contributor
There was a problem hiding this comment.
Maybe move it to utils? This might be used in other model pipes as well.
msluszniak
requested changes
May 26, 2026
Comment on lines
59
to
-138
| @@ -86,34 +87,37 @@ std::vector<types::Instance> BaseInstanceSegmentation::runInference( | |||
| auto instances = collectInstances( | |||
| forwardResult.get(), originalSize, modelInputSize, confidenceThreshold, | |||
| classIndices, returnMaskAtOriginalResolution); | |||
| return finalizeInstances(std::move(instances), iouThreshold, maxInstances); | |||
| return finalizeInstances(std::move(instances), iouThreshold, maxInstances, | |||
| useWeightedNms); | |||
| } | |||
|
|
|||
| std::vector<types::Instance> BaseInstanceSegmentation::generateFromString( | |||
| std::string imageSource, double confidenceThreshold, double iouThreshold, | |||
| int32_t maxInstances, std::vector<int32_t> classIndices, | |||
| bool returnMaskAtOriginalResolution, std::string methodName) { | |||
| bool returnMaskAtOriginalResolution, std::string methodName, | |||
| bool useWeightedNms) { | |||
|
|
|||
| cv::Mat imageBGR = image_processing::readImage(imageSource); | |||
| cv::Mat imageRGB; | |||
| cv::cvtColor(imageBGR, imageRGB, cv::COLOR_BGR2RGB); | |||
|
|
|||
| return runInference(imageRGB, confidenceThreshold, iouThreshold, maxInstances, | |||
| classIndices, returnMaskAtOriginalResolution, methodName); | |||
| classIndices, returnMaskAtOriginalResolution, methodName, | |||
| useWeightedNms); | |||
| } | |||
|
|
|||
| std::vector<types::Instance> BaseInstanceSegmentation::generateFromFrame( | |||
| jsi::Runtime &runtime, const jsi::Value &frameData, | |||
| double confidenceThreshold, double iouThreshold, int32_t maxInstances, | |||
| std::vector<int32_t> classIndices, bool returnMaskAtOriginalResolution, | |||
| std::string methodName) { | |||
| std::string methodName, bool useWeightedNms) { | |||
|
|
|||
| auto orient = ::rnexecutorch::utils::readFrameOrientation(runtime, frameData); | |||
| cv::Mat frame = extractFromFrame(runtime, frameData); | |||
| cv::Mat rotated = utils::rotateFrameForModel(frame, orient); | |||
| auto instances = | |||
| runInference(rotated, confidenceThreshold, iouThreshold, maxInstances, | |||
| classIndices, returnMaskAtOriginalResolution, methodName); | |||
| auto instances = runInference( | |||
| rotated, confidenceThreshold, iouThreshold, maxInstances, classIndices, | |||
| returnMaskAtOriginalResolution, methodName, useWeightedNms); | |||
| for (auto &inst : instances) { | |||
| utils::inverseRotateBbox(inst.bbox, orient, rotated.size()); | |||
| // Inverse-rotate the mask to match the screen orientation | |||
| @@ -131,11 +135,13 @@ std::vector<types::Instance> BaseInstanceSegmentation::generateFromFrame( | |||
| std::vector<types::Instance> BaseInstanceSegmentation::generateFromPixels( | |||
| JSTensorViewIn tensorView, double confidenceThreshold, double iouThreshold, | |||
| int32_t maxInstances, std::vector<int32_t> classIndices, | |||
| bool returnMaskAtOriginalResolution, std::string methodName) { | |||
| bool returnMaskAtOriginalResolution, std::string methodName, | |||
| bool useWeightedNms) { | |||
|
|
|||
| cv::Mat image = extractFromPixels(tensorView); | |||
| return runInference(image, confidenceThreshold, iouThreshold, maxInstances, | |||
| classIndices, returnMaskAtOriginalResolution, methodName); | |||
Member
There was a problem hiding this comment.
We are modifying the code that should be general (BASE instance segmentation) and we bloat it with some positional parameters that are useless for other models. This is super bad architectural design what we have right now. We need to discuss this internally how we should tackle that. Because this is not the problem of this PR only, but almost all models. I created RFC in discussion section how to deal with this one. See #1189
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Introduces a breaking change?
Type of change
Tested on
Testing instructions
Screenshots
Related issues
Checklist
Additional notes