How to pass in an initial clipRect?

[rjfjohnson]

First off - great project - hats off!

I am using ML Kit / Vision to detect a bounding box, such as on a face or document, etc. on an image.

Question: knowing a bounding box (x,y,width,height), how can I pass that in as an initial crop rect?

I have tried a few different ways such as:

transformSetup = MainEditorTransformSetup(
  transformConfigs: TransformConfigs.empty().copyWith(
    cropRect: Rect.fromLTWH(cropX, cropY, cropWidth, cropHeight),
  ),
);

ProImageEditor.memory(
      key: _editorKey,
      sourceData,
      configs: ProImageEditorConfigs(
        designMode: platformDesignMode,
        mainEditor: MainEditorConfigs(
          transformSetup: transformSetup,
        ), 
   ),
);

I am receiving very unpredictable behaviour - I presume the coordinates are relative to the native source image, and in pixels (not %, for example)?

I have also tried a post frame callback with something along the lines of:

_editorKey.currentState?.addHistory(
  transformConfigs: _editorKey
      .currentState
      ?.stateManager
      .transformConfigs
      .copyWith(
        cropRect: Rect.fromLTWH(
          cropX,
          cropY,
          cropWidth,
          cropHeight,
        ),
      ),
);
_editorKey.currentState?.setState(() => {});

But, again, I am finding this very unpredictable- I have verified the bounding box dimensions that I am passing in to be correct, but the widget is not behaving as expected.

I have also tried hard-coding a bounding box, say, 50,50,100,100 and this does in fact crop to a square but it is no where near the top left of the image, rather it is almost in the center.

Any pointers would be appreciated. I was not able to find an example, although I did do some thorough searching.

[hm21]

Hey @rjfjohnson, great to hear you like my package, really appreciate it!

Unfortunately, what you're trying to do is pretty tricky because the transformConfigs are mainly meant for internal handling and can be hard to understand. I added MainEditorTransformSetup specifically for cases where users first use the crop editor and then open the main editor with the same transformations.

If you want to transform your image to that exact position, you’d need to reverse those calculations, which is quite complex and honestly something I haven’t done myself. The main challenge is that the image expands automatically with BoxFit.contain, which makes the math more complicated. Still, here are a few tips I think you’ll need for the calculations:

• You can read the original image size via originalImageSize and the rendered (decoded) size via decodedImageSize inside SizesManager (editorKey.currentState!.sizesManager).
• Inside the mainEditorCallbacks, there’s a callback called onImageDecoded that lets you know when the editor has finished decoding the image and all sizes in the SizesManager are available and ready to use.
• The cropRect in TransformConfigs is needed when the aspect ratio changes. Depending on whether the image fits the width or height, you’ll need to adjust accordingly. Don’t forget to set cropEditorScreenRatio too.
• The offset in TransformConfigs defines how much the image is moved from the editor’s center, which is at position 0,0. For example, to center the image’s top-left corner, you'd use -Offset(renderedSize.width / 2, renderedSize.height / 2).
• The scaleUser defines how much zoom is needed for the image to fit the editor’s body size. This is tricky because you also need to factor in the aspect ratios of both the image and the editor body, as those affect how BoxFit scales the image.

Also interesting could be the following calculations cuz the editor has a callback called onCompleteWithParameters, which returns the crop position on the original image. That calculation is actually the exact reverse of what you need, so reversing those functions might help you get the result you're looking for.

/// Returns the top-left offset of the crop area in the original image
/// coordinates.
///
/// [rawImageSize] is the size of the unscaled original image.
/// Takes into account user transformations like pan and zoom.
Offset getCropStartOffset(Size rawImageSize) {
  double originalWidth = rawImageSize.width;
  double originalHeight = rawImageSize.height;

  double renderWidth = originalSize.width;
  double renderHeight = originalSize.height;

  Offset transformOffset = offset;

  double horizontalPadding = (renderWidth - cropRect.width / scaleUser) / 2;
  double verticalPadding = (renderHeight - cropRect.height / scaleUser) / 2;

  /// Calculate crop offset in original coordinates
  Offset cropOffset = Offset(
    horizontalPadding - transformOffset.dx,
    verticalPadding - transformOffset.dy,
  );

  /// Calculate scale factors for the offset
  double offsetXScale = renderWidth / cropOffset.dx;
  double offsetYScale = renderHeight / cropOffset.dy;

  return Offset(
    max(0, originalWidth / offsetXScale),
    max(0, originalHeight / offsetYScale),
  );
}

/// Returns the size of the crop area in the original image coordinates.
///
/// [rawImageSize] is the size of the unscaled original image.
/// The result is scaled based on the user's zoom level.
Size getCropSize(Size rawImageSize) {
  double originalWidth = rawImageSize.width;
  double originalHeight = rawImageSize.height;

  double renderWidth = originalSize.width;
  double renderHeight = originalSize.height;

  /// Calculate scale factors based on crop dimensions
  double widthScale = renderWidth / cropRect.width;
  double heightScale = renderHeight / cropRect.height;

  return Size(
        originalWidth / widthScale,
        originalHeight / heightScale,
      ) /
      scaleUser;
}

[rjfjohnson]

Thank you very much, some great pointers and that has saved me a significant amount of time!

I got there in the end...! FWIW I ended up with the following. I am using the widget constrained in other widgets so the full working example is difficult to extract, however, the crux of it is:

void _applyInitialCropTransform() {
    if (!_isImageDecoded) return;
    if (cropX == null ||
        cropY == null ||
        cropWidth == null ||
        cropHeight == null)
      return;

    final editorState = _editorKey.currentState;
    if (editorState == null) return;

    final sizesManager = editorState.sizesManager;
    final rawImageSize = sizesManager.originalImageSize;
    final decodedImageSize = sizesManager.decodedImageSize;

    // Calculate scale factor from raw to decoded
    final scaleX = decodedImageSize.width / rawImageSize!.width;
    final scaleY = decodedImageSize.height / rawImageSize!.height;

    // Convert bbox from raw coordinates to decoded coordinates
    final decodedBboxX = cropX! * scaleX;
    final decodedBboxY = cropY! * scaleY;
    final decodedBboxWidth = cropWidth! * scaleX;
    final decodedBboxHeight = cropHeight! * scaleY;

    // Calculate the center of the desired crop in decoded coordinates
    final bboxCenterX = decodedBboxX + decodedBboxWidth / 2;
    final bboxCenterY = decodedBboxY + decodedBboxHeight / 2;

    // Calculate the center of the full image
    final imageCenterX = decodedImageSize.width / 2;
    final imageCenterY = decodedImageSize.height / 2;

    // Get the editor body size (the viewport where the image is displayed)
    final editorBodySize = sizesManager.editorSize;

    // Calculate the aspect ratio of the BBOX (not the editor!)
    final bboxAspectRatio = decodedBboxWidth / decodedBboxHeight;
    final imageAspectRatio = decodedImageSize.width / decodedImageSize.height;

    double cropRectWidth, cropRectHeight;

    if (bboxAspectRatio > imageAspectRatio) {
      // crop bbox is wider than image, fit to image width
      cropRectWidth = decodedImageSize.width;
      cropRectHeight = cropRectWidth / bboxAspectRatio;
    } else {
      // fit to image height
      cropRectHeight = decodedImageSize.height;
      cropRectWidth = cropRectHeight * bboxAspectRatio;
    }

    // Calculate scaleUser:
    final scaleToFitWidth = cropRectWidth / decodedBboxWidth;
    final scaleToFitHeight = cropRectHeight / decodedBboxHeight;
    final scaleUser = min(scaleToFitWidth, scaleToFitHeight);

    // Calculate offset:
    final offsetX = imageCenterX - bboxCenterX;
    final offsetY = imageCenterY - bboxCenterY;

    // Build the cropRect, centered on the image
    final cropRectLeft = (decodedImageSize.width - cropRectWidth) / 2;
    final cropRectTop = (decodedImageSize.height - cropRectHeight) / 2;
    final cropRectRight = cropRectLeft + cropRectWidth;
    final cropRectBottom = cropRectTop + cropRectHeight;

    // Calculate cropEditorScreenRatio:
    final editorAspectRatio = editorBodySize.width / editorBodySize.height;
    double cropEditorScreenRatio;
    if (bboxAspectRatio > editorAspectRatio) {
      // Crop is wider, constrained by width
      cropEditorScreenRatio = editorBodySize.width / cropRectWidth;
    } else {
      cropEditorScreenRatio = editorBodySize.height / cropRectHeight;
    }

    // Create the new transform config
    final newTransform = TransformConfigs.fromMap({
      "angle": 0,
      "cropRect": {
        "left": cropRectLeft,
        "top": cropRectTop,
        "right": cropRectRight,
        "bottom": cropRectBottom,
      },
      "originalSize": {
        "width": decodedImageSize.width,
        "height": decodedImageSize.height,
      },
      "cropEditorScreenRatio": cropEditorScreenRatio,
      "scaleUser": scaleUser,
      "scaleRotation": 0,
      "aspectRatio": -1,
      "flipX": false,
      "flipY": false,
      "cropMode": "rectangular",
      "offset": {"dx": offsetX, "dy": offsetY},
    });

    editorState.addHistory(transformConfigs: newTransform);
    _hasAppliedInitialCrop = true;

    // Open the crop editor to let the user adjust
    WidgetsBinding.instance.addPostFrameCallback((_) {
      editorState.openCropRotateEditor();
    });
  }

Again, many thanks for both the project overall and in particular your pointers on this matter!

Cheers,

[hm21]

Thank you very much for sharing that solution. It could definitely be helpful for other users trying to do the same💪🏻