How to optimize the effect and inference speed for very high-resolution images?

[keydexofficial]

Assuming the input is a high-resolution image, such as 5000x4000, what can be done to achieve better performance metrics and inference speed? Possible solutions:

1. Use the BiRefNet_dynamic model;

2. Directly input it into the BiRefNet_HR series model;

3. Split the high-resolution image into smaller chunks and input them one by one into the BiRefNet_HR series model;

4. other methods etc.

May I ask if anyone has researched which method is the best practice?

[ZhengPeng7]

It depends on the characteristics of your data.
If the targets tend to be small in a large scenario, I prefer to detect them in a lower resolution, then crop the region of the target into a smaller image. Then send the cropped one to a BiRefNet model for HR images, such as BiRefNet_dynamic / BiRefNet_HR.
Otherwise, I recommend 1 / 2. Method 3 usually destroys the semantics of the target and should fail when intact distractors appear in the background.

[keydexofficial]

Thank you for your reply.

If the targets tend to be small in a large scenario, I prefer to detect them in a lower resolution, then crop the region of the target into a smaller image. Then send the cropped one to a BiRefNet model for HR images, such as BiRefNet_dynamic / BiRefNet_HR.

"detect them in a lower resolution" , if to use this method, do you have any recommended detection models?
Background: The images we input come from various industries and scenarios, with no specific context available for optimization.

[ZhengPeng7]

You can use a model for lower resolution because you only need the approximate location of the target in step one. But of course, you can also use an HR BiRefNet for it. For example, both use BiRefNet_HR in these two steps.

If the background is very general, you should try the other ways as I mentioned above.

[keydexofficial]

You can use a model for lower resolution because you only need the approximate location of the target in step one. But of course, you can also use an HR BiRefNet for it. For example, both use BiRefNet_HR in these two steps.

If the background is very general, you should try the other ways as I mentioned above.

Apologies, I misunderstood your previous meaning.
By "detect," you didn't mean using object detection models like MMDetection or YOLO, but rather lowering the resolution of the original image, then inputting it into, for example, BiRefNet_dynamic. After that, we would locate the results on the original image, crops, and then input them into BiRefNet_dynamic or BiRefNet_HR.
Is that the correct understanding?

[ZhengPeng7]

Yes. You can refer to my colab demo of box-guided segmentation in the README. That's a straightforward implementation of it.

[keydexofficial]

我想了下，对于原图的crop，大多数情况下应该会导致效果变差，因为crop变小之后，输入给模型的信息变少了，从而导致模型效果变差。

[ZhengPeng7]

你可以参考我说的colab里的代码. cropt选定, 还原到为resize的原图中的box, 然后用这个box去inference. 所以一般来讲会变好, 因为原图5000x4000, crop出来只有2000x2000, 那刚好很适合2048x2048训的HR模型.