SAM2 #155
SAM2 #155
Conversation
| float scale = GetScale(originalWidth, originalHeight); | ||
| int newWidth = imageSize; | ||
| int newHeight = imageSize; | ||
| Color32[] resizedImage = ResizeColor32( |
There was a problem hiding this comment.
In ResizeColor32, scaling is performed without maintaining the aspect ratio, ignoring scale. In ApplyCoordinateScaling, the coordinates of the Point are corrected with scale. A mismatch occurs here. Please provide the scale as an argument to ResizeColor32 and perform resizing while maintaining the aspect ratio.
There was a problem hiding this comment.
Thanks for the suggestion. The solution is to scale the width and height separately with target size(1024). Because the inference need the shape to be (3, 1024, 1024). So if scale ratio(GetScale) cause it become (3, 683, 1024),it will cause memory out error with the inference.
Please refer to line 1279.
There was a problem hiding this comment.
@larryaxinc Resize while maintaining the aspect ratio, and fill the margins with black. If this is not corrected, I think the problem where the clicked point and the segmented point do not align will not be resolved.
There was a problem hiding this comment.
Keep the aspect ration, and fill in other margin with black to make the shape keep(3, 1024, 1024).
But somehow this looked a bit wried on height(1200 in height, 1800 in width for original img).
| Application.temporaryCachePath, | ||
| promptProtoPath | ||
| ); | ||
| // string mlpPath = System.IO.Path.Combine( |
There was a problem hiding this comment.
Please delete unnecessary comments.
|
@larryaxinc I have filled in the feedback, so please respond. Also, please test whether it works in WEB camera mode. You can set camera_mode to true in AiliaImageSegmentationSample.cs to enable WEB camera mode. |
|
@larryaxinc The accuracy is lower compared to the Python version. Please dump the input and output float values of the vision encoder and calculate the mean squared error between the outputs of the Python and Unity versions to analyze where the issue lies. 
|
@kyakuno It has been fixed by the new commit. Please have a check. |
2 participants
#145 Implemented.
Also looks like the click point on image height is not fit well. It would be very appreciate if you could have some idea.