Clicks not registering in Unity

[Volko76]

Hi,
I'm trying to make an AR mobile app.
The interface is in HTML and is served by a server. It communicates via Unity via EvaluateJS.

I would like to be able to click on the red area to place object in the floor (using a raycast).

The red area is transparent (I set the webview to transparent too) and I see the Unity world here.
The issue is that when I click in the red area, my click is not registered by Unity (in android) so it doen't trigger the placement of the object at this position.

I tried to use https://github.com/gree/unity-webview/tree/experimental/unity-over-native-ui and it seems to work in the editor but not in android. I also tried with 32Bit and RendeOverNativeUi enabled and disabled and it changed nothing. I know it is in experimental.

Am I missing something ?

Thanks a lot for the project and the help provided
Wishing you the best day

[Volko76]

I tested on the master branch and I get the same results : it is working in the editor but not on android

[KojiNakamaru]

Transparency doesn't mean that events can be passed to unity. Every event on the webview rectangular region is treated by the webview. The experimental/unity-over-native-ui variant is for drawing unity's elements over webviews. For that variant, you need to specify masks to receive events in unity (cf. #584 (comment) ).

If you utilize the latest one on the master branch, you can receive an event on webview and send it to the unity side by Unity.call().

[Volko76]

Thanks a lot for the anwser @KojiNakamaru , it clarifies a lot of things.

However, I don't fuly understand how masks are working I think. Can you explain a little bit more how mask are working ?

I forked the experimental branch, enable Render Over UI and 32Bit in the android player settings. Then I allowed cleartext to use my local http server for debug.
I set the margins to 0, 0, 0, 0 and the mask to 0, 0, Screen.width, Screen.height to fill the whole screen but I still cannot register a click on a TMP Button.

Am I doing something wrong ?

Thanks a lot

[KojiNakamaru]

I put a modified sample app under https://github.com/gree/unity-webview/tree/example-for-1115/sample where modification commits are below:

4426a65...d8b69d6

In this sample, margins (0, 0, 0, 0) and a mask (0, 0, Screen.width, 100) are specified.

unity-webview/sample/Assets/Scripts/SampleWebView.cs

Lines 153 to 158 in d8b69d6

	webViewObject.SetMargins(0, 0, 0, 0);
	webViewObject.SetTextZoom(100); // android only. cf. https://stackoverflow.com/questions/21647641/android-webview-set-font-size-system-default/47017410#47017410
	webViewObject.SetVisibility(true);
	WebViewObject.ClearMasks();
	WebViewObject.AddMask(0, 0, Screen.width, 100);

We can press top buttons rendered by unity while interacticting the webview in other regions.

[Volko76]

Thanks a lot for the provided sample app. Unfortunatly, I don't manage to make this sample app to work 😥

This is really strange.
I clone this branch, open it up in Unity (6000.0.26f1), opened the sample scene, select android, select developpement build, click on "build and run".

And I can interract with the webview but not with the top buttons.

I'm using a rog phone 6 on android 14. It has 12 Go of ram and a Snapdragon 8+ Gen 1

[Volko76]

Okay, so I made my experiments and it turns out that in 6000 and in 2022, it does not work but in 2020 it's working.
I will try to investigate on why

[Volko76]

2021 does not work too

[Volko76]

I tested with 2020.3.48f1 like you, but also with 2020.3.47f1 to see if it was the "reimport" process of Unity that causes issues but they both work.

So the issue surely comes from the changes between 2020 and 2021

[Volko76]

I tried with iOS and it worked on all versions but not on android

[KojiNakamaru]

Thanks for testing various unity versions. I investigated what happened and found the following part "bringToBack" logic didn't work for recent unity versions.

unity-webview/plugins/Android/webview/src/main/java/net/gree/unitywebview/CWebViewPlugin.java

Lines 814 to 821 in d8b69d6

	{
	// cf. https://stackoverflow.com/questions/6759036/how-to-send-view-to-back-how-to-control-the-z-order-programmatically/19872801#19872801
	ViewGroup parent = (ViewGroup)layout.getParent();
	if (parent != null) {
	parent.removeView(layout);
	parent.addView(layout, 0);
	}
	}

This issue seems unavoidable, so I tweaked event handling logic:

d8b69d6...083dc00

The sample app on the latest https://github.com/gree/unity-webview/tree/example-for-1115 should work correctly. Please note that you need to import the package from this branch as I updated plugin code and binaries only on this branch.

[Volko76]

Thanks a lot, I will try it out

[Volko76]

@KojiNakamaru I tried it and it worked ! Thanks a lot ❤

Because it worked, I started to transfer my project from the webview "master" branch to the webview "example-for-1115" one and now I have a new issue.

I use the default unity AR project and it seems that it doesn't manage to show both the webview and the AR video feed (But it was working great on the master branch).

I seems also that when I enter in the application, I can see the webview for a fraction of a second and then it disappears.

Do you have an idea of how to solve this issue ?

[KojiNakamaru]

The AR mode should treat the framebuffer differently and perhaps it is not designed to be compatible with "Render Over Native UI". If the main branch still works, the following approach should be viable.

If you utilize the latest one on the master branch, you can receive an event on webview and send it to the unity side by Unity.call().

[Volko76]

The AR mode should treat the framebuffer differently and perhaps it is not designed to be compatible with "Render Over Native UI". If the main branch still works, the following approach should be viable.

If you utilize the latest one on the master branch, you can receive an event on webview and send it to the unity side by Unity.call().

Yes it is becomnig too difficult.
I ended up passing the drag values of the JS to Unity and adapt my UI even thought it's poop code :
document.addEventListener('pointermove', function (event) { //console.log(event) Unity.call("SendDragDelta/" + event.movementX + ";" + event.movementY); });

Thanks a lot for the help you provided ^^ 👍