ImageOverlayPlugin improvements

[xawill]

Question

Hi!
I played a bit with ImageOverlayPlugin and I have a few "issues" I would like to discuss with you (let's call them "questions" for now and based on your answers I can provide a repro if needed):

1. I was using the TextureOverlayPlugin available in the examples to texture my quantized mesh terrain (with a quick WMTS url replace for x,y,z tiles idx). What is the added value of ImageOverlayPlugin in your opinion? I see the architecture is more scalable; you can add a lot of different ImageOverlays, but why not just improve the existing TextureOverlayPlugin?
2. Because I am experiencing a big performance degradation after migrating TextureOverlayPlugin -> ImageOverlayPlugin (see the diffs here). It is really slower to load the overlay tiles. Is the new plugin still WIP and you expect to fix known performance issues or is there something wrong on my side?
3. Trying to overlay this 3d buildings tileset with ImageOverlayPlugin, I noticed only the root tile is loaded (note that this doesn't happen with other tilesets such as quantized mesh terrain I also use). Digging into the issue, I noticed that in the 3d buildings tileset, tile.__hasRenderableContent is set to false even though there are child tiles. This implies tile.__depthFromRenderedParent not being set properly and thus _calculateLevelFromOverlay returning a wrong value which stops the plugin from loading child tiles. It might be linked to you not being able to properly test functionality with "ADD" tilesets, as you wrote in TilesRenderer: "ADD" seems to always load all child tiles #1209. I don't know if this is the correct way to fix it, but currently I am using return tile.__depth - 1; instead of return tile.__depthFromRenderedParent - 1; (here) as a quick personal patch. Let me know how we can coordinate the fix, because this one I am pretty sure this is a bug of the tiles renderer.
4. I know this is currently not possible to share the same plugin among multiple tilesets. Imagine however the following use case. I have a quantized mesh terrain I overlay with aerial imagery using the ImageOverlayPlugin. I also overlay the same aerial imagery over another tileset of 3d buildings (to have real roof textures). Currently the aerial imagery is loaded twice and I haven't been able to find a way to synchronize both overlays so that they are not processed twice for nothing. Any hints?

Supplemental Data

No response

Library Version

v0.4.14 (latest)

Three.js Version

r179.1 (latest)

[gkjohnson]

1. ... you can add a lot of different ImageOverlays, but why not just improve the existing TextureOverlayPlugin?

Fundamentally the plugins work differently. The TextureOverlayPlugin can only load textures per tile that are already aligned with the existing UVs - eg for the types of overlays in the example that are generated specifically for the given tile set. The ImageOverlayPlugin is designed to allow for overlaying tiled image formats with different global CRS' on top of any tile set. EPSG:4326 and EPSG:3857 (Web Mercator) can be overlaid on top of each other on any tile set, including Quantized Mesh and Google Photorealistic tile sets, which doesn't adhere to any tiling scheme. The overlays in the demo use two different CRS'. In order for the TextureOverlayPlugin to do anything like this the tiled image set has to be exactly aligned with the tile set, which would typically not be guaranteed. Admittedly the naming of the two plugins could be changed to more clearly distinguish the two.

In order to support this the ImageOverlayPlugin is quite a bit more complicated and must generate new UVs to accomodate the different CRS distortion, compose a subset of image tiles into a render target per tile based on the coverage, and each image set must keep a separate render target to avoid precision issues in shaders.

In fact WMTS in particular is a more complex format and it may be the case that even if it's using the same CRS as a quantized mesh format the data will not align.

2. ... Because I am experiencing a big performance degradation after migrating TextureOverlayPlugin -> ImageOverlayPlugin (see the diffs here). It is really slower to load the overlay tiles. Is the new plugin still WIP and you expect to fix known performance issues or is there something wrong on my side?

Because of the differences in complexity there are likely some performance differences but without a simple set of examples to compare the two or an example of what "slow" means (framerate? time to download?) I can't say whether it can be fixed or what it might be coming from. If you'd like to open a new issue to investigate we can take a look.

One notable difference would be that the ImageOverlayPlugin is that it limits the number of tiled images that can be downloaded at once by using the TilesRenderer download priority queue whereas the TextureOverlayPlugin would simply kick off hundreds or thousands of downloads. You could try increasing the allowed number of downloadQueue jobs to see if that helps things.

3. ... tile.__hasRenderableContent is set to false even though there are child tiles. This implies tile.__depthFromRenderedParent not being set properly and thus _calculateLevelFromOverlay returning a wrong value which stops the plugin from loading child tiles.

I'm not following this. It may be a bug but "_calculateLevelFromOverlay" should just indicate which level of image tiles to use for the overlay of the tile and shouldn't stop tile geometry downloads. "__hasRenderableContent" also only refers to the current tiles "content" field and is not indicative of any child tile geometry.

Another simple demo in a dedicated issue would be helpful for understanding this. ADD tile sets are currently working according to the spec but I'm not testing them often. A demo in the project would help, I think (perhaps with the swiss data sets).

4. ... I know this is currently not possible to share the same plugin among multiple tilesets.

Yeah unfortunately this is a limitation of the plugin at the moment. It may be possible to change things so that multiple ImageOverlayPlugins can share overlay instances but it's not currently possible. I didn't really expect this to be a common use case. Let's make another dedicated issue for this and we can look into what's involved in this change.

[xawill]

1. Understood. Thank you for the explanation. Indeed, the tile sets alignement requirement is a big limitation. So if I understand correctly, it could be possible to support other projections (like Swiss EPSG:21781)? I checked a bit and would propose to have ProjectionScheme abstract and have a subclass per supported EPSG. Currently only two projections seem to be supported and the switch between both is not really scalable IMO. Would that make sense to you? If not, how could I for instance overlay an aerial imagery tileset in EPSG:21781?

2. Thank you for the insights. Increasing downloadQueue.maxJobs to something like 30 indeed improved performance, but I don't find the overall experience as "snappy" as before. I'll open an issue when I am ready to work on performance improvements ; currently not able to propose a useful repro case.

3. Yes, for me it is a bug; opening a dedicated issue with proper repro case.

4. Great; happy you are open to the proposal. Opening a separate issue for the feature request.

(Bonus) Could you explain in what #1277 is linked to this issue?

[gkjohnson]

it could be possible to support other projections (like Swiss EPSG:21781)?

Yes it's possible that other projections could be supported if there are data sets that are using it commonly. I'm collecting data sets and formats to support in #943. If there is example data with this projection I can add it to that list.

(Bonus) Could you explain in what #1277 is linked to this issue?

Yeah I intended to link #1206 which has the same name as this issue 😅

---

Thanks for making the issues - I'll try to respond to some of them and the PRs in the coming weeks. I'm currently very subscribed with projects and other work so sorry for delays in some responses.

[xawill]

No pressure! Take the time you need and thank you very much for the library ü