Improvements on Depth Testing for Billboards

[Beilinson]

Feature

Currently, disableDepthTestDistance=5000 by default. From my experience, this is too large of a value over our use case with relatively flat but slightly hilly terrain, resulting in too many billboards that should be hidden behind terrain to appear as if they were in front (video example 1). Due to this, we usually set disableDepthTestDistance=100, which our specific use case is an improvement on the visual pop-in that we otherwise experience. However, this leads to billboards being rendered below the ground for most zoom levels (see video example 2 below).

I decided to look around for similar issues and found a significant amount of issues with their end result being unclear to me:

1. Billboards on terrain - some problems #4686 tagged as high priority since 2018, have 1-4 been updated over the various billboard clamping changes that have occured since 1.114?
2. Billboard ordering incorrect when using disableDepthTestDistance #6838 I can reproduce this as well today, however I wonder if this a particularly relevant real life use case? This ticket led me to the biggest questions:
3. Enable improved depth testing for all billboards #6802 This was closed due to being blocked by Billboard ordering incorrect when using disableDepthTestDistance #6838 ("ideally"), and disableDepthTestDistance causes weird mouse interaction #6840, which seems to be fixed (from my testing and from the conversation in the ticket). Despite this, I clearly see that parts of Enable improved depth testing for all billboards #6802 made it into main, for example the depthsilon (which is indeed a fantastic variable name) logic exists in BillboardCollectionVS.glsl. What exactly happened here?
4. Label letters misaligned when zooming in #7376 was mentioned at the end of disableDepthTestDistance causes weird mouse interaction #6840 (which seems to be why it isn't closed yet?), however I cant reproduce this either and this hasn't been mentioned since 2017.
5. Separate pass for billboards #8810 was opened as a result of Enable improved depth testing for all billboards #6802 and disableDepthTestDistance causes weird mouse interaction #6840, but again it looks like parts of Enable improved depth testing for all billboards #6802 made it in so what exactly is the status of this?

Personally, point 5 from #4686 was marked OBE (overruled by events? I think), however it makes a whole lot of sense to me and wasn't really explained:

Please consider depth offset in the BillboardCollectionVS.glsl. In previous versions there were the following lines:

#ifdef CLAMPED_TO_GROUND
    // move slightly closer to camera to avoid depth issues.
    positionEC.z *= 0.995;
#endif    

It would be great if, in addition to eyeOffset paramter in billboard and label, there will be a depthOffset which will be expressed as a fraction of the camera distance to the vertex (0-1).

I played around with scaling the depth in the BillboardCollectionFS.glsl, and this actually resulted in a much nicer transition and behavior for billboards that are only barely occluded by the terrain.

Default Behavior (pop-in, billboards behind terrain appearing fully instead of partially occluded):

default.webm

disableDepthTestDistance=500 (still pop-in, all billboards are now partially occluded by the terrain beneath them)

depthDistance100.webm

disableDepthTestDistance=500 + Depth Scaling by factor of 0.97 (less pop-in, smoother transitions, billboards on top of terrain are fully visible while those behind hills are partially occluded as intended):

depthDistance+depthScaling.webm

#ifdef LOG_DEPTH
    float depth = czm_branchFreeTernary(v_depthFromNearPlusOne < 1.1, v_depthFromNearPlusOne, v_depthFromNearPlusOne * .97);
    czm_writeLogDepth(depth);
#endif

[Beilinson]

Another example (from #4510):

Main:

Regular disableDepthTestDistance + Depth scaling patch on PointPrimitiveCollectionFS

Obviously from another angle the billboards end up clipping into the ground because the distance between the bottom of the graphic the larger it is, which is why I would also support this being a custom attribute passed in per instance like suggested in #4686.

[Beilinson]

I just wanted to point out that this is relevant for points as well, because unlike with billboards which we can set the verticalOrigin: VerticalOrigin.BOTTOM , point primitives logically have no such option and so suffer from this problem always.

I'd also like to point out that as of now PointPrimitiveCollection is missing the following code which exists in BillboardCollection:

if (!defined(disableDepthTestDistance)) {
  disableDepthTestDistance = clampToGround ? 5000.0 : 0.0;
}

So points actually have a default disableDepthTestDistance=0.0 rather than 5000 like for billboards (and by extension labels).

[mzschwartz5]

Hey @Beilinson - I haven't fully digested your in-depth (hah) write up here, but check out this PR - like, literally check out and test out with. I think it should address these issues, but it's still in progress so I definitely want to know if it doesn't fully cover what you've identified.

[Beilinson]

Hey @mzschwartz5, I checked this out just now with your latest commit, this doesn't really address the issue/solution I mention here, at least not from a first look. This is what I see with your PR on the following sandcastle:

Default disableDepthTestDistance:

This looks the same as in main

disableDepthTestDistance=20:

It seems that billboards are overlapping, even more so that further away billboards are actually appearing on top of closer ones. Also it doesnt really look like disableDepthTestDistance is changing the distance at which the depth test happens,

disableDepthTestDistance=Infinity:

From what I understood the primary goal was to transfer the depth testing that exists in main to work better on global scales, and this does seem to be working nicely.

1. I mentioned above billboard being incorrectly culled #4510 (which links to Billboard multifrustum clipping #2139). Note that despite the title and description of those, the sandcastle uses a PointCollection which apparently has a separate pipeline from BillboardCollection, so currently this PR does not improve that. I replicated something similar to the sandbox from there, so you can compare as well:
   Point Depth test Sandcastle

2. I actually am prioritizing local scale depth testing. I would prefer to disable it entirely, and trust the fragment depth testing, as it works better for the most part on any real features that should occlude the billboard. Where it breaks in my opinion is when fighting with terrain that is directly beneath it. Please compare the above images to the below image, which just slightly nudges the depth of the billboard forward by a percentage based amount (so it works through all zoom levels consistently), which is what I mentioned above in my suggested solution:

All I've done is replaced:

cesium/packages/engine/Source/Shaders/BillboardCollectionFS.glsl

Lines 158 to 160 in 1dcf973

	#ifdef LOG_DEPTH
	czm_writeLogDepth();
	#endif

with:

#ifdef LOG_DEPTH
    // Nudge the depth forward by a factor of 3% unless it is too close
    float depth = czm_branchFreeTernary(v_depthFromNearPlusOne < 1.1, v_depthFromNearPlusOne, v_depthFromNearPlusOne * .97);
    czm_writeLogDepth(depth);
#endif

Note that unlike with the cesium disableDepthTest logic, this continues occluding billboards that are genuinely partially behind a hill, but keep the terrain directly under the billboard from occluding them:

Before:

After:

To me, the second is visually more appealing, and less nauseating when moving over hilly terrain. What do you think?

[mzschwartz5]

@Beilinson

There's a lot to take in here, so forgive me if I don't respond to everything, but here are a few thoughts:

1. Billboards and labels are both rendered using the BillboardCollectionVS/FS shaders. Make sure to test both because I've seen many "fixes" for billboards that don't work well on labels. I wasn't aware points were a separate pipeline, but whatever we do figure out for billboards can likely be applied there as well.
2. "It would be great if, in addition to eyeOffset paramter in billboard and label, there will be a depthOffset which will be expressed as a fraction of the camera distance to the vertex (0-1)." - mathematically speaking, there really shouldn't be any difference between adjusting in eye space vs. NDC space. Both approaches adjust depth in the end, but the former does it in linear space (much more intuitive) while the latter does it in log space (but not always! Only if #LOG_DEPTH is defined, which further complicates matters). So when you adjust by 3%, it's in log space, which can be quite a big nudge, and may cause more artifacts in some cases than it solves.
3. "From what I understood the primary goal was to transfer the depth testing that exists in main to work better on global scales" - this was only part of the goal. It turns out that there's a special test that was run on billboards at close proximity, which you can read about here. However, it did not behave as advertised. There were several bugs, and the behavior actually kind of worked backwards. More info in this comment.
4. I feel like there's generally something else going on in your above photos that we're not understanding... for example, when you set disableDepthTestDistance=20, I wouldn't expect that image to look any different than the first photo, yet it does. The camera is clearly further than 20m away from any billboard, so behavior should not change (at least, if you took those photos on my PR branch?)

[mzschwartz5]

Also, it might be worth revisiting this after we merge #12821, because it makes some pretty drastic changes that may void some points of our discussion here. Like, as I mentioned above, it does affect local-scale depth testing, albeit in a subtle way, and it especially affects the meaning of disableDepthTestDistance, which was previously being misinterpreted.

[Beilinson]

I feel you definitely touched on my main point. I think revisiting after #12821 is merged is fair.

Just a few minor responses:

"It would be great if, in addition to eyeOffset paramter in billboard and label, there will be a depthOffset which will be expressed as a fraction of the camera distance to the vertex (0-1)." - mathematically speaking, there really shouldn't be any difference between adjusting in eye space vs. NDC space. Both approaches adjust depth in the end, but the former does it in linear space (much more intuitive) while the latter does it in log space (but not always! Only if #LOG_DEPTH is defined, which further complicates matters). So when you adjust by 3%, it's in log space, which can be quite a big nudge, and may cause more artifacts in some cases than it solves.

depthOffset being percentage based in my opinion rather than a linear offset (depthMultiplier maybe is more accurate) is the strength of this solution, as at different zoom levels this would shift the depth of the billboard proportionately to the zoom level, just like terrain feature size being scaled in screen space proportionately to the zoom level.

I didn't test this with LOG_DEPTH disabled, so thats an interesting question, and you're correct that most likely this multiplier should be scaled logarithmically as well maybe?

I feel like there's generally something else going on in your above photos that we're not understanding... for example, when you set disableDepthTestDistance=20, I wouldn't expect that image to look any different than the first photo, yet it does. The camera is clearly further than 20m away from any billboard, so behavior should not change (at least, if you took those photos on my PR branch?)

Definitely weird to me as well, I indeed took this pictures on your branch running locally. I'll leave the sandbox here again for you to test:

https://sandcastle.cesium.com/#c=1Vdrbxo5FP0rXj4NysQMZBNIINGmJGoqkWRF2fZDWUVm5s5grbGRx5PAVvz3vZ4Xw6MNUXelrhQJuD73+L587PhKxoY8c3gBTS6JhBfSh5gnM/optTnjmp/+7itpGJegxzWXfB1LQgxojZaLwmGU/aahVrPPSosgNzh1dyxX9e5YZtvQ2AcJNBJqAjSAuZmOIDbXEUJjk/tgKEYngD5j6achghB8Hise4NIennK5WzhMuBATxXTQV0KAb7iS265zzWfc8GeIKQsCxyZVKcC7XQInTZyQ1P1ig8y1Kyubaj2NutEgMZp9VsSDIKzYQMmImyQAjOW4SU86py3v9NeTk9bZebt13t0CM1NgPXrWbrfPW2enHa/T6TTPTiq1iTQPPvPATBHY9Lxu1X4HPJqarQWN6TAZCbhjIvzI/8528DzvtFrxzXkYFj5pnbazOd7ldCu4IpFXYCXd0WF0+2G2ATGYEZ+BSozj1MnllR3ZUGniCDBkadPt4kevUqIuOTpa1rPRLpGLDLnIkWmRLXCRAzGitFai0ta8XvfMTKkAPXeAvuCAuwSnlNnPBWkQZ92zY9Ks25ArbOu+7yGLMampZZNK2y/Lgi5v9S7fXMU8n/9KQ/tMGxVpNp9y3ykTcMvdszm2LHuOUnpg8hKQcoOL9TGlfoV+pOxmuC2TToGtu4U3n7EIT9O4Rmkj/R43QuZzwc2SRjxEwSmQ0zTDIYSgQfpQSs/dpp32B9f3vz+NHp/eDx//eLgp/QMes4mAm0J0bniM02OJWl4OWmWlW6Fk4Z+LC563d6Dyw++zGWhGQ7EcKecrCZCVS5YXwwpCyTCWdqLKWqaWj0wGPg6FAFvQkVJiwvS7xBjUmnHtOgjWeKu7YSIzIXPy+eMhcX7JixBAiPocOKVHvRzS0oQT8K82s/Ah3xysrZPtbh1h1+oSSs96Gr47F9kOTwMVqSf1DFqwJZ3LqDIisc8EOnm0/eapGd4OrkcfPt3uDE5lJuoHtG0IM4ztgM69oXGVZumUvoLu7jY5kTnpIYHfg0ycLxaY72pgYWzdh2C1AFMxCuVPIWlZaNQVsAFd7GaWtu6g7DaiplttWovf633qFnyrvGHZbezuyakv2Gz+kyW0JVdvyOZBSfgpUnh4fLj9dtx/1ivvCrxq3msmiydH/ub4/pH6IH2NlyesL9ofOFT5E6uiUhjLptal6pYJXalk66qUV1hxy1ZgdP0UOLosct0+obRyI78uslX725ToBv6HZTv+z8tWc2u92CwFXFnrb3w2x5cUSbRw8KYxMJuj7OFlM0n8v8BQP46tU69RuPQC/kx4cLnnXyPiCxbHuBImQtjn6Lh21WsgfsNNKBZwGT1mF5iFTJtXg8xIKe018Oeul8maW2H8Bw

[mzschwartz5]

depthOffset being percentage based in my opinion rather than a linear offset (depthMultiplier maybe is more accurate) is the strength of this solution, as at different zoom levels this would shift the depth of the billboard proportionately to the zoom level, just like terrain feature size being scaled in screen space proportionately to the zoom level.

That's a good point, but I think you could do the same in linear depth by nudging the eye space position forward by a percentage of the near-to-far plane distance (rather than by a percentage of the eye space position itself). Thoughts?

edit- I don't think "zoom level" has any relevance to either approach. The NDC log depth-space nudge is not based on zoom level. It's based on the camera frustum, which has the same near and far plane whether or not you're zoomed in or out.

[Beilinson]

That's sounds like a good idea, and yes my bad when I mention zoom level I'm really talking about the near/far plane as you said.

Thanks for taking the time and being so responsive here, hope to continue after #12821 merges!